Starcraft AI blog

I’ve mentioned it briefly before, but here’s a longer post. It is past time to start writing serious machine learning methods into Steamhammer. I’ve been writing code for it behind the scenes, though none is in Steamhammer yet, even in the development version. I selected a combination of techniques that will learn fast, will run fast, and I hope will be accurate enough. Right now I need to fix a numerical precision problem (I’ve always hated numerical analysis), but soon it should be ready to start testing on non-toy problems. It won’t be in the next Steamhammer 3.1, but perhaps a version or two after that, if all goes well.

The first application will be a “will I win this game?” evaluation function. The idea of evaluation functions is very general: You can evaluate anything, “how good is this build order?” “am I likely to win this fight?” “which tactical maneuver is better, A or B?”—anything you want to measure or compare, really. The use of evaluators is also very general. Whenever you want to make a choice, if you have the right evaluator and you can provide it the right inputs, you can compare the choices and pick the one that looks best. That is what search is, and search is one of the most basic ideas in AI.

The “am I winning?” evaluator will take several hundred numbers as inputs, unit counts and things like that. You can see my 2018 analysis of LastOrder for some of the possibilities. The output will be an evaluation of how likely Steamhammer is to win from the game position, I think a probability or something that can be converted to a probability. My initial estimate is that it should run in under 5 milliseconds. It doesn’t need to be run often, so even if that’s optimistic it will be fast enough. If it works as well as I hope, it will be possible to specialize the evaluator for each opponent. If that succeeds, there will be a pre-learned evaluator for unfamiliar opponents, and learning data in the opponent model will update it to understand that player. I’m seriously expecting the learning to be fast enough for that to help, though we’ll see.

The first use of the evaluator will be to select openings. Right now Steamhammer keeps tabs on whether a given opening won or lost. The bot does not know, at least until it plays a lot of games, whether it won because the opening gave it a huge strategic advantage, or whether it was behind after the opening but managed to scrape a win anyway. The evaluator will tell it, and it will select better openings. For example, against a much stronger opponent Steamhammer rarely wins and falls back on trying builds at random, hoping to hit one that works. Most of the random choices are poor, but it is losing every game anyway so it can’t tell. The evaluator will tell it which tries are more nearly successful; it will try those more often and have better chances. That is only an example; I expect the evaluator to help against most opponents.

A later use of the evaluator will be to construct new builds. I have plans in mind. There is already code in Steamhammer—it’s not finished or working, but the nub of the idea is there—to simulate build orders. When that is in place, Steamhammer will be able to evaluate builds that it has never played in a real game and get an idea of whether they will work. “I got run over fast. If I substitute 12 pool for 12 hatch, am I ready in time?” If that succeeds, Steamhammer will be able to customize builds to counter specific opponents. The potential is great, and this evaluator is a key step on the way.

Starcraft gives the players many many choices. It’s not possible to search any large proportion of them. In the search/knowledge tradeoff, I think that means that knowledge is preferred: You want to search few choices (at least compared to how many there are), but select and evaluate the choices with a lot of knowledge. That’s why I think that knowledge-rich machine learning methods are the right way.