Progress on Features, Vectors and Game Recording

It took several days to optimize the architecture of my game capturing classes and the feature vector representation. Here is what I came up as a first approach to the overall goal of giving the bot a sense of decision making.

Spoiler: As a first exercise the data will be used to train a prediction model to predict the outcome of a game for Odysseus. In this process I’ll understand the data and if it’s enough or not.

Actually the code could be plugged in to any StarCraft bot to retrieve data in a representation easy for machine learning algorithms. I’m thinking of decoupling that part for easy integration. Stay tuned ;)

Representing Features in Odysseus

In order for a machine learning model to use the captured information from the Record and MatchRecord classes these have to be further processed into another representation. Basically I want to create a simple integer vector holding every information encoded in numerical values which can be exported to a .csv file for example.

Feature vectors are used in many machine learning models to represent a unified, numerical input that can be processed and optimized through a learning algorithm until we receive a desired model applying some transformation on an input vector to a reaction at the output. While the representation of an reaction is a complete different story, let us have a look at the feature representation first.

From an Engineering Perspective

• FeatureVector: Is a numerical vector representation of a Record in a model specific representation.

This class has two constructors and several vector operations like transpose(). It is residing within the Features namespace together will some helper structures:

• FeatureNames is a vector that maps string-keys to corresponding positions of the feature vector.
• UnitTypeNames this is a map that transforms between UnitType of the BWAPI::UnitTypes enumeration and string keys. This is used for all involved structures to identify them in a unified way.

I took some inspiration on that of the Facebook library TorchCraft¹.

In-depth View

Features

FeatureVector is a vector of numerical information. It has two different representations and an associated hard-coded revision number that identifies its compatibility with a computed machine learning model.

The two representations are captured by a type. One takes just a Record, which resolves in containing only information of one specific moment, without a link to any game in particular. The second takes a MatchRecord in addition to the Record and therefore leads to an interface where information about the match, like victory and scores, is also present in every training example.

Thoughts about the Decision Making Model

There are several ways how a decision making model could look like. In my implementation it will be hooked into existing code, receive features and compute some reaction with it. It should be capable of playing any race against any other race. Here are some of my ideas about that:

• Construct a full model which can play any race

  Advantage: Shared knowledge for specific tasks that are equal for all races.

  Disadvantage: AI could try to construct units that it cannot produce because of a different race. Information and possible actions of other races could confuse/distract the learning process.

• Construct a race based model with full inclusion of all other races

  Advantage: The bot learns based on its race and the races abilities to play against any other races with the same model.

  Disadvantage: Not as general. Three models have to be trained instead of one. Some of the learning will be redundant.

• Construct a Race vs. Race model which is specifically trained by match up

  Advantage: extremely specialized, with little distraction.

  Disadvantage: Need to train 9 models instead of 3 or 1. Heavy redundancy. Could be more inefficient than learning one model with distractions.

• Construct an opponent bot based model

  Advantage: direct counter measures against specific bots. More specific data, no distraction.

  Disadvantage: most likely to loose a lot of generalization, redundancy and possibly less efficient. Less data.

By distraction I’m talking about the problem, that the AI could decide to build a Protoss unit while playing Zerg, because it decides that this would be a good counter move in that situation. It then would have to learn from the reaction (which is none) coming from the game that there are better moves than that.

I would assume that I’m as a human player use all of these representations in the course of one game. Starting from the bottom and going to the top depending on the information I have about my enemy. The difference however lies within my capabilities of using an efficient model that is able to differentiate and mix knowledge from different matches. Basically I can merge my models. The AI would also need this ability to merge it’s models or experiences from different matches.

If a neural-net is trained with enough computational power and time the net should be able to overcome the distractions of a universal model and learn useful and not-so useful actions. But that’s fairly vague.

What’s next

Next the focus lies on getting to know my data. With several scripts it is planned to visualize and analyse the retrieved datasets.

After that I’ll have a look at the Steamhammer implementation of Strategy. I have to understand how opening books are defined and processed. Currently it seems I’ll stick with that idea. It’s also of great interest how the “dynamic” strategy decision making process looks right now. I divide the decision making in these two categories. Opening book and strategy generation.

When I understood the code and mechanics the fun part begins. Somewhere in those functions the actual machine learning brain interface will arise, which will take responsibility of some parts of the decision making, if not over all of it.

Off-topic

Yay, this is my first post with footnotes! Hope it works, cause I like that nifty, little thingy.