Progress on Features, Vectors and Game Recording
It took several days to optimize the architecture of my game capturing classes and the feature vector representation. Here is what I came up as a first approach to the overall goal of giving the bot a sense of decision making.
Spoiler: As a first exercise the data will be used to train a prediction model to predict the outcome of a game for Odysseus. In this process I’ll understand the data and if it’s enough or not.
Actually the code could be plugged in to any StarCraft bot to retrieve data in a representation easy for machine learning algorithms. I’m thinking of decoupling that part for easy integration. Stay tuned ;)
Representing Features in Odysseus
In order for a machine learning model to use the captured information from the
MatchRecord classes these have to be further processed into another representation. Basically I want to create a simple integer vector holding every information encoded in numerical values which can be exported to a
.csv file for example.
Feature vectors are used in many machine learning models to represent a unified, numerical input that can be processed and optimized through a learning algorithm until we receive a desired model applying some transformation on an input vector to a reaction at the output. While the representation of an reaction is a complete different story, let us have a look at the feature representation first.
From an Engineering Perspective
FeatureVector: Is a numerical vector representation of a
Recordin a model specific representation.
This class has two constructors and several vector operations like
transpose(). It is residing within the
Features namespace together will some helper structures:
FeatureNamesis a vector that maps string-keys to corresponding positions of the feature vector.
UnitTypeNamesthis is a map that transforms between
BWAPI::UnitTypesenumeration and string keys. This is used for all involved structures to identify them in a unified way.
I took some inspiration on that of the Facebook library TorchCraft1.
FeatureVector is a vector of numerical information. It has two different representations and an associated hard-coded revision number that identifies its compatibility with a computed machine learning model.
The two representations are captured by a type. One takes just a
Record, which resolves in containing only information of one specific moment, without a link to any game in particular. The second takes a
MatchRecord in addition to the
Record and therefore leads to an interface where information about the match, like victory and scores, is also present in every training example.
Thoughts about the Decision Making Model
There are several ways how a decision making model could look like. In my implementation it will be hooked into existing code, receive features and compute some reaction with it. It should be capable of playing any race against any other race. Here are some of my ideas about that:
Construct a full model which can play any race
Advantage: Shared knowledge for specific tasks that are equal for all races.
Disadvantage: AI could try to construct units that it cannot produce because of a different race. Information and possible actions of other races could confuse/distract the learning process.
Construct a race based model with full inclusion of all other races
Advantage: The bot learns based on its race and the races abilities to play against any other races with the same model.
Disadvantage: Not as general. Three models have to be trained instead of one. Some of the learning will be redundant.
Construct a Race vs. Race model which is specifically trained by match up
Advantage: extremely specialized, with little distraction.
Disadvantage: Need to train 9 models instead of 3 or 1. Heavy redundancy. Could be more inefficient than learning one model with distractions.
Construct an opponent bot based model
Advantage: direct counter measures against specific bots. More specific data, no distraction.
Disadvantage: most likely to loose a lot of generalization, redundancy and possibly less efficient. Less data.
By distraction I’m talking about the problem, that the AI could decide to build a Protoss unit while playing Zerg, because it decides that this would be a good counter move in that situation. It then would have to learn from the reaction (which is none) coming from the game that there are better moves than that.
I would assume that I’m as a human player use all of these representations in the course of one game. Starting from the bottom and going to the top depending on the information I have about my enemy. The difference however lies within my capabilities of using an efficient model that is able to differentiate and mix knowledge from different matches. Basically I can merge my models. The AI would also need this ability to merge it’s models or experiences from different matches.
If a neural-net is trained with enough computational power and time the net should be able to overcome the distractions of a universal model and learn useful and not-so useful actions. But that’s fairly vague.
Next the focus lies on getting to know my data. With several scripts it is planned to visualize and analyse the retrieved datasets.
After that I’ll have a look at the Steamhammer implementation of Strategy. I have to understand how opening books are defined and processed. Currently it seems I’ll stick with that idea. It’s also of great interest how the “dynamic” strategy decision making process looks right now. I divide the decision making in these two categories. Opening book and strategy generation.
When I understood the code and mechanics the fun part begins. Somewhere in those functions the actual machine learning brain interface will arise, which will take responsibility of some parts of the decision making, if not over all of it.
Yay, this is my first post with footnotes! Hope it works, cause I like that nifty, little thingy.
Any thoughts of your own?
Feel free to raise a discussion with me on Mastodon or drop me an email.