Progress on Features, Vectors and Game Recording

It took several days to optimize the architecture of my game capturing classes and the feature vector representation. Here is what I came up as a first approach to the overall goal of giving the bot a sense of decision making.

Spoiler: As a first exercise the data will be used to train a prediction model to predict the outcome of a game for Odysseus. In this process I’ll understand the data and if it’s enough or not.

Actually the code could be plugged in to any StarCraft bot to retrieve data in a representation easy for machine learning algorithms. I’m thinking of decoupling that part for easy integration. Stay tuned ;)

Representing Features in Odysseus

In order for a machine learning model to use the captured information from the Record and MatchRecord classes these have to be further processed into another representation. Basically I want to create a simple integer vector holding every information encoded in numerical values which can be exported to a .csv file for example.

Feature vectors are used in many machine learning models to represent a unified, numerical input that can be processed and optimized through a learning algorithm until we receive a desired model applying some transformation on an input vector to a reaction at the output. While the representation of an reaction is a complete different story, let us have a look at the feature representation first.

From an Engineering Perspective

This class has two constructors and several vector operations like transpose(). It is residing within the Features namespace together will some helper structures:

I took some inspiration on that of the Facebook library TorchCraft1.

In-depth View

Features

FeatureVector is a vector of numerical information. It has two different representations and an associated hard-coded revision number that identifies its compatibility with a computed machine learning model.

The two representations are captured by a type. One takes just a Record, which resolves in containing only information of one specific moment, without a link to any game in particular. The second takes a MatchRecord in addition to the Record and therefore leads to an interface where information about the match, like victory and scores, is also present in every training example.

Thoughts about the Decision Making Model

There are several ways how a decision making model could look like. In my implementation it will be hooked into existing code, receive features and compute some reaction with it. It should be capable of playing any race against any other race. Here are some of my ideas about that:

By distraction I’m talking about the problem, that the AI could decide to build a Protoss unit while playing Zerg, because it decides that this would be a good counter move in that situation. It then would have to learn from the reaction (which is none) coming from the game that there are better moves than that.

I would assume that I’m as a human player use all of these representations in the course of one game. Starting from the bottom and going to the top depending on the information I have about my enemy. The difference however lies within my capabilities of using an efficient model that is able to differentiate and mix knowledge from different matches. Basically I can merge my models. The AI would also need this ability to merge it’s models or experiences from different matches.

If a neural-net is trained with enough computational power and time the net should be able to overcome the distractions of a universal model and learn useful and not-so useful actions. But that’s fairly vague.

What’s next

Next the focus lies on getting to know my data. With several scripts it is planned to visualize and analyse the retrieved datasets.

After that I’ll have a look at the Steamhammer implementation of Strategy. I have to understand how opening books are defined and processed. Currently it seems I’ll stick with that idea. It’s also of great interest how the “dynamic” strategy decision making process looks right now. I divide the decision making in these two categories. Opening book and strategy generation.

When I understood the code and mechanics the fun part begins. Somewhere in those functions the actual machine learning brain interface will arise, which will take responsibility of some parts of the decision making, if not over all of it.

Off-topic

Yay, this is my first post with footnotes! Hope it works, cause I like that nifty, little thingy.

  1. https://github.com/TorchCraft/TorchCraft 

Any thoughts of your own?

Feel free to raise a discussion with me on Mastodon or drop me an email.