Embedding Python - An Odyssey

Now that I have decent training and validation sets and a first prediction model, the question is how to integrate it into the bot. As data scientist I work with python and tensorflow for a proof of concept. It is certainly possible to use other libraries and techniques than that for a real game development process. Maybe with full, fast pipeline support on Windows in C++. This is something for the future, when this turns out to work though.

Since I have to write my thesis based on the results so far, I will focus on that for the next week which should result in less practical work and fewer posts.

I also have the permission now to publish all code and the models as open source - so this will happen soon.

First Python Module integration is live!

Yesterday I developed a first extension point for python scripts - the predictor. The predictor provides different methods to use the captured game state in conjunction with a trained model to come up with some decent prediction. At the moment it just predicts the probability of winning the current game.

Logically the predictor helps the bot evaluating it’s current situation and performance. It also serves as a good tool for guiding the training of other components like decision making. When training a neural net it can be used as the gradient target.

Technically the predictor is the first of several extension points to provide custom behaviour for specific components. The extension point provides several parameters, an API for callbacks and the processing of an expected result. It checks if a module for the extension point is present. If no module is found or an error occurs it either silently calls a fallback module or notifies the user that it will be deactivated.

The development was mainly guided by the Python/C API Documentation, especially the embedding part, the Reference and several stackoverflow answers^{1 2}.

The biggest and most annoying problem was the actual import of the module from the disk. While embedding python into a C/C++ application is not complicated at all, especially with the great documentation, it got me struggling with really low-level, to unexperienced-problems. First I tried to import the module using a windows absolute path to the location of the DLL-library, so I can simply ship the provided extensions with the DLL without a need for the user to copy/paste them somewhere. I already did this successfully with the config file. Guess what? Pythons import is a real pain in the a**. I ended up by importing it with a relative import from bwapi-data/AI/modules/predictor.py. If someone knows how to do it with a path like C:\odysseus\bin\modules\predictor.py please let me know.