# VIDEOS

For best visualization of these movies, please select the full screen feature in your AVI player.

These videos demonstrate the results of the policies generated by our algorithm on the two Freecraft domains described in the paper:

# Tactical domain

## Planning world

#### AVI video

We planned in a model with 3 footmen and 3 enemies. These footmen coordinate their attack and are able to kill the enemies, losing only one footman in the process.

## Generalization world

#### AVI video

We generalized our value function to a problem with 4 footmen and 4 enemies. The coordination strategy generalizes very effectively. Note in the video that at first, all footmen attack one enemy. As this enemy weakens, they coordinate, decreasing the number of  footmen on this enemy, until only one footman is left. This footman finishes off the enemy, and moves on to the next one.

## Generalization world

#### AVI video

We then generalized our value function to a problem with 5 footmen and 5 enemies. Here, the footmen also demonstrated interesting coordination. For example, in the beginning 4 footmen attack one enemy and 1 attacks another. Our model says that the probability of killing an enemy with more than 4 footmen is the same as the one with 4 footmen. Although we never planned in a problem with 5 footmen, our generalization captures this characteristic of the model. Unfortunately, at this point, we do not take into account the position of the units. In this example, one unit spends a lot of time trying to reach an enemy. This inefficient causes our policy to lose to Freecraft's one in a close battle.

# Strategic domain

## Planning world

#### 2 peasants, 1 barrack, 2 footmen and 1 enemy

AVI video

We planned in a model with 2 peasants (a builder and a regular peasant), 1 barrack, 2 footmen, and an enemy. Note that only after both footmen are built, they start attacking the enemy. Nonetheless, both are killed by the stronger enemy. However, as the enemy becomes weak after this attack, in the next round, only one footman is sent to attack it.

## Generalization world

#### 9 peasants, 1 barrack, 3 footmen and 1 enemy

AVI video

We generalized to a model with 9 peasants (a builder and 8 regular peasants), 1 barrack, 3 footmen, and an enemy. Here our policy waits for the 3 footmen to be built before attacking. They are able to kill the enemy, and only one footman is lost.