AI -- the Thread

[LDiCesare]

Mark, any idea how I can check the ai build orders? I made it so that the ai gives some build orders, but I want to check if that works. The econ.txt file gives a lot of output, but mostly for player civ. Or where should I put a trace in the econ code to check the ai got the orders correctly?
I just want to get some reinforcement code to work before I send the code. This will probably not allow to breach walls yet, but there is a figure in the xml files to tweak the quantity of breachers and attackers the ai thinks they need, so it might be tunable (when I'm sure ai reinforcements work).

[Mark_Everson]

Originally posted by LDiCesare
Mark, any idea how I can check the ai build orders? I made it so that the ai gives some build orders, but I want to check if that works.

Hi Laurent:

The orders for each Economy object are in a Map that is held in the class GovtEconOrders. You get it by calling econ.getGovtEconOrders(). First you need to get econ from either the square, province or civ object to which you sent the order, but you should already have that. In case you haven't discovered it already, most of the stuff you will need is in the game.economics.orders package.

Once you have the GovtEconOrders object you can call getGovtEconOrder(String name), which returns null if the order name doesn't exist there. If it exists it returns the GovtEconOrder object that is the individual order. It may be that all you need is to verify the order exists where you think you put it! However, if you need more specifics on the order that is there, you can use a large number of public methods in GovtEconOrder to find out details about the order, if you want to go that far.

Note that any economy that has had the econ interface called up for it will have one of every type of order there already. Most of them will be '0%' build orders that do nothing. It was done that way to make the interface have a copy of each order that could be shown in the gui. I am just warning you about this since its possible you could use the interface to verify some orders, and get mislead about whether you have created an order or not. For these dud orders, calling getPartOfLocalTaxesToUse() will always return 0. You can use that as a crude discriminator of whether you put the order there, or the interface did.

I hope this is the info you need. As always, if you need more, just let me know.

[LDiCesare]

I sent the code

Here's what the ai now does (requires -newai on command line to run):

It simulates fights before it sends units away, not just when it is by the target square.
Although this is much better, you probably won't see the results of it because the ai doesn't cheat and so it doesn't see your units, unless the map is revealed. If it knows one of your cities is defended, it will do some computation before going kamikaze.

When it can help, the ai will change build orders to get some reinforcements. The reinforcements should come from a reserve plan, but that's not yet implemented. The ai does change build orders, however. It adds one build order for a given unit type each time it wants reinforcements. This will eventually have to change, as in a long game, its build list will grow huge for little gain, but in the meantime it works. The ai chooses one unit, which best looks like the average of what the opponent has, and, if it targets walls, has breaching power.
The exact algorithm for reinforcements is to bring back the same quantity/quality of troops as what the opponent had in the square at the end of a simulated lost fight. This figure is multiplied by a constant (10 by default) if the opponent is behind a wall, and catapults (or other units that can breach walls) are required to breach the wall too. The quantity in case of walls needs tuning.

There are two things which could vastly help the ai:
- Getting the reserve plan to work;
so the ai will build new units and assign them to this plan. If I can remember the profile of armies I put in the plan, I can remove them when they are no longer needed. I'd have to be able to know that a unit has been produced because of a given order for that, which means I'd like to have two build orders, one Legion and one Reserve Legion. I'm not yet sure how to handle that. I also need to be able to build more than one type of unit as reinforcements.

- Seeing enemy units;
Of course, I could cheat, but I don't want to. I'd also like to remove the knowledge of other cities locations which is still in the code. In order to be able to see enemy units far away, we need espionage/scouting. I'll check if I can unearth a thread related to that topic. If someone else finds one, they're welcome to bump it.

[Mark_Everson]

Originally posted by LDiCesare
- Seeing enemy units;
Of course, I could cheat, but I don't want to. I'd also like to remove the knowledge of other cities locations which is still in the code. In order to be able to see enemy units far away, we need espionage/scouting. I'll check if I can unearth a thread related to that topic. If someone else finds one, they're welcome to bump it.

Hi Laurent:

I got the code, although I haven't opened it yet. I'm dying to try it out! Unfortunately I am very busy today, so I may not have time for much more than this post. Then again I may be back before long!

Can the AI see the units just as the player does? I think that is what you are saying, but I just wanted to be sure. I agree that we should keep the standard as a non-cheating AI, since we want to emphasize how different the Clash AI is from all the other games. However it might be worthwhile to put in an -AIcheat flag or some such so that the player can, at their discretion Allow the AI to see everything. What do you think? Is there a huge amount of added work involved? It could also be useful in that it would allow you to see how your AI does with perfect info.

[LDiCesare]

Can the AI see the units just as the player does? I think that is what you are saying, but I just wanted to be sure.

Exactly.
You can see how the ai works with full info by removing the fog of war altogether. I did it in the scenario I added (check the list of scenarios, the last one is called 'reinforcement test', and is just that). So you don't really need a hard coded flag.

I'd like to have some spying set up so that both players and ai can see through the fog of war somehow. I think it would be nice for the ai and player both, and that is something which is not well done in any game that I know.

[LDiCesare]

Currently, the ai manages single objectives (attack square X or square Y), but we'll need to have more complex plans, like (attack and defend). Any idea of the value of defending a square?
Defending is worth the square value times the probability of an attack if the square was not guarded? What defense should be put and how much?
Any better ideas?

[LDiCesare]

I think here's how to handle defense vs. offense:
Currently I have only implemented low level ai (task force level). Each task force uses the same attitude, which means they'll all attack or all defend...
It will be better to implement defense by adding a layer of ai on top of the task forces, so that each task force is attached to a higher level plan, each plan considering it needs a gicen proportion of the existing armies (or something more complex) to be successful: Typically, an attack plan and a defense plan.
This way, I can leave the code that handles task forces as is, and provide a beginning of a higher level thinking.
I'm not yet there, though, but at least, I can safely postpone the problem.

[Mark_Everson]

Hi Laurent, I think that sounds good for now. I have to admit that I had been thinking about the problem, but hadn't come up with any particulary good near-term solutions.

Certainly you can start with static offensive and defensive apportionment of forces as a percentage of the total. Probably a bit more quality can be gained fairly easily by having that proportion be a simple function of relative combat strengths of the different civs involved. That will have to wait until you have some simple spying or other info-gathering functions implemented.

One thing I did way back in Demos 3 and 4 was to have code that determined what the overall military posture should be based on forces available (it knew total forces of all the combattants, but not where they were) and the relative diplomatic statuses of the civs. When you get further along, we could bring something like that back.

[LDiCesare]

One thing I did way back in Demos 3 and 4 was to have code that determined what the overall military posture should be based on forces available (it knew total forces of all the combattants, but not where they were) and the relative diplomatic statuses of the civs. When you get further along, we could bring something like that back.

That's the way they do it in Galciv.
It's prone to many abuses, like building lots of cheap units which aren't handled correctly in terms of power (many poor units won't beat one good unit with a lesser total strength in that game - probably too in Clash though to a lesser degree), and diplomatic statuses weren't ascertained correctly either (ai would go to war with a weakling even though said weakling had a very powerful ally).
That said, I may go for it, but I'd like to get some spying in place before.

On the code front, I'm in the process of refactoring the whole reinforcement code so that the units built to provide reinforcement will be better fit to the task (type of unit updated every turn based on what has already been sent), and so I won't clutter the economics with hundreds of obsolete reinforcement build orders.
I'm still struggling with reserve units. I may go the rich way, i.e. have no reserve but always build reinforcements when I need some, or consider that units which have the same goal as my current unit are good candidates for a merge (typically if I have surrounded a city with 2 TFs, they might want to merge together and take the city rather than ask independently for reinforcements or change objective).

[Mark_Everson]

I think your approach is good Laurent.

I have finally had the time to check out last week's version of the AI. You have achieved a lot! All I have done so far is check out the Carthage scenario playing Rome. If I do nothing as Rome the AI functions basically competently until it gets down near Rome. The Hannibal TF of course can't take Rome without reinforcements that can handle the walls. I played until turn 40 and the AI was still calling for reinforcements that were streaming as single units all the way from the Carthaginian homeland. I have several suggestions for things that might improve the AI, although I don't know how difficult they would be to implement. I am just going to go through my suggestions here, and you can evaluate how good they are, and how difficult they would be to implement given where you are now. It is possible that the AI could also perform much better just by changing the parameter files. I haven't tried to do that yet.

1. This is somewhat of an aside, but I thought I would mention it before I forget. In the wave of conquest that spreads across southern France and northern Italy, a few squares get left behind, still in control of the Romans. Since the squares are generally ungarrisoned and would be trivial to take, something in the AI that makes that happen would be valuable. One quick suggestion that I have is to make the value of a square increase if it is held by the enemy but adjacent to friendly squares. Something like a simple small addition to its value per friendly square adjacent might be the right combination. That would mean that an isolated unfriendly square would tend to become a much more desirable target, even if economically worthless, and be taken over fairly quickly.

2. It is very undesirable that Hannibal camps outside Rome for 30 turns when the entire southern part of Italy is open for the taking! I left the two cities adjacent to Rome completely stripped of troops, and Hannibal would not so much as move the single square necessary to take them. I know this is due to the current simple nature of the AI, but an uncomplicated fix might be able to handle it at least for the short term. One idea is that an attack plan should calculate the time needed for the required reinforcements to be built and get there. As this time increases there should be an increasing chance that that plan will be scrapped and replaced with some other objectives that can be achieved in a shorter timescale. For one thing, by the time the reinforcements get there the enemy may also have reinforced, leading to a way to essentially deadlocked the the capabilities of the AI fairly cheaply with a wall city used as bait.

As for how smart planning of this sort can be done well, while still sending the reinforcements for the long-term plan of capturing Rome, I don't know. I think as the scenario is currently set up, it should work somewhat like the real war in that Hannibal should be able to take over much of the Italian peninsula aside from Rome fairly easily. If the AI can accomplish that, you will have achieved a lot!

3. The way that reinforcements go where they are needed is also undesirable. The first problem is that all these individual units moving over the map Really slow down the game. It took many seconds to crank a turn on my fairly new computer. Secondly, they are tactically vulnerable to fairly small forces if they move as single units. For both these reasons I think that some way to group reinforcing units into reasonably-sized TF's needs to come fairly soon. Obviously the amount of groupings importance increases as the distance traveled increases. For now, using major cities that are not too far off the optimal march path as waypoints might be practical. I don't know...

Sorry to hit you with so many requests! You may even have had some of these in mind for the near future, so I don't want to belabor them. Let me know if you want to discuss these further, or have me think about refinements. Otherwise I will just leave you on your personal track of developing the AI, which has been very successful so far! I certainly don't want to slow down the overall progress on the AI by too much fine-tuning at this stage. On the other hand, it would certainly be good to get a reasonably playable AI out there in a demo fairly soon.

Thanks for all the hard work, and I think fairly soon we will be able to claim your AI as a significant further Fun factor in Clash!

[LDiCesare]

Since the squares are generally ungarrisoned and would be trivial to take, something in the AI that makes that happen would be valuable. One quick suggestion that I have is to make the value of a square increase if it is held by the enemy but adjacent to friendly squares. Something like a simple small addition to its value per friendly square adjacent might be the right combination.

I agree in principle, but I think such a method would cause the ai to work in circles around its own territory and the player currently doing this finds it cumbersome. It would probably be better if neither the ai nor the player had to mop up the squares, like for example, changing the whole province ownership when the province capital is taken (except where troops remain)?

2. It is very undesirable that Hannibal camps outside Rome for 30 turns when the entire southern part of Italy is open for the taking!

Yes. Right now, I'm even worse off with my current iteration of the code . I will try to do something like requesting reinforcements, and then choosing another plan on later turns (i.e. I will not accept plans that require reinforcements if I am already awaiting some). The problem will be to decide of a meeting point between the reinforcements and the to-be-reinforced task force.

3. The way that reinforcements go where they are needed is also undesirable. The first problem is that all these individual units moving over the map Really slow down the game. It took many seconds to crank a turn on my fairly new computer. Secondly, they are tactically vulnerable to fairly small forces if they move as single units.

I agree. One thing which can make things better is that currently the task force keeps asking for reinforcements until they reach it. This bogs down the economics and the ai, pathfinding, etc.
So I'm working on having the reinforcements known to the task force (currently they are known only by the plan), so the task force won't ask for the same thing over and over again. Then I must find a way to group the reinforcements before sending them, and find a meeting point. I have no idea how to do that right now. Using waypoints sounds like a good idea, but the AI would have to know the map well enough to do that, and that is not currently the case (it's more than pathfinding, it's finding the nearest city on the path to a moving unit - arg...).

[Mark_Everson]

Originally posted by LDiCesare
It would probably be better if neither the ai nor the player had to mop up the squares, like for example, changing the whole province ownership when the province capital is taken (except where troops remain)?

I agree, your suggestion is better overall. I had forgotten the discussions we had on this previously. Unfortunately it doesn't help with the Carthage scenario at all since many of the squares I was talking about are in Italy, and Rome is never taken. I'll think about what rules we might be able to use to streamline conquest.

[LDiCesare]

I was talking about are in Italy, and Rome is never taken.

I thought about Gaul.
For Italy, if the ai decides to do somethign when it is looking for reinforcements, I think it should fix the problem.

[LDiCesare]

Status

Here is the current status of the ai, after looking at your observations:

-The ai doesn't perform well in part because it doesn't know your units. If you play Delenda with fog of war off, you should see a different game.
This means I want some spying. I checked the models, and there is no spying model per se, only an allusion to it in the diplomacy model.
I can't search the forums right now to unearth a thread with the corresponding info, but I couldn't find any recent one. The best I found was the diplomatic UI thread, but after looking at the code vovan did, there is nothing to manage spying and very little UI coded.

-The ai takes a lot of time to compute moves when there are many task forces.
I cut down the number of simulations run (this can now be set in xml files, by setting the confidence of one simulation - I used 0.1 which means 10 simulations were run, and got up to 0.4 which means I'll need only 3).
Still, there are several things which can be done to avoid the slow down, which is indeed unbearable in Delenda:
Reduce time taken to wage/simulate a fight. This can be changed by simplifying the military combat model. While this could be done, I'm not keen on rewriting it. I can reduce the number of ticks taken by the simulation, too, to maybe 2 or 3 turns (configurable), which would divide the time by 3 or 5.
Reduce the number of lone task forces. Providing the task forces incentive to merge together when for example they are in the same square and have the same goal.

-Units move about one by one.
This problem is also to be addressed by mergeing the units together more often.

-The ai reinforcement build is not working very well in Delenda. I still have to see Carthage build catapults. I've to investigate more.

-Square values require the economics to provide some figures. I ran into null pointers when there is no publicSector in a given square. I consider that these null pointers mean there is no infrastructure so the value is 0, but hope that this will not cause any problem.

[LDiCesare]

Status

I have corrected some bugs in the map vision of the ai, so the ai now behaves better, though there's a bug there for the player (sometimes it takes 1 turn to uncover fog of war).
It currently takes about 100ms. on my computer to process new orders for one task force, and there are about 8 simulations which means each task force requires about 1 second to process. This means 1 second per ai task force... Way too long. The good use of threads would take care of nine tenths of this, but not on turns when the player immediately presses end turn.
I still can't see catapults built by Carthage.

Ill try to investigate about the time spent on orders, and the number of taskforces (though this number is lowered now because reinforcemetns work better).
I think it should be possible not to evaluate plans whose maximum value is lower than the currently most successful. This could allow to evaluate only 1 or 2 plans instead of 8 msot of the time.

Reinforcements use a poor algorihtm to reach their target: The target of the reinforcements can now move instead of staying stuck where it was. But the reinforcements go where the task force they want to help was when they were given the order. If they reach the square and the task force left, they will follow it. This means they may never merge, or may cross one another without merging. I increased the probability of staying in one place when asking for reinforcements, but that is probably not enough.