Exciting AI improvements.

[Blake]

Originally posted by couerdelion
Programming the AI to play a better game is going to start making their bonuses really pay off, even on the medium levels.

Next thing, you’ll be finding that you need to make friends at Monarch level for fear of the AI rush.

But it looks very impressive Blake. Does it significantly improve the AI performance or simply give you better cities to capture?

I'm sure it gives a small performance boost, but the the main performance gains will only come with the "Holy Trinity"
1) Good city placement.
2) Smart build choices to increase growth.
3) Smart workforce placement and use of whip.

Like at the moment, the AI doesn't priotize food enough for the whipping to really pay off. Altough it does get in unhealthy/unhappy situations where a bit of pruning will greatly improve economic health as well as providing an influx of hammers.

[zeace]

Awesome work Blake! I've wanted to do such a thing myself, but I've never gotten around to it. Somehow I've always found playing Civ to be more fun than improving it. Maybe you're work will inspire me and others to delve into the SDK...

[Blake]

Hi everyone.

I've added some more powerful reasoning ability to the AI. It now counts food! It looks at the resources (improved) and floodplains. It sums up the food surplus from those resources, and the food deficit to work dry resources. Positive food is good, negative food is bad (a gold mine with no food to work it is worse than a gold mine with food to work it). This means it will have a slight tendency to avoid clusters of silver mines (and such), if possible it should try and found cities at the edges of clusters of dry resources.
And of course it'll love any site rich in food, and sites with lots of food and lots of good food sinks.

I'm very happy both with continents and great plains behaivour and I've added everything I can think of to the decision making process.

So I'm now posting the newest version.

Download "CIV Improved AI" 0.3

By the way, at the start blue circles will jump around a bit. I'll put it like this. Blue circles do maphack, but they don't quite have full vision into black fog. So don't put too much faith in the blue circles until you've actually scouted the land.

Things to watch for - None of these SHOULD happen but they might.
1) Food resources going unclaimed by any city (it'll neglect useless furs and such).
2) Founding on top of resources without justifaction.
3) Weird start locations (the code is shared with start location code).
4) Excessive Slowdown during AI turns (I don't think this will be a problem, I've taken care to keep the code lean).


Something not to watch for:
Strategic error. The AI can't "dotmap". I've tried to preserve sanity to stop it going for super-super cities (it might found two cities to work a cluster of 4 food resources) but it'll still do things like found 1 city where 2 would be better and block resources from being ever worked.

Download "CIV Improved AI" 0.3

So on the agenda:
Worker automation AI (Food balance)
Build Governor AI (enhancing growth builds are good)
Workforce Governor (good tiles are good, food is good, whipping is good).

[binTravkin]

Wow, even more cool things coming up.
I hope this doesn't go unnoticed by Firaxis

[Swiss Pauli]

Woo and yay! This weekend, I shall be mostly testing these improvements.

[slgann]

Does this also contain your whipping and research overflow fix? I've been using that fix but I'm thinking that I'll switch to this one. I sounds really great.

slgann

[Blake]

It does indeed include those fixes.

[johnmcd]

It would be great if the city placement improvements could be matched by auto worker improvement, more for the AIs benefit than mine!

[Blake]

Oh now this is rich.

If I'm reading the 1.61 code right.

The AI does not whip Granaries.
The AI does only whips lighthouses if the city has no food.

In fact it'll never whip buildings unless there is special case code for them. So I'll fix up the lighthouse code and add granary code, and also add general case whip awareness so it'll whip everything it can (if it makes sense to do so).

edit: Done! The AI should now whip large builds (anything) in cities with unhappy or unhealthy population. Exact formula is if ((unhappy * 2 + unhealthy) >= 3) and (pop_cost >=2) and no existing anger then Whip Away!

Certain builds - Granaries, Lighthouses, Settlers, Workers, Workboats are considered essential builds, meaning they'll be whipped even if there is existing whip anger. Oh yeah, and if the city is in danger? That's considered essential too. The AI should now (hopefully) ravage it's population with the whip when invaded.
Ever thought "The AI could easily drive me back if it whipped a couple of axemen out of that massive city?" ...
It brings joy to my heart.

I might make some of this dependent on difficulty level (especially the war-ravaging thing), altough if you're going to play this AI then you should probably expect it to be both more efficient and more ruthless.

[Swiss Pauli]

Originally posted by Blake
The AI should now (hopefully) ravage it's population with the whip when invaded.
Ever thought "The AI could easily drive me back if it whipped a couple of axemen out of that massive city?" ...
It brings joy to my heart.

I might make some of this dependent on difficulty level (especially the war-ravaging thing), altough if you're going to play this AI then you should probably expect it to be both more efficient and more ruthless.

That's simply fantastic. Anything that makes warmongering more challenging is wonderful.

[johnmcd]

So are your whip fixes now available for download?

[Blake]

Everything I've done is available 3 posts up I still have more work to do on the AI.

I decided to run a test where play a duel with a single AI and put spies in the AI capital and watch it.

It starts out well training a bunch of warriors to sit on their thumbs, wihle founding Buddhism (so much for them always going for Hinduism, hah!).
He trains some archers and a worker, the worker starts building roads and farms.
Then he starts on a settler, since he gets Bronze Working, he switches to slavery and whips the settler. W00t. Good start. But after BW he gets Iron Working.
The settler heads out, upon founding the AI researches Polytheism and the new city gets Hinduism. The AI immediately starts a vital build - Hinudi Monastary!
I watch with growing horror as the AI persists in working unimproved hills while the worker builds more farms, despite the masses of already-unworked farms.

Thousands of years pass.



The AI is busily researching Calendar and STILL hasn't researched Pottery. I'm starting to understanding how the AI is just so damn weak.

I give up and research Pottery and offer him the gift, which he accepts.

Granaries DO get whipped into the outlying cities but the Capital doesn't deserve one.

Then the AI generates a prophet and builds the buddhist shrine. Of course upon generating a prophet you immediately... run 6 priests?!!!

And a while late the AI starts a Courthouse, in his CAPITAL. With the -2 upkeep. AND STILL HASN'T BUILT A GRANARY *weeps*. Or a single mine.

I have my work cut out for me, that's for sure.

I'm thinking I might have to nuke it from orbit and rebuild from the ground up (I mean workforce placement code) it's the only way to be sure.

[johnmcd]

Originally posted by Blake
I'm thinking I might have to nuke it from orbit and rebuild from the ground up (I mean workforce placement code) it's the only way to be sure.

You'd only need to nuke it from orbit it was a hugely efficient threat to survival, by the sounds of things a pointy stick should do it.

[ben04]

In XML technology files there's a proprety iAIWeight. What is that for? I played a bit around with it and though that the AI would prioritize techs with a high modifier. The AI seemed to pretty much ignore it.

Also I've found that the AI build much too much units and not enough buildings.

I'd love to help if I could only get that darn SDK to compile.

Also that AI code really looks bad. Functions that have about 1300 lines and not a single line of comment. No wonder that that code is buggy.

[couerdelion]

Originally posted by Blake
Ever thought "The AI could easily drive me back if it whipped a couple of axemen out of that massive city?" ...
It brings joy to my heart.

I might make some of this dependent on difficulty level (especially the war-ravaging thing), altough if you're going to play this AI then you should probably expect it to be both more efficient and more ruthless.

Then the thing to do is obviously fight more battles on home turf where you have the roads. Let the AI whip the cities down and then move in

Now if the AI could employ some basic battle tactics things would get scary and I would be probably be back-tracking all the way to Noble level

Edit: Just read your AI story.

But if it is playing this well then now I think MY play must be really bad.

[Blake]

Originally posted by ben04
In XML technology files there's a proprety iAIWeight. What is that for? I played a bit around with it and though that the AI would prioritize techs with a high modifier. The AI seemed to pretty much ignore it.

It's probably a matter of magnitude. From the SDK it seems that:
100 would be the minimum which might be noticable.
400 would make it simialler to a tech an AI likes (like economic AI's liking Banking).
1000 would be enough to make it very attractive.
3000 would lead to it being chosen in preference to founding religions and stuff.

Also I've found that the AI build much too much units and not enough buildings.

Indeed...
I've started working on that bit of the code. Part of the problem is the AI has no sense of priority, the buildings it build might be crap ones.

I'd love to help if I could only get that darn SDK to compile.

Also that AI code really looks bad. Functions that have about 1300 lines and not a single line of comment. No wonder that that code is buggy.

Have you tried the instructions in the creation forum for compiling it with Code::Blocks and the free VC compiler?

I don't find the AI code to be bad... I mean the code style is good and it's ultra readable despite lack of comments (the function names are generally descriptive), the code just isn't very well tuned for CIV4...

[sabrewolf]

Also I've found that the AI build much too much units and not enough buildings.

Indeed...
I've started working on that bit of the code. Part of the problem is the AI has no sense of priority, the buildings it build might be crap ones.

i don't quite agree here. the AI usually has too few units. or should i at least say too few strong units.

i see it too often that the AI may have 8 units protecting each of its cities. but when you start invading, hardly any of them move in as reinforcement.

the choice is also often bad. while they now build trebuchets and more catapults, they don't use it properly. cats have city attack promotions when they are used in defence. and even if my stack moves up on a city with 3 siege units they seldom use more than one of those to weaken the stack.

also, the AI always promotes the units immediately instead of a) waiting to see what's needed or b) recover from a battle quicker by halving the wounds.

last, honeypots: scouts and workers put undefended at your border will nearly always be captured/killed. if it's a lone cavalry it's much easier to deal with then, than when deep in enemy territory...

so i don't think there should be less units built, but the use/upgrade/promotion of the units should be improved...

[ben04]

Originally posted by Blake

It's probably a matter of magnitude. From the SDK it seems that:
100 would be the minimum which might be noticable.
400 would make it simialler to a tech an AI likes (like economic AI's liking Banking).
1000 would be enough to make it very attractive.
3000 would lead to it being chosen in preference to founding religions and stuff.

I needed to make it 100000 to have the AI do what I want. 1000 was the value I tried earlier.

It might be an idea to give the wheel and mining a 2000 points and pottery and bronzworking 100000 each. It makes the AI a bit too predicable but on the other hand better predicable than lame.

There are also such modifiers for buildings. Setting a granary to 100000 made a granary always being the second building. Baracs were always the frist building they have build. Except once where the second building was a library in a city without growth.

The Leader propretries do have an iBuildUnitProb however I didn't notice any difference in behaviour when setting it to 1 and 1000 and 100000.

Having lots of defensive units (longbows) made the AI prefer settlers and workers over other units but didn't boost buildings. It didn't know how to use gifted warriors for city defense, they simply walked around in the AI's territory without a target.

After the AI had completed a number of units, doesn't matter if civil or military, in a city it switches into building build mode where it as a first move always build a Barac.



Also your code doesn't get the city founding right either. It founds too often on a resource. It managed to found a city on the only occurence of silk on the entire map. That +1 happyness is wroth more. It also managed to found on a gold hill.

[ben04]

i don't quite agree here. the AI usually has too few units. or should i at least say too few strong units.

Before you build lots of units you should build a baracks. Early on you should also build a granary to boost growth so that the AI can make use of its higher happyness cap. In the long run you will have more and better units when prioritizing buildings.

also, the AI always promotes the units immediately instead of a) waiting to see what's needed or b) recover from a battle quicker by halving the wounds.

I suspect that the AI assigns promotions randomly. May I suggest the following algorithm:

1) If a units can't be attack nor wants to attack don't promote.
2) Otherwise evaluates the odds of combat and if under 75% choose the promotions that boosts the odds more.

When attacking it shouldn't be complicated to determine the odds. One could work stack considerations into the code. Meaning if there is no unit without promotion in that stack with more that 75% when attacking reconsider all units with promotions and choose the best. If there still is no unit above 75% reconsider the stack and choose the best unit with colateral damage. If there is none with collateral damage choose the best unit for attacking with promotion if the odds are above 20%. Otherwise retreat.

When defending it is probably a lot more difficult to work out the odds. An algorithms that considers all possibilities would probably be too slow. How about:
For enemy each unit n tile away that can move n or more tiles consider the odds. With n : 1<=n<=2. Ignore stack issues.

The downside of this algorithm is that when approaching an AI stack with a single unit it will trigger the promotions for each unit in that stack. This could be exploited by the user. For example approach an invasion force with an archer in the late game and all (damaged as the others will be over the 75% anyway) AI units will have anti archer promotion. Pretty much a wasted promotion when fighting against guns.

On the other hand you probably will need to know the details about the game mechanics or otherwise you wont even notice this exploit.

[sabrewolf]

Before you build lots of units you should build a baracks. Early on you should also build a granary to boost growth so that the AI can make use of its higher happyness cap. In the long run you will have more and better units when prioritizing buildings.

clear... some basic buildings are no-brainers. i just meant this in general, the AI doesn't build too many.

I suspect that the AI assigns promotions randomly.

not quite randomly. when a unit is built it gets assigned a AI-strategy. city defender, attacker, etc. so at the time. while some units are probably attack-only (swordsmen, all horse-based units), some are used for both (archers, axes, longbows, etc.). depending on what the built unit was intended for, an pikeman might get a city raider for isntance.

May I suggest the following algorithm:
1) If a units can't be attack nor wants to attack don't promote.
2) Otherwise evaluates the odds of combat and if under 75% choose the promotions that boosts the odds more.

1) --> certainly! but there probably is no flag/function called boolean couldBeAttackedAnyMomentNow()
and don't forget that with your idea he might upgrade better for defences, but not with foresight. if we see the AI approaching with 4 knights, we'll have some elephants, spears and pikes to defend and we'll ignore upgrading the axemen. can the AI do that too?

2) --> hardcoding is dangerous. 75% is a nice lead, but if you face cavalry vs. cavalry, giving combat I makes quite a big deal (10% more --> 68% chance of winning).


as for the downside example with the archer: the algorithm ofcourse would consider the archer NOT a threat and wouldn't bother promoting.


let's see what blake comes up with. so far it's been great

[Thedrin]

Originally posted by ben04
Also your code doesn't get the city founding right either. It founds too often on a resource. It managed to found a city on the only occurence of silk on the entire map. That +1 happyness is wroth more. It also managed to found on a gold hill.

Founding on a luxury or food resource does not prevent the civ from obtaining the happiness or health it provides once the appropriate technology has been discovered (calender in this case). As with founding on strategic resources, it only detracts from the tile bonuses provided by the worker improvement ordinarily used to access the resource.

Incidentally, the AI Persians settled a beautiful city on a plains cow in a game I played yesterday. Two gold mines and enough food (partially thanks to sugar) to work them both. I can't claim that the AI wouldn't have done so without the changes made but I find myself less likely to raise a city and resettle than I was before.

[LDiCesare]

Sounds great work Blake.

[johnmcd]

I think I'm seeing some of the expected behaviours, I wonder if the AI is a little too keen on coast now, it's keener than I am anyway. I've seen it put down some pretty crummy towns just to get a single clam when most of the interior is a mix of moutain and desert. That's fine when space is running out, but early on, it feels like it shouldn't be one of the first three cities.

I can see the whipping working for the AI though, it's very cool, suddenly strategy is needed to win wars...

Is Barbarian city placement affected?

[Blake]

You're more than welcome to post any screenshots of suspect behaivour (remember the AI WILL maphack so take it from World Builder to reveal all resources).
Screenshots of smart (Wow!) behaivour are also welcome .

I happen to find that coastal cities are VERY good investments, they are cheap to set up and provide a good profit and are often connected to the trade for free. Other changes should improve the AI's use of food which will make them benefit more from seafood.


My latest developments have been to fix the pottery thing somewhat - the AI now recognizes Pottery as enabling Towns rather than Cottages. I might need to do more. It also places more weight on Alphabet especially if tech trading is enabled.

I know everyone wants combat improvements, but unfortunately combat is less formulaistic than other aspects. What I WILL try to do is improve city siege. I'll try and make it so the AI first bombards down all the defenses (if there's enough siege) then pauses for the siege to recover moves, it'll then attack with all the siege followed by all the other units - none of that splitting up stuff.
I'll also make it pillage a lot less to reduce frivolous losses.


edit: Ohhh man. The Worker AI is grizzly. The problem is that it doesn't see unimproved tiles and figure out how to improve them... instead it has this idea that a city's terrain is "good enough" like with floodplains, it doesn't see a great oppurtunity to improve the tile, it basically sees a tile that doesn't need improving.
The windmill > mine thing seems to be plain faulty coding, basically it figures out if a city will be starving at size 20 - so in that screenshot earlier of the mongol capital, all the hills leads the AI to believe that the city is starving and needs more irrigation, I can bandaid this by basing it more on population but then I discovered the issue with unimproved floodplains. I need to make the AI emphasize improving the tiles it's working.
Anyway as you can see with the bandaid fix (a 1-line code change) the AI is improved somewhat:


The lucky bugger popped a copper...

[Dog of Justice]

Just wanted to join everyone else in saying, INCREDIBLE work, Blake.

Regarding building priorities, the first method that comes to mind is a systematic cost/benefit calculation.

[Bobby Chicken]

Why would you train the AI to whip? A lot of real people have success without whipping on Immortal and Deity, probably more so than whippers. I can maintain an edge all the way through the middle ages (where my skills begin to deteriorate; have to work on that) and have never found whipping that good. If I can be kicking the AI's ass on deity in 500 BC, I don't think the AI needs to adopt whipping. But congratulations Blake. I can imagine myself trying to figure out that code. It looks the same as me trying to read the rossetta stone. I think the AI should be programmed to build less units and more buildings when not planning for a war in the near future. They should also bring troops to the frontal cities instead of leaving them sitting 3-6 to a city to be taken out piece-meal. Maybe that's pretty much impossible. Don't know; it's all a big mixture of hierogliphics, greek, and Egyptian to me.

[Blake]

Whipping is a quick, painless cure for happiness/unhealthiness woes.

[couerdelion]

Originally posted by Bobby Chicken
A lot of real people have success without whipping on Immortal and Deity, probably more so than whippers. I can maintain an edge all the way through the middle ages (where my skills begin to deteriorate; have to work on that) and have never found whipping that good.

There are people playing Deity level that do not whip!!?

What seems strange though is that it is possible to catch up and gain an edge at this level and yet merely “maintain” an edge thereafter. Is it perhaps the case that the “edge” is acquired too late for it to be converted into a strong advantage and that this is only possible at lower levels like Emperor?

But in answering your question about teaching the AI to whip, this is a more effective way of gaining production in the early to middle stages of the game for any city with moderate to good food. The AI does not play at Immortal level so it will play better if it whips.

[MikeH]

I dunno about improving the AI, this thread has actually taught me a lot about city placement, whipping and build orders.

[Bobby Chicken]

Originally posted by couerdelion


There are people playing Deity level that do not whip!!?

What seems strange though is that it is possible to catch up and gain an edge at this level and yet merely “maintain” an edge thereafter. Is it perhaps the case that the “edge” is acquired too late for it to be converted into a strong advantage and that this is only possible at lower levels like Emperor?

But in answering your question about teaching the AI to whip, this is a more effective way of gaining production in the early to middle stages of the game for any city with moderate to good food. The AI does not play at Immortal level so it will play better if it whips.

OK, I think I got you. A granary is unecessary on deity because you do not want your poplulation to grow that fast. But if you ain't playin on emperor or above, you would benefit from the granary. So yah, teach 'em to whip. But yah, as far as whipping is concerned for players, pangea, deity, standard size world, shitty starting location, and I was kicking ass into the middle ages where I got mauled from eight sides being in the middle of the continent with those unfriendly AIs surrounding me. I still might have to try a little midieval whipping though, to jumpstart some new cities.