Exciting AI improvements.

[franlato]

Blake, can Firaxis pay you to improve the war AI for the next patch??

[Solver]

I have to agree here, it seems that an expert-based system is much more efficient for Civ4. Not to say that a genetic approach is useless, but it would probably indeed take an unreasonable amount of time to achieve any results. Blake explains very well why. And genetic testing would also likely see some good-algorithm AIs have poor fitness merely because of poor luck, as that's a part of Civ4.

[Diadem]

Well there's always luck involved when you evolve algoritms. That's why you need a lot of testruns per generation - or alternatively a very large population size and number of generations (which is the way that nature does it - it doesn't run one live a 1000 times, it runs millions of lives for millions of generations).

So for civ, expert systems are the way to go. At least until civ 7 or 8



Anyway, I have an idea, sort of. The AIs always seem to leave a lot of units in their cities. Often I wage wars where for example they have 10 cities with 6 units each, while I have 10 cities with 1 unit each, a reserve defence force of some 5-10 units and an attack force of 10-15 units.

In such a situation I easily obliterate the AI, with about half the units he has. I take out his cities one by one, without any problems.

If the AI would be less sticky with city garrissons, it would leave only 2 units in each city, throw 40 at my attack force, completely obliterate it, and then overrun my civilization.

I think there is major room for improvements here... The AI needs to be made much more flexible with city garrisons.

[fed1943]

It would be possible to say the computer something like:

Try and avoid to fight with a small portion of your forces against all or

a big portion of oponents'?

Best regards,

[zeace]

The real key to the expert system isn't even that one expert can beat a neural net. Blake has been drawing on the expertise of all of us. I enjoy going back and reading strategy threads from a year ago when the game came out: "What are we doing wrong? Why are we all bankrupt? Just when my army starts rolling it disbands?"

While I'm sure Blake has hundreds of hours of play time (Don't even try to calculate it, it's scary), the entire experience of everyone posting is in the millions of hours. And we've done lots of trial and error stuff just like an adaptive AI could do. But we can all see the obviously dumb stuff and avoid it, so our hours of play time correspond to hundreds of hours of simulation time. So now we have hundreds of millions of simulation hour equivalents.

Of course the AI plays a little faster than us, but our collective knowledge is staggering. Just ask the poor testers for this game. I'm sure they put thousands of hours into testing it, but that's nothing compared to the millions of hours we play it. Of course we're going to find more bugs.

PS. Thanks everyone for every post you've put on this forum so that Blake can learn as much as he can so he can make a great AI for all of us. Yay Blake

[kniteowl]

Hey Does the AI already know how to Specialise Cities after they've settled a city?

Like a City that mostly has tiles that have Food >=2? (basically grassland or floodplain tiles) to cottage as a commerce/research city?

or a Production City That basically has many Hills to mine suported by the Food Surplus from Floodpains and Food Resources?.

and So on etc

[Blake]

Nope.
I gave it some rudimentary understanding via my new worker automation code... it'll terraforming certain cities as farm/mine, if a city really needs food to work resources (like silver mines), it really gets farms! This on the whole works really quite well and is one of the more impressive changes I made.

Unfortuantely the rest of the City AI does not know what the city is terraformed as... it wont nessecarly build a wonder in the city terraformed as production... so more work is needed still.

[kniteowl]

Kool your a kiwi... didn't realise and here I thought you were an amazing person improving the AI overseas LOL

Another question Concerning Spiritual Civs

How often do they switch civics?

When I play a Spritual Civ I go Switch Crazy going from Slavery to Selfdom and back almost every 5 turns depending on the circumstances. Sometimes I also do it with Organized Religion and Pacifism.

[kniteowl]

Another question, do you plan to improve the AI on archipelago maps in the future?

AI's are terrible on those Maps, I had one AI settled it's Second City on the other Side of the world from it's Capital, that didn't make sense at all, The city was pretty Good, the logistics of defending that city would proabbly be a nightmare.

Also the Strategy in a archipelago game is totally different, you generally research using the commerce from sea and ocean tiles. Production Tiles are Quite Rare (eg forest and Hills) but there's Plenty of food so slavery is a must

the AI's usage of slavery has considerably improved by you as far as I can see.

Would code the AI to play Differently bettween normal maps and a archipelago maps?

[Tattila the Hun]

Originally posted by Diadem
Anyway, I have an idea, sort of. The AIs always seem to leave a lot of units in their cities. Often I wage wars where for example they have 10 cities with 6 units each, while I have 10 cities with 1 unit each, a reserve defence force of some 5-10 units and an attack force of 10-15 units.

In such a situation I easily obliterate the AI, with about half the units he has. I take out his cities one by one, without any problems.

If the AI would be less sticky with city garrissons, it would leave only 2 units in each city, throw 40 at my attack force, completely obliterate it, and then overrun my civilization.

I think there is major room for improvements here... The AI needs to be made much more flexible with city garrisons.

Sometimes it's because of happines problems...

[Diadem]

Well of course there are sometimes reasons to have many units in your cities. But in general, the AI is just terribly inefficient with its units.

If I have 1/3 of the units the AI has, I will win. It won't even be a particularly hard war. That's just terrible.

Improving the AI effectiveness in warfare would be a major improvement. If they can use their units more effectively, we can do away with a lot of the bonusses they now need for units - making things more equal.

AI city placement and worker useage used to be a disaster. But Blake really improved this. The AI is now also smarter with civics, including usage of the whip. That means the AI can now build a decent economy. Good. I really think warfare is the next step.

The AI is so terribly dumb in using its units...

[Addled Platypus]

Originally posted by Diadem


The AI is so terribly dumb in using its units...

I think we have all seen the pinball movement too many times, this is a good change !

[kniteowl]

Another Nice Improvement to the AI would be it's usage of Great People, Have you guys seen an AI have a golden age in the early middles ages... WHAT A WASTE OF GREAT PEOPLE, their better off Becoming Super Specialist or lightbulbing techs.

should make it so the AI can only use Great People for Golden ages after or during the construction of the Apollo Program, it would make them a whole lot more competitive for the space race.

[JackRudd]

Trouble with that one is that it falls down spectacularly in No-Spaceship games.

[kniteowl]

Originally posted by JackRudd
Trouble with that one is that it falls down spectacularly in No-Spaceship games.

Then make the AI more likely to have Golden Ages during the Late Modern age or during thel ast 50 turns of a game when Space race is turned off.

It just makes sense that golden ages are much more productive during the late stages of the game.

[Blake]

Another Nice Improvement to the AI would be it's usage of Great People, Have you guys seen an AI have a golden age in the early middles ages... WHAT A WASTE OF GREAT PEOPLE, their better off Becoming Super Specialist or lightbulbing techs.

I shall improve the GP usage. As you've noted at the moment it is quite bad, and looking at the code I will have no trouble improving it.

This will be a rewrite moreso than a tweaking, what I'm doing is adding a function to estimate the value of a Golden Age which will then be compared with the lightbulb value.

Lightbulbing code is also getting an overhaul, in short the AI will be less dumb about lightbulbing religious techs (it seems at the moment it'll lightbulb such a tech regardless of wastage), it'll be smarter about lightbulbing techs which it wants anyway. To be precise, the way I'm doing it, it'll always lightbulb the tech it is currently researching if it gets at least 90% of the maximum lightbulb value (ie it'll use GP's to generally hurry it's research along). It will lightbulb other techs (off of the beeline) if doing so will complete the tech without much wastage (max 10%), more wastage is tolerated if there's a freebie on the tech (max 30%).

Final logic has the AI compare the golden age value with the discover value. First it tries to lightbulb a complete free-stuff tech should doing so be non-wasteful. Then it tries to create a special building if doing so makes sense.

If the golden age value is low and there's nothing good to discover it'll settle the GP. If the GA value is medium, it'll wait for something good to lightbulb, or start a GA, whatever comes first. If the GA value is High it'll save it for the GA no matter what.

I note with despair that there is no "Cash Bomb" AI for the Great Merchant. I thus comandeer some missionary code and quickly write the cash bomb code, I test it in game by gifting Louie a GM and it scurries over to my capital and generates a pile of cash for the french... which I promptly fleece from the AI. Maybe some anti scamz0rs code... yeah at the moment they'll happily run 60% science despite having a 1500g stockpile... so I'll just give them the default of 100% science and let them adjust that down if they need the gold (I dunno why default logic isn't like that) and I'll make them keep some gold on hand for deficit research.


Anyway, I know everyone wants military improvements but that stuff is a lot harder than economics, feel free to suggest stuff which looks like could use improvement.

By the way I'll definitely be releasing a Warlords Patch DLL very soon after the Warlords patch comes out (no, I don't know when that will be), I may or may not release a CIV1.61 DLL before the Warlords patch, but I'll definitely release one after the Warlords patch.

[Diadem]

So the AI will never settle great people? That's interesting. I guess with AI effectiveness as low as it is that's not really worth it. Incidently, what happens if you capture a city with settled great people? Do you get them, or do they disappear?

Yeah economic improvements are probably a lot easier than military ones. To be honest I wouldn't really know an easy method to improve AI military intelligence. It would have to learn how to defend cities properly, with fewer defenders in cities that won't be attacked and fluid defences that can be quickly moved to where they are needed.

We'll have to teach it about using units efficiently. When to attack and when not to attack. Often now if you arrive with a Stack of Doom it will suicide several artillery units at it, heavily injuring your stack, and then it will only kill one or two units in the stack with normal attack. That's just such a waste.

It needs to learn about pillaging. When to pillage, and when not to pillage. So it needs to be able to estimate it's strength and chances, and it shouldn't pillage when it has good odds of taking a city.

Those are all complex improvements. Not at all easy.

Perhaps we should sit down with a group of people who know a lot about the game and at least a bit about coding, and discuss ideas. A specific topic for this purpose is perhaps an idea?

[Xmudder]

Originally posted by Blake
Anyway, I know everyone wants military improvements but that stuff is a lot harder than economics, feel free to suggest stuff which looks like could use improvement.

AI colonizing the new world on Terra Maps would be great. More generally colonizing on any other continent / island (Archipeligo). But holding their continental posessions is more important.

By the way I'll definitely be releasing a Warlords Patch DLL very soon after the Warlords patch comes out (no, I don't know when that will be), I may or may not release a CIV1.61 DLL before the Warlords patch, but I'll definitely release one after the Warlords patch.

Could you release a warlords dll before the official patch? It could be a week or more, and I'd love to give your changes a try before then.

[popejubal]

Originally posted by Blake
This will be a rewrite moreso than a tweaking, what I'm doing is adding a function to estimate the value of a Golden Age which will then be compared with the lightbulb value.

I'd just like to say how very impressed I am with your initiative here.

Instead of just saying, "no Golden Ages", you decided to actually figure out when a Golden Age is worthwhile. Honestly, that's something I've never bothered to do since it's more work than I'm willing to put into a game that I'm playing.

Be careful that you don't make the AI too good.

You could end up giving the AI a good strategy, good tactics and forcing it to do all of the decision making calculations that we humans can't be bothered to do.

[Quillan]

Hehe, and what's wrong with that? If the AI gets that good, we can take away all the freebies the AI gets now to make up for its incompetence.

[Forwarn45]

I'm not sure how the AI military intelligence (or lack thereof) is currently coded. But, on the defense and attack, I would like the AI to have objective algorithms that are prioritized and that it checks every turn. On attack, one would be the "can I take the city with the current military forces I am sending at it or at least wittle it down?" In that equation, the AI should consider whether it will be able to do this with available forces once the city defense value is weakend with siege. On the defense: "can I take out one of those stacks of doom heading my way by an all-out attack, first with siege units, then nearly ALL units in the area." Another defense priority should be: "If none of my cities are threatened with sufficient forces to take them but I am still somewhat weak compared to the attacker, I will consider sending raiding parties to weak areas of enemy territory and/or setting up some stronger defenses at key points (not just one unit on a critical resource), including defense chokepoints like hills next to the city.

[popejubal]

Originally posted by Quillan
If the AI gets that good, we can take away all the freebies the AI gets now to make up for its incompetence.

Nah, that would be fine. I'm just afraid of Blake making the AI good enough that I won't be able to beat it even after we take away the freebies...

[Tattila the Hun]

It will be good enough to beat me, even if I'm not playing...

[kniteowl]

Will The AI know how to flip cities using a Great Artist? I've never seen an AI flip a City using a Great Artist.

Edit :The AI will Lightbulb Techs before Creating a Special Building???

What about GS? Wouldn't they generally be better off Building Academcies in the early Game, unless they have tech Slignshot to Early Philosophy

And GEs? I'd definitely want the AI to Save them to rush Wonders no matter what, the Time I lightbulb GEs is with Machinery for Early Maces and during the late game when there are no wonder's left to rush.

[couerdelion]

The code for settling GPs might need some thought since, as human players, we tend to have a fairly good idea where we will be building Oxford or Wall Street – OK well Wall Street is perhaps easy enough so the settled Prophets or GM should be no problem. In fact, the more I think about it, GS would tend to settle with the Academy so it’s almost certain to be the capital. Likewise, the bonus for hammers from Bureaucracy would suggest that GEs go to the capital as well. With that sorted, it’s no surprise where OU will go.

With GS the comparison also has to be made with more Academies. I don’t think the calculations are that difficult. Likewise, the decision to create a second or third shrine will need some comparison to settling or generating a Golden Age.

The last problem with a Golden Age is that it takes two GPs. But how the calculation ought to factor in the time delay is tricky since, by the time the next arrives, you’ll have more production/commerce tiles for the Golden Age to improve.

[sjm]

I'm currently coming to a close in my first game with the improved AI. It looks to me like the AI has got a lot better at founding cities in more intelligent places, although it still has a tendency to try and squeeze in that one extra city on the last available tile on the continent that is unclaimed, and exactly 3 squares from the nearest city.

There was far less cottage spam, and far more farm spam going on. That's probably not a bad thing, as the AI would previously turn almost every possible plains square into a cottage that would never get worked, as the city never grew past size 7 or so. Plains cities seem to be growing better, and thus working more tiles. The AI still seems to prefer windmills over mines - even in towns that (at least post-biology) were absolutely overflowing with food production. I saw Hatty building the iron works in a city surrounded by plains (not necessarily a bad thing, but ALL farmed), a couple of hills (with windmills) and a couple of inland lake tiles (3 food as the city was on the coast). Given that excess food supply there (about 15 food/turn), it really would have made sense to mine those hills!

Anyway, one other thing I did notice, that I don't think I've ever seen happen before, is that despite the AI grabbing 6 of the religions between them, they failed to build a single shrine by 1750-ish (marathon game). I even ended up building the hindu shrine myself, after capturing it post-1700. Only one AI ended up with 2 holy cities, so there were 5 AIs (actually 6, Monty lost the christian holy city to Saladin's culture at some point) that between them could not build a shrine in about 5,000 years. Sure, I've played games before, where only some of the shrines were built - and some only quite late in the game. But I've never seen none built at all.

This would suggest to me, that for one reason or another, the AI has stopped producing as many great prophets (or great people in general?). I don't know if this is a result of different AI (governor?) behaviour, or just a fluke. However, it would be nice to see the AI make a concerted effort to pop a prophet, if it's got a holy city and the religion has a decent following. As it was, Hinduism was the strongest religion in the game, shared by 1/3 of the civs in the game (all the warmongers, so furthrmore, they were all running theocracy! - so it was generally their ONLY religion as well). Upon building the shrine, it was worth a good 60 gpt - so I'd definitely think it was worth a prophet!

[lockstep]

Originally posted by kniteowl
Will The AI know how to flip cities using a Great Artist? I've never seen an AI flip a City using a Great Artist.

I can't prove it, but I think I've seen such a thing. In one of my first games, when I was playing as Washington, my neighbor AI civ to the north - Isabella - was gradually being conquered by Bismarck (an AI still farther north). Eventually - I was a builder then - I decided to join the party and to claim some of Isabellas cities for myself. A few turns after I had captured the third Spanish city (and sent Isabella to an island exile), I noticed that another former Spanish city, now owned by Bismarck, had a very large border and threatened to flip my latest 'Spanish' city. Another few turns later, the city actually flipped to Bismarck.

I wasn't familiar with the exact rules of cultural border expansion then, but looking back, it seems very likely that the AI culture bombed me.

[Arrian]

I'm pretty sure the AI has used culture bombs on me a few times.

-Arrian

[Soren Johnson]

nice work, Blake. This is definitely exhibit A for why we wanted to release the AI code.

[lockstep]

Originally posted by Soren Johnson Firaxis
nice work, Blake. This is definitely exhibit A for why we wanted to release the AI code.

Sounds like signature material for you, Blake.