Exciting AI improvements.

Blake · October 14, 2006, 20:46

However, one thing I noticed: for hills the AI does not seem to be able to make up its mind: mine or windmill. It happened a lot that I saw 5 workers mine all the hills around a city and 10 turns after that they were building windmills. Once these were build, they switched back to mines, etc

This is a known problem and it's very tangly. In short the AI doesn't know it's already adjusting the terraforming of a city - so it sees a starved city and gets a worker to change a mine to a windmill... but it repeats this x5. Then once that's done, it sees an excessive food surplus so starts converting windmills to mines...
There are possible solutions I'm looking into.

Maybe you haven't touched that code, but I don't remember the blue circles being this misdirected in the past. Here's one where I went worker first on a highlands map. I started with hunting and researched animal husbandry, but I guess it wants me to irrigate the turndra?

I have absoutely no idea why it would suggest the tundra.
Look into it immediately I shall.

edit: uhhh... okay that was easily reproduced. I don't really understand WHY but ... actually I think I do understand why...
This shouldn't be hard to fix. It's a return of the "farm bug" where the AI wouldn't build farms, because it saw them as having no yield (a farm has no yield until biology, it has a special effect of +1f with irrigation - I had to manually check or that special effect. it's the same deal with camps/pastures - they produce no yield, it's special yield with a bonus only, i need to check for that manually.)
Thanks for finding the bug. It wont effect the AI badly ebcause they always hook up resources with high priority, but it will be fixed.

johnmcd · October 15, 2006, 03:05

I've just won my first full game since the new AI, down a level and it certainly was a different experience.

The reason I won it though was primarily due to some straightforward old fashioned AI incompetence.

I wanted a space race win, as I thought that was the best way to go toe to toe with the AIs new productivity and commerce know-how. While I led (I came from behind) the game was far from sewn up.

Until my two major rivals were prepared to enter a devestating war with each other in exchange for Democracy.

Really, those guys should have locked down their win method a millenia earlier and started weighting options to match it.

I guess that is a massive recoding of prioritisation and the like but might be something to bare in mind.

Much better game than usual though!

Fosse · October 17, 2006, 15:25

Does this change Barbarian city placement as well?

dacole · October 19, 2006, 16:08

Originally posted by Blake

Goverment Civics:

HRULE: Total_Population * 0.2
REP: 12 * City_Count (up to max # cities effected) + Total_Population * 0.5
POLICE: Wartime: 5 * City_Count (Units) + 2.5 * City_Count (WW).
POLICE: Peacetime: 1.25 * City_Count (Units) + .5 * City_Count (WW).
US: 3.3 per Town + 8.3 per City_Count (16.7 if avoiding science).

HRule is valued very, very lowly (really I should add one of those long very very very chains to indicate just how lowly it is valued) - it has the dubious honor of being valued lower than police state in peacetime if cities are smaller than size 9.
Rep is twice as valuable than HRule just from the specialist effect, the happy effect is smaller - but the value from the happy effect alone means you need a population of 360 for HRule to be more valuable. Rep value is not based on specialists themselves.
Police State is always worth less than US and usually worth less than Rep. In wartime it's worth more than HRule.
US is worth more than Rep under most cases and is always worth more than the other two civics.

The goverment column essentially uses this logic:

US > Rep > Police State > HRule.
There are two glaring flaws:
HRule is less valuable for empires with small cities - in reality it's most useful for small empires (ie should be based on city count rather than population), HRule is so worthless the AI might not see it as worth the Anarchy.
Police State is always less valuable than US regardless of *anything*.

You know I'm not sure I agree with the above and would like to know what you are basing it on. Maybe I war to much but I always run into problems with war wariness causing my people to become unhappy above and beyond what I can spend on happiness additions. Usually the loss of money going from a medium to high value civic is more than made up from the people who now work squares. As well, especially early in the game you don't have many towns (at least I don't maybe 1 or 2 per city since they take so long to build up) so though I freely admit the extra hammer from them is wonderful and requiered in the late game if at all possible especially in the early game I would see police state as being a better choice..

Blake · October 19, 2006, 17:40

That's what the existing 1.61 / 2.00 code does.

MortVader · October 20, 2006, 04:23

I've been playing a few games with this new AI .dll now, and I must say that it plays really well. The AI gives much better resistance against my world-domination now! I am really looking forward to both an improved combat/invasion AI, and also a port of this excellent "mod" for the Warlords expansion.

Well done, Blake!
Excellent work!

binTravkin · October 20, 2006, 05:05

I just yesterday attempted a CS slingshot just to found out someone had researched CS before me through conventional means

MortVader · October 20, 2006, 06:08

CS slingshot? Caste System slingshot?

We havent heard about this from under my rock. What is that?

Thedrin · October 20, 2006, 06:17

Civil service slingshot. Research code of laws and then use the oracle or a great prophet to get civil service. This allows the civic beuracracy which is, in general, more powerful the earlier in the game you access it.

MortVader · October 20, 2006, 06:25

oh.. thanks

Arrian · October 20, 2006, 08:16

Originally posted by binTravkin
I just yesterday attempted a CS slingshot just to found out someone had researched CS before me through conventional means

is right. Wow.

-Arrian

kiky · October 20, 2006, 23:46

Originally posted by binTravkin
I just yesterday attempted a CS slingshot just to found out someone had researched CS before me through conventional means

What year was it? and what difficulty?

binTravkin · October 21, 2006, 05:00

It was emperor, dont remember the year tho.
It might've been a bit of my fault being slow to take off, but still impressive.
Im guessing the AI could've been using a GP, though Im not sure it does at all.

Senethro · October 22, 2006, 14:43

Playing against Warlords AI with more of blakes additions implemented and Monarch is suddenly quite challenging.

Played as Julius, Praet-rushed the 2nd and 3rd place civs to gain a total of 9 cities on a small map in 1200AD. I'm not on top of the scoreboard, but 5th out of 7 in tech.

I'm now clueless as to whether I should bully the smaller civs, attack Mansa Musa (who is in danger of running away with the game) or sit tight and build up.

binTravkin · October 23, 2006, 12:34

Blake, your modifications in worker-building priorities seem to have striked back.
In my last game I had a civ (Bismarck) who never grew past 3 settlers, while having room for at least 3 more. I suspect it being caused by building a lot of workers while the AI 'settler time' is, thus denying the valuable resources to expansion, and then just giving up.
Also the AI razing algorythm is simply nuts (though it's not your fault) - once it gets rampant, it razes just about every city.

Blake · October 23, 2006, 12:44

In my last game I had a civ (Bismarck) who never grew past 3 settlers, while having room for at least 3 more. I suspect it being caused by building a lot of workers while the AI 'settler time' is, thus denying the valuable resources to expansion, and then just giving up.

That doesn't sound right. AI's with discounts and large cities build workers so fast. It should literally take them no longer than 10 turns to fill their worker capacity (first to half (up to 1.5 workers per city), later to full (up to 3 workers per city)). If the AI didn't found more cities that means it didn't find the city sites desirable (for some reason).

Also the AI razing algorythm is simply nuts (though it's not your fault) - once it gets rampant, it razes just about every city.

Part of this is the Major Jerk factor - Napoleon for example is simply nuts when it comes to burning every city in his path. I'm inclined to agree the AI's are a bit happy to raze cities later in the game... havn't touched the code for it though because i'm not really sure what the criteria should be... humans usually raze cities because they dont like the placement, but AI's can't care about that... the jerk factor seems as good as any...

binTravkin · October 23, 2006, 12:47

Part of this is the Major Jerk factor - Napoleon for example is simply nuts when it comes to burning every city in his path. I'm inclined to agree the AI's are a bit happy to raze cities later in the game..

Hope that gets changed soon.
What's the use of two powerful AIs fighting each other and one being eliminated while the other gains nothing but plunder..

johnmcd · October 23, 2006, 15:06

I was on another continent but watched this happen.

Napoleon razing Aztec cities every turn, some brilliant, cities too. Then he sends off his settlers to colonise the land, not by walking them ten tiles west, but by loading them on galleons and sailing a hundred tiles east...

Tattila the Hun · October 23, 2006, 15:50

Well, as long as he has them boats, it's good that he uses them, no?

Shigga · October 24, 2006, 08:36

Blake,

stunning work so far. It absolutely deserves being implemented by Firaxis in the upcoming patch. Looking forward to see what you will do to the combat code!

Good move there, Firaxians. I know it's not out of pure acknowledgment but will save you from some work to be done, but at least it's a start on aktive work with the community.

rjmatsleepers · October 24, 2006, 12:15

Is there a "in game" way to tell which core dll you are using? I now have the original, Blake's improved AI and my own compiled version of the SDK code. I can imagine the possibiltiy of getting confused

RJM

Impaler[WrG] · October 25, 2006, 16:32

Howdy Blake, just found out about your AI moding, I've been moding Civ4 practicaly from day one but I (and it seems most moders) use CivFanatics, wan't aware of any major moding going on at Poly.

I think you should cross post a thread their to expose your mod to more people.

Theirs also a guy over their by the name of "The Great Apple" who has been doing AI moding as well but his system is coming from the oposite direction (massive programing skill vs massive gameplay skills) its based on "evolving" the AI by replacing every hard coded weighting value in the AI functions with a call to a global matrix of values which is mutated from the original values and piped into the game by python. This allows each empire in the game to have different weighting values and to thus display different behavior. Using some code he borrowed/expanded on by another moder named jdog5000 he sets up a game of Civ4 inwhich their are no human players, only AI's and which is capable of running through the whole game non-stop. Statistics from several such games are collected and the AI's ranked. The fitest AI's in the form of these weight tables are then mutated and the cycle continues. Due to the huge amount of processor time required he's farmed out much of it to volunteers at CivFanatics who run it at night and send back the statistics. Theirs been some improvment in the AI, though I'm not in a position to compare your mods as I've played neither.

It dose seem to me that WITH YOUR POWERS COMBINED! (and some Heart) the AI could be brought to new hights of awsomness. You could splice in weighting factors throughout the game, their are also some function modifications similar to your own. By using a function pointer array or matrix your modified functions could be spliced into the game and branched to only by a particular empire allowing the direct battle between your modified AI and a vanilla Firaxis AI.

The home thread and source code is here >>> http://forums.civfanatics.com/showthread.php?t=184128

Diadem · October 25, 2006, 16:54

Evolutionary algoritms eh? Always fun.

But I wonder how suitable civ4 is for such an approach. Parameters that work for one civ may not work for another. So you have to use a different algoritm for each AI. Which dramaticly increases the number of testruns you need - and this number was already very, very high.

Furthermore AIs respond differently to humans then to other AIs. Of course, in an ideal game they wouldn't, but I doubt we can get the AI intelligent enough to do away with things like anytime soon.

Also you still have the problem that such an AI is still limited to behavior you put in. It might change the value of its own parameters, but it can't change the parameters itself. Which is probably what you need to really optimize the AI.

Finally, by evolving AIs like this you are killing off their personalities. We might end up with a quiet, builder-type Montezuma. Or a raving lunatic Gandhi ...

I firmly believe that evolutionary algoritms have a great future ahead of them, and I have no doubt that AIs for civ 7 or 8 will be programmed like that. But for the near future... I think the best option is to just clone Blake a couple of times

sophist · October 25, 2006, 18:42

Finally, by evolving AIs like this you are killing off their personalities. We might end up with a quiet, builder-type Montezuma. Or a raving lunatic Gandhi ...

That's unlikely, but if it happened, it would be a good thing. What if it turned out that Gandhi's leader traits were actually suited for warmongering rather than what we assumed? Surely you'd want the AI to give the strongest challenge it could without cheating... If we end up with a smiles-and-hugs Montezuma, then the easy solution is to reshuffle the traits, not to force the AI to play a strategy that's not suited for its traits.

Xmudder · October 26, 2006, 15:26

Excellent work.

Any idea what would happen if I tried to play Warlords with your mod installed?

Swiss Pauli · October 26, 2006, 15:50

Download the patch when it arrives and you'll find out.

Blake · October 27, 2006, 00:15

An evolved AI for CIV4 is never going to compete with an expert sytem AI or in any case it would take over a hundred thousands years of CPU time to develop an AI close to an expert system AI. While soime projects may be able to muster up that amount of CPU time.

For example as an expert I know how to optimally grow cities with seafood, or floodplain cities, or any other type of city I take the time to analysis. An evolving AI *may* stumple across the optimal way to develop a seafood city, but there's no guarantee that the little single aspect of improving a seafood city will be enough to increase it's fitness - at least without thousands of trials.
But as an expert I can simply teach the AI how to develop a seafood city - there, done.

I'm not dismissing the capability for genetic alrgorithms to actually work, like a genetic algorithm probably could determine the optimal solution to developing cities, if that was the only problem it had to solve and it had good scoring system to assess the progress of a city. Unfortunately doing this sort of thing on the fly would be too computationally intensive and caching the results wouldn't work with mods any better than a hardcoded solution.

The project is more interesting in terms of AI personalities altough I believe that CIV play doesn't come down that much to personality so much as leverage of traits, terrain etc, and that's what I train the AI to do. In terms of points of personality, I think what the existing leaders have is plenty enough, tendency to declare war and so on. In some ways it's too much like AI tendency to build certain terrain improvements (terrain should generally be terraformed optimally rather than to some preference...)

LDiCesare · October 27, 2006, 02:05

Blake's right. Genetic algorithms are goos for optimiing a problem when you've already modelled that problem correctly. There's no way to be sure that the variables available to the ai are the best suited for optimisation, and some behaviours may be held in the code and not in variables. So a genetic algo will only work so far.
btw Blake, any idea of how much of your work will get in the next patch?

Impaler[WrG] · October 27, 2006, 05:16

I generaly agree with the assesments on genetic AI's as implemented. You would realy have to have some kind of Nural Net architecture with the potential to selectivly access any information in the entire game to realy have a chance to evolved behavior that is in anyway deeply strategic. Ofcorse with that many parameters your talking huge amounts of time to evolve even simple behaviors.

My personal theory on AI is that it needs to be modled more closly on human strategic thinking which is actualy very Top down (I need to Defeat Montizuma, thus I need to start the invasion in 3 turns, thus I need that Causack rushed, thus I'll move the worker onto that mine). The AI's current thinking is bottom up, each individual unit and city trying to do what is "optimum" based on pre-defined weightings. Theirs no real strategic master plan.

If I were dreaming up the AI from scratch I would have a heirarchical structure of Objects derived from a base "Minister" class. At the top is the King who analysis the games overall state and sets empire wide policy, thouse policies come in the form of instantiating sub-ministers of a particular type each type coresponding with a policy. Each one is narrowly programed to achive a particular goal such as making lots of Gold, creating GreatPeople etc etc. Each minister ranks assets belonging to the Kingdom (units & cities) in their usefullness in achiving its goals. The King allocates the resorces among the various ministers its created giving thouse with goals it deems most pressing the lions share. The ministers then deside how to actualy run everything. Military oriented ministers aka generals comand units and conduct warfare. Economic ministers aka Govenors generaly comand Cities and Workers. New ministers can be programed to adress specific AI defficiencies as they are discovered as each Ministers is in essense an encapsulated and cohesive strategy much along the lines as you have been programing.

To quote the Art of War
"A state is not commanded from without, and an army is not commanded from within"

Diadem · October 27, 2006, 08:59

Blake explains the point about evolutionary AIs much more clearly then I did. Thanks

I think that in a couple of decades we'll be able to develop frightingly intelligent AIs with evolutionary algoritms. But not right now, for a problem as complex as civ4.

Simply running the game in the background, pitting AIs against eachother, as the civfanatics project does, is probably a terribly time-inefficient method of developping AIs, by the way. If you want to do this seriously you need to develop a civ-simulator specificly build for this purpose. All fancy bits taken out, everything that has to do with the player taken out, and entirely optimized for speed. And then you need a huge grid to run the simulator.

So let's stick to the expert approach. Because I still want a more intelligent AI.

Yesterday I played a game of OCC with permanent allies enabled. I want my ally to win a domination victory, so I want him to conquer cities. We were at war with India, and I had my units next to Delhi. I asked him to attack it as well. First he attacks with 2 infantry, which got slaughtered. Then he brought in another 2 infantry and 4 artillery. Meanwhile I had the city down to 1, heavily injured, defender.

What does the AI do with his units, when he sees a city with 1 heavily injured defender and 5 of my units next to it? He pillages the country-side, of course. I had to keep the city at 1 defender for 10 turns before he finally took it, after pillaging all his own future improvements...

*sigh*

I can forgive him not understanding the nature of OCC, so thinking I wanted the city for myself. But then he still should not pillage! And anyway I asked him to attack it...