Exciting AI improvements.

[Blake]

In my last game I had a civ (Bismarck) who never grew past 3 settlers, while having room for at least 3 more. I suspect it being caused by building a lot of workers while the AI 'settler time' is, thus denying the valuable resources to expansion, and then just giving up.

That doesn't sound right. AI's with discounts and large cities build workers so fast. It should literally take them no longer than 10 turns to fill their worker capacity (first to half (up to 1.5 workers per city), later to full (up to 3 workers per city)). If the AI didn't found more cities that means it didn't find the city sites desirable (for some reason).

Also the AI razing algorythm is simply nuts (though it's not your fault) - once it gets rampant, it razes just about every city.

Part of this is the Major Jerk factor - Napoleon for example is simply nuts when it comes to burning every city in his path. I'm inclined to agree the AI's are a bit happy to raze cities later in the game... havn't touched the code for it though because i'm not really sure what the criteria should be... humans usually raze cities because they dont like the placement, but AI's can't care about that... the jerk factor seems as good as any...

[binTravkin]

Part of this is the Major Jerk factor - Napoleon for example is simply nuts when it comes to burning every city in his path. I'm inclined to agree the AI's are a bit happy to raze cities later in the game..

Hope that gets changed soon.
What's the use of two powerful AIs fighting each other and one being eliminated while the other gains nothing but plunder..

[johnmcd]

I was on another continent but watched this happen.

Napoleon razing Aztec cities every turn, some brilliant, cities too. Then he sends off his settlers to colonise the land, not by walking them ten tiles west, but by loading them on galleons and sailing a hundred tiles east...

[Tattila the Hun]

Well, as long as he has them boats, it's good that he uses them, no?

[Shigga]

Blake,

stunning work so far. It absolutely deserves being implemented by Firaxis in the upcoming patch. Looking forward to see what you will do to the combat code!

Good move there, Firaxians. I know it's not out of pure acknowledgment but will save you from some work to be done, but at least it's a start on aktive work with the community.

[rjmatsleepers]

Is there a "in game" way to tell which core dll you are using? I now have the original, Blake's improved AI and my own compiled version of the SDK code. I can imagine the possibiltiy of getting confused

RJM

[Impaler[WrG]]

Howdy Blake, just found out about your AI moding, I've been moding Civ4 practicaly from day one but I (and it seems most moders) use CivFanatics, wan't aware of any major moding going on at Poly.

I think you should cross post a thread their to expose your mod to more people.

Theirs also a guy over their by the name of "The Great Apple" who has been doing AI moding as well but his system is coming from the oposite direction (massive programing skill vs massive gameplay skills) its based on "evolving" the AI by replacing every hard coded weighting value in the AI functions with a call to a global matrix of values which is mutated from the original values and piped into the game by python. This allows each empire in the game to have different weighting values and to thus display different behavior. Using some code he borrowed/expanded on by another moder named jdog5000 he sets up a game of Civ4 inwhich their are no human players, only AI's and which is capable of running through the whole game non-stop. Statistics from several such games are collected and the AI's ranked. The fitest AI's in the form of these weight tables are then mutated and the cycle continues. Due to the huge amount of processor time required he's farmed out much of it to volunteers at CivFanatics who run it at night and send back the statistics. Theirs been some improvment in the AI, though I'm not in a position to compare your mods as I've played neither.

It dose seem to me that WITH YOUR POWERS COMBINED! (and some Heart) the AI could be brought to new hights of awsomness. You could splice in weighting factors throughout the game, their are also some function modifications similar to your own. By using a function pointer array or matrix your modified functions could be spliced into the game and branched to only by a particular empire allowing the direct battle between your modified AI and a vanilla Firaxis AI.

The home thread and source code is here >>> http://forums.civfanatics.com/showthread.php?t=184128

[Diadem]

Evolutionary algoritms eh? Always fun.

But I wonder how suitable civ4 is for such an approach. Parameters that work for one civ may not work for another. So you have to use a different algoritm for each AI. Which dramaticly increases the number of testruns you need - and this number was already very, very high.

Furthermore AIs respond differently to humans then to other AIs. Of course, in an ideal game they wouldn't, but I doubt we can get the AI intelligent enough to do away with things like anytime soon.

Also you still have the problem that such an AI is still limited to behavior you put in. It might change the value of its own parameters, but it can't change the parameters itself. Which is probably what you need to really optimize the AI.

Finally, by evolving AIs like this you are killing off their personalities. We might end up with a quiet, builder-type Montezuma. Or a raving lunatic Gandhi ...

I firmly believe that evolutionary algoritms have a great future ahead of them, and I have no doubt that AIs for civ 7 or 8 will be programmed like that. But for the near future... I think the best option is to just clone Blake a couple of times

[sophist]

Finally, by evolving AIs like this you are killing off their personalities. We might end up with a quiet, builder-type Montezuma. Or a raving lunatic Gandhi ...

That's unlikely, but if it happened, it would be a good thing. What if it turned out that Gandhi's leader traits were actually suited for warmongering rather than what we assumed? Surely you'd want the AI to give the strongest challenge it could without cheating... If we end up with a smiles-and-hugs Montezuma, then the easy solution is to reshuffle the traits, not to force the AI to play a strategy that's not suited for its traits.

[Xmudder]

Excellent work.

Any idea what would happen if I tried to play Warlords with your mod installed?

[Swiss Pauli]

Download the patch when it arrives and you'll find out.

[Blake]

An evolved AI for CIV4 is never going to compete with an expert sytem AI or in any case it would take over a hundred thousands years of CPU time to develop an AI close to an expert system AI. While soime projects may be able to muster up that amount of CPU time.

For example as an expert I know how to optimally grow cities with seafood, or floodplain cities, or any other type of city I take the time to analysis. An evolving AI *may* stumple across the optimal way to develop a seafood city, but there's no guarantee that the little single aspect of improving a seafood city will be enough to increase it's fitness - at least without thousands of trials.
But as an expert I can simply teach the AI how to develop a seafood city - there, done.

I'm not dismissing the capability for genetic alrgorithms to actually work, like a genetic algorithm probably could determine the optimal solution to developing cities, if that was the only problem it had to solve and it had good scoring system to assess the progress of a city. Unfortunately doing this sort of thing on the fly would be too computationally intensive and caching the results wouldn't work with mods any better than a hardcoded solution.

The project is more interesting in terms of AI personalities altough I believe that CIV play doesn't come down that much to personality so much as leverage of traits, terrain etc, and that's what I train the AI to do. In terms of points of personality, I think what the existing leaders have is plenty enough, tendency to declare war and so on. In some ways it's too much like AI tendency to build certain terrain improvements (terrain should generally be terraformed optimally rather than to some preference...)

[LDiCesare]

Blake's right. Genetic algorithms are goos for optimiing a problem when you've already modelled that problem correctly. There's no way to be sure that the variables available to the ai are the best suited for optimisation, and some behaviours may be held in the code and not in variables. So a genetic algo will only work so far.
btw Blake, any idea of how much of your work will get in the next patch?

[Impaler[WrG]]

I generaly agree with the assesments on genetic AI's as implemented. You would realy have to have some kind of Nural Net architecture with the potential to selectivly access any information in the entire game to realy have a chance to evolved behavior that is in anyway deeply strategic. Ofcorse with that many parameters your talking huge amounts of time to evolve even simple behaviors.

My personal theory on AI is that it needs to be modled more closly on human strategic thinking which is actualy very Top down (I need to Defeat Montizuma, thus I need to start the invasion in 3 turns, thus I need that Causack rushed, thus I'll move the worker onto that mine). The AI's current thinking is bottom up, each individual unit and city trying to do what is "optimum" based on pre-defined weightings. Theirs no real strategic master plan.

If I were dreaming up the AI from scratch I would have a heirarchical structure of Objects derived from a base "Minister" class. At the top is the King who analysis the games overall state and sets empire wide policy, thouse policies come in the form of instantiating sub-ministers of a particular type each type coresponding with a policy. Each one is narrowly programed to achive a particular goal such as making lots of Gold, creating GreatPeople etc etc. Each minister ranks assets belonging to the Kingdom (units & cities) in their usefullness in achiving its goals. The King allocates the resorces among the various ministers its created giving thouse with goals it deems most pressing the lions share. The ministers then deside how to actualy run everything. Military oriented ministers aka generals comand units and conduct warfare. Economic ministers aka Govenors generaly comand Cities and Workers. New ministers can be programed to adress specific AI defficiencies as they are discovered as each Ministers is in essense an encapsulated and cohesive strategy much along the lines as you have been programing.

To quote the Art of War
"A state is not commanded from without, and an army is not commanded from within"

[Diadem]

Blake explains the point about evolutionary AIs much more clearly then I did. Thanks

I think that in a couple of decades we'll be able to develop frightingly intelligent AIs with evolutionary algoritms. But not right now, for a problem as complex as civ4.

Simply running the game in the background, pitting AIs against eachother, as the civfanatics project does, is probably a terribly time-inefficient method of developping AIs, by the way. If you want to do this seriously you need to develop a civ-simulator specificly build for this purpose. All fancy bits taken out, everything that has to do with the player taken out, and entirely optimized for speed. And then you need a huge grid to run the simulator.

So let's stick to the expert approach. Because I still want a more intelligent AI.

Yesterday I played a game of OCC with permanent allies enabled. I want my ally to win a domination victory, so I want him to conquer cities. We were at war with India, and I had my units next to Delhi. I asked him to attack it as well. First he attacks with 2 infantry, which got slaughtered. Then he brought in another 2 infantry and 4 artillery. Meanwhile I had the city down to 1, heavily injured, defender.

What does the AI do with his units, when he sees a city with 1 heavily injured defender and 5 of my units next to it? He pillages the country-side, of course. I had to keep the city at 1 defender for 10 turns before he finally took it, after pillaging all his own future improvements...

*sigh*

I can forgive him not understanding the nature of OCC, so thinking I wanted the city for myself. But then he still should not pillage! And anyway I asked him to attack it...