DESIGN: AI Improvement discussion

[MrBaggins]

Yes... A* is a great pathfinding algorithm.

However, the threat map i'm doing doesn't use paths... its based on areas. It depicts collaborative 'influence' of units. Its purpose is to show how vaguely grouped units can threaten particular locations.

[hexagonian]

Originally posted by MrBaggins
My thought as to a better solution to the science problem, is an "elastic lead" or "knowledge spread".

Consider if there were 8 players in a particular game. Civilization 1 researches a particular tech. All other civs researching that tech gain 1/7 of the total necessary points to research the goal. If another civ starts to research it, they gain a bonus at the start of research equal to (x/7) of the goal, where x is the number of civs already having researched the technology.

Actually, the MAX_BEHIND, MIN_BEHIND settings simulate this to some degree, but with a subtle difference.

The following is opinion and nobody may agree with it - still I will throw it out for discussion.

First off, the game boils down to the player against all other AI civs - that is the premise of the game - to beat all other civs.

The problem with this model is that it removes much of the incentive for the human player to maximize science - if this model also applies to the player. Fall behind and the human player can coast because he receive a bonus for being behind (...cheaper tech cost). Push ahead and the player actually gives the AI opponent more help.

Its the first aspect of the setup that I'm not too crazy about (falling behind...). The second one is not a problem for me because it does not remove any incentive - in fact it makes sure that I stay focused on maintaining a science lead, and to me, it simulates the feeling of tech by osmosis.

I know that this should cut both ways - the player should also benefit from tech by osmosis, but its the whole idea of removing incentive to do your best that bothers me.

I also know that there is the incentive to reach the most powerful units first, as well as the wonders, but the way to handle tech differences between civs is to make the benefits more gradual (more advanced units have small differences in stats over the slightly earlier ones, but as you get further ahead, the differences become greater) Thus there are no gamebreakers unless you have units that are greatly advanced over your rivals - as it should be. The battle for domination becomes more subtle, and takes more time to achieve, without sacrificing the incentive to do so.

(And you make sure that the AI can upgrade all obsolete units when it reaches the required tech - in my mind this is most critical.)

civ3 used this knowledge spread model and coupled it with an all-too effective tech/gold trading system which ended up wiping out most incentive to focus on science research.

[Dale]

Originally posted by MrBaggins
Yes... A* is a great pathfinding algorithm.

However, the threat map i'm doing doesn't use paths... its based on areas. It depicts collaborative 'influence' of units. Its purpose is to show how vaguely grouped units can threaten particular locations.

Try this.......

1. Generate your enemy threat influence map (based on areas, or sites as I call them, of highest enemy threat).
2. AI determines which site to send unit to.
3. Combine enemy threat & terrain influence maps together (to get the influence of enemies and terrain obstacles).
4. Through the generated influence map into A* and watch it weave a path from source to target avoiding all other threats and obstacles (to ensure the unit reaches the intended site) till it reaches the target site.

This is the path I'm taking.

EDIT: The bonus of combining the enemy threat and terrain influence maps, is it'll tell you which sites have enemies AND high-terrain. No one wins a war fighting uphill against fortified units.

[marc420]

Hello,

I'm enjoying reading this discussion, and as I haven't done any modding, take what I say with a grain of salt. But I'm thinking of two different sets of AI systems here, so I thought I'd add them to the mix.

1) Mulitple solutions with result-scoring.
An early message mentioned the problem of knowing when to change a global slider versus when to change the number of entertainers in a city/cities. When I thought about how I sometimes solve this problem, it occurred to me that an AI could probably use the same method to solve the problem.

The way I go through and do this while playing some game is to set the sliders on one setting, then go around and tweak each city till its the way I like it. Then go write down empire-wide totals. Then go change the global slider, and repeat the process of tweaking each city. Then again look at the empire-wide results.

If the AI "knows" at the start of this that it is trying to maximize production, or balance production and science, then their aren't so many slider settings that it couldn't go through this sort of process for each slider setting and compare the results.

2) AI that learns.
This is the one I've really wanted to see in a game some time, an AI that learns over time.

Give each civ's AI a "rating" for a particular strategy. Maybe its a strategy that tries to balance offense versus defense. Each civ in a game has an AI that has a different rating for this.

At first this may be almost random, or it may be random but centered around the ai-writer's best guess. Then, at the end of the game, the AI system looks at the scores/final results of each of its AIs. The ones that did poorest get a low score, the ones that did best get a high score.

Next game, the civ's in the game all get AI's generated again, but the now the generated ratings are more tilted towards the one's with a higher score based on previous results.

Two things seem nice about this approach. One is that the AI-writer doesn't need to arbitrarily setup the perfect AI. Getting it close as a starting point would help, but then the system will adjust (hopefully). The second is that this sort of system would tend to adapt itself to the strategies that work against the player on that computer.

And one general last note. Once I had a chess program where the strength of the AI was controlled by the amount of time you gave it to makes its move. It used a tree system like the one I mentioned about the sliders above. If you gave it more time to make its move, it could study more alternatives and go deeper down the tree. So more time = better AI.

I've always been curious if a strategy game could do something like that. Out of the box its set to play pretty quickly. But as you beat the AI and wanted tougher games, you would move up the amount of time the AI had to think, and thus give the AI time to look at more alternatives.

Personally, after I've played one of these games for awhile, I wouldn't mind a slower/smarter AI. Even if you got up to letting the AI think 15min or 30 min in doing its turn(s) (plural for all the AIs in the game, 30 min might mean 3 min each for 10 civs), I'd be willing to take that delay in exchange for a more challenging game.

Those 30 min delays would still be a lot faster than PBEM.

just a few of ideas I wanted to throw out.....

[J Bytheway]

With regard to the fuzzy logic, there's a Word document about how it worked in CTP1 on my website (link at the top of this post) and I also have some lecture notes on the subject of fuzzy logic and neural nets in general (in pdf format) if anyone wants them.

If you want learning AI and fuzzy logic, then one option is to go is with fuzzy neural nets. If AI learning goes from game to game, it might be nice if they stuck more or less to the same civ, so that the civs developed their own personalities and you could grow to love and hate them. Of course, this will be an extra advantage to the human, who knows the AIs personalities. Another thing to consider is providing the option to just run games containing only AI as fast as possible, so you can leave them learning while you're not there.

But there's a lot of "if"s and "might"s there, so I won't get too carried away.

I guess I know enough to at least implement a fuzzy logic system, but I don't know enough AI to plug in all the right numbers, choose the right defuzzification methods, etc.

I know that in CTP1 it was found that increasing the time allocated to the AI (which, fortunately, could be altered in Const.txt) improved its performance. It would be nice to have the AI thinking during the humans turn, too.

[Tamerlin]

Originally posted by Solver
But point is, yes, AI is either an 8-city puke or a 30-city monster with 65 happiness at them. But, with the source, we can give the AI, say, a 0.2 multiplier for unhappiness bonuses...

Enough with the cheatings, we all know the CtP2 AI is cheating but too many cheatings are spoiling the game. I would prefer an AI that would be taught to respect the City cap, to build efficient combined armies and to launch an amphibious assault rather than an AI that would be strengthened by another cheating. The more an AI respects the rules the better IMO.

[MrBaggins]

I think the issue with any learning AI we might consider is insufficient sample size. Also it seems to me that we will find it difficult to measure success.... or more specifically slight degrees of less or more success.

[Solver]

Tamerlin, there can be different cheatings. Decreasing unhappiness bonus is a subtle cheat, it's not immediately visible. Giving the AI two extra Settlers at start is too visible, as it absolutely outexpands you, no matter how hard you try.

Marc,

Ahh, the AI that learns... many have wanted to create that, yet there's no game AI that learns properly . I am sure that creating it with our limited resources is impossible.

Hex,

Yes, Civ 3 uses the knowledge spread model, and does it well somewhat. That I like. Still, you are right about the problem where human players (actually, they have no other choice on higher difficulties) turn off science and simply buy all their techs through gold.

There are, however, solutions to this we can implement. First, give the AI better comparative values. In Civ 3, the key is to research techs that AI doesn't - and, if you have 1 tech it doesn't, the AI will happily give you 3 techs that everyone else has for it. So, prevent the CtP2 AI from giving techs away for cheap merely because they are researched by many civs already. Also, don't make it grab any new tech from human just for the sake of it - do some evaluation. And, give it a concept of science lead, which would force the AI into, to a degree at least, keeping its science lead. Civ 3 AI is never in the science lead, because you can get all its techs with money... CtP2 AI must be able to think like "if I give human that tech, he gets Cavalry, but if I don't, my units are far superior, so I should not".

[Ekmek]

As far as science leading and valuing tech leads etc, could we code it so these are different personalities/strategies. Such as one AI valuing science so it will aggressively pursue tech leads, or a expansionist pursuing more terrain, or an economic one that tries to acquire more goods?

These would make the game challenging in the respect that the player would have to contend with not hard thinking AI but have to deal diplomatically or strategically with AI that have different goals, so its not just a conquer all other civs game it could be a science race, resource monopoly, etc game.

(The goods idea also creates a possible economic victory option like controling all of one ressource can be a victory condition i.e. oil).

[Solver]

Different AI personalities are already, to a degree, included in the game, so I guess yes, we can built further on those.

[Ekmek]

Thats good news, different personalities would be better than the civ3 style attributes. I guess if the computer loads a specific AI personality it would not be a big leap to assign special technologies or even wonder-like atributes to its palace... just the possibility starts putting civ3 to shame.

[Solver]

Well, look at the current CtP2 AI, as said, it's *supposed* to act differently depending on its personality... these differences, though are currently highly subtle and barely visible.

[st_swithin]

A* is the WORST pathfinding algorithm EVER!!!

Actually, A* should have been [0] in the array of 'required bug fixes.'

What a piece of poo! And I thought I was a poor programmer! I'M NOT THE WORST!!! I'M NOT THE WORST!!!

[hexagonian]

Originally posted by Solver
CtP2 AI must be able to think like "if I give human that tech, he gets Cavalry, but if I don't, my units are far superior, so I should not".

In strategies.txt, there are techs that you can prevent the AI from trading with you. The AI will not trade away Gunpowder in the default game

However, I'm not sure if the AI is currently able to recognize the different levels of tech... 'Here, I'll give you Agriculture for Fusion'

[Solver]

Yup, comparative values... that's what I'm saying.

As for strategies.txt techs not to trade, that's all stupid. It means that AI won't take Tank Construction for Gunpowder... silly!