Arguments why its nearly impossible to program an "almost human" AI.

[Ralf]

About AI city-foundation strategies:

I often started by founding a base of 3-4 nearby cities - usually by then i uncovered much territory relatively far away with help of scouting warrior-units. I could then overview the map and make a foreseeing strategic decision:
I will move my next two settlers far, far away to that fertile and promising patch of land over there, and found two new cities. From there I will create a new base of cities – then i can expand both city-bases inwards, like burning the candle at both ends.

Above type of foreseeing city-expanding strategies is very hard/ impossible for the AI-programmers to mimic, right? What could be done (not 100% sure, giving my lack of hands-on programming experience) is to mimic the following strategy:

I usually (about 4 times out of 5) bump into a coastline with my first settler, within 2 turns. If i do i found a coastal city (if not i found one anyway, inland). After I founded my first c-city i always “stay glued” to the coastal line with the following settlers. After I founded about 10-12 coastal cities, i start to work my way inwards.
This strategy works very effective on big islands and even small continents – it’s often the best way to make use the land most effectively. Lets not however argue if the latter is correct or not – my point is: it should be possible to tune the AI to do this. Right or wrong?

About the AI unit “shuffle”:

He, He!! I have to smile then I read this, Steve. When I played Civ-2, i always played the land only expansionist/ perfectionist who had naval attack-units very far down on the priority-list. That didn’t hinder the AI to “shuffle” (sometimes 2-3 naval-units) about 2-3 squares outside many of my coustal-defence/city-wall/tanks defended C-cities anyway. If they declared war, the AI just kept smashing destroyers and what not, again and again – to little effect.

Also: If it’s possible (I didn’t know that) for the AI to “know” if the human player (HP) has naval-attack units harbored somewhere, it should also be able to “know” other average unit or defence-improvement data as well - and from this having rules for, then its meaningful to continue bombarding/attacking, or not. How many AI-units is allowed to be lost (without killing HP-units in return), before AI truce- and peace-negotiations are urgent.
In my case the AI-threat represented by those shuffling naval-units often was totally contra-productive. The build-/maintain-resource cost + the unit-away-from-home unhappiness, must have been a very heavy burden on the AI-civs.

Anyway, I see your point then it comes to shuffling in order to “detect” HP-units (but why shuffle in its OWN tracks SEVERAL times – WITHIN that same turn?).


DaveV: “So while scripting and/or templates are a good idea, they *must* be changeable. As strategies mature and new schemes are developed, the AIs should be able to keep up. This also would lessen the problem of the AI becoming predictable and therefore beatable”.

I totally agree – I want those templates editable.

[This message has been edited by Ralf (edited June 26, 2000).]

[UltraSonix]

I actually have nothing to contribute to this thread 'cause I'm not a very experienced programmer, but I've got to say that, guys keep it up! Some of these stuff provide fascinating insights into how a game can be built.

------------------
No, in Australia we don't live with kangaroos and koalas in our backyards...

[Ralf]

Not only do I want those templates (and/or scripting) editable – I want them to be individually and uniquely different according any AI civ temperament/strategy-style. I want LOTS of variants to randomly choose from. I want to test out other hardnosed civ-veterans most successful (and for me unknown) .txt templates, downloadable through the Internet – one for each AI-civ, as well - and test them ALL out - simultaneously.

The words “Templates” (or scripting) sounds like something rigid and predictable, and therefore boring - something that doesn’t change from game to game. I really don’t have to be that way. Far from it.

One can’t always “blame the developers” however. I remember about a guy who complained about the ICS-strategy, and how boring it was for him to play the game that way, again and again (although effective).
The answer from another guy was: Well, just don’t play the game that particular way.

[Steve Clark]

quote: Originally posted by TheLimey on 06-25-2000 01:15 AM Also, most AI programming is inclusive rather than exclusive when describing hinting for behaviour. I.E. Telling units what to do, rather than 'not do'.

There alot of good points made here. I think the above is one of the best ones. What this means specifically to Civ is that the AI, when starting out, builds their cities based on a set of rules (3 cities 7 squares apart along the nearby river) depending on some variables (terrain, proximity to others, etc.). Further growth or building of units/improvements is primarily based on a set of conditions (whether under attack, demographic rankings, powergraph rankings, etc.). In others words, 'if A and B equals X, then it will do Y'. It is not going to be programmed like a human to say 'will plan to do X in about 10 turns, therefore must do A and B now, and C if Y happens in the meantime'. Because of the incredible amount of conditions and variables in Civ, it really can't be programmed to think ahead very much.

In response to programming a few lines to fix the AI units 'shuffle'. What would you have it do instead? For example, the AI knows that you have built or are building a naval attack unit in city A. It is programmed to try to intercept that coming out of your port and the only way in can do it is by shuffling between a few squares. It would be difficult to program it as how a human plays it.

Simplistically put, the AI cannot 'see' the game as we see it. It plays the game only with numbers representing terrain values, unit values and coordinates. Even if you don't know programming logic. Take out a piece of paper and create a 20x40 grid. Then place a settler in one grid and then write down all of the factors going into deciding what and where to build - using only numbers representing everything in the game. Then multiply that 1000 fold to add in all of the other complexities of warfare, trading, diplomacy, etc. Then you will have any idea of why it can only do 'if A and B equals X, then it will do Y'.

[DaveV]

Several comments:

The ultimate test of any Artificial Intelligence upgrade would be the Turing test: if you can't tell whether you're playing an AI or a human. That goal is unreachable, but some relatively simple programming tricks could improve the AI vastly.

One of the early AI programs was Eliza, which would reply to your conversation by looking for key words and giving scripted replies. People were amazed by the seeming intelligence of a rather simple program.

Part of the problem here is the Sherlock Holmes effect. At the end of the story, when Holmes reveals his train of reasoning, everyone says "well, that's obvious!". But without knowing the details, the answer seems to appear by magic. Similarly, the AI programmers in Civ II have laid their work bare for all to inspect via the cheat menu. Not only does this allow people to say, "that stupid AI was building a library when it should have been building a temple", but it allows people to analyze the AI algorithms. The human players' intimate knowledge of the AI techniques leads to a style of gameplay specifically geared to exploit the AIs' weaknesses. Plus, what may have seemed like optimal strategy at the time the game was released has been trumped by new techniques.

So while scripting and/or templates are a good idea, they *must* be changeable. As strategies mature and new schemes are developed, the AIs should be able to keep up. This also would lessen the problem of the AI becoming predictable and therefore beatable.

To sum up - the AIs don't need to *be* intelligent, just to *appear* intelligent. There is a huge difference. No AI will ever be able to compete one-on-one with a top human player, but they can put up a much better challenge than they do now. Although Civ is much more complicated than chess, the AIs in Civ have several advantages over their brothers in the chess world: they outnumber the human player, they can have handicaps in production/research, the human player has limited information, and there are many more roads to a winning game.

[Urban Ranger]

A very interesting method of determining advantages and disadvantages is the field (or influence) model first proposed by Chris Crawford (for a game at least).

In the original article Chris proposed to use a "field" to model the influence projected by any given unit in a game. This "field" is the same as the fields in classical physics, and can be represented by: I = S/R**2, where I is influence, S is strength of the unit, and R is distance. You can modify the fomula to have the field strength dropping faster or slower.

This can be expanded in Civ3 to include influences of specials, terrain, cities, and whatnot. You can have different sorts of influences: food production, shield production, etc. To take a concrete example, the computer builds a city at a particular location after it has optimized a number of influences, say, grain production and shield production.

A couple more points on AI programming. First, you don't have to recompute everything whenver something changes. The program can store the results of the previous computations and make modifications according to the changes that have been made. That will save a lot of time.

Second, it can make use of a database just like Deep Blue. It can store previous moves and strategies, thus cutting back on computation requirements by a huge amount.

[TheLimey]

You make a very good point with regard to weighting using fields; its a great idea to have a number set that we can 'see' and 'use'.
Regarding your last statment though, while it *could* store previous moves and strategies... what would that achieve;

To be useful in branch prediction, you'd need to know (also) about the map, and the moves of other players too. You'd also need to know the success or otherwise of certain actions (combat for one, for instance).

This imposes several factors elsewhere in the game; for instance you need to make unique identifiers for units, to insure that they can be identified as the same entity by whatever processing you do later. As you introduce these unique entities you need to classify them.

There are a bunch of other considerations, such as gold level, tax settings and so on that also need to be stored on a turn by turn basis.

Whist this isn't an overwhelming amount of information, it is a lot, and would slow down the processes. How significantly? I'm not sure.

The analysis definitely won't be possible on an in game basis. It will have to be between games.

Then the question becomes, how do you interpret these logs you generate. How does a computer know that its moving towards unit 20 towards location X1,Y1 rather than X,Y, was a tactical mistake. Humans can reduce the situation to identifiable parts and use visualization to identify strategies and their outcomes.

Of course, you could always copy a human; do what you think, but save the state of the analysis each turn. If the success values aren't high enough go back a number of turns depending on what the goal was, and retry the same situation using the next permutation of strategy. Quantum gameplay

[Stuff2]

I don't think the AI has to store all of the history. It should instead concentrate on the current turn.

First of all. The AI - should have some kind of knowledge of it's power compared to other civs atleast those where a embassy is established.

This power should be a few clear defined variables like:
Economic strength
Economic growth
Military strength
Military growth
Scientific strength
Sceintific growth
Population
Popualtion growth
Overall strength
Overall growth

The thing is this.
Every civ tries to get the biggest number in all these areas. But also they are carefully not to get involved in someone that is too powerful. In those cases where the AI "don't know" it simply puts a guess into the variable.

This variables are what makes the AI-player decide if you are strong enough to be a threat or weak enough to be able to defeat.

Then reputation, nationalism, ideologies, governmentform, and the "feeling" for other civs is what determines if they actually will declare war.

Then after this is done the AI - will use it's geographical knowledge of you for determing which cities they should invade.

Then after this is done it will worry about actual troop movements.

What i mean is that the AI don't need to consider everything at the same time and it doesn't need to store old moves. It simply has a number of variables telling it how things are going, another bunch of variables telling it what to do about it, another bunch of varibales telling it how to do it and finnaly a bunch of varibles for doing it.

Every event and action will in some way influence some of these variables. Some kind of randomness should also be incorporated and then the AI-player will seem almost intelligent. Or atleast not as stupid as it is now. As it is now it seems like every unit and every city has it's own agenda. But all you need for overall AI - decisions is overall variables and that is not so hard to get.

[yin26]

On this topic of AI, I have seen more than one programmer/designer (including Brian Reynolds) say something like this:

"Basically I play the game and think about what I would do in a given situation. This then gets programmed and tested. If the result makes for a more challenging AI in a given situation, the instructions are kept, otherwise tossed. Repeat repeat repeat."

In other words, this is a kind of asynchronous learning process where it's the programmer doing the learning and "instructing" his program to do better next time. The limitations of this are glaring:

** The AI will never be better than the programmer himself (and let's face it, most programmers simply can't play the game as much as dedicated fans do).

** Once the game is released, the AI is more or less unable to learn anything new since the programmer can't possibly continue to tweak the AI according to what develops in the "real world" of Civ gaming. Thus, a trick that beat the AI one game will beat it forever. Find enough tricks and the game is no longer a challenge.

Some possible solutions have been these:

** The .txt files, as far as I know, were originally made popular by Brian Reynolds. This is an extention of the programmer making the AI better...except it's the gamer himself who is the programmer. Obviously the gaming community itself can much more readily tweak the AI to address the strategy of the week, and I hope to see this ability expanded for Civ3--but it's still only a patch-work solution.

** A more promising solution, the AI Holy Grail, is to eliminate the need for the programmer/gamer in this process entirely and find a why for the AI itself to analyze and correct the decisions that lead to its defeat. One simple step is the rock-scissor-paper method whereby the comp "remembers" what you attacked with at any given point in the game and makes the proper counter for the next battle or even the next game (if you allow it to build and build on its record of how you play). Again, this can be beaten by simply understanding that the AI will now expect you do to A, B, C. So doing D, E, F will likely result in an easy win. In part, this is how Big Blue is beaten--unorthodoxed (but, of course, clever) chess moves simply throw the AI for a loop.

This "memory" is still better than nothing, though, as most players simply stick to a pattern themselves, and if the AI can record and react to it over several games, you would in effect be training the computer to beat your specific game plan. I have to admit, this has appeal if only in that I would need to keep trying different things in order to keep the AI off-balance. By the way, that works well against human players as well.

But the big piece is still missing: How do you program an AI to intelligently expect the unexpected, to cleverly try the unexpected itself, and to learn not only from what has actually happened in previous games (which can be done in at least a simple way) but to predict what might happen beyond that stored experience (which would require, at last, "intelligence"--keeing in mind that random responses might occasionally look smart but will generally fail, so its more than just being unpredictable)?

Most of us aren't actually all that intelligent in this sense either when you think about it. That's good, in a way, since the AI could learn to beat us given enough exposure to our patterns of play (though I suppose we'd need user profiles so that your brother doesn't come on and confuse the poor AI with something different from your strategy).

But true intelligence is more than this, obviously, and to expect it from computer code when its rare to find in human beings themselves is a grand hope indeed. Personally, I think if computers ever become THAT smart, [insert scary Doomsday sentence here]...

Until then, a strong on-line community with the tools to tweak the game is still our best bet for a Civ3 that will evolve along with us.
[This message has been edited by yin26 (edited June 28, 2000).]

[Grier]

Well the only thing that I can say to follow Yin26s' excellent post is that the AI doesnt have to be better than (or even equal to) a human player to give a good challenging game, cos remeber there are often 7 (or more) civs against you, which puts the odds against you anyway.

My only comment on improving the AI is that if you get rid of ICS then you also get rid of the easiest way to beat the AI, and that will make the game more challenging.
[This message has been edited by Grier (edited June 29, 2000).]

[UltraSonix]

You know, Grier's right! I hadn't thought of that - perhaps a near-human AI is not needed after all...

------------------
No, in Australia we don't live with kangaroos and koalas in our backyards...

[yin26]

It's true that being ganged up on or giving the AI more resources to work with, etc. is a workaround. In Age of Kings, for example, putting the AI on "Hardest" gives it such a jump on you that you end up playing a totally different kind of game to survive and win. Not that this is bad in its own way, but once again its a matter of finding how best to let the AI screw up than it is executing great strategy...though some of what you develop could translate well to multiplayer.

[Ralf]

To summarize it all: I can live with the overall “unit-navigation problem” - IF Firaxis instead concentrates its efforts on…

* The overall AI City-management problem.
(Use force guiding scripts/ templates instead – and many variants of them, so no game is the same).

* The ICS-problem (and not only the extreme variant of it – also building 12-15 cities and 2-3 early Wonders BEFORE building basic temple/marketplace structures should be combated).

* The implementation of the “Rise and fall of empires” idea. TOO successful and far ahead should also have its distinct disadvantages. (By the way, is there a thread about this?)

… Then above alone should ensure a much more interesting game. Repeating SMAC with different graphics just isn’t good enough.

[Grier]

IMO the rise and fall of empires is the most essential change required for civ3, as its one of the only ideas that will actually make civ3 a different game without losing the cannon civ elements.

[Steve Clark]

That along with its companion idea, 'different modes of winning'.