I think I've managed to figure out how the AIPs work. The AIPs certainly have a lot of information about the inner workings of the AI. Read and enjoy.

AIP COMMENTS (based on default.aip)

Default.aip is the first aip loaded at game start and contains a number of strings which are not found in the other personality aips. If repeated in the other aips, the new values take precedence over the default.

// The next line was used in Dark Reign to specify a likelihood that a spy
// would be seen... We can use that stuff in Civ3, too
int infiltrator_period = 2;

No clue.


PRIORITIES

These probably set the weighting for AI priorities. Most of the priorities default to the figures loaded at start from default.aip so I will only cover those which change in the other aips.

double build_buildings_priority = 20000000.0; except citywall.aip which sets this to 1998000.0
double build_troops_priority = 20000000.0; except build_troops_off.aip (1998000.0), unit_focus.aip (20002000.0)
double sally_priority = 0.0; except nosally.aip (300000.0), sally.aip (459000.0)
double seige_priority = 405000.0 for all except siege_off.aip = 0
takecity.aip = 415000.0
Determines priority assigned to conquering cities.
double attack_troops_priority = 405000.0 for all except takecity.aip = 415000.0
Determines priority assigned to building attack_troops?
double bombard_priority = 407000.0 for all except takecity.aip = 415000.0
Determines priority assigned to building bombard troops?

Actually, there are lots more - I just can't be bothered to go through the lot. This is my guess on how it works. The higher the number, the higher the priority assigned to it.

From default.aip, it seems that buildings and troops have equal priority, though if unit_focus.aip is loaded, then units take priority, if build_troops_off.aip is loaded, then buildings take precedence. Wonders are at 1997500.0 so they will always be 3rd. The AI also prioritises the ALP components on par with units and buildings, and interestingly, capitalisation also gets a 2000000.0 billing. Settle_priority is 1700000.0.

MODIFIERS

These modify the above priorities, depending on the situation. Just some of the more intresting ones:
double human_target_raw_bonus = 5000.0; //Nice to be human
double defenseless_city_raw_bonus = 50000.0; //Juicy targets are hard to pass up.
double danger_diminishment = 0.5; //This is interesting - it is set to 0.5 for all AIPs except slaver, which has it set at 80000.0. No idea what it does.
double per_capita_modifier = 500.0; //It looks like the bigger the city, the more attractive the target. Defaults to 500.0 except for gather.aip, where it is set to 0. My guess is that while gathering troops, the AI doesn't want to be distracted by hitting cities until gathering is complete.

// WANDER CONTROL
double wander_density = 0.05;
int wander_rounds_from_target = 3;
Hehe, cool stuff.

// FORCE MATCHING
double min_defense_matching_force_ratio = 1.2; //changes depending on military.aip
double min_attack_matching_force_ratio = 1.2; //changes depending on military.aip
int num_city_defenders = 1;
Sets the goal for force matching ratios. I believe num_city_defenders sets the minimum number of city defenders (more are allowed). Pull_units_into_city.aip increases minimum city defenders to 5. Barbarians have this set to 0, and yes, I've seen Barbarians leave cities undefended.

// MAX BUILD TURNS
int max_build_building_rounds = 15.0;
int max_build_unit_rounds = 20.0;
int max_build_wonder_rounds = 50.0;
Good stuff - it makes sure that the AI doesn't start on a building or unit that will take forever to finish. I've looked further down the default.aip file and I've found that the AIP considers the X-Lab an improvement, not a Wonder. What I will do is to move the X-Lab and Embryo Tank into the Wonder category, so that it will treat it as a Wonder, not an Improvement. Because if I don't do so - the AI will never be able to build the X-Lab since it goes over the specified 15 turns. - DONE TLL 14/6/99 - note that this s only necessary for default.aip - the rest do not override this value.

Doing this also lays one of my other major concerns to rest - the AI will not be stupid enough to build the X-Lab until it can do it within 50 turns, which is not so easy to do even under Technocracy. It is not impossible, however - a good sized AI city with lots of mines can do the job. Just not very quick. This also means that if the AI can't complete the X-Lab in 50 rounds it won't try, and will focus on building units and other improvements instead. Another worry put to rest.

// MAX GOALS PER CITY
// These values are used for an AI team to define the maximum number of each
// type of goal to consider in a given round. The number is defined per city
// in the empire, so if the empire had 10 cities, and MAX_ATTACK_GOALS was
// 0.5, we'd only allow 5 attacks per round

Most of the following are self-explanatory, but the question here is ... what's the difference between the positive and the negative numbers? Take for example:
#define MAX_EVAL_SEIGE_GOALS -20 (default)
Cleric, SciFew, SciMany (0.5), MilFew, MilMany (2), Slaver (-5), Barbarian, Naval, Supernaval (-10), Fallback, Gather, Takecity (-15), Siege_off (0)
I certainly don't get the significance of the negative numbers - it would make sense that the Warrior types would consider more conquest options, but shouldn't it also apply to Fallback, Gather and Takecity, which are war AIPs? Or does the AI only consider the integer figure, with the negative number meaning something else to the AI? e.g. for Takecity it will still consider 15 targets per city in empire, but the negative number has some other significance?
This would make sense, because siege_off, which tells the AI to stop conquering cities is a flat 0.
So the -ve numbers signify something, but its the integer that gets evaluated.

#define MAX_EXEC_REFORM_CITY_GOALS 0.5 (default) Set to 0 for Barbarian
//TLL 14/6/99 Changed this to 0.25 for default.aip - AI is reforming cities needlessly, spending 1000 gold too early and too often.

// The Build Data
// This specifies a number of lists, from each of which EXACTLY ONE type of
// unit will try to get built (in whatever quantity is specified)
// Troops_Primary take precedence over Improvements and Wonders.
// Troops_Core are interleaved with Improvements and Wonders. Wonders
// score just over 8000, Improvements just under 8000. The Troops_Core
// score is doubled for the first unit then bonus and given as is for the last
// requested unit, so bids over 4000 drown out buildings for part of the build list.
// Unit bids over 8000 drown out all buildings and wonders until fulfilled.

I'm not sure how these priorities interact with the earlier priorities above. I have a strong hunch that these are also modifiers and add directly to the priorities above.

Troops_Primary would then take precedence over Wonders by virtue of their big modifiers, added to the 2500 bonus they already have over Wonders above.

Then the first unit in Troops_Core is considered - and since the average Wonder has a priority modifier of 9,100, less the 2500 difference = 6600, any doubled bid of 3500 (not 4000) for the first Core unit called will take precedence over an Average Wonder. For improvements, because build_building_priority is 2000000.0, a doubled bid of 4000 is required to drown out buildings.
It will then build either an Improvement or Wonder, and then a Troops_Core and then interleave, until all bids are fulfilled. It all really depends on the priority assigned to the Wonder or the Improvement. Then it will start building Overflow troops, which are only built if everything else has been fulfilled.

I wonder if this emphasis on troops is what is causing the AI to overswarm. What you might want to do is to leave the priorities alone, but to reduce the actual number of troops built per city. This would free up space for more improvements. Watch what you do though, you want to make sure that you do not cripple the AI's ability to make war. Check to see how all these interact with the War aips.

"Scout_Troops_Primary", 25000.0 default for all //Highest priority

"Land_Settler_Troops_Primary", 10500.0; masssettle_sandbag.aip (20500), survival_mode.aip (15000)
It would seem here that masssettle_sandbag (only used in Chieftain-Warlord and if found early) prioritises Settling more. masssettle.aip (used for King-Deity and if found early) leaves it at default. I assume this means that on lower difficulty levels the AI tries to build more cities if found early. Survival_mode is a specialised IP that only kicks in if the human_capitol is within 15 tiles and the turn number is 16-34. I assume again that it will try to build more cities in order to prevent being wiped out quickly.

"Sea_Settler_Troops_Primary", 11000.0; default
"Space_Settler_Troops_Primary", 11000.0; default //Note how both are higher than Land - in other words, if sea or space colonisation is possible, they take precedence.

"Land_Troops_Primary", 6000.0; default except for getarmy, survival_mode (7900.0)
"Land_Strike_Troops_Primary", 6000.0; default except for getarmy, survival_mode (7900.0)
"Land_Ranged_Troops_Primary", 6000.0; default except for getarmy, survival_mode (7900.0) //Makes sense, survival_mode will also want to prioritise units after settlers.

"Sea_Troops_Primary", 5000.0; default except for naval, supernaval (6000.0)
"Air_Defender_Troops_Primary", 10000.0; default //Looks like air defense is a priority if needed. Good call by programmers.
"Space_Defense_Troops_Primary", 10000.0; //default

"Settler_Troops_Core", 3500.0; default
"Slaver_Troops_Core", 6001; default except for masssettle.aip (900), Slaver (6500) //Slaver troops include Clerics and Televangelists (the Cleric AI will build these instead of Slavers)
"Abolitionist_Troops_Core", 6001; default except for massettle.aip (900) //Interesting, it looks like masssettle.aip (loaded King-Deity and if found early) will focus on buildings and units instead of special actions. Makes sense. In any case, it will leave masssettle and masssettle_sandbag once it gets more than 5 cities. I am beginning to wonder whether it is advisable to change this to 4 cities. Civs that are next to you hardly have a chance to play to their personalities - in fact, with Slavers, you can basically keep them at less than 6 cities for the early part of the game, in effect dampening their personality prospects. - DONE TLL 14/6/99

"Diplomatic_Troops_Core", 4500.0; default except for getarmy (0.1), masssettle, masssettle_sandbag, survival_mode (900), naval, supernaval, scimany (2500.0) //I guess Peaceful personalities wait for other civilisations to contact them? No biggie, gives it more personality this way.

"Scout_Troops_Core", 2900.0; default except for getarmy (0.1), masssettle, masssettle_sandbag, survival_mode (900), naval, supernaval, scimany (2500.0) //Mmmm, other than initial exploration, Peaceful personalities also prefer to stick to their own turf. Cool.

"Spy_Troops_Core", 4500.0; default except for getarmy (0.1), masssettle, masssettle_sandbag, survival_mode (900), naval, supernaval, scimany (2500.0) //Ditto
"Infector_Troops_Core", 4500.0; default except for getarmy (0.1), masssettle, masssettle_sandbag, survival_mode (900), naval, supernaval, scimany (2500.0) //Ditto
"Ecoterrorist_Troops_Core", 4500.0; default except for getarmy (0.1), masssettle, masssettle_sandbag, survival_mode (900), naval, supernaval, scimany (2500.0) //Ditto

"Defend_Troops_Core", 3500.0; default
"Assault_Troops_Core", 3500.0; default
"Ranged_Troops_Core", 3500.0; default
"Naval_Attack_Troops_Core", 1750.0; except for barbarian (1000.0), naval, supernaval (3050.0)
"Naval_Defense_Troops_Core", 1750.0; except for barbarian (1000.0), naval, supernaval (3050.0)
"Naval_Tranport_Troops_Core", 4501.0; except for barbarian (1000.0), masssettle (2750.0)
"Carrier_Naval_Transport_Troops_Core", 1650.0; except for barbarian (1000.0), naval, supernaval (3050.0)
"Naval_Stealth_Troops_Core", 1750.0; except for barbarian (1000.0), naval, supernaval (3050.0)
"Air_Attack_Troops_Core", 2000.0; except for barbarian (1000.0)
"Air_Bomber_Troops_Core", 2000.0; except for barbarian (1000.0)
"Air_Defender_Troops_Core", 2000.0; except for barbarian (1000.0)
"Space_Assault_Troops_Core", 2000.0; except for barbarian (1000.0)
"Space_Defense_Troops_Core", 2000.0; except for barbarian (1000.0)
"Space_Bomber_Troops_Core", 2000.0; except for barbarian (1000.0)
"Nuke_Troops_Core", 4500.0; except for barbarian (1000.0)

"Lawyer_Troops_Core",5000.0; except for barbarian (1000.0) //Ah, no!!!!! Ok, I'm going to have to do something about this. It prioritises Lawyers!! ahead of units? omg - no wonder the AI is so )*&*)(! annoying.
"Branch_Troops_Core", 4500.0; except for barbarian (1000.0) //Grrrr. Either we have to change this so that FRIENDLY AIs won't Corp Branch you (in diplomacy flis) OR we reduce this priority.
"Ad_Troops_Core", 4500.0; except for barbarian (1000.0) //Ditto for Subneural Ads, if the game gets that far.
"Terror_Troops_Core", 4500.0 except for barbarian (1000.0) //I wouldn't mind seeing more Ecoterrorists and Infectors actually. But trade troops - oh man. They should play a role, yes - but not to such a huge extent.

The rest of the list is for overflow troops, which get prioritised last and so aren't important. Only interesting high priority is "Wormhole_Probe_Troops", 100000.0, which takes precedence I suppose if it ever becomes available.

BUILD LISTS

What follows next is a series of build lists. For each list, the AI will only build ONE type of troop in the list (probably the one at the top of the list, unless it is no longer available - though this is just my guess). After deciding which unit it wants to build, it will then build the specified number of units per city.

Which brings me back to the question of negative numbers. What on earth do they mean? Looking through the list, this is my best guess - negative numbers refer to the most they will ever try to build of a unit at any given time. Meaning that for eg.

Build_List_Element Land_Settler_Troops_Primary[MAX_ELEMENTS];

#DATA
//
//--------------------------------------------------------------------
"UNIT_SETTLER", -2;

It will only ever build 2 settlers in all its cities at any given time, instead of building 2 cities. This would also tie in with the MAX_EXEC and MAX_EVAL modifiers above, with the AI only ever evaluating or executing so many goals at a certain time.

This is not to say that the AI will never build more than 2 settlers or 1 nuke etc - it simply limits the total number that can be under construction. This is my best guess.

Positive numbers like:

Build_List_Element Land_Troops_Primary[MAX_ELEMENTS];

#DATA
//
//--------------------------------------------------------------------
"UNIT_WARRIOR", 1.0;
"UNIT_HOPLITE", 1.0;
"UNIT_PIKEMEN", 2.0;
"UNIT_MUSKETEERS", 2.5;
"UNIT_MACHINE_GUNNER", 1.0;
"UNIT_PLASMATICA", 1.0;
"UNIT_LEVIATHAN", 0.1;
#END_DATA

Means that it will build that many units per city, depending on what unit it chose. So for example, if it chose to build Pikemen, it would try and build 2.5 Pikemen for every city in the empire. If it chose Leviathans, it would only build 1 Leviathan per 10 cities. And in all cases, it will only build units if it is capable of finishing them within 20 turns (based on int max_build_unit_rounds = 20.0 above). So in other words, the AI won't assign units to be built haphazardly.

Anyway, this is just my best guess.

WONDER AND IMPROVEMENT LISTS

Then we get down to Wonder and Improvement Lists. Again, this is my best guess, but the AI has internal routines that tells it to rank what type of improvement it presently needs. Once it goes ABOVE that threshold, it will choose a building from that list. This is borne out by the behaviour of citywall.aip.

double Wonders_List_Threshhold = 0.8; except for Barbarian (0.99) and citywall (0.6)
double Transportation_List_Threshhold = 0.6; except for Barbarian (0.99), default (0.8), citywall (0.7)
double Growth_List_Threshhold = 0.2; except for barbarian (0.99), default, citywall (0.3)
double Happiness_List_Threshhold = 0.2; except for barbarian (0.99), citywall (0.5)
double Production_List_Threshhold = 0.4; except for barbarian (0.99), default (0.2)
double Gold_List_Threshhold = 0.4; except for barbarian (0.99), default, citywall (0.3)
double Science_List_Threshhold = 0.2; except for barbarian (0.99)
double Defense_List_Threshhold = 0.2; except for barbarian (0.99), citywall (0.0)
double Miscellaneous_List_Threshhold = 0.0; // equals 1 - best_rank default for all
double End_Game_Object_List_Threshhold = 0.0; // equals 1 - best_rank 0.0;
double Capitalization_List_Threshhold = 0.0; // equals 1 - best_rank 0.0;

The // are the programmers' own comments. End_Game_Object and Capitalization only appear in default.aip. Now following the above logic, it would seem that you'd choose Miscellaneous, End_Game_Object or Capitalization pretty easily. However, that weird comment about 1 - best_rank makes me wonder if the situation is reversed for these three. But I am pretty sure all the way from Wonders to Defense it will choose a building from the list if it exceeds the threshold.

Actually, I think Miscellaneous follows the same rule as well - I've looked into the Miscellaneous list and it contains first a Granary, then City Walls, then a Market, Courthouse and Population Monitor. So it will default to building these buildings first if it doesn't exceed the other thresholds.

Growth and Science seem to take precedence over Gold, so it looks like the AI will not need any tweaking in this regard. It looks like yes, the AI will know how to build Science Improvements when Science slows down (the AI keeps a separate count, accessible only if you Spy on an AI city of "City Science" meaning how much Science is being produced in a particular city - this includes Science from Gold).

What follows after all this are the actual build lists, once again I assume the AI will choose the one with the highest priority, unless it is no longer available for building. If priorities are equal, it will pick the highest on the list.

Some changes will need to be made - for Science buildings, you want it to build a Publishing House before a University, since the University is pretty hopeless. - DONE TLL 14/6/99

ADVANCEMENT LIST

Finally, right at the very bottom is the research path the AI will take, and it differs from personality to personality. It will research the highest available on the list, and then work its way down.

You'll have to go through the entire list of AIPs to figure out the other nuances, but this is the general gist of how the AIPs work.

END OF COMMENTS TLL 14/6/99