Controlling Settlers?

[BureauBert]

I am posting this to continue discussing a question that (re-)appeared in the Frenzy AI revisited thread . Talking about AI strategies and concepts we came to the settling question: How to make the AI settle more intelligently?

Originally posted by Martin Gühmann
... you just need to set the settling priority very high and the map will be filled with AI cities ...

Well, yes, that's one thing I did - and I wrote

• a set of settling strategies plus
• a small framework of SLIC to determine the AI's choice of settling strategies using
  [**] their government-specific city-max (thanks to player.government[] this is much easier now)
  [**] the "average" human's number of cities to determine the AI's "desired number of cities"
  [**] and a little bit of variation accoring to gamestage
  [**] (additional parameters to be added / additional refining in future)
  
  The problem with this set of measures is, that they will "fill the map with cities", but they won't necessarily do it more intelligently. They will still settle too close or too far, they will still try to settle in foreign territory, if they are neighbored by a superior HUMAN they will still not dare settling close to him (at least sometimes and for some reason I haven't yet researched), and they will stop settling at all at some point between ancient age and renaissance no matter how high I set their settling priority in strategies.txt and how ridicoulosly small their "empire" is.
  First thing I did, because I really hated having to expel each and every foreign settler, the AI nonetheless immediately settling between my cities if I forgot to expel one of their "little dudes":
  
• wrote a small handler 'DontSettleHere': simply return STOP at SettleOrder if on foreign territory (this goes for the HumanPlayer, too, of course)
  
  The consequence was, that the AI would leave their settler there forever because it obviously can't manage to assign a new goal to this settler (again: no matter which priority is set). The same thing happens if a settler is hindered in settling at his original goal by some other reason. I tried to overcome this problem by
  
• stealing a function from IW I think (but I can't find the source at the moment, please don't beat me if I am not correct) to evaluate settling locations (of course it would be useful to get SLIC-access to the terrain's SettleScore, too)
• moving settlers to good settling locations (like any other army, I thought) and
• force them to settle there using Settle() or SettleOrder()
  
  This didn't work out and so I
  
• wrote a function to simulate settling: SettleNow()
  
  OK this works. The problem would be still, that once the AI has sent out a settler his goal seems to be determined once and forever and they will never re-task him whatever happens. Therefore I had to write
  
• a function to keep track of the "first three" settlers of each AI civ to detect if they are moving at all, to force settling if they accidentially stumble over a good settling tile, and to
• "beam" paralyzed settlers home (actually the paralyzed guy is killed and a new one is created in a random city - giving the AI the chance to assign to the new unit a different goal, raising the chance of accidentially stumbling across a good settling location).

This works, too, and it seems pretty efficient, but it is not extremely elegant and I would still prefer being able to take influence on settler's moves and orders. We could then develop "settling tactics" similar to other unit's/armie's tactics.

Has anybody tried to SLIC settlers to move and settle? Successfully?

My impression is, that settlers are moving during a different turn phase than all other units and we simply don't have SLIC-access to this turn phase at all. Therefore, none of the event handlers I tried to use to control settlers using MovePathOrder()/MoveToOrder() ... as well as Settle()/SettleOrder() did work.

If some people can confirm this theory we could possibly put this issue on the fix list for the source code project.

However - my "SLICing the settlers"-experiment can be examined here: MoT_s_settling.slc and here: MoT_strategies.txt (near the bottom of the latter).

[Peter Triggs]

My impression is, that settlers are moving during a different turn phase than all other units

Not exactly. Because the raw priority for the settle goal (~900k) is so much higher than any other raw priority, matches for this goal are always executed first (so it may look like a different turn phase but it really isn't).

Has anybody tried to SLIC settlers to move and settle? Successfully?

I don't think so. And, in fact, if you want to change settling behaviour, I think it may be necessary to do it via the source code.

[BureauBert]

... always executed first (so it may look like a different turn phase but it really isn't) ...

Ah, yes this would explain my impression in actually watching the settlers. It still won't explain why triggering events like e.g. MovePathOrder() on events like e.g. BeginTurn does work for any army except for settlers . Anyways it seems like this really needs some fix

... via the source code.

If one or two additional, experienced SLICers scecond this point of view I will post a "DEBUG: Controlling Settlers - Events (Orders)"-thread in the Source Code forum.

[Martin Gühmann]

Originally posted by BureauBert
It still won't explain why triggering events like e.g. MovePathOrder() on events like e.g. BeginTurn does work for any army except for settlers . Anyways it seems like this really needs some fix

It doesn't need to be fixed. Your problem is that you use the wrong events, the events you used are just the orders and not the event themself. If you give an order to an army it waits with the execution until it is the turn of this army. So you can give an order this turn to an army but the order cannot be executed this turn, therefore the army has to wait for the next turn. And if an order is given to an army it can be cleared until it is executed. Look into the MG_BetterAI.slc and you know how to move units. And there is also an Settle event.

City spacing can be determined in strategies.txt. Settle targets can be determined in goals.txt or orders.txt, and also the target owner.

-Martin

[BureauBert]

Your problem is that you use the wrong events

I have tried several events to trigger on, among those e.g. BeginTurn.
The event I tried to trigger has been: Event:MovePathOrder(army_t, location_t) (I am pretty sure I did try Event:MoveToOrder for testing, too, though this one would not be very useful to move a settler to a good location ).
This combo usually does work to move any army anywhere (on the same landmass/ocean, of course) since e.g. good old Frenzy AI works exactly this way. When trying to move settlers in exactly the same way, it didn't work.
Also, triggering Event:SettleOrder(army_t) (ok, this is just an order) or Event:Settle(army_t) didn't work (i would assume the latter should trigger immediate settling, if there's supposed to be a difference between those two).
I did not say that it would be impossible to trigger on Settle - like you are doing (or planning to do - "//") in MG_BetterAI.slc:

//HandleEvent(Settle)'MG_SaveSettleLocation'pre{
//	MGSettleLoc = army[0].location;
//}

But taking into consideration your argument:

if an order is given to an army it can be cleared until it is executed

and Peter's:

Because the raw priority for the settle goal (~900k) is so much higher than any other raw priority, matches for this goal are always executed first

I could think of something like the hard-coded tasking logic not only moving settlers before all other armies (and thereby fooling me ) but also overriding the orders given by SLIC due to the high base priority assigned to the settling goal. In other words: The exe would always override all orders given to settlers through SLIC by its own logic because the base priority for settling (moving to location, settling there) is set too high?
Still this sounds a bit weird to me: why shouldn't the exe override my SLIC-orders when it's able to match a goal with say priority 450000? A match is a match, isn't it? Consequently orders given to armies through SLIC should always take effect only if the army isn't and will not be tasked by the exe until the next turn (BeginTurnExecute or whatever) and so forth.
But on the other hand it's a fascinating theory . When I have time (-> the weekend) I will try to lower the settling goal and see if I will then be able to SLIC them.
Well anyway, I will give it a try - if nobody tells me this theory is complete rubbish in the meantime .
Additional expertise would be appreciated!

[Dale]

I don't know if anyone remembers, but I was the only one (from memory) to SLIC settler stuff. In AOM I had the colonisation thing happening.

BureauBert, take a look at this SLIC (ignore the WhatGovtAmI and SetupSomethings) as the rest deals with my attempt to make the AI settle well.

The script works by simulating the automatic settling by the population (non-Govt settling). When a city pop increases, there's a chance it'll be grabbed by the script (influenced by what settling policy you enact). When the script grabs a pop it determines a good site to settle (influenced by whether you select coastal or inland policies) and then creates a city there. The pop is not added to the original city but the new city simulates the new pop starting a new city.

Anyways, something to look at that worked, and surprisingly worked well. One thing though, I would suggest changing the script so when it looks for a good site to settle that it looks at cells in the radius distances of 4 to 7. At present the script only looks at radius 4.

Dale

[Dale]

Crud, forgot the file. Here it is.............

[child of Thor]

Originally posted by Dale
I don't know if anyone remembers, but I was the only one (from memory) to SLIC settler stuff. In AOM I had the colonisation thing happening.

I remember played one of the early releases of AOM for a couple of months in one of my old work sites, what about 1.5 years ago was it? - it certainly made the game feel very different with the AI's behaviour. Was there also a problem with AI cities revolting all the time? this could have been from another mod i was playing........it was all so long ago

[BureauBert]

I don't know if anyone remembers, but I was the only one (from memory) to SLIC settler stuff. In AOM I had the colonisation thing happening ...

Yes, yes NOW I remember - i have stolen ColSiteScore() from you then
Abstracting settling is indeed a very clever and interesting concept and probably the only one that can currently be SLICed in a satisfying way. I even remember the maps you posted in your thread at this time to proof that the AI are actually doing well - the maps looked really good! Additionally this concept can easily be tied to various "strategies" and "policies" in the context of different Mods.
The only thing that keeps me off using this approach is my personal preference: I just like exploring, sending out the "little dudes", competing in claiming the land by actually moving units. I even thought of abstracting settling for the AI while sticking with "conventional" settling for the HUMAN - but I think this would be too annoying by lack of plausibility and it would be difficult to balance, too.
This is my reason for trying to SLIC the AI to better use of their settlers despite the problems I ran into.
But your successful efforts have encouraged and helped me a lot in actually trying it.
Did you come up with the abstracted settling model after having tried to do it the "conventional" way or didn't you ever try moving the settlers?

[Martin Gühmann]

Originally posted by BureauBert
I have tried several events to trigger on, among those e.g. BeginTurn.
The event I tried to trigger has been: Event:MovePathOrder(army_t, location_t) (I am pretty sure I did try Event:MoveToOrder for testing, too, though this one would not be very useful to move a settler to a good location ).

And again order events do not help you. The only purpose of an order event is to tell an army to do something on the next occasion. And if the AI thinks in the meanwhile that the orders are not good then it changes the orders. Instead of just giving orders you have to call the events that actual do you want to be done.

Originally posted by BureauBert
This combo usually does work to move any army anywhere (on the same landmass/ocean, of course) since e.g. good old Frenzy AI works exactly this way.

That's wrong this combo doesn't work very well, actual you should exspect that you give the MovePathOrder and the army moves along the given path until it reachs the end of the path, that doesn't work. It does work in frenzy for some extend, because the order is given on the BeginTurn event and it is given on the MoveUnits and on the EntrenchOrder event as well. So the order is given on every turn again and pre whenever the army might do something else then it should do. That is not a very efficient way to do it. And in fact it can be done better and MG_BetterAI.slc does it better.

Instead of using these move order events use the MoveArmy event, and before this event use the ClearOrders event to clear all previous given orders. With the MoveArmy event you can also make an army attack another army or city, with all these move orders you cannot do it. And another thing never use the MoveUnits event that event has no legality checks, you can do a lot of things with it, you can move an army into a forign city without attacking it, or move some units into a suary with hostile units and group them into an army afterwards, but don't expect that the game does not crash when you try to use this hybrid army, or move an army into the sea.

Originally posted by BureauBert
When trying to move settlers in exactly the same way, it didn't work.
Also, triggering Event:SettleOrder(army_t) (ok, this is just an order) or Event:Settle(army_t) didn't work (i would assume the latter should trigger immediate settling, if there's supposed to be a difference between those two).

The Settle event should do the job, of course only if the settler has enough move points left, so that it can settle. So you have to do it when the settler got his move points for the current turn and before it actual moves.

Originally posted by BureauBert
I did not say that it would be impossible to trigger on Settle - like you are doing (or planning to do - "//") in MG_BetterAI.slc:

//HandleEvent(Settle)'MG_SaveSettleLocation'pre{
//	MGSettleLoc = army[0].location;
//}

I don't know why you want trigger on the Settle event, I would assume that you would rather call the Settle event. However you can only see the code fragment only in combination with the next code fragment:

//HandleEvent(DisbandArmyOrder)'MG_NoDisband'pre{
//	if(GetGroupLandUnitsAtLocation(army[0].location) > 0 
//	&& GetNonGroupLandUnitsAtLocation(army[0].location) > 0
//	&& army[0].location != MGSettleLoc){
//		return STOP;
//	}
//}

Both code fragments deal with the problem that the AI cannot stack its units properly. This fragment should prevent the AI from disbanding its armies when it should not merge its armies, e.g. foot soldiers and air units. One side effect of forbiding army disbanding at all is that settlers cannot settle, and therfore the settling detection was necessary. However I came to the conclusion that this was not a good idea to handle the problem.

Originally posted by BureauBert
I could think of something like the hard-coded tasking logic not only moving settlers before all other armies (and thereby fooling me ) but also overriding the orders given by SLIC due to the high base priority assigned to the settling goal. In other words: The exe would always override all orders given to settlers through SLIC by its own logic because the base priority for settling (moving to location, settling there) is set too high?

That is rather wrong, the higher the priority the more likely it is that the AI selects the settle goal, however another difference makes it how offen the AI takes the settling goal into consideration and execution.

Originally posted by BureauBert Still this sounds a bit weird to me: why shouldn't the exe override my SLIC-orders when it's able to match a goal with say priority 450000? A match is a match, isn't it? Consequently orders given to armies through SLIC should always take effect only if the army isn't and will not be tasked by the exe until the next turn (BeginTurnExecute or whatever) and so forth.

First BeginTurnExecute isn't called once per turn it is called once per army. And for the other question, your orders will be overwritten if the AI find another target that fits better. You have to call the real execution events Settle and MoveArmy.

And of course you have to explore orders.txt and goals.txt in orders.txt you find which unit can execute which orders and you find which are the target of an specific order and in goals.txt you find also the target type of an goal and you find also the target owner, for instance if you add the line: TargetOwner:HotEnemy to the SETTLE_LAND_GOAL the AI does settle in territory of an opponet with that the AI is on war.

-Martin

[BureauBert]

Hi there!

Finally and thanks to the informations given by Peter Triggs and Martin Gühmann I have been able to get this script working: Better Settling 1.0 (in the middle of my humble "download section" ).
I tested it as a stand-alone version on a not-quite-ultra gigantic map (115 x 230), watched it in automated /rnd mode for about 150 turns on a "regular" gigantic map and as for my taste it runs fine, with one exception: The exe still trying to move my settlers elsewhere at times occasionally causes a clueless settler moving back and forth for several turns until the situation changes and the exe "releases" the poor guy (this preferably happens along rivers, must have something to do with movement points left - I couldn't find a workaround for this one). Well the entire script has been my lesson in making compromises with the exe - I think the result is ok so far (maybe the strategies settings will need some tweaking as well as the script's logic for the evaluation of settling sites). However - these are the main purposes of the script:

• AI will use settling strategies according to their need for additional cities, they will also compete with the HUMAN player
• They will select locations for settling and move their settlers somewhat more intelligently.
• They will never settle on foreign territory (NOTE: this is also prohibited for the HUMAN player)
• They will generally build compact empires (without placing their cities too close) and they will not settle far away from home as long as there is enough land to settle near the center of their empire

EDIT & UPDATE:

• REQUIRES: CTP2 Apolyton Edition playtest version fom 2004.06.15 onwards (successful implementation of player.government).
• I found a working workaround for the problem mentioned above (settler moving back and forth because of concurring orders from the exe), the problem is not resolved at the root, but the settlers do no longer get stuck for several turns.

My apolgies go to the two people who already downloaded the script - I have updated the ZIP-file, please download it again from the same link, thank you!
If more people would be willing to take a look at the stuff and to test it (along with testing the CTP AE playtest version/s) I would be glad and I would greatly appreciate any feed-back and suggestions.

[Martin Gühmann]

I just took a look into your code and as you use the playtest build anyway, these things may also improve your code:

// - g.num_of_players: Gets the number of players in the current game
// - g.last_player: Gets the last player, the player with the highest
// index in the game
// - g.max_player: Gets the maximum number of players that is in a game
// possible, currently it is 32 including the Barbarians.

The above is an iprovement, because you cannot really trust the values in the userprofile.txt:

	tmpMaxPlayers = preference("MaxPlayers");
	tmpNumPlayers = preference("NumPlayers");

The other function you can use is this new slic function:

// - IsWater Gets whether a location is water.

Unfortunatly I don't have much time now so that I cannot tell you how you can bypass the land settler check, but it is possible, it is in one of the slic files I uploaded.

-Martin

[BureauBert]

Thank you Martin for looking at the code and also for pointing me to the new SLIC functions. Actually I have been so glad to get the settling code working at all that I didn't care too much for additional code optimization . But on the other hand I did indeed experience some trouble with the SLIC-function using preference("MaxPlayers") when I changed this value using ModManager between two tests to a lower value. In my reloaded playtest game all cycles through the players were incomplete and it took me some hours to figure out that there was no bug in the settling functions but in the screwed up preference value .
On this occasion I actually tried to replace the preference-stuff with one of the new g.*_player variables (I am not quite sure but I think it was g.max_player) but it gave me a SLIC-error at startup (I don't remember the exact wording, I think it was something like "unknown", "unimplemented" or similar). Well I took it out without further testing just because I was too busy with my own stuff at that moment.
But I will give the new variables and also the IsWater-function a try as soon as I get back to coding.
BTW I opened a new thread PROJECT: New SLIC & DB-Constants to help people willing to test new SLIC-stuff . In some cases it could be useful to add extended descriptions - for example I am not quite sure if g.max_player is supposed to contain the absolute maximum of players the exe currently allows (this is what the current description suggests), or if it is supposed to be a more reliable replacement for Preference("MaxPlayers"), hence containing the MaxPlayers value of the current game.
Another question I didn't yet dare to ask in the Playtest II-thread (and it's a bit off-topic here): How are the chances of broken database-access being repaired soon (as far as I noticed AdvanceDB and BuildingDB are affected)? I don't want to hurry anyone and I know that many people are on vacation now, but I am somehow stuck with testing because as it stands I would need to decide whether to use advance- and buildings-db-access or player.government ... and this would be a hard decision
Ah, yes, bypassing the (explicit) land settler check: I remember having seen something like this - it would be something along the lines of

UnitDB(unit[0].type).SettleSize > 0

- right? I will study the code you posted as soon as possible As far as I know from Mr Ogre's famous text only integers in DB-records are accessible - would it nonetheless be possible to check if a flag is set or not? Like:

if(UnitDB(unit[0].type).VictoryEnslavement) {
     // this unit is a slaver
}

Well I will do some own research, too - just in case you or someone else could give a quick answer ...

[Martin Gühmann]

Originally posted by BureauBert
But on the other hand I did indeed experience some trouble with the SLIC-function using preference("MaxPlayers") when I changed this value using ModManager between two tests to a lower value. In my reloaded playtest game all cycles through the players were incomplete and it took me some hours to figure out that there was no bug in the settling functions but in the screwed up preference value .

Yes this is the problem MaxPlayers is not save, you can change it during the game, and it doesn't give you any clue how many players are really in the game.

Originally posted by BureauBert
On this occasion I actually tried to replace the preference-stuff with one of the new g.*_player variables (I am not quite sure but I think it was g.max_player) but it gave me a SLIC-error at startup (I don't remember the exact wording, I think it was something like "unknown", "unimplemented" or similar). Well I took it out without further testing just because I was too busy with my own stuff at that moment.

Maybe you used the wrong *.exe or you misspelled it. Another possibility could be that there is something screwed up in the source code project itsself.

Originally posted by BureauBert
In some cases it could be useful to add extended descriptions - for example I am not quite sure if g.max_player is supposed to contain the absolute maximum of players the exe currently allows (this is what the current description suggests), or if it is supposed to be a more reliable replacement for Preference("MaxPlayers"), hence containing the MaxPlayers value of the current game.

It is supposed to contain the absolute maximum of players the ctp2.exe currently supports. And actual I don't know which use some reliable replacement should have. Because it doesn't tell you anything about how many players are really in the game or how many players can be in the game. I think there might be situations were you need the absolute maximum. But actual in most cases you need to know who is the player with the highest index in the game.

Originally posted by BureauBert
Another question I didn't yet dare to ask in the Playtest II-thread (and it's a bit off-topic here): How are the chances of broken database-access being repaired soon (as far as I noticed AdvanceDB and BuildingDB are affected)?

Well I waited for you to tell us if the problem still exists in the lastest ctp2.exe. It would surprise me if the problem would just affect two of the databases. The database record files are all generated by a program from a script, and the database accessors use all the same code. And another problem is that I tested it on my setup that is not identical with the playtest setup, and I wasn't able to reproduce the problem, but I think I tested UnitDB and GovernmentDB.

Originally posted by BureauBert
Ah, yes, bypassing the (explicit) land settler check: I remember having seen something like this - it would be something along the lines of

UnitDB(unit[0].type).SettleSize > 0

- right? I will study the code you posted as soon as possible

Right, if you just want to know if the unit is able to settle, to figuere out wether it is a land or a sea settler, you have to check SettleCityType. However it is not the best solution, but no mod changes this so far. And the answer on your question that a unit type is not an integer is that the database index of the unit type is returned.

Originally posted by BureauBert
As far as I know from Mr Ogre's famous text only integers in DB-records are accessible - would it nonetheless be possible to check if a flag is set or not? Like:

if(UnitDB(unit[0].type).VictoryEnslavement) {
     // this unit is a slaver
}

That is a boolean and booleans in slic are represented by 1 and 0. And that is also true for such flags, if they are present 1 is returned otherwise 0. You can also check the other flags. However in your example you cannot really sure if the unit is a slaver, because VictoryEnslavement means whenever this units takes succesfully part of an battle a slave is taken. In that case you can use the function IsSlaver.

-Martin

[BureauBert]

Good evening!

Maybe you used the wrong *.exe ...

and

Well I waited for you to tell us if the problem still exists in the lastest ctp2.exe ...

... ah, yes ... while I have been waiting for the next playtest version, because the database issue hasn't been mentioned in the changelog for the 2004.06.28 build ... I just downloaded the latest build, will probably give it a try tonight and post my observations to the appropriate thread (don't want to completely threadjack my own thread)

I think I tested UnitDB and GovernmentDB

GovernmentDB.TooManyCitiesThreshold already worked fine with the 2004.06.15 build ...

Thank you for explaining g.max_player. I agree with you that this will be the most reliable variable to use for cycling through all players, and g.last_player would be the most economic one and reliable in most cases, too.

And actual I don't know which use some reliable replacement should have. Because it doesn't tell you anything about how many players are really in the game or how many players can be in the game.

I just wonder how the latter (g.last_player) would work for the Diplomod script which needs to set up these huge arrays for storing e.g. the "has contact" information. For this setup a definite number of (max-) players is needed, as far as my understanding goes . I suspect that using g.last_player for this purpose would screw things up as soon as an additional (splinter-) civ appears. At least I think this is what already caused a lot of problems with the original version of Diplomod, and caused me to modify the setup of the array:
I changed it from

DIP_hascontact[(DIP_NumOfPlayers * DIP_NumOfPlayers) + DIP_NumOfPlayers]

to

DIP_hascontact[(MOD_MAXPLAYERS * MOD_MAXPLAYERS) + MOD_MAXPLAYERS]

... where MOD_MAXPLAYERS comes from the known function using preference("MaxPlayers"). For being on the safe side I could have used e.g. "28" instead of "MOD_MAXPLAYERS" as well, but I have been afraid that this would blow up the array enormously and need a lot of memory. Using g.max_player for this purpose would probably have this effect, too (but I really don't know anything about memory allocation). In this case I imagine a reliable variable containing the original preference value (= the maximum of players in the current game, and keeping this value even if the userprofile settings are changed in the meantime) could be useful. But certainly Dale or someone else who worked more intensely with Diplomod could tell if this actually is the case.
I guess the difference between the "exe's MaxPlayers" and the "current game's MaxPlayer's" wouldn't hurt so much regarding the size of other arrays that need to be set up at gamestart and indexed by players - so it might be preferable to use g.max_player for these.
Anyways - for anything I might want to do myself in the near future I will certainly be fine with what is already there .

You can also check the other flags.

Ha! That's great, this means I will need to explore the databases more seriously in future (well, yes I should do that anyway ).