How Supply and Demand Lists Are Determined

[samson]

In the 725BC save I have, Philadelphia does not demand spice. If it did once, it doesn't now. All spice demands are on continent 13 and are ODD.

You said the second largest continent has 300 tiles? Oddly, that's the breakpoint for a 1/2 reduction in spice supply I detected. One more tile on that continent would reduce the SQ of cities there and thus favor demand of spice over supply.

I think we're getting close with Spice. Cloth Demand, on the other hand, is still giving me fits. Under 10 techs, the simple terrain formula works well. But after 10 techs, the demand increases at 10 tech boundaries. This indicates a formula something like:

Demand = Forest x 4 + Hills x 4 + Techs/10 x (???)

??? is different for different cities. CitySize doesn't seem to have any effect on the DQ.

[solo]

Yes, when Philadelphia was founded, it demanded spice, but this went away on its first cycle turn. By the way, 3 new cities in row had this happen, but this is probably just a coincidence. Spice calculations are holding up, so far in further tests, after I got the continent count. On continent #13, its 34 at size 1-3 is fairly decent and the 68 at size 4 and higher gives it a good shot at making demand lists.

By the way, I did find another EVEN spice in another game, but it's continent looked like it was #2 in size. I did not want to count two continents to check just one example, though. I have found much to confirm ODD only on the #1 sized continent in other saves. No exceptions to that yet.

You may want to count up tiles on continent #1 just as double check, since just 1 tile cuts things very close. I am not including internal oceans in my counts, since they are numbered different.

Cloth demand may be hinging on another factor we have yet to consider, so I will keep an eye on this, to see if I can spot something unique happening with cities demanding cloth as I continue through the ones in my test game. One thing maybe worth a try is changing dates as another way of simulating the passage of time. I doubt it, but from what we've seen so far, you never know.

Another thing I have been forgetting to mention is that when there is a tie among 2 or more commodities on a roster, the dearest commodity is given the nod, just as you experienced when there is a tie at zero. This consistency is no surprise, but worth noting.

I'm off to check the lists for the Babylonians.

[solo]

While doing Babylon, I have had the first chance to use rivers and to have gold in contention, with which there is a slight problem. Babylon(37,33) (gems,silver) continent #13.

grass = 4
desert = 2
Plains = 5 + buffalo = 9
hills = 3 + wine = 7
mountains = 2
ocean = 3
rivers = 4

2100 coal,gems,copper dye,silver,wool size 2, techs:20
1300 coal,gems,gold dye,silver,silk size 3, techs:21
750 coal,gems,gold dye,silver,spice size 4 techs: 22

(techs now include the one being researched)

First, the demand side works okay, as long as one assumes that copper gains enough steam to get under the silver wiildcard in 750. By size 4, it usually does.

On the supply side, coal is guaranteed for slot #1, since it is way ahead of the others. However, I calculate gold to be 78, all three years, with no competitors at that level, the closest ones being:

2100 hides = 48 and copper = 45
1300 copper = 45 and wine = 38 (hides down to 24)
750 wine = 38 and hides = 24 (copper now in demand)

For things to work better, I would like gold to be half of what it is, which might be possible if the x 1/2 modifier for mountains < 2 were adjusted to read <= 2. This would change gold to be 39. With gold at 39 we would get everything right for 2100 and 1300, and with a little fiddling with either the wine or gold equation, we could have 750 come out right, too.

One consideration is that wool has a SQ of 40 all three years, but this is wiped out by its DQ of 44 each time. If there could be less demand for wool at size 4, then its supply would inch out gold at 39, putting gold in slot 3 where it belongs. This scenario requires no fiddling to the gold or wine equations.

(A final note, is that hides can not be on the supply list in 2100 UNLESS techs are 20, and for this to happen, the tech currently being researched has to be included in tech totals, as you suggested might be necessary, in your previous post.)

(Note 2: I have moved on to Ellipi, and think I have found a definite problem with the wine formula. The LesserOf function should take it off the supply roster, but does not. I may be mis-interpreting how it works, but Ellipi has no rivers and IS supplying wine)

[samson]

Solo,

Did the Babylonians have any starting techs? The numbers you give indicate that you are counting all techs. You'll need to go back to a 4000BC save to see what they had. I suspect they had some and that is throwing the calculations off.

Yes, the gold modifier for mountains should be <3: 1/2.

I've also seen the ties are resolved by choosing the commodoties with the highest ID#.

For continent #1, I count 310 tiles.

[samson]

Running the numbers, I get that Babylon must have had 5 starting techs. This would give the following supply lists (with wildcards removed):

2100: coal, hides, copper
1300: coal, wine, gold
750: coal, wine, gold

[solo]

Yes, the Babylonians had 5 starting techs. So to get tech totals are you saying:

1) Get total from science advisor screen
2) Add one for the tech currently being researched
3) Then subtract the number of starting techs

I haven't tried doing it this way yet. If so, I'll have to re-check the Roman cities to see if they still are working correctly. Before they only seemed to be working if staring techs were ignored.

Other starting tech totals for that map:

1) Romans - 3
2) Germans - 0
3) Egyptians - 5
4) Americans - 3
5) Greeks - 5
6) Indians - 4

How do starting techs affect the SQ's of wine or gold for Babylon? Looks to me that this only changes hides.

What numbers are you getting for Babylon supply commodities of hides, wine and gold? Please give details on the wine calculation, as I am not sure I am doing it as intended.


310 tiles is close enough for me. This means /10 for spice yields 3 then, using either count. Were you adding in lakes? They have a different ID.

[samson]

Yes, the Babylonians had 5 starting techs. So to get tech totals are you saying:

1) Get total from science advisor screen
2) Add one for the tech currently being researched
3) Then subtract the number of starting techs

Yes, acquired techs +1.

I also checked the current research cost of each civ and calculated how many techs they were paying for, subtractiing this from the total to get their starting techs. My numbers were:

Romans - 2
Babylonians - 5
Germans - 0
Egyptians - 5
Americans - 3
Greeks - 5
Indians - 4

Thesea are the same as your numbers except for the Romans which you list as 3. Just to be sure we're on the same page, I am in year 725BC, Babylonians turn.



My supply numbers for Babylon:

2100BC

Coal: 131
Hides: 96
Copper: 45
Wool: 40
Gold: 39
Dye: 30
Silver: 27
Wine: 25
Beads: 17
Salt: 15
Cloth: 12
Spice: 10
Gems: 9
Silk: 8
Oil: 2


1200BC

Coal: 131
Wine: 50
Copper: 45
Wool: 40
Gold: 39
Dye: 30
Silver: 27
Cloth: 25
Hides: 24
Beads:17
Salt:15
Spice: 10
Gems: 9
Silk: 8
Oil: 2

725BC

Coal: 131
Wine: 50
Copper: 45
Wool: 40
Gold: 39
Dye: 30
Silver: 27
Cloth: 25
Hides: 24
Beads:17
Salt:15
Spice: 10
Gems: 9
Silk: 8
Oil: 2

In 1200 and 725, Copper and Wool have higher DQs and so Gold pops onto the Supply list.

Wine = 36 (plains x 4) or 16 (Rivers x 5 - Grass)
(16 is the lesser of two)
+ DistanceToDateLine/4 (37/4) 16+9 = 25
x 2 (Northern Hemisphere) = 50

In 2100 at size 2, Wine is 50/2 = 25. When size goes to 3 it jumps to 50.

[solo]

Thanks, for those details. A lot was clarified for me. You are right about the Romans, as starting techs were 2, not 3. My earliest save was 3950 BC, from which I noted 3 techs on the scoreboard. However, now my written log of the game reminds me the Romans opened a hut to get a tech on the 4000 BC turn! Good grief!

Anyways, earlier Roman results still tally okay when taking this into account.

I was off a bit on wine, so thanks for that details.

I think we are on the same page now in all departments, and can check off Babylon as another city that tallies up okay.

Is cloth still giving fits? Are all cities coming out different from each other?

If it's not city size, perhaps it's some calculation based on the cities' locations. An EVEN, ODD or reaminder kind of thing, or perhaps something new like distance of the city to its capital, or the order that cities were founded in. Could be related to the cycle year in some way, too, since this is usually different for each city in each civ. This would give up to 16 different variations.

[solo]

One more question about wine.

In Ellipi, there are no rivers, but there are 3 grass, with a -3 result which is defintely lowest, because there are plenty of plains. Does the LesserOf function really yield -3 or do you just count it as supply = 0?

The reason I ask is because the DQ turns out to be 16.

If -3 is used for wine supply, then when it's adjusted for location, the result is 10, otherwise it is 16.

However, wine ends up on the supply list.

[samson]

Wine ... yes, it looks like the (Rivers x 5 - Grass) part of the equation is zero-limited.

Also on wine, the continent# modifier extends beyond continent # 10: 2,6,10, 14, 18, etc. ... the series continues. I guess this would be expressed as RemainderOf(Continent# /4 ) = 2.

Re: Cloth. I don't have an answer yet, but I've designed a testing plan.

First, there are two formula templates that could be at work here. One is:

Forest x 4 + Hills x 4 + ( Techs/10 x ???)

the other is:

GreaterOf( Forest x 4 + Hills x 4, (Techs/10) x ???)

The second seems likely since so many factors and formulas have opposites and we already have a LesserOf relation in Wine.

In either case my testing plan is to create cities without Forest and Hills to eliminate the known factors, then tackle the unknown as if it was formula itself, testing for the usual factors: terrain, location, continent#, etc.

I may pump the techs up to 100 to get a 10x multiplier so Cloth appears more easily. Since CitySize doesn't affect Cloth Demand I can use the Silver/Gold/Gems triad with varying city size to measure Cloth within +/- 2.

If this approach doesn't find anything, then we'll have to consider unknown factors.

[solo]

Nineveh and Ur have been checked.

Ur is an EVEN city, and would want to demand spice in 600 BC when it reached size 4, with a count of 68, but hides, with a count of 58, appeared in the third slot on the demand list, instead. So it really looks like only ODD cities on the largest continent are eligible for demanding spice, since it was screened from Ur's demand list. Hopefully, this will happen to more cities on continent #13, and help confirm this theory.

No other problems, as lists went as predicted.

I like your plan for attacking the unknown factor(s) in cloth. Without terrain influencing the equation, an observable pattern may emerge. Good luck!

A thought I had was that the basic terrain formula may change every 10 techs, but it looks like you will be testing for this. Adjusting the terrain formula every ten techs would make each city's increase in demand pretty much unique. For example, after the first 10 techs go by, another terrain type might be added into the terrain equation. 10 more techs, another terrain type is introduced, and so on.

[solo]

Uruk, Akkad and Ashur check out okay, to finish to Babylonian cities.

[samson]

I have a formula for Cloth Demand. The testing plan worked pretty well, with a few modifications. Turns out it didn't use a GreaterOf relation. Also, no new factors were needed to explain Cloth demand, just terrain and location in a new ordering with the Techs/10 modifier. In the process of setting up the tests I also found corrections to the Hides supply and Coal Demand formulas. Now posted.

[solo]

Nice job of finding the rest of the cloth demand formula!

I have tested it with Pompeii, where cloth (16 with the original forumla) needed to pick up some points to make it past the value of wool (20). Having a lot of mountains and a tundra tile was enough to give tile 6 more points, resulting in a total of 22.

Back when the game was being developed and when they were coding the formula for cloth demand, they probably chuckled and said, "here's one they'll never figure out!" It's a good thing that a similar scheme was not used for the other commodities.

Since my last post I have moved on to German cities and have checked out Berlin and Frankfurt, with no further problems. It could be that we now have most of the kinks out of the forumas pertaining to smaller cities.

Without a way of deciphering the values of dye and copper, it looks like the list is finally complete, and just needs to be tested some more as cities grow in size and techs accumulate. I will be continuing tests, but in the meantime have a few notes that might be useful for users of the formulae, and you might want to post some of these in that section of this thread:

1) You mentioned that wine supply gets a x 3/2 multiplier for every 4th continent. The entry needs to be updated to reflect this.

2) Under techs it is worth mentioning how the tech total is determined.

3) A note that integer arithmetic is in effect and how this affects calculations.

4) For Land, note that this is a straight count, excluding rivers and specials.

5) For other formulae, rivers AND their underlying terrain are both used.

6) In the case of ties, on rosters, the most valuable commodity is given preference.

7) Give an example or two to illustrate the geographical formulae, especially for the ones that involve longitude.

8) It wouldn't hurt to illustrate commodity formulae either, with a few examples.

9) I'm not sure about this, but at some time I think I remember you mentioning that you found that wool demand can be influenced by city size. I can not find that post, so I might just be mistaken about this.

[samson]

All good points, I'll include them in the final writeup.

As for Dye and Copper, I think their behavior needs to be defined even if no formula is possible. They certainly act differently on MGE from 2.4.2. It may even be possible to derive formulas for 2.4.2 (with a list of exceptions once they are understood.) I may give them a little more time before closing the civlab casebook on this project.

Also, I'm still not sure when Oil and Uranium are used as wildcards and how to deternine which is used. They may be a few more issues, I'll have to check my notes.

But in general, yes, I think the main questions of how supply and demand commodities are selected and what factors cause them to change are pretty much answered. We probably need to write something up about known triggers and include that information here as well just so everything about Supply and Demand is in one reference place.

A fun project it was.

[solo]

Working on this has been a pleasure, too.

I may have spoke too soon about kinks, as when checking Hamburg, wool presents a problem, coming on to the supply list 2 times, though its DQ surpasses its SQ.

Hamburg(20,54) (wine,wine) continent #21, iron working unknown

plains = 7
hills = 3
grass = 4
ocean = 7
rivers = 4

1650 hides,coal,wool dye,wine,beads size 3 techs:15
900 trade for Pottery
800 coal,wool,salt dye,copper,wine size 3 techs: 18

For both years I'm getting an SQ of 24 and a DQ of 26 for wool, putting it on the demand roster.

This may be due to the fact that I interpret temperate zone offsets to be distances, which are usually expressed as positive numbers. If negative offsets are possible, then wool will work correctly here.

In any event, I think this is an example of something that should be clarified in the geographical definitions.

[samson]

I count 5 grass and 6 plains at Hamburg which gives an SQ of 26 and a DQ of 24. I think that was the problem, a grass/plains crossover.

Temperate Zone Offsets are distances and should not be negative. You're interpretting the concept correctly.

[Elephant]

I come back from a week away and find you have solved it all. I'm so disappointed...

Now we need to "interpret" the results - where is a good place to plant an early city to get Dye or Hides, where is a good place to plant mid-game cities to get Oil, and later Uranium? Lets ponder some of these formulas and come up with some practical play suggestions. A few Excel spreadsheets might be helpful too...

[samson]

Hey, Elephant, nice to see you again. Well, there's still Dye and Copper demand, heh heh.

There's another thread already going (Strategic Implications of Understanding Supply and Demand) to address just the kinds of issues you raise. Feel free to contribute. I have some more ideas I'll be adding in a day or so.

[solo]

Samson,

One more thing we need more info on is about how spice demand works on continents other than the largest one.

Yes, I got those counts wrong for Hamburg, and I'm embarassed to admit that this still happened even after double-checking them! Maybe it's time give this a brief rest before resuming checks. I've been going at it pretty hard, but it's been a lot of fun.

Welcome back, Elephant!

[solo]

I've done a little more research on spice. After getting a spice demander from an EVEN city on my second largest continent, I decided to use settlers there to found cities to see if more spice demanders would appear on this continent. I was able to get another one fairly quickly with an American settler, but this time the city was ODD, giving an ODD and an EVEN spice demander on the same continent.

Attached is a zip file containing two hot saves, one illustrating spice demand by Philadelphia, an EVEN city and the second illustrating spice demand by an ODD city, Chicago. Both are on the second largest continent.

My conclusion from this is that spice demand is limited to ODD cities on the largest continent. On any other continent, any city may demand spice. This makes sense from a design viewpoint, since spice demand is determined by continent size. Spice demand would probably appear on most, if not all, demand lists early on for the largest continent, espcially in panagea map games, so the ODD rule was used to limit spice demand for the largest continent. On smaller continents, where spice DQ is less competitive, there was no need to limit its demand in this artificial way.

[samson]

Solo,

The "odd" spice rule could apply just to the largest continent, but I suspect that there's a size breakpoint at work here. By which I mean that if a continent's size is greater than a certain number, the odd-rule is invoked. For instance, what if there were two very large continents, like Eurasia and the Americas on your giga map; both would likely produce many spice-demanders. If the odd-rule only applied the largest, Eurasia, then the Americas would have many more spice-demanders.

On the "EVEN" hot save, Philadelphia seems to have been founded on this turn and immediately demands spice. However, when I run the numbers, it shouldn't be demanding spice. And if I deliver a caravan or use Reveal Map, the spice demand goes away and the "correct" commodities appear on the S/D lists.

This raises a question: is the founding of a city a "trigger event"? If so, then why is Philly demanding Spice?

[Elephant]

Originally posted by samson
Hey, Elephant, nice to see you again. Well, there's still Dye and Copper demand, heh heh.

There's another thread already going (Strategic Implications of Understanding Supply and Demand) to address just the kinds of issues you raise. Feel free to contribute. I have some more ideas I'll be adding in a day or so.

Leaving the toughest for last, huh?

My apologies for missing the "Strategic Implications" thread. I had just read through the week I missed (as well as 350 "business" emails...) and popped a comment in at the end. Then I started looking at your formulas. I hit your Implications thread around 12:30 last night...

I'll post some thoughts on Implications over there after I finish digesting the formulas (and maybe a bit of turkey? ). Just wanted to post a quick but heart-felt THANKS for all your hard work on this issue. Between the Solo Cycle and the Samson S/D Formulas I hope we can reduce the frustration of bringing that Gold caravan up to Babylon's gates only to have the demand disappear at the last minute. There ought to be an award for this kind of achievement...

[solo]

Samson,

When I run the numbers on Philly, I'm getting spice = 15, putting in the third slot, after hides and wine. The closest competitor I see is cloth, which I compute as 11. Silver and gems come in at 8 each, even lower. Silk, dye, wool and salt are all on the supply roster.

Philly(65,23) (gems,beads), continent #1

my terrain counts (known to be in error in the past!)

grass = 7
mountains = 2
forest = 2
plains = 1
ocean = 9
rivers = 2

dye,gems,salt hides,wine,spice size 1 techs: 14

looks ok to me for 1550. (beads is under gems)

In 1050, Philly's cycle turn, the lists were:

beads,gems,silk dye,hides,wine size 2 techs: 14

with dye pushing spice off the demand list. (salt under gems)

It looks to me like formulas are being obeyed when cities are founded. I had the same thing happen to two cities on the largest continent. When founded, they showed dye in slot #3, but on their first cycle turn, dye displaced spice from their demand lists. Original numbers were wanting spice on the foundation date demand lists.

If your numbers prove to be correct, then I agree that something weird may be going on with spice, or that there was a glitch to the list when Philly was founded. We have seen evidence of glitches in lists, such as when Washington was allowed to build a dye caravan, when dye was not in supply. Elephant, here is your golden opportunity! Please figure out what's going on.

Your other point about North America on the giga map is well taken. Although there were not many cities on that continent, I would expect at least 1 or 2 spice demanders because of its size. As usual, the plot thickens, and there may be more to spice demand than seems apparent now.

[samson]

You're right. Philly's spice demand is legit.
And city-founding is a trigger, so the initial S/D lists should be correct.

Re: Triggers.

As you know, on a solo-year every city whose city number plus the turn number is Mod-16 will have its S/D list updated automatically at the beginning of the turn. But every viewing of the City Display screen on a city's solo-year is also a trigger that updates the S/D lists.

For example, if you trade for Nuclear Fission, this doesn't automatically change any of your cities' S/D lists. But if you then view the City Display of a solo-year city, this act will update the demand list with Uranium.

The solo-year automatic update appears to occur before other updates to a city on the pre-turn cycle. For example, if a city's size increases during its solo-year any changes this might have on the S/D lists are not automatically updated. But if you view the City Display, then the size-related changes are triggered and are applied. However, if you don't view the City Display that year but wait until the next turn (when City Diplay viewing is not a trigger) then you'll see the S/D list status pre-size change. Tricky isn't it?

[solo]

Wow! I did a few tests on this with a few cities going through size changes, and it's even trickier than what you describe. For example, Hamburg turned size 3 on its cycle turn, and built a temple on that turn, too. Here are the various lists:

before 1650 hides,coal,wool dye,beads,wine
1650 zoom for temple hides,dye,coal beads,wine,silk
1650 after zoom hides,coal,wool dye,wine,beads

Now if you don't zoom in 1650 and just look later, you get 1650 after zoom, but if you do not zoom and also do not look, the following turn you get what you saw for the 1650 zoom!

Another city I looked at was Veii, that turned size 4 on its cycle turn of 800 without building anything. If you look at Veii during that turn, nothing has changed, but if you do not look at Veii and wait until the turn after its cycle turn, the lists change in a radical way:

before 800 silk,salt,silver dye,copper,spice
look during 800 same as above
wait and look on next turn silk,salt,dye spice,copper,beads

This is really weird, because dye, usually solidly entrenched as the #1 demand, has moved onto the supply lists for both Veii and Hamburg! This may explain how Washington was able to build dye with a caravan completed on a cycle turn, when I never saw dye supplied. If zooms were built into caravan builds, I bet I would have seen dye on Washington's supply list during the zoom.

During previous tests, I have been looking on every zoom and also looking at every city on every turn, but now it seems that not looking postpones the cycle trigger! I wonder how long? I'll have to do some more checking into what not looking can do!

The implications here are a bit mind boggling. I see the possibility of timing your trigger to when you want to have it occur by not looking, and another possibility of manipulating lists by postponing looks. There seems to be a definite glitch at work here, and the fact that I was able to move dye out of demand with a postponed look may suggest that dye is not stuck on the demand list by purpose, but instead, accidently with cycle turn looks.

Maybe the lists only work correctly if cycle turn looks are omitted. I remember you saying that dye did not behave "correctly" during games, according to observations made in your tests.

[samson]

Solo,

Both the cases you describe are following the solo-year trigger-law exactly as I stated it. It's just that when using MGE you have to contend with the Dye/Copper bug as well.

What you see in Hamburg when Zooming after the temple build is the result of the automatic solo-year update. This actually corrects the Dye/Copper problem and Dye shows up on the Supply list where it belongs. However, using Display City during a solo-year causes another update and, unfortunately, on MGE this re-instates the Dye/Copper demand distortion. If you don't use Display City in a solo-year, then the "correct" values will fix for the next 15 turns. Unless, of course, the city is involved in a caravan delivery which re-invokes the Dye/Copper bug.

but now it seems that not looking postpones the cycle trigger!

No. A city's lists are updated automatically only during its cycle-year. The trigger isn't postponed. When you look at a city in a non-cycle year it has no effect, you are merely seeing the cycle-year updates for the first time. Try using Trade Advisor during a cycle year instead of Display City, it will show the automatic updates without triggering the Dye/Copper bug.

[solo]

A brief update on this. For cities that have list changes on cycle turns, but do not have size changes or tech changes on these turns, delaying a look until after the cycle turn produces different lists, again with dye moving from demand to supply. Ditto for a delay of another turn. Looking on cycle turns makes something different happen. My observations are from a hot seat MP game. Things may work differently in SP, and even in 2.42 games.

[samson]

A brief update on this. For cities that have list changes on cycle turns, but do not have size changes or tech changes on these turns, delaying a look until after the cycle turn produces different lists, again with dye moving from demand to supply. Ditto for a delay of another turn. Looking on cycle turns makes something different happen.

Solo, you are observing the same phenomenon I described but attributing a different cause to it. What's happening is just what I said: Viewing the City Display on a cycle-year is a trigger-event. So is the automatic update at the beginning of a cycle-year turn. The automatic update corrects the Dye/Copper distortion, but Viewing the City Display in that year reinstates it. It works the same way on 2.4.2 except without the Dye/Copper bug. [Actually 2.4.2 has its own Dye/Copper problem but it manifests differently.]

There's a bit of Heisenberg at work here, but it's not entirely unpredictable.

[solo]

Okay, I did not see your reply to my first post on this. What you say makes sense to me, but if dye-copper bug is not implemented by cycle turn looks, then their "correct" demand values should be lower, right?

In any event, I can alter or "correct" lists by choosing to delay looks until after cycle turns in MGE games, or use cycle turn looks if I want "bugged" lists.

From what you say, in 2.42, a cycle turn look always gives a "correct" and updated list. If so, this is a radical difference between game versions.