Model Organization

[F_Smith]

Just wanted to make a quick suggestion:

Should the models be reorganized along 'functional' lines?

In developing the game code, it has to work like this --

[*]Map modelling -- everything to be stored in a mapsquare object, map generation, ecology, that sort of thing.
[*]Resources modelling -- all 'resource' data/methods.
[*]Population modelling -- everything to be stored in a population ('ethnic group') object.
[*]Population interaction modelling -- governments, econ, society, combat.[/list=a]

Anything I'm forgetting?

I propose that in getting ready for the next demo/prototype release, we appoint 4 people to manage/gather the current models in each area together. This would really help speed coding of the next release.

Does that sound okay? Any volunteers?
[This message has been edited by F_Smith (edited September 14, 2000).]

[Mark_Everson]

F_Smith:

I'm not trying to be snide, but what exactly is this supposed to accomplish?

You have jammed most of the game into item 4... by the apparent decision that every time people of any sort interact in any way it should be #4. I'm sure it makes some sense in the way that you look at programming, but I don't see how this type of organization will achieve anything useful.

I personally think the existing 'functional' area organization works much better than what you propose. What's your reasoning for how your proposal would help?

[F_Smith]

Hi, Mark:

I'm sorry, I should have been more clear. I'm stretched pretty thin this week, and it's making me kinda loopy.

These are the steps required to build the models. We can't do resources until the map is done. We can't really do population until the map and resources are done. We can't do govt, etc, until the population is done.

In effect, I'm trying to convert these 'game' models that have been developed into the 'computer game' models necessary to build a game program.

To do so, it'll be necessary to organize them along these functional lines, according to the data each model will require. That would help the model designers as well as the coders, wouldn't it?

When building any 'computer game' model, absolutely the first thing to define is what specific data the model will need, agreed?

Then we'll need to carefully arange/architect the data model that will store that data. Any models that work with the same data should be developed jointly.

Because 'data integrity' is the biggest bugaboo we'll face when trying to merge these vast, complex systems into one program. The data has to have a common format, be available to all parts of the program that require it, and has to be consistent. That is not easy to do.

Failure to architect the data first will vastly complicate any attempt to code this game, in my experience, and quite possibly doom the project.

Making a computer game isn't like making a paper board game, of course. When making a computer game (or any program, for that matter), the focus must be on the data itself. In a board game, the 'models' don't have any actual knowledge of each other, so data integrity is a non-issue. The Combat Results Table just uses whatever numbers the player decides on.

And then the very *last* step in building a computer game program will be models in 'item 4', coding the actual methods fleshing out the interactions (a.k.a. "business logic", or in our case "game logic") that manipulate the data.

[roquijad]

I'm so ignorant about all these things.....


The data structure seems to be very important to how model methods will be implemented. My question is, if we decide later to change objects from their initial position in the data structure to another place in it (which I assume is easy), would it be difficult to adapt methods to the new structure? In other words, is the structure so decisive that implementing methods should wait for what we feel is the very best data structure?

I ask it because currently FE ethnic groups are coded as parts of a mapsquare object. Mapquares are way down the data structure, as I see it. In my original model I suggested EGs in the civ level (top in the structure), so I wonder how difficult it would be to move EGs objects up in the hierarchy concerning methods. It's not that I want to move EG to the position I gave them in my original plan... I just want to know how important and definitive decisions like this are right now.

[Mark_Everson]

Well, for my notion of what should go into the various map classes I of course nominate what is in the existing code. It doesn't have everything, but it has a Lot of stuff.

Is there a point to my posting it here?

[F_Smith]

Rodrigo:

Yes, it is vital to have the basic data structure at least. Or else you won't even be able to code the game at all. When you try to merge the complex systems, you will end up with a myriad data integrity problems.

Fortunately, I think we have the basic data structure now. It still needs tweaking, and adding to, but it's scalable enough to go up or down.

As far as storing ethnic groups -- it is absolutely impossible to store this info at the civ level in the program. It has to do with the nature of 'objects'. And how you model complex, dynamic systems.

Please believe me. It would so hopelessly complicate so many other things in other models, the game would not ever get coded and debugged. Because other models need mapsquare level population info. So it'd require lots of kludgy, hard to write, hard to maintain algorithms just to figure out what happens at the mapsquare level. That's terrible OO design. I'd be happy to explain more, in detail, but this isn't really the thread.

Just remember -- that has absolutely nothing to do with the game's rules. The player can deal with ethnic groups on the civ level if you prefer. Or the province level.

* * *

Mark:

Yes, perfect place to start.

Please do post the list here, if you have a chance.

[Richard Bruns]

This is in reply to F_Smith's post in the social model thread that discussed monolithic vs. component programming:

Yes, I am a procedural programmer. I designed the models along those lines. Even though I now know better, the old habits are vary hard to break. I find myself writing procedural Java code using static variables and a single main{} loop. (Maybe I can eventually challenge F_Smith's claim to writing "the best procedural Java code you've ever seen" )

Perhaps that is why I simply cannot comprehend the advantages of a "component" approach. How is it more efficient to get all the inputs and run the long and complex tech equations for every single square, and then "build up?" That seems like a horrible waste when we could just run the equations once and then apply the results to the individual squares.

The same philosophy seems to be used in the social model with EG's as well. This thread is as good a place as any to discuss all of this.

Basically, I need to understand why running a set of equations 3000 times is more efficient than running them once.

[Beör]

quote: Originally posted by Richard Bruns on 09-19-2000 11:29 PM This thread is as good a place as any to discuss all of this.

I agree. For the last week or so there has been avid discussion in the Social and Tech model threads on the nature of EGs, their aggregation into larger EGs, the storage of tech level in EGs, what happens if EGs migrate or get conquered etc. (Actually there has been so many posts that we would reach the 6000 'postmark' very soon!)

The reason for this is that a lot of the ideas for this game has started in level 4 of F_Smiths original post in this thread: The complex modeling of society, while the basic data structure has been more elusive.

The basic data structure may be in place, but it seems to me that not everybody is familiar with it ( I know I am not ). F_Smith has adopted component modeling and I couldn't agree more (from a laymans point of view). To what depth the components should be modeled is a result of what the smallest entity is that could influence game play. The entire game is really based on interactions between two distinct ressources: physical surroundings (the map with geographical ressources) and the people living there (population) guided by some rules (the models) and a context (the civ). It has been decided that the basic unit of territory is the square (there could be options for playing the game at the province-level like Risk, but the default game is square-based). Since ownership of territory is based on the square, population must also be modeled at the square level. If all people were alike, this would be the lowest level of modeling, but all people are not alike: It has been decided that people differ along ethnic boundaries so we have to track Ethnic Groups at the square level. Somewhere I saw a suggestion (probably LGJ - he's really into detail ) that people should be modelled occupying different parts of a square. If this was adopted we would have to model EGs at the square-location level!

So what I am saying here - and I'm sure F_Smith will agree - is that we have two basic storage containers for the game data (one for each basic ressource): The mapsquare and the EG (= the number of people of one particular ethnicity living in a mapsquare). These are the basic game objects from which the whole game is assembled. The basic task is to decide which attributes and behaviours should apply for these objects. I suspect that F_Smith has done much of this already.

But now it begins to become more complicated. At many points in the game it will be necessary to aggregate these different objects into larger objects:

For map squares this is intuitive: One or more mapsquares form a province based on the players preferences. One or more provinces form a civ, the size of which is based on the players military and political actions. One or more civs form the planet (actually there might be a catch here: Barbarians - will they be rogue civs or will they have their own class and AI? Rodrigo and I discussed it briefly in the social thread). So far so good.

For EGs it is apparently not so intuitive how to perform aggregation. For some time now I have been trying to draaaaaag out of Rodrigo and F_Smith how they intend to aggregate the square-based EGs into FE an EG at the province level.

F_Smith suggested that EGs should be aggregated if they share a common culture object. I was not happy with this: It works fine as long as the EGs remains in the same civ, but if a square changes hands, the culture object would have to be divided in two, because the culture of an EG can change every turn, in a direction decided by who owns the square. The EGs thus no longer share a common culture object and the twain shall remain forever apart (sounded rather archaic don't you think). If a lot of squares changed hands, shortly we would not be able to aggregate any EGs because every one of them would have their own unique culture object (these objects might have very similar or even identical attributes, but since they are not shared, but separate entities, it is not possible to aggregate by them). And maybe even more important: When EGs migrate to another civ they would also have to have their shared cultural object split from the original. 'So what' I hear you say, but concider this: If we start with say 100 shared cultural objects (=100 separate flavours of ethnicity, probably too low), in a while each may be divided on average into 5-10 cultural objects (a conquest here, a migration there). We now have 500-1000 cultural objects on which to aggregate EGs. With vivid migration in theory we could have at least 50 of these represented in an average map square. 50 times 10000 land map squares is 500k square level EGs. In my opninion these estimates are very conservative. In particular the 500-1000 shared cultural objects might be low by a factor 10 later in the game. These 500-5000 k EGs might not represent a serious computational problem, but it is highly unrealistic. Some of these effects could be mitigated by the recent allowing for a minimal EG size. Small EGs would be assimilated into larger EGs, and thus they would not have a pointer to a culture object, and if this was the last pointer the cultural object would be destroyed. Nevertheless, basically migration screws things up. It is no accident that this has not been modelled in any civ-like game so far. In fact most civlike games don't even model population in individual squares! I think that the basic problem with this approach is that a cultural object is not an entity in itself, but rather an attribute of an EG (just as tech=knowledge is an attribute of an EG - right F_Smith )

I am still waiting for Rodrigos response to my latest query in the social thread, but I suspect that he will answer along one of the following lines:

EGs are aggregated if they have identical cultural attributes. Every turn each EG is compared to every other EG in the province. If they have identical attrubutes they are aggregated when the need arises during this turn. This is easily done by comparing the cultural objects in a 2x2 matrix. But there are 9 cultural attributes each with 101 levels, so you theoretically you have 101^9= 1.093685272684e+18 possible values of the culture object. If a random culture object was chosen for the EG the chance of having two EGs aggregate is virtually zero. Granted, the culture of an EG is not chosen at random, it evolves gradually from a common origin, but players will tend to choose more radical cultural attributes than in real life, and I strongly suspect that in the dynamic environment of Clash very shortly there will be no two identical EGs to aggregate. From a scale-theoretical point of view the combination of 9 ordinal scales with 101 discrete classes is equivalent to a 9-dimensional continuous scale where identity is not an option.

EGs are aggregated if they have similar cultural attributes. This adresses the issue that, eventhough no two cultural objects will be alike, they will certainly be similar in some ways. Every turn all EGs in the province are grouped acording to their proximity on the 9-dimensional continuous scale. This can be done using hierarchical cluster analysis, but I do not know any details of how complicated this is. Probably this would get the job done, but one possible complication is that in a large province by pure chance an EG in the far eastern part might be similar to an EG in the far western part, they would aggregate, and wouldn't that look funky.

EGs are aggegated if they have the same entry in the attribute Nationality that Rodrigo included mainly for identification purposes (this is actually what I have been trying to lure him into saying all along ). The attribute Nationality is very different from all the other cultural attributes: It is measured on a nominal scale, meaning that it is really a classification system. The good thing is that there is a limited number of classes/cathegories (=the number of ethnic groups/nationalities included at game start + any we choose to create during gameplay) making it inevitable that a lot of EGs will be aggregated in each cathegory. Another good thing is that for a given EG it is constant (at least with the present definition), not subject to change as the EG changes status by migration or conquest. What Rodrigo didn't like was that I suggested replacing the txt constant Nationality with a pointer to an object with the sole attribute being a txt constant Nation (I might not have stated it this clearly at the time, primarily because I didn't have the clarity of mind I have now (regarding the game design that is )). The advantage of this would be that in the Nation class you could have methods allowing the splitting of a nation if EGs drifted apart geographically or culturally (this is not posible with the present definition of nationality). Another advantage is that we could include a dynamic array in the Nation class, pointing to every EG having this nationality, thus having easy access to all EGs pointing to the same Nation, without having to loop through all EGs to check their Nationality. The disadvantage would be that once in a while we would have to check if two nations were close enough and culturally similar enough to make it desirable to merge the two into a new nation. I never intended to have the Nation class have any independent behaviour (except from the splitting) - I mentioned (wish I hadn't - it confused the issue) the option of having meta-cultural attributes on the national level, but this is not a good idea.

This has become rather lengthy and I have so much more to say in the tech/EG area, but I will forego that for now.

Entering sarcastic mode
One last thing. There has been some concern as to whether the modeling of the gameworld in these small components might strain computer resources too much. Not to worry. At the pace this is proceding, we'll all have 10 GHz processors, 1 G Ram, 1T harddisks and Jave ver 5.0 - in our wristwhatches!
Exiting sarcastic mode

For cheater s here's a resume:
There are two basic storage containers for the game data: The Square and the EG
The game is a function of these two basic objects at various levels of aggregation, ruled by the models, and seen in the framework of the played civ.
The rules controlling aggregation of squares are pretty much set.
The rules controlling aggrgation of EGs is in my opinion not set yet.
Migration and conquest screw things up, and the effects of these two should be considered in detail in every modeling effort.

[F_Smith]

Richard:

I do understand your frustration.

First, your assumption that you can avoid calculations by using a 'monolithic' approach is not typically correct. You end up doing the same calculations, just in different places at different times. And end up doing calculations that the component approach will *not* need (when things happen at the mapsquare level, for instance). And with a 'component' approach, I can much more easily control when and where calculations get done, because things are already broken down into smaller 'components'.

Second, and more importantly, you gain efficiency, reusability, flexibility and power with a 'component' approach. Imagine casting a rifle barrel by hand, as opposed to machine-tooling one that is designed as a 'component'. The 'component' can be used in a variety of guns, easily replaced, easily upgraded, etc.

So a 'component' approach allows us to replace each 'component'. When we learn/think of a new, better way to do something, then we can easily plug that in. A 'component' approach allows us to easily offer 'game options' for each component. It allows us to easily reuse components, to avoid code duplication (big, big plus!). And since each component has specific jobs, we can perfect and fine tune each component until it's ideal -- hence we will see performance gains over a 'monolithic' design.

Third, (and this is the *really* big reason for a 'component' approach), the biggest savings is in programmer's time. This is the real key here . . . building a bunch of components, then assembling them like a bike, can easily be done by a team. Each only has to know about the component they're working on. You don't need to know anything about the trigger of the gun to design a screw for the barrel. With a 'monolithic' approach, since there is *only* one thing, you have to understand that thing.

It requires a massive amount of 'gear up' time for new programmers. That's why I made the comment to Mark about programmers not being able to make contributions because of the 'methodology'. The very fact that the person that was going to code ya'lls tech model was going to start from scratch should have been a *big* flag that the coding methodology is waaaaaay out of whack. A 'component' approach will fix that.

Whew -- I can talk about this all day. Bear with me if I'm not explaining it correctly, I'm doing my best.

If you can get a copy of the book 'Extreme Programming Explained' you'll see this very discussion, and an explanation of good OO design practices, put in a much better way than I can explain.

One great example -- an 'Operating System', like Windows. A 'monolithic' design would have one big program with a 'spreadsheet', 'word processor', etc, capabilities built in.

A 'component' approach would have a seperate 'OS kernal' component, 'spreasheet' component, 'word processor' component, etc. Like Windows.

[F_Smith]

Boer:

An 'Ethnic Group' minimum size is 1000. I should have answered you before on this, but I kept getting sidetracked.

Yes, there will be thousands of 'Ethnic Groups' in a full, planet-sized game. We've been assuming something on the order of 10,000 populated squares as a maximum, with something like 3 to 5 ethnic groups to a square. Altho many will share cultures, this still may mean as many as 30,000 different cultures.

I personally don't think this is 'unrealistic', to be honest -- we've got at least a half-dozen cultures here in my part of Texas (Mexican-American, African-American, 'Country/Redneck', German-American, Jewish-American, 'Typical Ugly American' (me), 'Native American', etc).

The cause, as you pointed out, is cultural diffusion. 'New' cultures arise when an 'ethnic group' grows away from the 'base' culture due to migration, conflict, etc. 'Mexican-Americans' have a very different culture from 'Mexicans', altho there are similarities. That sort of thing.

In game terms, I think the different cultures simply become varied inputs into equations. So the more the merrier. That way, the game won't get boring after the 200th time you've played it -- hopefully you won't have 'seen it all'. And then, for scenarios (which hopefully will be a big part of the game), there's enough depth to really make it interesting.

[manurein]

Hi all

I designed a social model for this game some times ago.
I think that this model answers a few questions Böer is raising.
You can see it here : http://apolyton.net/forums/Forum21/HTML/000151.html

See the mixing and the migration sub-models.
This model was based on the assumption that cultures tend to converge when they are in neighbouring places for some time.

A few remarks : first, I have left the Clash team for lack of time, but i still try to follow it, and i really want to play it : keep the pressure guys!

Concerning the procedural vs OO design, it's obvious that OO is far superior in such a project. This said, i think that you make a mistake when you try to design directly OO models; IMHO you are missing a step : OO design concerns the implementation of the model; what you are doing now is deciding what you models will do ; this is the specification phase, and OO has nothing to do with specification. First decide what your models must do, give the equations they will implement, etc. When your model is ready, then implement it, which means first designing it then coding and testing it. Clash is a very complex project, far more complex than most commercial games. If you mix the specification and the design phase, you overcomplicate the job needlessly.

Also, you should try to design each model as a "black box" : the other models dont need to know "how" they do, they need to know "what" they do.
What I mean is, all a model needs to know about the other models is the datas that the other models can give him for integrating in their computations. They dont need to know how these datas are produced.
If a model does what it is intended to do, then it can compute ant value concerning what it does, even if it does not use it internally.
Then, each model designer should design his model knowing what the other models are intended to do. Then ask the other model designers to supply him with the datas he needs for his computations. And supply the others with the datas they ask.

Thats all guys, good luck!

[Richard Bruns]

F_Smith:

The problem is that modeling tech at the square level does nothing to make the system simpler. The equations you would run at the square level are exactly the same as the ones you would run at the civ level. RP's will be gathered and distributed, and then tech change will be calculated. All you are doing is taking that "monolithic" system and running 10,000 copies of it!

[Beör]

F_Smith

I sort of agree

If the lower limit of EG size is 1000 people, the havock caused by migration will not be as large as I fortold. You could probably keep it in the 20.000-30.000 range, which I find high but acceptable. I read the other day that there is between 5000 and 6000 languages being spoken in the world, and people speaking these languages will be subdivided into many groups. But mind you many of these languages (I believe it was a little more than half) are on the edge of extinction, being spoken by less than 10000 people. We will have to test this. I say that if more than these 30.000 cultures are created we have to consider an alternative.
Actually there is another thing about several of these aggregation techniques (common object, identical attributes, similar attributes) that I don't like and that I think points to a problem with the method:

YOU WILL NOT KNOW WHAT TO CALL THE AGGREGATE.

If you use the nationality approach you will call the aggegate 'Germans' or 'Germans in XX province' or 'Germans in YY civ'.

With the other methods it will be:
'All persons that were originally Ggerman, but are now something else characterised by their cultural attributes a,b,c,d,e,f,g,h,i'
or
'All persons with cultural attributes a,b,c,d,e,f,g,h,i'
or
'All persons with cultural attributes close to a,b,c,d,e,f,g,h,i'

Grouping by nationality will preserve the flavor, while still allowing EGs to have differing cultural attributes. The aggregates will be larger and more heterogenous, but isn't that closer to reality?

[F_Smith]

Richard:

I think we're discussing that in the other thread, so I won't repeat here.

* * *

Boer:

Actually, the player can easily filter/aggregate by each and every EG attribute.

Since the player won't have god-like control, the only reason the player will need to know what civ-level aggregates are would be for whatever game models require. So the aggregate problem is a matter of the 'civ' level modelling, not 'square' level 'ethnic groups'. Even if we store 'ethnic group' info at the civ level, we'll have the same 'aggregate' issue.

I will have the filter available for a player to see how many people of a single ethnicity, how many people of a single culture, etc, are in their civ.

* * *

Manurein:

Hi. Welcome back.

[This message has been edited by F_Smith (edited September 21, 2000).]

[Beör]

manurein

Read your old model - it was not manureout (OK you probably heard that one before!).

There were several things I liked (although I am glad we got rid if the race thing ). And you are right: Many of your thoughts on grouping people in 'agents' and 'meta-agents' are valuable in the present discussion.

I would like to emphasize one thing from your model that I think is worth discussing (and I am sorry: this will relate to scale theory): The use of nominal scale (cathegories) instead of ordinal scales with so many steps that they are in fact continuous has several advantages:

1. You now what to call your groups: the Green Race, the Blue Race.

2. By using nominal scales you don't imply that one category is superior to another: the Blues are not better than the Greens, they are just different.

3. You don't imply that they are placed on a continuum. That somewhere between the Blue and the Green there is a Blue-Green. And between the Blue-Green and the Blue there is a Blue-Green-Blue.

The weakness of nominal scales are just as obvious:

1. You have to find a meaningful name for all categories.

2. You cannot perform arithmetic on nominal scales. Thus there is no average or median. The mixing of people gets diffcult, because you cannot estimate the 'typical' value of the merged people, other than choosing the value of the largest number of people.

So when we want to aggregate people it is necessary to use at least an ordinal scale, where arithmetic operations are possible. But this leads to the problems of no names, lacking identity and establishing proximity (particularly when you have to do this in several dimensions).

Fortunately F_Smith has opened my eyes:

F_Smith

Of course you are right EGs can be aggregated by any of the attributes, and the player will rarely if ever have to do that. I was going of on a tangent because of the discussion I had with Rodrigo in the social thread.

I guess the filter you mention will translate values on the 0-100 scale into another scale with much broader intervals (FE 0-20, 20-40. 40-60, 60-80, 80-100 or maybe even wider) which we could then give a name and use this name to describe the people to the player: '15% of your people are very agressive' or 'On average your population is moderately nationalistic'. Beneath this we would still have the actual numbers thus allowing aggregation of EGs either on the detailed scale with the disadvantages I have outlined or on the less detailed scale.

The other kind of aggregation is the one performed by actions/events in the game, and here we can decide ourselves which method we find most appropriate in the circumstances

Ingeneous - why didn't I think of that .

Whisper mode

F_smith: That's Beör - not Boer. B E Ö R

Funny how you can identify with a silly alias acquired about a month ago.
Whisper mode off
[This message has been edited by Beör (edited September 21, 2000).]

[Beör]

BTW does anybody know how to perform hierarchical cluster analysis. Is it feasible to perform it in this context of aggregating EGs - or other objects with several attributes measured on continuous scales?

[F_Smith]

I am so sorry for mis-spelling your signature.

Beor -- forgive me.

[Richard Bruns]

Shouldn't all of this be in the social model thread? Important social discussion is being put in this thread where it will probably be lost. Meanwhile, I am losing track of the model organization discussion. Mark, could you move the recent posts to the proper thread, and then delete this one?

[Mark_Everson]

Can someone just copy all this stuff to the appropriate thread(s)? When its done let me know and I'll delete the excess here.

[Richard Bruns]

I thought moderators could shuffle posts around. This stuff would go in the Social Model v.2 thread.

[Mark_Everson]

I would do it Exactly the same way anyone else can. Edit each post and copy that to a new post in the relevant thread. The only think I can do that you can't in this instance is delete the posts once they're moved.

[Richard Bruns]

Okay, I'll copy them all tomorrow unless someone objects to that in the meantime.

[Richard Bruns]

Manurein's thread and all of the discussion after it have been copied to the social model thread.

[Mark_Everson]

Thanks Richard!

Unless someone objects, I will, 24 hours from now, delete all the posts that were transferred to the social thread.

If you object, speak up now...

[Beör]

OK by me