anybody interested in....

[Velociryx]

One of the things that makes SMAC such a great game is the fact that there’s simply no way you can do everything at once. Every decision made has at least one, and oftentimes several opportunity costs. While I have played the game a ton of times and pursued every strategy discussed on the boards (yes, much as I hate to admit it, even the Rover Rush), I have not yet sat down with another player and compared detailed notes about the inner functionings of an empire, and I think it could prove quite enlightening, especially in light of the recent “attack” thread.

What I propose then, is the creation of a “test map” for the purposes of an experiment. We generate or have someone build a test map, make it available for download, and then have interested people play a game out, taking careful notes about each turn’s activities. If we set such a test map up, it will be of vital importance to map out specifically what information we’re looking for. Here’s a list to get us started, but by all means, feel free to add more!
1) A basic description of each player’s style
2) Population and number of bases
3) Total net mineral counts, empire wide
4) Total support costs, empire wide
5) SE settings run and cash/lab/psych allocations
6) Per turn income
7) Per turn lab outputs
8) Tech Advances gained
9) The results of popped pods

In order to remove as much randomness from the test as possible, I think we should use no pod scattering and no random events, and use directed research. We can use any map size, but obviously the larger the map, the more pronounced the differences will be, which might be the way to go.

The goal here is not to see who can win the game faster or more spectacularly, or to see who can transcend first, but to illustrate various game approaches, comparing them with real game data. As results begin coming in, we’ll also get the opportunity to get an “under the hood view” of how different strategies work, and how similar or pronounced the differences really are. It’s one thing to say that “a Momentum player will have a slower rate of research.” It’s quite another to be able to look at results that have been turned in and say, “Well, in comparing these two games we can see that the Momentum player was generating 82 research points per turn, while the Hybrid guy was cranking out 117.”

I dunno….I think it’s a pretty cool idea. If anybody else is interested, reply here. If anyone knows of a decent map we can use for testing, lemme know, although I don’t see a problem with just using a randomly generated one.

As far as results go, I think we should keep track every turn, in order to capture the most detailed data possible, but some people might be a bit overwhelmed by that.

Thoughts?

-=Vel=-

[Avenoct]

Velociryx,

This sounds like a wonderful idea! I volunteer to get the map going. I'd try several tests of random maps untill one seemed most likely to 'play fair' to the players and AI's strengths. As far as turn tracking, are you volunteering to measure up each players turn? This seems like a vast undertaking! I could help, but I'm wary of marking every turn. That would mean opening each game and taking notes. How about every 5 or 10 turns? We could even have each player take notes and append them to the emailed game to indicate lucky wins or losses or overall feelings about how the game is going.

Just some thoughts,

Smack

[big_canuk]

Vel:

Your idea sounds very similar to the challenges that have been played. Perhaps the challenge format with very detailed reporting, would be the way to go. I only did one challenge, the Lal challenge, and tried at the beginning to give detail and reasons for much of what I did:

http://apolyton.net/forums/showthread.php?s=&threadid=7309&highlight=Lal+challenge

Googlie's site makes availiable some of the old challenge maps. Possibly one of these should be used.

http://googlie2.tripod.com/spartanchronicles/id12.html

I think the DT series also has some recording of game statistics.

Another good way to compare and learn, is to play team games. I have learned much from team games I have played with Googlie, and have already learned a bit with only 2 turns gone by in FW from Misotu.

I really like the idea of comparing notes, but don't know if I could put the time into it right now, with FW starting, and no games finishing too soon. I know from the Lal challenge, that even the minimal recording and *posting* is fairly time consuming.

[Googlie]

Vel - and Smack:

Following up on big_canuk's comments above, I could certainly recommend either the Roze map/game (smax) or the Morgan one (smac) from the CGN challenge series.

Both were at transcend level, and for both I tweaked the AI considerably (using the Editor to set policy and to have ab-initio pacts between the AI factions, and even starting them off with some forested tiles and a crawler or two - but not the tech)

The Lal challenge was at intermediate level, and the Gaian one at entry-level.

Presumably you are talking about SP compare games from the same start position/conditions/factions for all players?

Or were you suggesting SP compare games on the same map/AI factions, but get to choose the faction you want to complement your own playstyle?

G.

[Ken Hinds]

I understand the core of Vel's idea. It's very similar, as far as the recording goes, to what is done for some of the CIV II comparison games.

What I'm not clear on and it sounds like this may apply to others also, is are we testing a map and the way it effects different styles of play or are we testing a fixed set of starting conditions, i.e., everyone gets the same factions, and starting location for those factions.

I not sure how much it would be a valid comparison of play styles to just have the rules and map fixed in stone, but leave the number of factions, native life, and starting locations set to random conditions. If several people got and played through some very poor or very excellent starting situations then it might indeed have a value for trashy players like myself on how to deal with rude neigbors on your doorstep 10 or 15 turns into the game. Otherwise I'm not sure that such a setup would produce the desired results.

Ken

[Blake]

Doesn't sound like too bad an idea, reporting every turn is too much work (tm), prehaps every 10 turns for the first 100 years, then every 5 turns, with extra reports if something exceptional happens (say building a SP which has a noteworthy effect on your empire). Also saving regulary, and prehaps every participant submits a turn 50, 100, 150, 200, ... , victory save game, which could then be collected at some website...

Seeing it isn't a challenge I suggest Ironman OFF, this is good for people with crash problems, and is great if you forget to save at a year, because you can use an autosave instead. Also seeing it isn't a challenge I'm not sure if a challenge map is appropriate?

What faction would the player be?
My choice would be Aki, her faction can run pretty much anything (greater variety of viable SE thanks to +2 effic), and can pop-boom with some difficulty, has some momentum benefits, pretty much a well rounded faction which doesn't eliminate any particular strategy.

Anyway, altough I officially dont play smax anymore I would be willing to play a comparison game, in either SMAC or SMAX.

[Avenoct]

I agree that the important thing for a compare game is to limit the variables as much as possible. To this end I suggest that:

Player is a predetermined faction in a predetermined spot.

Difficulty is possibly NOT Transcend to encourage everyone to play and further distinguish effective strategies from one another rather between those that succeed and those that fail.

SMAC rather than SMAX as there are those of us who can't use SMAX

Standard random map that perhaps has been trolled by the game initiator to make it as 'wide open' as possible (ie not starting on a tiny island, close to one or two factions, not near the monsoon jungle...)

Not ironman. This would be for compare rather than challenge.

Further, I think the choice of faction is important as some players like certain styles more suited to differing factions. I wonder if a 'Vanilla' faction could be created for this? No bonuses or penalties whatsoever....+0 across the board. One could even use the 'Alpha Base, Beta Base' series to further help diagnose the game.

Now we just need to see who's going to keep track of this

-Smack

[Velociryx]

Making use of one of the Challenge Maps

Hiya Brother Googlie! And that is an excellent idea, ‘bout using the challenge maps for comparisons of various strategies!

If we begin with the maps already mentioned here, that gives us Lal, Morgan, and Roze. All three of these are well suited to a variety of play styles, and if someone would care to volunteer to set up similar test maps for the other factions, we can begin with these, and post results while the other maps are being set up.

I think tests of this nature have a number of benefits, including:
1) It truly forces each player to think about every action, knowing that the results will be posted for comparison
2) It is beneficial to players newer to the game, giving them benchmarks to shoot for in improving their own games.
3) A much clearer illustration of the tradeoffs between making “guns” and making “butter.”

Having said that, the most illustrative results of these tests will be brought forth if each player simply plays his/her game out like always, and records the results. Of course there are ways to boost your empire’s stats, but the real purpose here is to see how each player’s “native gaming style” does when placed side by side with others.

I have also been thinking further about exactly what kinds of data would be most useful for comparisons, and here’s an updated list of stuff we ought to be on the lookout for. Again, if anybody else has any ideas about exactly what to measure, please, add to this!

1) One sentence synopsis of the player’s game style
2) Population and number of bases
3) Total Gross Mineral Counts, Support Costs, and Net Mineral Counts, empire wide
4) SE settings used, and economic allocations (labs, cash, psych)
5) Income per turn
6) Lab outputs per turn
7) Tech advances gained
8) Results of pods popped
9) Total # and type of Armed Forces
10) Total # and type of Former
11) Total # and type of Probes

Comparative Measures of Productivity
1) Average Industrial Output per Pop. Point (Net Minerals/Population) (we could do this on an individual base, or an empire wide basis)
2) Average Energy (cash) Output per Pop. Point (Net Energy/Population) (as above)
3) Average Lab Output per Pop. Point (Net Labs/Population) (as above)

The overall effect of these measures will (hopefully) demonstrate the nuts and bolts of why various strategies work, and how effective they are when stacked side by side. We can assume that the Momentum strategy sees its gains in these various measures by “Investing” in the Empire’s Armed Forces and using those forces to force early submissions and acquire bases by force, rather than by simply building them. What will be MOST interesting is to take a peek at the different approaches in time, and to that end, I have to agree that a “per turn” record keeping would be waaaay too much work, and not be much more instructive than say, an “every ten turn” report on progress (with notes tossed in as important landmarks are reached—completion of an SP, conquest of an enemy base, or what have you)

Conquest notes: It would be very instructive to see notes on the exact numbers, composition, and mineral costs of attack forces used. This would include transports, escorts, and air cover where applicable. Anything used to attack, support the attack, or transport the attackers must be considered as a part of the cost of attacking. This will give us a real sense of just how cheap or expensive it is to make war at various points in the game.

Also, by having such a precise measure of the costs associated with attacking, we can see precisely what the “dividends” of said attack force are (well, not precisely, because these measures do not take into account the amount of harm that attacking does to an enemy, rather, they focus exclusively on measurable benefits to the PLAYER’s empire).

Other thoughts or ideas?
-=Vel=-

[Iskandar Reza]

hey vel, you could save that effort for civIII. why tire yourself now on smac/x?

[Velociryx]

Hmmm….that’s a good idea, but I think I’ll consider this a “dry run” for CivIII – I suspect that the “Culture” option in the new Civ game is going to open up all SORTS of interesting possibilities….not the least of which will include the acquisition of neighboring towns without ever firing a shot….

-=Vel=-

[Robby Boy]

I'd definitely be interested. Stick me down for it.

[Fitz]

Vel, I'm definately in favor, especially in a week or two when I have some time (in class 6 hours + working 8 right now for a week).

Questions/Comments:

1) Map or scenario? If just a map, then you would have to specify Faction, other factions (in SMACX) etc. I would actually prefer this, but it increases variables. Even if you use all the same settings, obviously the locations will be different. How about this: Start a game for each faction, preferably on the map/huge map of planet. Save the game. Mail them to Googlie, along with criteria for recording, for him to post on the website.

2) How about people record the info in notepad, and email them to you? That way, I don't have to make the extra effort to come here and post unless I feel like it, and you have text files to work with instead of cut&paste from here. Also, you can then specify a text format layout, to allow to be sucked into a database:

Start each turn with MY MissionYear. Next place the item description # followed by ). Then put the appropriate number for the item in question. Hit return and enter the next, etc etc. LEave a blank line between Mission years.

So if I change the 1) to pop and two to bases, my text file might read as follows:

Fitz.
Peaceful builder that tries to get all facilities in every base and attacks without coordinating his units.

MY 2101
1) Pop=1
2) Base=1
3) Minerals=3/3, Support=0
4) SE=Frontier/Simple/None/None, Alloc=50/0/50
5) NetIncome=0
6) Labs=1
7) Techs=CentauriEcology
8) Pods=None
9) Infantry=1, Rovers=0, Planes=0, Boats=0, Copters=0
10) Former=1, SeaFormer=0
11) Probe=0, SeaProbe=0
12) AvgMin=3
13) AvgEnergy=0
14) AvgLabs=1

Obviously, there needs to be some thought into how general/specific to make 7 through 11.

Edit: I'm willing to work with you on this in designing a database to read in and analyze the results. I imagine that your fairly savvy on making databases, but if you want to farm out some of the work, or discuss this at some length, send me an email at junioros@hotmail.com and I'll send you my work email. I have a fair amount of time I can kill there.

[Blake]

I'll pariticipate if it's going to happen in the next month, after that I'll be too busy doing assignments and whatsnot.

My play style is minimal-MM builder, severly limiting the number of crawlers I use (to about 2 or 3 per base), and building lots of boreholes. I also use understand all the ED stuff better than average so play with more of a mineral/energy hybrid than standard energy strategy (which means big bases working lots of high producing tiles). Almost completely non militaristic, I preffer to run circles diplomatically around the AI and use a combination of passive and semi-active defense (iow I turtle).

One definite is pods must be removed, example in case when I played the morgan SMAC challenge (the recent one) I popped an earthquake which raised a landbridge to miriam land, making the early game somewhat more challenging than was presumably intended. (instead of being seperated from miriam I shared a continent, courtesy of the pod). Altough unlikey such events really can change the whole game dramatically.

[theohall]

Just a thought for Vel

With a few of us involved in team matches, some of us are able to work very closely with our teammates when executing turns.

Would you be interested in hearing about our team strategies in a more private format (say email) vice the more public posting.

The reason for this is in my current team match, I believe my team is attempting to execute something not seen before.

We may lose, but the idea sounds neat and has some merit.

[Velociryx]

Hey guys, and my apologies for not getting back to this thread sooner. Got caught up in a couple other conversations and let this one slide way down the list!

To Fitz’s questions first:
1) Looks like we’ve got several “challenge maps” that are faction specific, and quite usable for testing purposes. We don’t have maps for all the factions as yet, but what we’ve got is a good start, and eventually, if we get enough people interested, those good with the editor might lend a hand where that is concerned.
2) Folks mailing me the results is a good idea. Lots of people, each posting in a slightly different format might make the data harder to use, and I don’t mind that at all! Good thinking! Regarding items 7-11, I think a vanilla reporting of raw numbers is fine, but again, it depends on what sorts of information we’re looking to glean from the experiment. If the interest is there, as far as seeing the precise compositions of military forces, then the vanilla numbers won’t cut it, and we’ll need to break it down more closely than that….as an aside, I try to do that in my games anyway. In the design workshop, I’ve got some classic names I tend to stick with: Standard Garrison 1.0, 2.0, etc. AAA Garrison 1.0, 2.0, The “Battle Buggy” series of attack rovers, and so forth. That way, I can tell at a glance what iteration I’m on with regards to upgrading units, and it helps keep the workshop cleaner.
3) A database. Excellent idea! I’ve not put one together in a while, but I think I still ‘member the nuts and bolts of it….enough to get something going anyway….would be most useful for analysis, too!

Totally agree with Blake re: no pods. I think pods at the landing site are okay, but even better would be no pods at all.

To Theohall….excellent idea ‘bout more private communication of results. My e-mail addy is: Webmaster@velociryx.every1.net, and if you write, give me a hint ‘bout your plan! Sounds sneeky, and I like it already!

-=Vel=-