PROJECT: Source Code: Project Team

[Matthew Hayden]

Just dropping a line to voice my enthusiasm for Activision releasing the source code, and hey, good luck with it! I can't program or create sprites to save my life, all I have done is play around with the text files, whose structure is actually decided by the pandora's box Apolytoners are about to recieve from Activision.

Just out of interest, does anyone have any ideas (Keygen I'm looking in your direction) about how long the various steps will need - getting around the code - fixing glaring bugs - making further adjustments - addition of new features. Any timeframe on that? And will each stage be released on ACS?





Does anyone know how the AI actually works in CTP2? I mean, is each civ an AIEntity or is it just a bad program running everything.

[Keygen]

Originally posted by Matthew Hayden

Just out of interest, does anyone have any ideas (Keygen I'm looking in your direction) about how long the various steps will need - getting around the code - fixing glaring bugs - making further adjustments - addition of new features. Any timeframe on that? And will each stage be released on ACS?

With no resources (source code, people that will work on it and their abilities) on my hands I can hardly do any prediction at all but I can try to guess some parameteres and qive a draft.

a) About the code release.
Mr Ogre mentioned that will have to strip off the comments from the code and send it back to Activision. Some things are not clear. For example if all comments will gone or the ones that have references to copywrited and trademarked stuff within the code related to other companies or possibly offencive to the public comments. If Mr Ogre has to eveluate one by one all the comments all over the code it would take months but I beleive a more sofisticated method would be used like searhing using keywords and wild cards to ensure comments that contain specific words (even mispelled) would be found and removed. If all commend are to be removed that would be even more easier. I am not sure what would take in the Activision's part, perhaps evaluate Mr Orge's work and further analyze the profits and law stuff.

I certainly don't see any code before September and the most fast prediction I can make is somewhere within September. But taking in mind the above it could easily be released somewhere between the last quarter of 2003 and the first quarter of 2004. Perhaps not at all...

b) About fixing bugs and making improvements.
Well, if I don't see the code first I can't say. If it's well organized and structured it will obviously take less time than a bad one. If it's completely stripped off of comments then we go in blind. If we have inside help from people that worked in the code and/or design we will certainly have faster results. And of cource the availability, quality and quantity of people that will work on the code will greatly affect a timeframe.

I would say from several months to few years.

c) About adding new features.
If they are going to be small ones I would include them in the above (b) section among bugs and improvements all part of a patch. If we are talking about major ones add 1-3 years and you might get some.

Originally posted by Matthew Hayden

Does anyone know how the AI actually works in CTP2? I mean, is each civ an AIEntity or is it just a bad program running everything.

Not sure but I think it works mostly with configuration files that include several parameters like civilization's personality, regard toward you depending on specific AI civ's personality and exchanges and actions between you and lot other things. Thus there is a limited AI entity for each civ but not complete.

Some of the moders around here could explain it much better.

[WesW]

From reading the response to my suggestion in the source code thread, and some of the other info that has surfaced, I would suggest some changes to the general way things seem to be taking shape.

I think it was Ray Park who mentioned that the code was only divided into a few major sections. One or two of those sections are probably not going to be a very big part of the whole, so you are going to have sections, probably the important ones, that have around a quarter million lines of code in them. Take a moment and try to visualize a file that big............ok, now hopefully you can see that I would have everyone work on the same general area of the game, rather than splitting up over the entire game.

Suggestions for the general approach to documenting, verifying and fixing:

You are going to have to figure out how the source code links to the game objects or text files, in addition to simply figuring out what part of the game the code is involved in.
You are most likely going to need frequent communication to figure out and/or confirm the code's purpose, and also to track down the bits of code that were added in at different times of the game's development. For Total War, they set deadlines as they went through the game's development, and once the initial development period was up, presumably the date to get a working beta done, any additional code had to be entered below that of the previous period for the entire file. This means that the code for a given part of the game is going to potentially be scattered throughout its section. I don't know if they set up mini-sections within the major ones, but if they followed the same procedure as the Total War team did for the text files, you are potentially going to have bits of code for a given function strung out over the entire quarter-million section.

For Total War, and most any other game, I imagine, the code was only documented and organized to the point that the person working on it could keep track of where things were. Once he got his section working properly, it would be combined with all the other work done during that period for the next beta, and so on until the game was released. This means that additions and fixes and work-arounds for a given function can be scattered all over the place. Remember that everyone was rushed throughout the game's development, and how Winnie spoke of living at the office for the last few months, and imagine how well things are going to be tidied up inside.

Also, for Total War, some of the comments made in the code are no longer true, or are misleading, because of changes made later on in the game's development. This means that everything you think you figure out is going to have to be verified. You are also probably going to need frequent conferences or PM availibility to discuss questions that arise about specific functions as you go along.

Tip: the first thing you should check is to see if you can cut-n-paste code from one place to another. If you can, you need to have a policy of grouping and organizing the code as you go along. You will be going back to the code again and again for various reasons, and the better it is grouped, the easier it will be to see how things are tied together. This is going to be absolutely crucial when you get into adjusting the AI, where so many things interact.
I would recommend you follow a sequence of finding, marking and grouping the various functions. Don't begin altering code until you have found all the code pertaining to a big enough part of the game so that it should be pretty much self-contained. This way, you can see specifically what your changes are doing.

When you get to the point where a self-contained section has been identified, go ahead and start making the necessary fixes to get it to working properly, or significantly approved anyway. Then you can release a first version of the new and improved game. This will help keep interest in the game, since you will be releasing things as you fix them, rather than waiting until the whole thing is done. It will also be good for you guys, as it will give you a sense of accomplishment after the tedious and frustrating work described above.

For most of my work, I have had to do a lot of stuff on my own, since I was the only guy interested or with enough free time to do it. I would not recommend this policy if you can keep from it. I would have everyone work on separate but related functions, perhaps in teams of two if you can have regular times where you can both work on it, so you can PM each other as necessary. This way you get knowledge accumulated at a much faster rate, since you will be better able to help each other when you get stuck, and the time spent being stuck is one of the biggest time consumers in modding.
The first time I got into serious modding of the Ctp 1 AI, a team developed of me and another guy working on the files, and two other play-testers. Tom and I would make changes and see if they worked or not, and then we would send out copies of the altered files to the other three. (Tom and I were working on different parts of the game.) We traded almost daily emails with either modded texts or play-testing results, and these were crucial to our successes, such as they were.
Things like this also form bonds between the participants, and it's these bonds that I remember most fondly from the modding experiences I have had over the last four years, and not necessarily the successes themselves.

Well, I didn't set out to write this much, but I hope you will take it all to heart from a guy who has been there more than once. My general message in this is to work together as much as possible. I would risk duplicating some efforts for the advantage of having more than one head working on things the size of which you are going to run into.
Put as many people as necessary to make steady progress on a given function, set priorities, communicate as often as possible, and shift trouble-shooters around as necessary to help with particularly confusing problems.
But the first law should be steady progress. If you put your heads together for a day or two and still can't figure something else- document it throughly, including the current theories on fixing it, and move on. When you get through the section in this manner, start at the top again and go through as many iterations as necessary until you get everything done that you can. Document the remaining problems for the entire section as you did for individual functions, and move to the next section.

IIRC, Ctp 2 didn't use the fuzzy logic of Ctp 1, but people like Peter Triggs can be invaluable in helping you figure out the mathematical theories the programmers used, and also perhaps with the overall structure of the coding. You will need someone heading the entire effort who is either a professional co-ordinator or has been the author of a large, multi-person mod.
If I was still into Ctp, I would volunteer, but right now I am up to my armpits in the Medmod IV. However, I will be happy to be an outside consultant for general questions and approaches, like the stuff in this post. There is also not much about Ctp 2 that I didn't work on pretty extensively, so I also might be able to help when changes don't seem to be having the effect on the game that you anticipated.

Well, I hope this can help you guys if and when you get the code, and I'm just down the street, so to speak, if you think I can occasionally be of any help.

[LDiCesare]

any additional code had to be entered below that of the previous period for the entire file.

In text fiels maybe, but in C++ code, that's both almost impossible and totally dumb. Versionning can be achieved through cvs or other means.
Plus a section in C++ is certainly not a 250,000 lines file. There are probably thousands of files, some of which may span several 10,000 lines.
WesW, what you say is true of all programming projects, and I expect programmers to already know that.
The part about cut-n-paste is one refactoring pattern. Anyone interested in coding should check out the refactoring books (I think there is a web site with lots of pointers inside somewhere - google it if interested).

[child of Thor]

hi WesW, hope you are well

Nice to see you keeping an eye on this as it developes, im sure your help will be requested at some point - probably when the sheer size of the project hits home and the code appears to be all over the place

This should be a really instructive/educational project for all who get involved enough

[Martin the Dane]

As some of you know, I have attempted to create a sprite-viewer for ctp, with little success. When the news of the possible release of the tcp2 source-code hit, I desided to give the sprite-file-format another go. This time I started by analysing a couple of simple .spr files, the result was that I finaly cracked the format. (I havn't nailed down all aspects yet, but enough to resume my work on a sprite-viewer/extractor).

For my earlier attempts I had tried to use the information released by Activission in the sprite tool documentation. This information contains commented code, and is the only code available to us, untill the full sourcecode is released.
What I have learnd from analyzing that code, is that it was far easier to crak the .spr file than it was to figure out what the code did. Now that I have the output-format of the code analyzing it is much easier.

What I want to say with this example is that IMO analyzing the code should start in sections where the in-put and out-put is known. This will let the analysts get a feel of the "dialect" and the way the code is structured.

I do have some experience in figuring out other peoples code, and since every programer has his/her own particular "dialect" digging into "new" code is allways a chalenge.

Take a look at the following to examples (they are in pascal):

function Blend(C1,C2,b:Byte): Byte;
begin
  Result := (b * C1 + (255 - b) * C2) div 255;
end;

and the same code:

function BlendColors(aBgColor,aColor,aBlendValue:Byte): Byte;
begin
  Result := (aBlendValue * aBgColor+ (255 - aBlendValue) * aColor) div 255;
end;

This is a very simple example, where none of the examples have any comments, but the last version of the code is much easier to understand. (And I have seen much worse examples that the first version).

Martin the Dane

[J Bytheway]

A sprite viewer would indeed be a wonderful thing. Good news, Martin .

[Keygen]

Wes, your experience would greatly help to undertake such a project so when and if the code is released any help you could provide would be most useful

Martin, first congratulations for you success
I hope more people get inspired like you did from the event of the source code release and do similar things

As for your code example I think in such a sizeable project, parameters, variables, constants etc. would be self-explained and easily understandable. Take a look at the slic scripts in CtP1. I am pretty sure that all names used in CtP2 source code will be clear and easy to remember.

[Dale]

Originally posted by Keygen
As for your code example I think in such a sizeable project, parameters, variables, constants etc. would be self-explained and easily understandable. Take a look at the slic scripts in CtP1. I am pretty sure that all names used in CtP2 source code will be clear and easy to remember.

Not necessarily. Maybe the global variables and classes might be self-explanatory, but I know a lot of programmers who have local variables not. EG: You work on a function for a month, then realise you need a temp int to hold a value from one of the calculations. What do you call it? TempValueHolder, TempNum, a, Num. Locals are usually left to whoever did the function where globals are usually decided before the codewriting starts.

[Keygen]

I don't doubt, there might be some such ones like a function Winnie mentioned (if wasn't joking at all) with a nasty name

But I have the fealing that the vast majority will be clear.
It would be hell for the programmers themselves to remember what each function does and what value each variable holds in such a huge code unless it has an easily recognizable name and/or a comment.

Besides TempValueHolder would be quite an explanatory name and variables serving as counters in loops also would probably have simple names but these type of variables are usually easy to recognize when bounded in local range within a function.

I only hope we get that code pretty soon because we all simply assume things here

[Martin the Dane]

I only hope we get that code pretty soon because we all simply assume things here

Exactly.

My point was not to discuss the coding strategy used in CTP2 (I have not seen enough code to discuss that), but to suggest that the analyzing starts in kode that produces known output from known input.

[WesW]

That's good advice, Martin. It kind of fits in with my advice of staying with similar functions, at least at the start.

I hope the optimists here are right about the structure and neatness of the code, but I would be somewhat surprised there.
You see, I remembered what Don said about the Ctp 1 data when he hacked into it. He said that it was no wonder there were so many bugs and broken features in the game. Said it was a wonder the game worked at all with the way everything was so disorganized inside, with bits of unused code lying around everywhere. He may have said some other things, too, but I remember his closing remark was that it looked like something put together by a bunch of kids who had never made a game before.
They may have cleaned things up during the making of the sequel, but I doubt it, since instead of fixing broken features, they actually broke some working ones when they expanded the code for the sequel. (Undersea tunnels are one example.)
So, I wish you guys all the luck, but I am afraid you are going to need it.

[Pyaray]

Originally posted by WesW
...
He may have said some other things, too, but I remember his closing remark was that it looked like something put together by a bunch of kids who had never made a game before.
...

It's really easy to criticize someone elses work when you have all the time in the world to review it. It's another entirely when you're trying to get something done by a specific ship date. When you have a month to rewrite one function is different than when you have a day to get it working.

When I worked in aerospace, the mission critical code got developed at a rate of about 1 line of code per hour. In the game industry, it's several hundred times that.

I've been on both ends of the spectrum, where getting it "perfect" was more important than getting it done. That's not how it works in the game industry. If it were, games would be bug free, but costs hundreds, or thousands, of dollars per copy, and take a decade to develop.

When I was in aerospace the largest program I ever worked on was 2 million lines of code. It took a team of 20 people 7 years to write it. And I assure you, it cost more than any game that has ever been written.

The people I have worked with in the gaming industry has been some of the best and most professional people anywhere. We do what we need to do to get the game done, in the best way we can, and hopefully still make a profit. We're capable of any level of quality that the finances for the game can support, but do you want to pay $500 per game? Like the military is willing to pay $500 per toilet seat or hammer?

And just so its clear, this post is not made in anger. It's a simple reality check, with a bit of food for thought thrown in.

Pyaray

[Peter Triggs]

Remark: Pyaray's already replied before I had a chance to post this.

In order to get a better idea of what to expect I've been browsing the sources of some of the open source games on the market. CTP2's code can't be any more confusing than some of the stuff I've seen. And if there's one thing I've learnt over the past couple of years where I've been trying to program with SLIC, it's that programming is not easy, so I'm in no hurry to criticize anybody's efforts (not even some of the CTP2 mods: and, btw, I have to admit that some of my stuff is really inelegant and downright sloppy ). But we'll just have to wait and see about how comprehensible CTP2's source will turn out to be.

In the meantime, here's a question for MrOgre or Pyaray, should they happen to peruse this thread. This is a quote from an interesting article on the Linux Game Development Center:

Prolog/LISP based if/then constructs are totally unfamiliar to a lot of programmers when used for more than just writing down a couple of goals. Procedural languages, such as C, are much more widespread. So it seems to make sense to cook up some sort of procedural language by yourself, write a parser and interpreter for it, and have this script control your game whatever way it wants, reading values, copying stuff around, and calling whatever functions necessary.

In other words, instead of writing the game logic in C, you write it in some language which you invent yourself[my emph] (typically a heavily simplified C or Pascal variant) and make sure it can be executed by your game by building the scripting environment for it.

The big advantage here is total power for the user. When implemented to the fullest, these scripting environments can give the end user as much power as if he had access to the original C source code. The entire game can be turned upside down : a ground unit can start flying and vice versa; a weapon may be changed, removed, or a new one may appear; and so on.. Any particular function you have written which may be of interest for anybody willing to modify the gameplay, may be given access to from within the script by fixing the interpreter code. The same goes for any variables, such as score, health, power, etc.

The downside here of course is that this is a lot of work.

The bit that really intrigues me is that part about the game logic being written in the scripting language rather than C. I've often wondered about how SLIC works: this quote seems to suggest that we can expect to find SLIC functions (as C++ functions ?) in the source. That would be very nice, but am I reading too much into it?

[Pyaray]

Originally posted by Peter Triggs The bit that really intrigues me is that part about the game logic being written in the scripting language rather than C. I've often wondered about how SLIC works: this quote seems to suggest that we can expect to find SLIC functions (as C++ functions ?) in the source. That would be very nice, but am I reading too much into it?

Yeah, that's pretty much how it works. There is a corelation between the SLIC function and its C++ counterpart to process it, I don't remember if that's a one to one relationship, or if there are multiple interpretations of each function (it has been almost 3 years since I looked at that code). But yes, every SLIC function is processed in the C++.

Pyaray