Understanding Tot Sprite File Format

[Angelo Scotto]

It's a pity that Tot spr format is unknown, i usually play Civ without animations (since i removed spr files a lot of time ago to play scenarios), some days ago i reinstalled Tot and started a game: i was delighted looking at animations of tanks or bombers.
Unfortunately it seems that Draco eventually lost interest in Tot (but i still hope to see him surface here again) so it seems we have to do the whole work alone.
A couple of days ago i started peeking the SPR files and attached you'll find a translated (and polished) version of my notes, they're far from complete but i think they're a starting point.

Greetings,
Angelo

[techumseh]

Hooray for Angelo! If there's anything I ,as a computer illiterate, can do to support you, just let me know.

[ravagon]

Ditto for me.
Downloaded your notes but I'm already I'm afraid, so I'm not likely to be too much help.

[Mercator]

Great work... I think I've found a little more thanks to your info . I'm not too familiar with all the correct terms or common "techniques" in file formats, so I just rant a little and guess away:

STATIC.SPR

The header consists of a whole set of DWORDs (or long integers, a sequence of 4 bytes in any case), most of them are offset pointers or so. I'll list them
1. 0 (for whatever reason )
2. 12, pointer to the next "block" at offset 12
3. 1636, a pointer to the end of the "header"
4. 0, offset of the first image (starting after the header, add value (3) to get the "real" offset)
5. - 408. remaining image offsets 405 in total
409. + (3) = end of file

And you didn't miscount the number of occurrences of "4000 0000 4000 0000" it is indeed 406, but near offset 139111 (0x21F67) there are two closely following eachother, one of them is either superfluous (an empty image not entirely removed correctly?) or perhaps just a coincidental occurrence of the sequence as part of the image encoding. In any case, the second of these two (at offset 139127 or 0x21F77) is NOT represented in the offset listing in the header.

UNIT01.SPR

Again lots of DWORDs, but not all offsets this time:

1. 12, pointer to byte 12 or DWORD (4)
2. 1324, pointer to the start of the image offset list in the header
3. 1928, pointer to the end of the header
328 DWORDs I haven't investigated yet
151 DWORDs, being the offsets of the images (works the same as with Static.spr: the first offset is 0 and is relative to the end of the header, so add (3) to get the "real" offset)

Edit: the last offset in the list always points to the end of the file, so for the UNIT01.SPR that would mean there are 150 images.

[Angelo Scotto]

Mercator, this is a great news!
Yes, offsets! i tried to use them directly as pointers but i fell in useless places of the file, sure, i had to sum header offset, it's clear now.

Unfortunately, the format in which is coded the image is still unknown, i wasn't able to find a single match with any standard image format.

I peeked in the Tot exe file and found that the real name of sprite file format is SPRT... but this is not very useful after all

Always in the exe there were references to several graphic formats: PCX, IFF ILBM (Used mainly on Amiga!!!), GIF and obviously BMP... this tells us that probably Microprose used a pre-made graphic library but i don't think this is useful too...

Greetings,
Angelo

[Mercator]

I haven't had any luck with the images either. I copied one presumed image to a separate file, and tried to open it as every possible image file PSP recognized... But still no luck. Then again, it might well be that it requires some little extra fixed header information added in front... Maybe DracoOmega (the one who said he was working on an editor) can give us just a little more info.

[Zarion]

This is Draco Omega here (under a different name due to login problems).

I figure everyone must either hate me now or think that I made the whole thing up and never figured it out. The truth is that I started writing the program several times with the best of intentions, but it was taking a while and it ended up getting abandoned. I guess I'll never end up finishing it myself so I'll just give the format out and let you guys handle that end of things. Again I apologize....

I had almost all of the format figured out within the first 3 days of looking at it, but my notes were sketchy to begin with and they probably got thrown out when I moved to university last month. (Also my old computer got beat up my the postal service and is currently inaccesable to me, but if I can manage to get a look at my old source code, some more things will become clear)This will be pretty much off the top of my head and incomplete, although it should probably help a little. And I'll post any more details as I remember them.

Firstly, there is no sort of compression used on the images, nor are they a standard format that I am aware of. They are 16-bit images, with each pixel taking to bytes (obviously) but not quite in the standard fashion. The bits are in standard 5-5-5 R-G-B format, but the unused bit is used here to represent whether that pixel should be painted with the team color (adjusted naturally to the shade of the pixel)

The image is divided into a number of rows of pixels, of which each row has a 2 byte header. I'm not 100% sure if this order is correct, but I think the first byte is double how many pixels this row is offset from the far left of the image. The second is the length in bytes of the row.
They could be in the other order, though.

Each row does not be the same length, but starts at the first non-transparent pixel of that visiable row and ends with the last one. This basically saves the space that would be wasted with the large ammount of transparent space on the left and right of the sprite.

There is also a header to each image that is either 45 or 46 bytes in length (I can't remember which) Unfortunatly, I can't remember the order of the entried at all, but I can at least tell you what some of them are, so other may be able to find them.

There is a value stating the true top, bottom, left, and right bounds of the image (with a maximum of 64x64 as in units.bmp) Also, is the size of the main image data. Some other, less used values relating to the position of the health bar are there as well.

Darn... As I'm writing this, I realize how little use it is and how little I remember. Well, I'll trie to get the source code off my old computer as soon as possible and post a more detailed, accurate, and complete version of the format, but I'm afraid that this will have to do until then, such as it is...... Oh, and if you have any questions about my poorly written description, feel free to ask.

[Boco]

Each row does not be the same length, but starts at the first non-transparent pixel of that visiable row and ends with the last one. This basically saves the space that would be wasted with the large ammount of transparent space on the left and right of the sprite.

Thanks DO/Z! How does the format store the offset of the first non-transparent pixel for each row?

[Mercator]

[...] each row has a 2 byte header. [...] I think the first byte is double how many pixels this row is offset from the far left of the image. The second is the length in bytes of the row.
They could be in the other order, though.

Marvellous! I think that's more than enough information to get us started (although any more is highly appreciated). Thanks Draco/Zarion!

[techumseh]

Any progress? I'm just a cheerleader, I guess, but I'm sure hoping you guys can solve this mystery!

[Angelo Scotto]

It's great to hear you again Draco/Zarion !
Thank you very much, you've saved us from searching tonnes of image formats (unfortunately i've already looked hundreds of them ...)

Actually i still have to find some time to work on this, but i think our first goal should be to define a correct format of a single image in the .spr,
We already know where to look and, thanks to Draco, we also know what to look, so it shouldn't be very difficult.

After that... well, we will see ...

Greetings,
Angelo

[SMIFFGIG]

I take it no one is bothering with this then???

Why didnt Microprose publish information on it in hte first place to encourage editing & making new units?

So are there no animated custom units for this game i take it, made by fans

[techumseh]

I think people are still working on it, SMIFFGIG. It may take some time to decode the thing.

I don't really know why Microprose didn't include a sprite editor or at least provide some way to modify the sprites. I think they had Starcraft in mind, where existing graphics are used in scenarios to represent things that look entirely different. But Civ is a very different game. IMHO, had they done so, ToT would have been much more popular. Indeed, the graphics (both animated and not) are it's greatest downfall.

[SMIFFGIG]

think they had Starcraft in mind, where existing graphics are used in scenarios to represent things that look entirely different

Lol i know EXACTLY what you mean (no no thats not a world war 2 spitfire it is a fusion powered space cruiser)

had they done so, ToT would have been much more popular

Without a doubt, had the potential to what civ2 was in modability and fan created scenarios

I think people are still working on it, SMIFFGIG. It may take some time to decode the thing.

Well good luck to them, who ever they are. I look forward to the results

P.S. It would certainly open alot of doors for ToT

[Mercator]

* bump *

I found out some more (attached)... And more importantly, I've made a little utility. So far it can only export all images in the Static.spr files to 24-bit bitmaps though (the info from Zarion was more than enough to figure out how to "decode" the images )...
But at least I'm getting somewhere.

Zarion (or Angelo?)... Do you by any chance have more info on the header sections of the Resource and Units sprite files? That seems to be the only thing I haven't figured out yet, but it's a rather important bit I'd say.