COMPILE: Linux Port

[cap601]

Originally posted by lynx_abraxas


Yes. Use gdb for determining the point where the segmentation fault appears and then step back the function stack to find the call of the function that has a wrong pointer.
Did any graphic turn up till the segmentation fault? A lot segmentation faults I solved came from a very bad usage (reinterpret_cast) of void pointers that are used for storing integer index values of lists. There are other segmentation faults that originate from pointer mess in the battle view. (I didn't find out the real problem concerning proper drawing of the sprites but I managed to eliminate a segmentation fault)

Good luck searching

I'm not sure if this is the correct syntax but my first attempt went as follows:

josh@josh:~/CTP2/ctp2_program/ctp$ gdb
GNU gdb 6.6-debian
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
(gdb) run ctp2
Starting program:  ctp2
No executable file specified.
Use the "file" or "exec-file" command.
(gdb) quit
josh@josh:~/CTP2/ctp2_program/ctp$ gdb ctp2
GNU gdb 6.6-debian
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu"...
Using host libthread_db library "/lib/libthread_db.so.1".
(gdb) run
Starting program: /home/josh/CTP2/ctp2_program/ctp/ctp2
[Thread debugging using libthread_db enabled]
[New Thread 47331906192720 (LWP 6369)]
[New Thread 1082132816 (LWP 6372)]
[New Thread 1090525520 (LWP 6374)]
No medium found
All resolutions available.
ui/aui_ctp2/c3blitter.cpp L719: MMX-Test succeded!
[New Thread 1098918224 (LWP 6375)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1098918224 (LWP 6375)]
0x000000000050f434 in C3Blitter::Blt16To16FastMMX (this=0x18e4560, destSurf=0x19b4750, destRect=0x41801f70, srcSurf=0x18d7880, srcRect=0x41801f80, flags=17)
    at ui/aui_ctp2/c3blitter.cpp:402
402                             );
(gdb)

That leaves me with a completely black window titled "Ctp2" open. Guessing at commands:

(gdb) continue
Continuing.
[Thread 1090525520 (LWP 6374) exited]

Program received signal SIGSEGV, Segmentation fault.
0x000000000050f434 in C3Blitter::Blt16To16FastMMX (this=0x18e4560, destSurf=0x19b4750, destRect=0x41801f70, srcSurf=0x18d7880, srcRect=0x41801f80, flags=17)
    at ui/aui_ctp2/c3blitter.cpp:402
402                             );
(gdb) continue
Continuing.

Program terminated with signal SIGSEGV, Segmentation fault.
The program no longer exists.
(gdb)

And that removes the window and the program ends.

Looking at line 402 of c3blitter.cpp, it appears to be the end of a section of assembly code but I have no idea what to do with it.

[lynx_abraxas]

Originally posted by cap601


That leaves me with a completely black window titled "Ctp2" open. Guessing at commands:

(gdb) continue
Continuing.
[Thread 1090525520 (LWP 6374) exited]

Program received signal SIGSEGV, Segmentation fault.
0x000000000050f434 in C3Blitter::Blt16To16FastMMX (this=0x18e4560, destSurf=0x19b4750, destRect=0x41801f70, srcSurf=0x18d7880, srcRect=0x41801f80, flags=17)
    at ui/aui_ctp2/c3blitter.cpp:402
402                             );
(gdb) continue
Continuing.

Program terminated with signal SIGSEGV, Segmentation fault.
The program no longer exists.
(gdb)

And that removes the window and the program ends.

Looking at line 402 of c3blitter.cpp, it appears to be the end of a section of assembly code but I have no idea what to do with it.

I'm sorry, I can't get deeper in to this at the moment. (My ctp2 pc isn't connected to the internet at the moment) I can remember mplayer has a very short introduction into gdb. The Blt16To16FastMMX is kind of a painting function. I fear it doesn't work for you although MMX-test succeded because of your 64-bit machine. Try changing the MMX-test to fail for your pc or make it use the c-coded blit-function or best try changing the at&t asm code to work with 64-bit I haven't got any experience with that though.
It doesn't look like the funktion, in which the segmentation fault appears in to have bad pointer in its parameters so I'd say the segmentation fault is produced by some code in that function itself.

As far as I know You'll need a special assembly debugger for that inline asm-functions not gdb. Tell me, if You found a nice one

[lynx_abraxas]

Originally posted by loig
Hi !
thank's cap601 make bootstrap did the trick.

cap601 would You mind updating the install instructions for linux concerning this? Include Your notes and tricks for 64-bit, if You don't mind. My system with ctp2 isn't connected to the internet at the moment.

lynx_abraxas, you spoke about a bad feeling about perf and SDL.
do you have explored more the different possibilities namely SDL-gl, plain GL or an other game framework on top of GL ?
what about the quantity of work needed for this kind of change ?

on what do you all think it is better to work first, so I can (try to) help ?

I ran out of time/motivation before I checked SDL-gl. It would be easey though. In theory, one would only need to give the initialisatoin function of SDL the SDL-gl parameter. I haven't looked into GL. I fear the porting from SDL to GL would be quiet some work and one should really concider to start from scratch with the newest win-version of ctp2 (trunk) if planning to start a port to GL. This would solve a lot of graphic bugs and so and would make the linux port much more atractive!

So if You have the will, I can just incurage You to get started

[cap601]

I've managed to get past my problem by changing all register names to have an r as the first character instead of an e. I also had to remove he final l on all the instructions that had it (e.g. movl %%eax,%%ecx -> mov %%rax,%%rcx). Not sure if it'll have any unintended consequences...

This allows running to continue to where the intro movie should appear (it doesn't). Clicking then loads the main screen but with many of the images missing (the console complains about not finding a large number of .rim files). I can then try to start a game, at which point it crashed out with the error:

Messagebox(Map Generator Error): Error while trying to load library 'dll/map/geometric.la': 'file not found'.  Using builtin map generator

Continue?
Mouse PeepEvents failed:
Mouse PeepEvents failed:
Mouse PeepEvents failed:
Mouse PeepEvents failed:
Mouse PeepEvents failed:
Mouse PeepEvents failed:
Mouse PeepEvents failed:
Mouse PeepEvents failed:
Mouse PeepEvents failed:
Mouse PeepEvents failed:
Mouse PeepEvents failed:
Mouse PeepEvents failed:
Mouse PeepEvents failed:
Mouse PeepEvents failed:
Segmentation fault (core dumped)

Copying the files stored in [CTP dir]/libs to the aforementioned directory (which included geometric.la) gives the error:

terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted (core dumped)

Running it through gdb doesn't appear to give much information:

terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc

Program received signal SIGABRT, Aborted.
[Switching to Thread 47420878234960 (LWP 24129)]
0x00002b2106788765 in raise () from /lib/libc.so.6
(gdb) continue
Continuing.

Program terminated with signal SIGABRT, Aborted.
The program no longer exists.
(gdb)

Attached is a full console log from running it normally if anyone's interested. If someone could tell me where to find the .rim files it would be appreciated.

@lynx_abraxas - I'll have a look at working out how to upload a changed file.

[lynx_abraxas]

Ah well, I have forgotten so much about this...
I think the rim files are on the original CD. You either have to specify a mounted windows dir where ctp2 is installed or mention the dir on the CD in one of the text config files in the dir where the ctp2 executable is executed. Somthing *ctp2*path* or so. I might have mentioned that some where in the thread I might have time to check that at home and tell you next week.

start ctp2 with nointromovie. The linux version by the way has quite a few parameters like fullscreen. Check the file containing main(). I can't remeber the command I used for extracting them...

Concerning your asm changes: They make sense. e in front of a register name means extended and instructs to use a 32bit register. On a 64bit cpu it makes sense they introduced a new naming sheme for the 64bit registers.
The some how dreadfull but also powerful at&t asm-syntax then still demands that you specify the amount of copied bytes with a postfix (l for 32bit) I'm not sure if no prefix means 64bits. You should check that. gcc has info on that for its gas (which does the inline assambly).

Concerning the termination: It's not a segmentation fault. Try stepping up the function stack in gdb instead of continuing. help in gdb is your friend Find the function in ctp2 that has a call to a function in libc. Then check the source code to find out what is ment to be done and repair it. I know it sounds easier than it is.

When I have my home connected to the internet again I might be of more help.

[lynx_abraxas]

Here is a link concerning amd64 asm compaired to 32bit:

http://www.tortall.net/projects/yasm/wiki/AMD64

[cap601]

I believe the .rim files are located within a series of .zfs files ([CTP dir]/ctp2_data/default/graphics/pictures/pic565.zfs and pic555.zfs). Unfortunately I'm not sure how to identify whether the problem is being unable to read the files or being unable to find them.

My crash is apparently due to :

terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc

Program received signal SIGABRT, Aborted.
[Switching to Thread 47732217451856 (LWP 11845)]
0x00002b6983bbf765 in raise () from /lib/libc.so.6
(gdb) up
#1  0x00002b6983bc11c0 in abort () from /lib/libc.so.6
(gdb) up
#2  0x00002b69834b97b4 in __gnu_cxx::__verbose_terminate_handler ()
   from /usr/lib/libstdc++.so.6
(gdb) up
#3  0x00002b69834b7746 in ?? () from /usr/lib/libstdc++.so.6
(gdb) up
#4  0x00002b69834b7773 in std::terminate () from /usr/lib/libstdc++.so.6
(gdb) up
#5  0x00002b69834b785a in __cxa_throw () from /usr/lib/libstdc++.so.6
(gdb) up
#6  0x00002b69834b7d19 in operator new () from /usr/lib/libstdc++.so.6
(gdb) up
#7  0x00002b69834b7dd9 in operator new[] () from /usr/lib/libstdc++.so.6
(gdb) up
#8  0x000000000079b3ac in SpriteFile::ReadFacedSpriteDataBasic (this=0x2551760,
    s=0x2553ae0) at spritefile.cpp:629
629                     Pixel16 *   CompressedData = (Pixel16 *)new uint8[size];
(gdb)

Unfortunately I can't see what's wrong (if anything) with that piece of code.

With regards to the assembly code I think I've correctly changed it. Unfortunately I think the only way of confirming that either way is if something breaks.

I'll keep looking to see if there is something I can do.

EDIT - I've also uploaded a revised readme to include the missing steps.

[lynx_abraxas]

I have updated a readme I still had, too. It's attached so that you can compare it with your updates.
I also did a new install at home and noticed that the rim-files are in tga-files. Those are mainly in the civlang.ctp and civmain.ctp zip-archives. Those have to be readable and in my opinion on a case-insensitive fs. There are instructions for that in the readme. I't should work without a mounted CD but it also wouldn't hurt.
As it happend before I couldn't resist fixing something, so I made movies play with mplayer. Those would need a mounted CD but I won't be able to do the svn update until I get my home connected (hopefully next month).

Concerning your error I would have to check that at home. You can try strace to view the system calls.

[cap601]

The main difference between the two readmes is the addition of the vfat partion information. Trying it doesn't help find the missing .rim's unfortunately. On that note it's also worth mentioning that the problematic .rim's are not present as .tga's in the zip files. In fact the only references that I've found are either in .ldl's (i.e. when they're used) or in the .zfs's above.

Attached is a log from running strace. Hopefully you can make more sense of it than I can. For what it's worth it also shows that the game is looking for the .zfs files but I can't see what happens next.

EDIT: I've solved the .rim issue. I ran ReadZFSFile using wine and extracted the relevant files as .tga's. This allows the game to read the files (baring case-sensitivity issues). Unfortunately I still can't start a game for the same reasons as above and we still don't know what the problems with the .zfs's was in the first place.

[lynx_abraxas]

Hm, I fear I don't fully get what you mean with the rim tga and zfs files. If I get missing rim file errors I can see in the strace that ctp tries to open tga files and then yields these messages. I got the tga files from the civmain.ctp and civlang.ctp files which I inflated with zip (not gzip). I'm not sure if the base name of the rim files is anyhow connected to the base name of the tga...
Concerning the bad_alloc error: I compiled a new SDL library for the openglblit option and by incident noticed in the change log they mention a bad_alloc bug being solved. Pherhaps you should try the newest version of SDL, too.

[cap601]

My point is that, although many of the .rim files are present in the .ctp files (as .tga's), not all of them are. The rest of them are stored in .zfs archive files; these include the problematic ones. This can be avoided by manually extracting the contents of the .zfs files, although it would be preferable to find out why the game isn't reading them correctly.

I'll have a look into updating my SDL library. Which version has the potential fix in it?

[lynx_abraxas]

I had a look at your strace. I'd say it shows that your ctp2 finds the .tga and .zfs files but after opening them it is not happy with what it finds in them and goes on searching for the rim files untill it gives up and tells you. There fore I'd expect there is a bug concerning 64bit architecture in the reading or interpreting routines in the code. I think I left a comment where I found the artefact (555-color) bug. That should be close to the routines. Perhaps its assembly code again. Did you search the whole code for gcc inline assembly tokens? ( eg: find -iregex ".*\.[hc]p*" -exec grep -Hn "__asm" {} \; )

I used this one: http://www.libsdl.org/release/SDL-1.2.13.tar.gz

I also noticed that using SDL with fbcon graphics the game performs much, much better. Even in 1400x1050 it doesn't seem any slower. To get it, start the game in a framebuffered console with: SDL_VIDEODRIVER=fbcon ./ctp2 (ftp://ptah.lnf.kth.se/pub/misc/sdl-env-vars)
It will automatically be in fullscreen mode and you should make sure (when I have committed my changes) that your mplayer can display under the console. By the way: X doesn't have to be running for that

[lynx_abraxas]

Forgot to tell you my first thoughts:

Put a line like this into the code and check if size gets below 1 or really high:


size = ssizes[j][0];
printf("%s L%d: size= %d; sizeof(uint8)= %d; sizeof(Pixel16)= %d!\n", __FILE__, __LINE__, size, sizeof(uint8), sizeof(Pixel16));
Pixel16 * CompressedData = (Pixel16 *)new uint8[size];

I get outputs like these after starting a new game or loading one (not before):

spritefile.cpp L628: size= 5408; sizeof(uint8)= 1; sizeof(Pixel16)= 2!
spritefile.cpp L628: size= 4832; sizeof(uint8)= 1; sizeof(Pixel16)= 2!
spritefile.cpp L628: size= 3920; sizeof(uint8)= 1; sizeof(Pixel16)= 2!

If that's not the case I'd say there might be a problem with 64bit and the supposingly architecture independent typedefs. eg: uint8, Pixel16, sint32 ...
You can find some (not necessary all with:
find -regex ".*\.[hc]p*" -exec grep -Hn "typedef.*sint32[:space:]*;" {} \;
find -regex ".*\.[hc]p*" -exec grep -Hn "typedef.*Pixel16[:space:]*;" {} \;
I see richardh has already change some to a more secure definition (ctp2_code/os/include/ctp2_inttypes.h)
My avr installation gave me a man page about these typedefs (avr_stdint.3.gz).

[cap601]

On the asm: the search finds:

./ctp2_code/gfx/tilesys/tiledraw.cpp (Claims asm has been translated into C code.  None of the remaining asm should be compiled on linux)
./ctp2_code/ctp/debugtools/breakpoint.h (shouldn't be relevent)
./ctp2_code/ctp/debugtools/debugcallstack.cpp (ditto)
./ctp2_code/libs/anet/src/dp/ptimer.c (All if #if 0 blocks)
./ctp2_code/libs/anet/src/dp/ptimer.h (Seems fine)
./ctp2_code/libs/anet/src/aeh/aeh.c (May need fixing but shouldn't be involved in current problem)
./ctp2_code/gs/database/StrDB.cpp (in #ifdef WIN32 bock)
./ctp2_code/ui/aui_ctp2/c3blitter.cpp (dealt with)

On ZFS things: I think I've found your comment (c3imageformats.cpp) but I can't see anything wrong. On the other hand I'm not an expert on what's going on.


On SDL: I'm already using 1.2.13 sothat's not the problem.


On fbcon: how do I make sure I have a framebuffered console?


On adding the extra code I get the outputs (on multiple runs):
spritefile.cpp L628: size= 4896; sizeof(uint8)= 1; sizeof(Pixel16)= 2!
spritefile.cpp L628: size= 4896; sizeof(uint8)= 1; sizeof(Pixel16)= 2!
spritefile.cpp L628: size= 4896; sizeof(uint8)= 1; sizeof(Pixel16)= 2!

These only appear on starting the game and immediately before a crash. Not sure why size appears to be constant in my case but not yours.


Pixel16 is defined in pixeltypes.h as an unsigned short int. I've converted the whole file to the same format as ctp2_intypes.h (i.e. using int32_t and family if present). Unfortunately this doesn't affect the current problem.

sint32 appears to defined in four places (ctp2_inttypes.h, XY_Coordinates.cpp, CTP2Combat.h and ctp-types.h) I'm not sure what is actually used, particularly as the definitions are different (int v. long v. int32_t [not in order]). For what it's worth int32_t is defined in /usr/include/stdint.h as int.


Not sure what to do next...

[lynx_abraxas]

Hm, I'm running out of ideas... The only thing I can think of is that there is something wrong with your tga loading. You could delete the rim files and try to figure out why it is not able to extract them from the tga files. I wonder why it wants to load sprites long before it needs them for displaying. Does an strace reveal the rims beeing opend/loaded?
You could also check with a few printfs or a good usage of gdb what the ssize array is like. (If I remember right the size var is set from that)