COMPILE: Linux Port

[ctplinuxfan]

Before i forget it: The slic_debug linker errors may be due to
- inlining (to try: gcc switch -fno-default-inline)
- missing ld_flag -lstdc++ (don't think it's that, though)

I have no time to test it, though. In the first case we need to move inline code in the headers to declarations in the .cpp files later on.

[J Bytheway]

Originally posted by ctplinuxfan
Yes, i got that once as well, but waited for a possible addition of a conveniance library target for that dir as well. Perhaps we should meanwhile add a target for the directory ctp2_code/ui/ldl dependent on ctp2_code/ui dependent on ctp2_code.

I think that there already is such a target, but it gets ignored when you make because of something to do with VPATH... I didn't investigate thoroughly.

You need to install libtool's libltdl.so and development files (on debian, it's in package libltdl3).

I did have that installed. I've now made it link, but I had to explicitly add -lltdl to CTP2_LDFLAGS in Makefile.common.

Now it's time to fix missing implementations (i get intended assertions of _splitpath() calls, which should be converted to string operations e.g. when just extracting the file extension to check if a filename ends with ".foo").

If I try running the executable I just get a segfault. Is it necessary to do anything special to make asserts noticable?

First only to get ctp2 to link. I'm not sure, whether the conveniance libraries may contain unresolved links on any platform (which they all do), if they may we can switch to that again later.

IIRC, I read in the libtool docs about them, which suggests that they're fairly platform-independant.

For the next steps, we need a copy of a ctp2 installation on a FAT16 or FAT32 partition mounted writable for the user.
These changes must be made to the linux copy:
- copy mapgen/.libs/*.so to the mapgen plugin directory in ctp2_program
- copy ctp2 executable to the ctp2_program/bin? path (where the .exe is located)
- copy ctp2_data/* files to the ctp2_data dir of the linux copy
- modify civ_paths.txt and userprofile.txt to contain / instead of \ and point to the plugin libraries ending with .so instead of .dll

I'll look into this.

TODO now

Well, it still looks slightly daunting, but we're certainly making progress.

Hope i didn't forget anything. And thanks again for your help so far!

Well, it's fun .

Something you might want to check: In aui_SDLMouse::GetInput I got a "control reaches end of non-void function" warning. I fixed it by putting a "return AUI_ERRCODE_OK" statement at the end, but I'm not sure that's the correct behaviour - I was unable to figure out what the function was actually trying to do.

[J Bytheway]

One other thing - the executable is huge. Are we using dynamic linking?

[J Bytheway]

Originally posted by J Bytheway
If I try running the executable I just get a segfault. Is it necessary to do anything special to make asserts noticable?

Well, further investigation reveals that if I put the executable into the correct directory then I get an assert in _fullpath. Nevertheless, the segfault does indicate some kind of problem.

[J Bytheway]

Rather than alter civpaths.txt, I thought it would be better to simply read it in and change the backslashes to forward slashes in software.

Also, by using Fromafar's new userprofile.txt entry to provide alternate sources of gamedata, I can use a ctp2_data directory on a (readonly) NTFS partition, which saves copying. Symlinks might work too. It's also worth noting that there's no need to copy the executable to the usual directory, so long as you run it from there.

In this way, I've bypassed the file access errors, but am now getting this segfault:

#0  0x40490214 in SDL_ListModes () from /usr/lib/libSDL-1.2.so.0
#1  0x0817a672 in display_EnumerateDisplayModes ()
    at ctp2_code/ctp/display.cpp:152
#2  0x0817a8ca in display_Initialize (hInstance=0x0, iCmdShow=0)
    at ctp2_code/ctp/display.cpp:334
#3  0x08174901 in CivApp::InitializeApp (this=0x8b2eb00, hInstance=0x0,
    iCmdShow=0) at ctp2_code/ctp/civapp.cpp:1531
#4  0x0816f9c3 in CivMain (argc=1, argv=0xbffff8f4)
    at ctp2_code/ctp/civ3_main.cpp:2155
#5  0x0816f58a in main (argc=1, argv=0xbffff8f4)
    at ctp2_code/ctp/civ3_main.cpp:1921

I believe this is due to not calling SDL_Init before this, but I wan't able to figure out when would be an appropriate time to call it, or how to do so, since I'm not really clear on how the classes of the UI system work together yet.

[ctplinuxfan]

The executable is dynamically linked. The debug information increases c++ executables at a min of 10 times. ctp2 is about 10-12M without debug information.

Originally posted by J Bytheway
Rather than alter civpaths.txt, I thought it would be better to simply read it in and change the backslashes to forward slashes in software.

Yes, that's a great idea, indeed.

However, i use a copy of the data dir so far, in case any files don't work without modifications. E.g., the code generated by byacc/flex doesn't parse files with lineendings other than specified by the target platform. As the target platform is linux, it expects LF endings, but gets CRLF endings.

For this, i executed on the installation copy

find ctp2_data -type f -iname "*.ldl" -o -iname "*.txt" -o -iname "*.slc" -exec dos2unix -p {} \;

I'd appreciate a solution similar to yours on the file seperators, but i have had no time yet to spend on how yacc/lex generated parsers work. Ideally, we'd block read the contents of the files in a buffer, strip out the cr's and let the parser operate on the buffer instead on file streams. So before i introduce more bugs than i normally do, i try to fix the rest, first.

I believe this is due to not calling SDL_Init before this, but I wan't able to figure out when would be an appropriate time to call it, or how to do so, since I'm not really clear on how the classes of the UI system work together yet.

Yes, i moved the SDL_Init for video to display.cpp, because first the modes are enumerated.
For now, i get a black SDL screen, so it seems the executable remains in the gameloop. So next would be to get the starting screen drawn. I guess there is a problem in the surface or the blitting code, because absolutely nothing gets drawn. Another cause with less probability may be wrong initialisation order or drawing functions get called too early.
Perhaps another problem may be in resolving .zfs file entries, at least i see some .tgas which are not found and aren't located on the file system, so i guess they may be in the compressed .zfs files.

[J Bytheway]

Originally posted by ctplinuxfan
The executable is dynamically linked. The debug information increases c++ executables at a min of 10 times. ctp2 is about 10-12M without debug information.

Right. I feared this might be the case. It's somewhat annoying, because the executable is larger than the amount of RAM I have, so it takes forever to start gdb (a couple of minutes).

However, i use a copy of the data dir so far, in case any files don't work without modifications. E.g., the code generated by byacc/flex doesn't parse files with lineendings other than specified by the target platform. As the target platform is linux, it expects LF endings, but gets CRLF endings.

For this, i executed on the installation copy

find ctp2_data -type f -iname "*.ldl" -o -iname "*.txt" -o -iname "*.slc" -exec dos2unix -p {} \;

I'd appreciate a solution similar to yours on the file seperators, but i have had no time yet to spend on how yacc/lex generated parsers work. Ideally, we'd block read the contents of the files in a buffer, strip out the cr's and let the parser operate on the buffer instead on file streams. So before i introduce more bugs than i normally do, i try to fix the rest, first.

Well, that sounds like the sort of thing I might be able to fix, so I'll look into it.

Yes, i moved the SDL_Init for video to display.cpp, because first the modes are enumerated.
For now, i get a black SDL screen,

Hooray!

so it seems the executable remains in the gameloop. So next would be to get the starting screen drawn. I guess there is a problem in the surface or the blitting code, because absolutely nothing gets drawn. Another cause with less probability may be wrong initialisation order or drawing functions get called too early.
Perhaps another problem may be in resolving .zfs file entries, at least i see some .tgas which are not found and aren't located on the file system, so i guess they may be in the compressed .zfs files.

Well, the main starting screen background is certainly not in a zfs file, but probably some of the other graphics on the screen are.

[J Bytheway]

Well, I've fixed the lexers to handle the alternate line ending format - that was easy enough. Then I had to copy some fonts across from Windows to allow the game to find them (will this cause problems?).

Now I also get a black window opened, and messaages about missing .tga files. I diagnosed this problem as far as determining that the ProjectFile object in question has no data stored in it at all, so I guess the reading in of the .zfs file is failing.

After I commit the changes I have now, I think I'll go back to trying to make things compile with debugging symbols defined, because I feel that will reveal many problems, as well as providing debugging output.

[ctplinuxfan]

Originally posted by J Bytheway
Well, I've fixed the lexers to handle the alternate line ending format - that was easy enough. Then I had to copy some fonts across from Windows to allow the game to find them (will this cause problems?).

Great! No, this won't be a problem. The fonts are freely available at corefonts.sf.net, so either you haven't installed them on linux or your distribution doesn't include the path of the .ttf files in the font path (when using a font server, or a font managing tool on debian like defoma which doesn't copy the lowercase .ttf files to the truetype font path). Perhaps we should collect the directories of the corefonts of various distributions and add them to the bitmap font search path regardless if other font dirs are included or not (on debian, it's /usr/share/fonts/truetype/msttcorefonts/).

Now I also get a black window opened, and messaages about missing .tga files. I diagnosed this problem as far as determining that the ProjectFile object in question has no data stored in it at all, so I guess the reading in of the .zfs file is failing.

Well, this could exactly be the problem. I'll have a look at that.

After I commit the changes I have now, I think I'll go back to trying to make things compile with debugging symbols defined, because I feel that will reveal many problems, as well as providing debugging output.

Yes, that would be great. I started with guarding unsupported windows debug code (e.g. _crt functions) while reviewing the sdl code, but i still focus on getting SDL to work.

[J Bytheway]

Originally posted by ctplinuxfan
Great! No, this won't be a problem. The fonts are freely available at corefonts.sf.net, so either you haven't installed them on linux or your distribution doesn't include the path of the .ttf files in the font path (when using a font server, or a font managing tool on debian like defoma which doesn't copy the lowercase .ttf files to the truetype font path). Perhaps we should collect the directories of the corefonts of various distributions and add them to the bitmap font search path regardless if other font dirs are included or not (on debian, it's /usr/share/fonts/truetype/msttcorefonts/).

Searching the debian (testing) packages for "times.ttf" yields only

usr/share/doc/libclanlib2/examples/TTF/TIMES.TTF.gz	    doc/clanlib-examples
usr/share/fonts/truetype/ttf-tamil-fonts/TSCu_Times.ttf	    x11/ttf-tamil-fonts

and neither of these really seems promising - can these fonts be installed through apt, or do I need to do something else?

Yes, that would be great. I started with guarding unsupported windows debug code (e.g. _crt functions) while reviewing the sdl code, but i still focus on getting SDL to work.

One thing that puzzles me - why have you entered so many different definitions of C_WARNINGS and CXX_WARNINGS into configure.in? Is this simply to easily allow changing which warnings are activated?

And, a last minor question - was there any particular reason you chose to use the name configure.in rather than configure.ac?

[ctplinuxfan]

The fonts are in the package msttcorefonts in the x11 category within section contrib. When installing the package, you need cabextract, too, and a internet connection so the install script can download the font installers from sourceforge.net (you can also provide a directory containing the downloaded .exes, but i'm not sure wether it persists).

So apt-get install msttfcorefonts should be enough.

Originally posted by J Bytheway
One thing that puzzles me - why have you entered so many different definitions of C_WARNINGS and CXX_WARNINGS into configure.in? Is this simply to easily allow changing which warnings are activated?

Yes, it's for switching purposes and maybe to support different a different warning level for the non-debug build later on or get -Werror included.

And, a last minor question - was there any particular reason you chose to use the name configure.in rather than configure.ac?

No, i'm just used to that name; perhaps this might enable somebody to compile ctp2 with older versions, too, but we could also rename it to the newer configure.ac, as most systems support the new naming style afaik.

[ctplinuxfan]

Further progress: On linux for 32-bit x86, images get displayed wrong, either due to blitting, image loading, rgb convertion or wrong chroma keys. Anything you choose from the main menu leads to a segfault. But there's one exception: The credits screen.

Finally, everything compiles and links on linux x86_64, too, but the anet test seems to remain in an endless loop (aborted that) and all you get when launching ctp2 is a black screen.

I totally forgot these things about the installation:
- the userprofile.txt must contain a path to the .la files of the map plugins
(the path within userprofile.txt is chosen with the filename converted to lowercase with the ending .la instead of .dll)
- the plugins (at least *.la and *.so) must be copied to the plugin dir (i used libltdl for the plugins, not libdl... )
- If you run the executable from another directory than the installation, you must create the plugin directory there, too

Additionally, you need to copy Setup/data/Add/ctp2_program/ctp/texture.dat from your ctp2 cd to the directory where the executable you run is located.

edit: userprofile.txt

[J Bytheway]

Well, I finished the changes to get it compiling with --enable-debug (defining _DEBUG and_PLAYTEST). I've made asserts immediately fatal, but we may not want that (just take the assert(0) out of ctp/ctp2_utils/c3debug.cpp line 303 to make them non-fatal).

The debug version fails with an assert reading civpaths.txt. I haven't looked into the cause yet.

When running the default version I just get a black screen, but I haven't performed the last few tasks you've just listed.

[ctplinuxfan]

Hi,

sorry for the long delay, i have so much work atm. that i even don't find a minute to do anything else.

Originally posted by J Bytheway
Well, I finished the changes to get it compiling with --enable-debug (defining _DEBUG and_PLAYTEST). I've made asserts immediately fatal, but we may not want that (just take the assert(0) out of ctp/ctp2_utils/c3debug.cpp line 303 to make them non-fatal).

At a first glance, i'd see 3 options:
1. leave them as they are (deactivated)
2. make them fatal (perhaps with an off switch in userprofile.txt)
3. interactive handling (dialog like microsoft handles them)

For a short term, i tend to 1, but your solution is right and asserts must be addressed either via 2 or 3 (last one perhaps involving licensing issues of graphic library used for the dialogs; additionally there maybe initialization issues (hidden event based main loop for graphical toolkits like gtk), thus on low prio).

When running the default version I just get a black screen, but I haven't performed the last few tasks you've just listed.

One thing to add is that you might get a black screen until you press the left mouse button. I guess this is a problem of the deactivation of movies (intro movie would be displayed under windows).

I hope you find a few minutes more to get things going; as long as my tassk list is as long as it's now it'll last some time from one of my commits to the other.

[J Bytheway]

At a first glance, i'd see 3 options:
1. leave them as they are (deactivated)
2. make them fatal (perhaps with an off switch in userprofile.txt)
3. interactive handling (dialog like microsoft handles them)

For a short term, i tend to 1, but your solution is right and asserts must be addressed either via 2 or 3 (last one perhaps involving licensing issues of graphic library used for the dialogs; additionally there maybe initialization issues (hidden event based main loop for graphical toolkits like gtk), thus on low prio).

My instinct is always to try to find errors as early as possible, which the fatal asserts allows. OTOH, most of them probably aren't real problems. If I can get as far as the main menu and some relatively easily trackable errors in the form of segfaults then I'll probably ignore the asserts also, but they do provide a good store of problems to be solved when the main problem of the hour is incomprehensible.

One thing to add is that you might get a black screen until you press the left mouse button. I guess this is a problem of the deactivation of movies (intro movie would be displayed under windows).

Thanks for the hint - I'll try that.

I hope you find a few minutes more to get things going; as long as my tassk list is as long as it's now it'll last some time from one of my commits to the other.

Well, I slowed for a while of end-of-year exams, but my present problem is that I'm on dialup and my modem doesn't work under linux, so I can't update or commit. If I move my working copy to a shared FAT32 partition then perhaps I can get somewhere... I do have free time at the moment, so I'll see what I can do.