COMPILE: Linux Port

[J Bytheway]

Just a quick note to say that the following thread suggests a way to fix all the warnings about non-POD types:

http://forums.devshed.com/showthread.php?t=181647&page=1&pp=15

[ctplinuxfan]

My idea goes in this direction. However, the ptr trick is ugly because it's unclear when unneeded temporaries of function parameters are destroyed (not a reference of the object is passed, but a pointer to the object. the pointer still holds an adress, after the object is destroyed, but dereferencing the pointer will lead into a crash, because it points to an invalid address).
My idea goes in that direction:

class Param {
public:
   Param(const Unit &u) { m_type = typeUnit; m_u = u; }
   Param(const City &a) { m_type = typeCity; m_c = c; }

   ... 
   const type getType() const;
   City &getCity();
   Unit &getUnit();

private:
   ...
};

class EventCall {
public:
   int call(const Param &p1);
   int call(const Param &p1, const Param &p2);
   int call(const Param &p1, const Param &p2, const Param &p3);
   ...
}

City c(id);
Unit u(id2);
EventCall evc;

evc.call(SEND_UNIT_TO_CITY, u, c);

however, this might lead into performance issues, because additional time is needed for creating temporary Param objects on stack. An alternative might be a C convention, for most classes have a unique id to allow distribution of the gamestate e.g. over network:

Function with 2x param count like this:

int evc callback(EVENT_TYPE type, ...)

with an odd number of variable length arguments following: a list of pairs of arguments. The first argument of such a pair is an uint32 describing the type of the argument following. The second argument of a pair is the Event-Argument converted to a POD-Type (maybe even a void * pointer to a type). This would need additional time decoding the argument types, but perhaps less time then creating temporary objects on stack which must be destroyed after calling.

Both have the need of decoding the argument list passed to determine the objects passed.

A last possibility would be to write a call function for any valid argument type list that could be passed to it, i.e.

class XY {
  int call(EVENT_TYPE, const Unit &, const City &);
  int call(EVENT_TYPE, const Army &, const City &);
  int call(EVENT_TYPE, const Army &, const Army &);
  ...
};

[Frog_Gamer]

CTP2 on Linux

I am a complete newbie as far as Linux is concerned, but I have played CTP2 for years now.

Due to the fact that I will have heart surgery next month, I should have plenty of time available to help.

In what way may I be of help?

My goal would be in the end to play CTP2 on my Linux Mandrake 10.2 system

regards to all and keeep up the good effort

regards to all

[J Bytheway]

Originally posted by ctplinuxfan
An alternative might be a C convention, for most classes have a unique id to allow distribution of the gamestate e.g. over network:

Function with 2x param count like this:

int evc callback(EVENT_TYPE type, ...)

with an odd number of variable length arguments following: a list of pairs of arguments. The first argument of such a pair is an uint32 describing the type of the argument following. The second argument of a pair is the Event-Argument converted to a POD-Type (maybe even a void * pointer to a type). This would need additional time decoding the argument types, but perhaps less time then creating temporary objects on stack which must be destroyed after calling.

Both have the need of decoding the argument list passed to determine the objects passed.

A last possibility would be to write a call function for any valid argument type list that could be passed to it, i.e.

class XY {
  int call(EVENT_TYPE, const Unit &, const City &);
  int call(EVENT_TYPE, const Army &, const City &);
  int call(EVENT_TYPE, const Army &, const Army &);
  ...
};

Since all the non-POD types being passed inherit from ID, we could combine these two solutions to have something like

class XY {
  int call(EVENT_TYPE, ARG_TYPE, const ID &);
  int call(EVENT_TYPE, ARG_TYPE, const ID &, ARG_TYPE, const ID &);
  ...
};

Where we have doubled argument lists as before to distinguish between different inheritors from ID, but we're able to pass the actual objects. This needs only one function for each possible number of arguments, rather than one function for each possible combination of arguments.

[ctplinuxfan]

Re: CTP2 on Linux

Originally posted by Frog_Gamer
I am a complete newbie as far as Linux is concerned, but I have played CTP2 for years now.

Due to the fact that I will have heart surgery next month, I should have plenty of time available to help.

In what way may I be of help?

My goal would be in the end to play CTP2 on my Linux Mandrake 10.2 system

regards to all and keeep up the good effort

regards to all

Great that you want to help.
For now, you could help us in porting, i.e. program to get CTP2 running on linux. If you want to do so, read the EULA, write a personal message or mail to kaan with a username and password you chose to access the source (and that you agree to the eula). If you are new to c++ programming, get a book at your library and feel free to ask what you don't understand.
Once your user has been setup, install subversion and run

svn co --username yourusername svn://ctp2.kaan.dk/ctp2/branches/linux ctp2-br-linux

(where yourusername is the username you chose).
You can run ./autogen.sh ; ./configure ; make to start compilation of ctp2.
For coding, first, the errors which prevent the code from compilation have to be fixed, second the compiler warnings which may indicate bugs and finally when ctp2 runs probably fix runtime bugs and implement more and more features left out at the moment (networking, videos, ...).

As a first step, you could start compilation and post where it stops with the errors you get (not the warnings). I try to get it run on debian testing (gcc 3.3.5) and gentoo 2005.0 (gcc 3.4.3), but maybe things are also different on mandrake.

The other thing you could do is playtesting once ctp2 compiles under linux, but it will still last some time until we're at this point.

Get well soon and welcome to the team!

[ctplinuxfan]

Originally posted by J Bytheway
Since all the non-POD types being passed inherit from ID, we could combine these two solutions to have something like

class XY {
  int call(EVENT_TYPE, ARG_TYPE, const ID &);
  int call(EVENT_TYPE, ARG_TYPE, const ID &, ARG_TYPE, const ID &);
  ...
};

Where we have doubled argument lists as before to distinguish between different inheritors from ID, but we're able to pass the actual objects. This needs only one function for each possible number of arguments, rather than one function for each possible combination of arguments.

In my opinion, this is the best idea.
Could you try to implement that by and by for the slic and gameevent handler? I will continue with the ui networking stuff once i have some time again.

[J Bytheway]

Originally posted by ctplinuxfan
In my opinion, this is the best idea.
Could you try to implement that by and by for the slic and gameevent handler? I will continue with the ui networking stuff once i have some time again.

Yes, I'll give it a try.

[ctplinuxfan]

Just 3 things:
- seems the anet changes did not break linux code for IA32/AMD32 (although it doesnt work on AMD64 yet
- now hopefully everything compiles
- ctp2 doesn't link

3 new things to do (in addition to implement things guarded out by and by...):
- Some linker errors are due to gameevent system, which i tried not to touch.
- Some SDL code needs to be rechecked (few symbols missing here).
- some further ctp2 code nneds to be reviewed for the missing symbols

[J Bytheway]

Well, I think I've finished reworking the SLIC-related varargs code, and now I'm working on the events-system related stuff. Once I'm through that, hopefully my codebase will compile and I'll commit the changes.

Can you do anything about the fact that anet runs its regression tests every time you 'make'? It's quite annoying.

[ctplinuxfan]

I added a linux86.tar.gz target dependent on the anet Makefile, which should prohibit that (caveat is, that you need to be in the anet directory if you change something and want to compile the changes).

I couldn't prevent previous changes to the slic code from also being backmerged (merged trunk to branches/linux a few days ago).
If you stumble over any conflicts, take your file, please.

[J Bytheway]

OK, I've got my version compiling too, so I submitted revision 414 with my changes.

Along the way I fixed as many compiler warnings as I could, so there are rather a lot of changes. It would probably be wise to look them over.

Now the only warnings that need disabling to get the code to compile (at least in the C++ code) are -Wno-multichar -Wno-unused -Wno-format -Wno-switch -Wno-switch-enum -Wno-non-template-friend.

I'm now seeing linking errors mostly in the lexer/parser code complaining of duplicate definitions.

A couple of other points:

When I was comiling in a seperate tree, it fails to create the ctp2_code/ui/ldl directory, and so one of the calls to byacc fails.

Do we need to worry at this point about what symbols we are defining, e.g. _DEBUG?

[J Bytheway]

I'm also getting "cannot find -lSDL_image" - I have two possible candidates:
/usr/lib/libSDL_image-1.2.so.0
/usr/lib/libSDL_image-1.2.so.0.1.3
Should it be finding one of those with that argument, or should it be looking for -lSDL_image-1.2?

[J Bytheway]

OK, I've fixed all the linking errors I had before by installing the missing library, altering the lexer/parser code and removing a couple of duplicate definitions. Now I'm facing a long list of unresolved links.

I'll commit my changes in the morning when I'm less tired and more coherent.

[J Bytheway]

OK, after fixing those I could, I'm left with the following linking errors:

civ3_main.o(.text+0x7fd): In function `ui_Initialize()':
ctp2_code/ctp/civ3_main.cpp:508: undefined reference to `aui_SDLMouse::aui_SDLMouse[in-charge](AUI_ERRCODE*, char*, unsigned)'
CityData.o(.text+0xf3a7): In function `CityData::RemoveBorders()':
ctp2_code/gs/gameobj/CityData.cpp:6325: undefined reference to `terrainutil_RemoveBorders(MapPoint const&, int, int, int, Unit const&)'
wldgen.o(.text+0xb38): In function `World::~World [not-in-charge]()':
ctp2_code/gs/world/wldgen.cpp:271: undefined reference to `lt_dlclose'
wldgen.o(.text+0xb3d):ctp2_code/gs/world/wldgen.cpp:272: undefined reference to `lt_dlexit'
wldgen.o(.text+0xbf4): In function `World::~World [in-charge]()':
ctp2_code/gs/world/wldgen.cpp:271: undefined reference to `lt_dlclose'
wldgen.o(.text+0xbf9):ctp2_code/gs/world/wldgen.cpp:272: undefined reference to `lt_dlexit'
wldgen.o(.text+0xcb0): In function `World::~World [in-charge deleting]()':
ctp2_code/gs/world/wldgen.cpp:271: undefined reference to `lt_dlclose'
wldgen.o(.text+0xcb5):ctp2_code/gs/world/wldgen.cpp:272: undefined reference to `lt_dlexit'
wldgen.o(.text+0x7cb5): In function `World::LoadMapPlugin(int)':
ctp2_code/gs/world/wldgen.cpp:2800: undefined reference to `lt_dlinit'
wldgen.o(.text+0x7cd5):ctp2_code/gs/world/wldgen.cpp:2804: undefined reference to `lt_dlopen'
wldgen.o(.text+0x7cfe):ctp2_code/gs/world/wldgen.cpp:2809: undefined reference to `lt_dlexit'
wldgen.o(.text+0x7d1d):ctp2_code/gs/world/wldgen.cpp:2816: undefined reference to `lt_dlsym'
wldgen.o(.text+0x7d31):ctp2_code/gs/world/wldgen.cpp:2822: undefined reference to `lt_dlclose'
wldgen.o(.text+0x7d36):ctp2_code/gs/world/wldgen.cpp:2823: undefined reference to `lt_dlexit'
wldgen.o(.text+0x7d87):ctp2_code/gs/world/wldgen.cpp:2839: undefined reference to `lt_dlclose'
wldgen.o(.text+0x7d8c):ctp2_code/gs/world/wldgen.cpp:2840: undefined reference to `lt_dlexit'
wldgen.o(.text+0x7dc3): In function `World::FreeMapPlugin()':
ctp2_code/gs/world/wldgen.cpp:2865: undefined reference to `lt_dlclose'
wldgen.o(.text+0x7dc8):ctp2_code/gs/world/wldgen.cpp:2866: undefined reference to `lt_dlexit'
WrlEnv.o(.text+0x1960): In function `World::CutImprovements(MapPoint const&)':
ctp2_code/gs/world/WrlEnv.cpp:897: undefined reference to `terrainutil_RemoveBorders(MapPoint const&, int, int, int, Unit const&)'
spritefile.o(.text+0x698): In function `SpriteFile::WriteFacedSpriteData(FacedSprite*)':
ctp2_code/gfx/spritesys/spritefile.cpp:212: undefined reference to `FacedSprite::GetFrameDataSize(unsigned short, unsigned short)'
spritefile.o(.text+0x6d4):ctp2_code/gfx/spritesys/spritefile.cpp:213: undefined reference to `FacedSprite::GetMiniFrameDataSize(unsigned short, unsigned short)'
spritefile.o(.text+0xbda): In function `SpriteFile::WriteFacedSpriteWshadowData(FacedSpriteWshadow*)':
ctp2_code/gfx/spritesys/spritefile.cpp:311: undefined reference to `FacedSpriteWshadow::GetFrameDataSize(unsigned short, unsigned short)'
spritefile.o(.text+0xc56):ctp2_code/gfx/spritesys/spritefile.cpp:327: undefined reference to `FacedSpriteWshadow::GetMiniFrameDataSize(unsigned short, unsigned short)'
spritefile.o(.text+0xce5):ctp2_code/gfx/spritesys/spritefile.cpp:344: undefined reference to `FacedSpriteWshadow::GetShadowFrameDataSize(unsigned short, unsigned short)'
spritefile.o(.text+0xd61):ctp2_code/gfx/spritesys/spritefile.cpp:359: undefined reference to `FacedSpriteWshadow::GetMiniShadowFrameDataSize(unsigned short, unsigned short)'
spritefile.o(.text+0xdf3):ctp2_code/gfx/spritesys/spritefile.cpp:375: undefined reference to `FacedSpriteWshadow::GetFrameDataSize(unsigned short, unsigned short)'
spritefile.o(.text+0xe8a):ctp2_code/gfx/spritesys/spritefile.cpp:386: undefined reference to `FacedSpriteWshadow::GetMiniFrameDataSize(unsigned short, unsigned short)'
spritefile.o(.text+0xf34):ctp2_code/gfx/spritesys/spritefile.cpp:398: undefined reference to `FacedSpriteWshadow::GetShadowFrameDataSize(unsigned short, unsigned short)'
spritefile.o(.text+0xfcb):ctp2_code/gfx/spritesys/spritefile.cpp:410: undefined reference to `FacedSpriteWshadow::GetMiniShadowFrameDataSize(unsigned short, unsigned short)'
aui_surface.o(.gnu.linkonce.r._ZTV11aui_Surface+0x20):ctp2_code/ui/aui_common/aui_surface.cpp:411: undefined reference to `aui_Surface::GetDC(hdc_t**)'
aui_surface.o(.gnu.linkonce.r._ZTV11aui_Surface+0x24):ctp2_code/ui/aui_common/aui_surface.cpp:411: undefined reference to `aui_Surface::ReleaseDC(hdc_t*)'
aui_ui.o(.gnu.linkonce.r._ZTV6aui_UI+0xf0):ctp2_code/ui/aui_common/aui_ui.cpp:65: undefined reference to `aui_UI::HandleWindowsMessage(hwnd_t*, unsigned, int, int)'
aui_Factory.o(.gnu.linkonce.t._ZN7aui_SDL2DDEv+0x4): In function `aui_SDL::DD()':
ctp2_code/ui/aui_sdl/aui_sdl.h:31: undefined reference to `aui_SDL::m_lpdd'
aui_sdlsurface.o(.gnu.linkonce.r._ZTV14aui_SDLSurface+0x20):ctp2_code/ui/aui_sdl/aui_sdl.h:18: undefined reference to `aui_Surface::GetDC(hdc_t**)'
aui_sdlsurface.o(.gnu.linkonce.r._ZTV14aui_SDLSurface+0x24):ctp2_code/ui/aui_sdl/aui_sdl.h:18: undefined reference to `aui_Surface::ReleaseDC(hdc_t*)'
aui_sdlui.o(.text+0x4b2): In function `aui_SDLUI::CreateScreen(unsigned)':
ctp2_code/ui/aui_sdl/aui_sdlui.cpp:144: undefined reference to `aui_SDL::m_lpdd'
aui_sdlui.o(.text+0x915): In function `aui_SDLUI::RestoreMouse()':
ctp2_code/ui/aui_sdl/aui_sdlui.cpp:214: undefined reference to `aui_SDLMouse::aui_SDLMouse[in-charge](AUI_ERRCODE*, char*, unsigned)'
aui_sdlui.o(.gnu.linkonce.r._ZTV9aui_SDLUI+0xf0):ctp2_code/ui/aui_sdl/aui_sdlui.cpp:88: undefined reference to `aui_UI::HandleWindowsMessage(hwnd_t*, unsigned, int, int)'
c3ui.o(.gnu.linkonce.r._ZTV4C3UI+0xf0):ctp2_code/ui/aui_common/aui_resource.h:183: undefined reference to `aui_UI::HandleWindowsMessage(hwnd_t*, unsigned, int, int)'
segmentlist.o(.text+0xc1): In function `SegmentList::SegmentList[not-in-charge](void (*)(int), char*)':
ctp2_code/ui/slic_debug/segmentlist.cpp:69: undefined reference to `vtable for SegmentList'
segmentlist.o(.text+0x2df): In function `SegmentList::SegmentList[in-charge](void (*)(int), char*)':
ctp2_code/ui/slic_debug/segmentlist.cpp:69: undefined reference to `vtable for SegmentList'
sourcelist.o(.text+0xef): In function `SourceList::SourceList[not-in-charge](void (*)(int), char*)':
ctp2_code/ui/slic_debug/sourcelist.cpp:100: undefined reference to `vtable for SourceList'
sourcelist.o(.text+0x303): In function `SourceList::SourceList[in-charge](void (*)(int), char*)':
ctp2_code/ui/slic_debug/sourcelist.cpp:100: undefined reference to `vtable for SourceList'
watchlist.o(.text+0xf1): In function `WatchList::WatchList[not-in-charge](void (*)(int), char*)':
ctp2_code/ui/slic_debug/watchlist.cpp:86: undefined reference to `vtable for WatchList'
watchlist.o(.text+0x305): In function `WatchList::WatchList[in-charge](void (*)(int), char*)':
ctp2_code/ui/slic_debug/watchlist.cpp:86: undefined reference to `vtable for WatchList'
collect2: ld returned 1 exit status

These seem to fall into a few categories:
- UI stuff which isn'tthere because the appropriate SDL code is missing
- terrainutil_RemoveBorders not working, for reasons which escape me
- Lots of functions starting 'lt_dl' being missing
- Some functions in FacedSprite which are undefined (I checked and they seem to really be unimplmented - I think perhaps it's supposed to be calling the versions on the parent class).
- Peculiar problems with lists which I don't really understand.

Also, you seem to have abandoned the "helper library" idea - any particular reason?

[ctplinuxfan]

Wow, great work!

Originally posted by J Bytheway
When I was comiling in a seperate tree, it fails to create the ctp2_code/ui/ldl directory, and so one of the calls to byacc fails.

Do we need to worry at this point about what symbols we are defining, e.g. _DEBUG?

Yes, i got that once as well, but waited for a possible addition of a conveniance library target for that dir as well. Perhaps we should meanwhile add a target for the directory ctp2_code/ui/ldl dependent on ctp2_code/ui dependent on ctp2_code.

For the makros we define, i think we should first look that everything works out of the box. We just need to ensure the cd check works, and later have to adopt as many flags as possible for the windows debug target (./configure --enable-debug) and the final build for release (no debug).

OK, after fixing those I could, I'm left with the following linking errors:

wldgen.o(.text+0x7cfe):ctp2_code/gs/world/wldgen.cpp:2809: undefined reference to `lt_dlexit'
...
collect2: ld returned 1 exit status

You need to install libtool's libltdl.so and development files (on debian, it's in package libltdl3).

I merged further SDL related code, deactivated the slic_debug code and got a linked executable.
Now it's time to fix missing implementations (i get intended assertions of _splitpath() calls, which should be converted to string operations e.g. when just extracting the file extension to check if a filename ends with ".foo").

Also, you seem to have abandoned the "helper library" idea - any particular reason?

First only to get ctp2 to link. I'm not sure, whether the conveniance libraries may contain unresolved links on any platform (which they all do), if they may we can switch to that again later.

For the next steps, we need a copy of a ctp2 installation on a FAT16 or FAT32 partition mounted writable for the user.
These changes must be made to the linux copy:
- copy mapgen/.libs/*.so to the mapgen plugin directory in ctp2_program
- copy ctp2 executable to the ctp2_program/bin? path (where the .exe is located)
- copy ctp2_data/* files to the ctp2_data dir of the linux copy
- modify civ_paths.txt and userprofile.txt to contain / instead of \ and point to the plugin libraries ending with .so instead of .dll

TODO now is:
- replace _splitpath() calls by appropiate c code (preferably implement something like BOOL string_endswith(char *, char *))
- implement essential parts of mainloop (ctp/*)
- fix any assertion errors we find
- review font related code in case we get no text displayed
- review our last changes (mainly type related warnings we corrected; we must ensure the code behaves the same)

When we get a running game (mouse + sound only so far):
- make sure the cd check works under linux as well
- copy everything to a case sensitive filesystem and make game work there (something like convert every filename to lowercase when reading ldl files or accessing filesystem, and convert installation to lowercase)
- write xml file for loki_setup
- generate Makefile.am to copy files needed to path for installer image
- sync makro definitions win/linux and make sure code compiles as well (perhaps despite guarded code for linux like debugging aids and too system specific code)
- fix code to compile on windows again and merge back to /trunk (this will become a very large commit i fear)

After that, we reach milestone 1

Hope i didn't forget anything. And thanks again for your help so far!