DESIGN:Help with NETINFO protocol

[MrBaggins]

As everyone knows you're limited to 64 wonders, endgame objects, and feats.

These are stored in a uint64 ('64 bit' int, unsigned.)

While this is a crap design decision, and downright limiting for modding, it did allow them certain shortcuts in coding.

The essential data structure solution is easy... just use a bitvector, which is extensible, and functionality can be built in, to do whatever a uint64 can do, and can easily include serialization.

The trouble in paradise comes in the form of MP implementation.

E.G.

g_network.Enqueue(new NetInfo
(NET_INFO_CODE_ACHIEVEMENTS,
(uint32)(m_achievements & 0xffffffff),
(uint32)(m_achievements >> 32)));

What this does is send a single "packet" with a code to indicate what its transmitting, then it splits (using a slightly obtuse method,) the uint64 into 2 uint32s.

The networking design included 5 optional data parameters, but this obviously limits us... to 160 bits of data. Still not enough... and in fact, the only thing that makes sense is making an extensible system for sending extended data such as this.

We have to design some form of protocol for dynamic encoding, and decoding, presumably with some extra control messages.

I have some thoughts and an overview of a solution... but I thought I'd open the discussion up.

So... thoughts?

[MrBaggins]

Here's my current thoughts on implementation.

NETINFO (
NET_INFO_CODE_ACHIEVEMENTS,
Index (x) + number of cells,
cell x+0
cell x+1
cell x+2
cell x+3
);

Here's the explanation.

Bitvectors (can) use 32-bit ints to store data, which is obviously useful as they can coincide with the datatype of NETINFO.

There is effectively an array of these 32-bit ints, that are accessed by index (which is normally used to find which 32 bit segment to do a bit operand on.)

We can use these indexes to our advantage when we are sending the data over the net...

E.G.

The source machine needs to send data about the current state of the 288 possible achievements to the target machine. (9 cells)

It would send the data in 3 groups: 4, 4 then 1, finally.

The first time, it would send with an index of 0...

It would send cells 0,1,2 & 3 the in the first enqueued item, then 4,5,6,7 in the next, and 8 in the last.

The target machine would receive these and insert them into the bit vector. The bitvector would automatically grow to accomodate data beyond the current bounds.

Since you use unsigned long ints, for storing array values, and there are no "spare" bits for determining an unused cell for the receiving machine, we have to send this encoded with the index sint32, since we might not always be sending data, in all 4 subsequent sint32's.

What we do to resolve this is encode how many cells we are sending, in a particular batch, in the first 2 bits (lowest significance, bit0 and bit1) of the index.

To encode it, you just set the value to number of cells -1, then add (index *4)

To decode it, you just "and" (&) the index with 3, and add 1, to get the number of cells sent, and shift the value right by 2 (or divide by 4) to get the actual index.

========
This should provide us with a robust extensible method, and be time independent, so that packets could be dealt with out of original order.

So, any noticable flaws, shortcomings, better ways of doing this?

[ctplinuxfan]

Hi,

this is a great idea.
There could be functions/methods for sending/receiving or encoding/decoding stuff of a bitvector/bitset, just in case we need to deal with the endianess of the data later (e.g. for different platforms on a mp game).

Ciao
Holger

[MrBaggins]

I have the bitvector working completely with g_theAchievementTracker

It used uint64's, with serialization and network send/receiving, just like Wonders, Buildings and Feats.

I've tested it, as far as starting a new game, playing a bit and loading and saving.

I'm going to test MP functionality, before I continue.

If all goes well, I'll implement on the rest of the limited "uint64" objects.