Multiplayer Coding/Issues

[Osiris]

Well, first off I am the guy in charge of multiplayer programming, so anything you want to know ask me or put it on this thread.

>
Again please give me your feedback. But one thing no matter what we do with multiplayer
so far I will have to no EVERY single variable that makes up world, so multiplayer can make
a couple of its own fake worlds for what I just explained above.
>
well, I think if you need to know every single variable that makes up the world, then we're
not using a proper object-oriented approach. We need to think hard about this, because I'm
convinced there is an "easy" way to do this, and Lots of hard ways .
>
Mark, you cant to the following with sockets though... 'whatever.write(socket,world)'
You can ONLY send character strings through a socket connection. So for example to send the population data to one of the clients the host would do the following....
'whatever.write(socket,"SET POP ProvinceA 1200")'.
Basically you cannot send over entire objects through networking. I might be wrong on this because this comes from C++ socket programming, but I have scanned through Java socket programming and it seems the same.
That is why I would need to know every variable, so each computer could send some or all of the maps data through strings like the example I should you.
Overall the best way to explain what a socket is is that it is nothing but a fancy file desriptor, and you can only read text period.

[mca]

There is definitely a way to send objects over the network in Java, there are even several ways I can think of, each with their own complications.

1: Object Serialization: Communicating through sockets and java.io.Object(Input|Output)Streams. Simple, but lots of typecasts and manual typechecks are needed, AFAIK, plus a bit of bandwidth overhead. I never used this.
It looks something like objectStream.writeObject(someObject);

2: Java Remote Method Invokation (RMI). Transparently invoking methods over the network, with automatic object serialization. Elegant and Real Easy to use, but costs a bit more bandwidth overhead.
It looks like any other method invokation: someReferenceToAnObjectOnAnotherMachine.someMethod (someParameters);

3: CORBA. Much the same as RMI, but is language independent. Perhaps less bandwith expensive than RMI but harder to use.

4: Writing a custom object communication protocol, using sockets and ByteArray streams or something similar in nature. Very Risky, IMHO, but can probably be less bandwith expensive than any of the above, if done right. I never tried to do this.

I estimate that the bandwidth overhead of any of the above methods is less than something like whatever.write(socket,"SET POP ProvinceA 1200");, but of course, experiments need to be done to be sure.

Martin
[This message has been edited by mca (edited September 09, 1999).]

[mca]

Re: synchronizing map (and other) info.

I suggest to use the Observer pattern, which is just like the Listeners and Events in the AWT and Swing.

(from Gamma, e.a.: Design Patterns)

Intent:
Define a one-to-many dependency between objects, so that when one object changes state, all its dependents are notified and updated automatically.

The idea is to have one "authoritative" model (e.g. an object of class GlobalMap ), where all changes take place, and that sends info on the changes to interested parties (e.g. PlayerMap objects, that show the player specific maps on the client machines). A PlayerMap only has the info that the player can actually see on the screen.

For example, when a player has moved an exploring unit and uncovers some new tiles of the map, the GlobalMap is told to update the authoritative data structure (perhaps world.moveTF(tf, from, to);). The moveTF will update the player's map info, and, after noting the change in some internal data structure, it sends a tf.getCiv().getPlayerMap().updateMap(tile) (or somethiing like that) request to the client for each of the newly discovered tiles, and also the two other tiles that changed: the tile the unit moved from and the one it moved to (perhaps even more, if there is some fog of war)

Continuing the example, the GlobalMap also checks if other players should be notified of the move. If the unit moved into or out of a tile that is seen by one of the other players, then that player's PlayerMap also gets an update or two: anotherPlayerMap.updateMap(tileThatTheUnitMovedInt o);
anotherPlayerMap.updateMap(tileThatTheUnitMovedOut Of);

Martin
[This message has been edited by mca (edited September 09, 1999).]

[Mikael]

Osiris,

Concerning multiplayer modes, will it be possible to implement hotseat? You know, when two or more guys play against each other on the same computer? Or PBEM, when player A sends his turn to player B, then player B plays his turn, saves it, zips it, and finally sends it back to player A?
I'm a big fan of these possibilities, as they make you save LOTS of money compared to network gaming.

[Mark_Everson]

Osiris:

Here's an example of using sockets and serialization to send objects. It is very simple, but shows the idea. Martin, I think it only requires one type cast per object sent. If we send a World there would be virtually no overhead in coding effort. (I'm not up on all this stuff so I could be wrong, but it looks easy) What the Speed of a non-trivial operation is I have no idea... Its Here. Just follow the links to client.java and server.java.

[This message has been edited by Mark_Everson (edited September 09, 1999).]

[mca]

Re: object serialization

Writing an object to a stream is really easy,
but finding out what to do with it on the other end might take a bit more coding effort. The receiver reads some object, but has to find out whether it is a World, a Treaty, a Battle, a Message, or whatever type we may choose to send across the network, cast it to the correct type, and then decide exactly what to do with it. So there's only one typecast per transaction, but there are a lot of different typecasts in the receiving code.

But as I said, I never really worked with object streams, so there may be some really elegant way to do it that I haven't thought of.

Martin

[Mark_Everson]

mca: Ahhh, Now I see what you mean.

[Osiris]

Mikael, YES I am a very big fan of PBEM myself, I dont know about multiplayer on one computer.

Ok, I was wrong, I haven't even read anything on networking for java yet, so objects it will be. I will make a very precise model for multiplayer by the weekend after this (sep 18). I need to learn about java networking first, I will do this in a couple days(might even make a partial model for multiplayer). I will try and make the first workable source for multiplayer object by the end of this month.

Although even though I know you can send objects over to the other side, I still want to do my origional aproach. I will need to know all the variable workup of world, BUT I will not be able to access the actual world itself, so this idea is no big problem with OO. although it would still be harder.
The newer model I am making is VERY much leaning towards each computer does map processing and give each other players orders, so in effect there wouldn't be any server in this model(everyone would be equal ). Most newer games have this model and they say it makes it much more efficient in ways like worrying about the server crashing(did some research in other games multiplayer model).

[Mark_Everson]

Osiris:

Yes, peer-to-peer was the way I was going to pitch also, although I didn't do any research. Keep us posted!

[manurein]

I will not code the multiplayer of Clash, and I'm not very aware network programming, so dont take my words too seriously.

Anyway, I have a few remarks :
- I fully support the no-server option, but IMO it will pose some problems we will have to solve : as we all know, the main limitation and trouble source in MP is the bandwidth. IMO, the low-level method or protocol used to send datas is not the main issue in this regard. The main issue is the amount of datas u send.
So if you want to have an efficient MP code, u first have to define very carefully which datas u send (which obviously wont be all datas) and when u send them. For this purpose, using a server is very effective, cause it can have a global vision of the game and decide accurately which data needs to be sent and which data does not.
If we dont have a server, we have to work to find some effective methods to limit the amount of data we send, knowing that each computer has a very limited vision of the global world.
- I may be wrong, but I think I remember that turns in MP will be simultaneous; and also that army movements will be drawn during the turn and implemented at the end of the turn, showing the results next turn. If this is the case, this will mean that all data exchanges will occur between the end of a turn and the beginning of the next, except for chat and diplomatic exchanges.
- Once again I may be wrong, but it seems to me that one effective method to reduce unneeded data transfers is to use a request model. With this, each computer will tell the others what it needs when it needs it.
If what is above is true (I'm not sure ...), this means that for example in the case of map revelation, each turn a computer will request each other to tell him what movements have occured in a given set a provinces (those provinces this computer knows). The other computers will send an answer, where there will be datas only for the provinces in this set where things have changed since the last turn, and nothing for the other provinces.
All this probably means that we would have a central messenging system for each computer (and no global central messaging system) that would get requests from every part and every model of the game, and would process them toward the other computers, filtering unneeded and redondant data, adressing to the right computer or broadcasting when nessary etc. It could prove efficient to reduce data transfers, and, once again, IMO its a more important issue than the low level transfer protocol.
One last note : if we choose a solution resembling that, we might have to design a high level protocol for the requests and the answers.

What u think?

[mca]

Osiris:

In your model, will each computer know all game data or will it be partitioned in some way, so that each computer is responsible for just part of the data?

manurein:

Will each computer have to keep enough knowledge of the other computers to decide for itself which computer to request, or does it need to request it from all computers in turn in order to find the computer that has the data?

How will it know which data it needs to update? Will it have to poll all the other computers for info on e.g. which tiles have changed since last turn, or is there another way?

If Manurein is right in that the choice of low-level protocol won't affect bandwidth consumption much, and that the high-level protocol is much more important, then we should go for the low-level protocol that has the least coding overhead (definitely RMI, IMHO).

Martin

[Mark_Everson]

Good productive discussion gentlemen. I'm new to the net area, and so am learning new rules-of-thumb every day.

I think that every computer Must have at least a Near-Current copy of the whole game available. I talk about what near-current means below.

Each comp would have a chunk of the map and associated civs that it is Responsible for in addition to the rest of the info. Each peer will know what the mapping is between computers responsible for them and map areas & civs. To the extent that most civs will stick to their own areas, the main difficulty is how to re-assemble the area information so everyone starts the turn (or near the start of the turn) with the same data.

My hope is that we can use RMI/request brokering for diplomacy, merchant trade and such. (You wouldn't need brokered requests for map info since each system already will have a current map shortly after turn start) Then there would be a class that covers the movement of military units (which needs to get there before combat can be cranked). Possibly another like this for map changes. Finally, a World or sub-World class for sending all information to each computer, which could be relatively slow. Other than the military/map information we might be able to string along all the other information through the first 2-10 seconds of the turn, taking some of the BW problems away. Also when some players finish their turn earlier than others, the complete information for their sector (possibly excepting military units of civs outside their area of responsibility) can be sent as soon as they are done. Or at least shortly thereafter to allow for any last-minute actions by the AI.

BTW Manu you are right in all your suppositions so far as I can recall.

Tell me if i'm rambling... I'm at work, I Must be rambling

[This message has been edited by Mark_Everson (edited September 10, 1999).]

[manurein]

mca : well, u ask the right questions ...

The problem is as follows : we have no server, so wether each computer knows everything, or each computer knows only its part and relies on other computers to know the rest.

The first case is clearly unhandlable, since it would that we transmit every piece of data to every computer at the end of every turn ...

The second case seems to be the most workable, but it is clear, as I stated and as u confirm by ur questions, that we have to design efficient methods to reduce as much as possible the overhead. And this represents work, real work men!
The example I gave is just an illustartion. It was not really thought out, and I'm sure there are more efficient ways to do the same, based on the same concept.
The advantage of the request model is that no data is sent to a computer that recquires no data. Of course, if we also want a computer to reduce the amount of data it recquires each turn, we have to design a system able to wether forcast some of the datas that wont be needed (and thus that wont be recquired) or give each computer a limited global knowledge.
Here is for example an immediate method for reducing transfered data in the example I gave in the last post :
instead of sending each turn to each computer a list containing all the provinces where updated data is needed, one computer could only send the difference between the set of provinces that was used last turn, and the set of provinces that will be used next turn. This supposes that the provinces where any computer has a look on must be memorized by every computer.
Thus, the first turn each computer tells every other computer every provinces it can know what happens in. Then, the other computers will send it each turn updated datas about those provinces; as long as the computer does not know about other provinces, it wont transfer any list when it recquires the data. When it discovbers a new province in a given turn, it will request updated data to all computers, joining a list containing the new province it knows. Of course it works i both ways, a computer may tell other computers to remove a given province from list it must update.
As u see, what we have to do is to developp many mechanism like this one that will aloow to reduce bandwidth usage for every piece of data that is susceptible to be transfered.
As a general rule, using a "difference" algorithm seems to be a good technique (difference algorithm means that u dont transfer all the data at a given time, u just transfer the differences between the state of the data this turn and the state of the data next turn. This means that a whole set of data is transfered only once, in the initialization phase). Various flavors of this algorithm are widely used in compression algorithms, includind TIFF, fax, MPEG ...

One last word, concerning bandwidhty usage/low level protocol : depending on the protocol, the amount of useful data vs. non-useful , protocol related data is between 10 and 100 for 1. This means that u will reuce bandwith usage a lot more by reducieng the amount of data u send than by choosing the most economic low-level protocol.

[Mark_Everson]

I think Manu's estimate is right that sending the whole game information each turn is prohibitive. We have Much more information in Clash than in most games. However, a lot of it only changes relatively slowly. Perhaps we can do something like what I mentioned above, but only update things that have Really changed. Movement of military forces is probably the most important. After that a slow trickle of background data can pass between the computers to keep the rest of the world (the part a peer is not responsible for) relatively up to date. This is probably Good Enough.

The reason I think we need things to be somewhat accurate for the parts of the world that the player's comp doesn't handle is for things like panning out the map. If the player suddenly decides to look in more detail at some remote part of the map, that information should be available Now. Its too much information to wait for from another computer. However, the information needn't be all that accurate for that stuff anyway. So we don't need to really keep it up to date all that well.

[Glak.]

just stopped by thought I'd mention that the SC model seemed appropriate, except I modified the first part because clash has random stuff. Here are the basics:

All computers in the game generate a random number. Then they send the number to all the other computers. The numbers are added or whatever to generate the random seed for game creation (thus no one can cheat because you have no idea what numbers they will send you)

All players enter commands and so forth
All commands are sent to all other players, along with a random seed. Nothing besides the commands and the random number is sent, nothing at all. Note that the commands are not sent in a clump at the end of the turn. If they are sent as the player issues them then bandwidth cannot be a problem. It is inconcivible that a human can enter info into a computer faster than a computer can send it. A 14.4 modem would thus be more than sufficient. Time between turns would be .5-1.5 seconds plus processing time as if it were a single player game.

All commands are processed on each computer independantly. Since all computers have exactly the same information and random variables the result on each computer will be indentical. If it is not than the offending player is thrown from the game.

Advantages: low bandwidth and short time between turns, most types of cheats (such as those that give you free money) are impossible

Disadvantages: Cheats that reveal information will be easy. However most cheaters will make mistakes (trying to do something that the game won't allow, such as trying to attack an army out of range) which will cause them to be booted from the game.