1: Object Serialization: Communicating through sockets and java.io.Object(Input|Output)Streams. Simple, but lots of typecasts and manual typechecks are needed, AFAIK, plus a bit of bandwidth overhead. I never used this.
It looks something like objectStream.writeObject(someObject);

2: Java Remote Method Invokation (RMI). Transparently invoking methods over the network, with automatic object serialization. Elegant and Real Easy to use, but costs a bit more bandwidth overhead.
It looks like any other method invokation: someReferenceToAnObjectOnAnotherMachine.someMethod(someParameters);

3: CORBA. Much the same as RMI, but is language independent. Perhaps less bandwith expensive than RMI but harder to use.

4: Writing a custom object communication protocol, using sockets and ByteArray streams or something similar in nature. Very Risky, IMHO, but can probably be less bandwith expensive than any of the above, if done right. I never tried to do this.

I estimate that the bandwidth overhead of any of the above methods is less than something like whatever.write(socket,"SET POP ProvinceA 1200");, but of course, experiments need to be done to be sure.

I suggest to use the Observer pattern, which is just like the Listeners and Events in the AWT and Swing.

quote:

---

(from Gamma, e.a.: Design Patterns)

Intent:
Define a one-to-many dependency between objects, so that when one object changes state, all its dependents are notified and updated automatically.

---

The idea is to have one "authoritative" model (e.g. an object of class GlobalMap ), where all changes take place, and that sends info on the changes to interested parties (e.g. PlayerMap objects, that show the player specific maps on the client machines). A PlayerMap only has the info that the player can actually see on the screen.

For example, when a player has moved an exploring unit and uncovers some new tiles of the map, the GlobalMap is told to update the authoritative data structure (perhaps world.moveTF(tf, from, to);). The moveTF will update the player's map info, and, after noting the change in some internal data structure, it sends a tf.getCiv().getPlayerMap().updateMap(tile) (or somethiing like that) request to the client for each of the newly discovered tiles, and also the two other tiles that changed: the tile the unit moved from and the one it moved to (perhaps even more, if there is some fog of war)

Continuing the example, the GlobalMap also checks if other players should be notified of the move. If the unit moved into or out of a tile that is seen by one of the other players, then that player's PlayerMap also gets an update or two: anotherPlayerMap.updateMap(tileThatTheUnitMovedInto);
anotherPlayerMap.updateMap(tileThatTheUnitMovedOutOf);

Writing an object to a stream is really easy,
but finding out what to do with it on the other end might take a bit more coding effort. The receiver reads some object, but has to find out whether it is a World, a Treaty, a Battle, a Message, or whatever type we may choose to send across the network, cast it to the correct type, and then decide exactly what to do with it. So there's only one typecast per transaction, but there are a lot of different typecasts in the receiving code.

But as I said, I never really worked with object streams, so there may be some really elegant way to do it that I haven't thought of.

Ok, I was wrong, I haven't even read anything on networking for java yet, so objects it will be. I will make a very precise model for multiplayer by the weekend after this (sep 18). I need to learn about java networking first, I will do this in a couple days(might even make a partial model for multiplayer). I will try and make the first workable source for multiplayer object by the end of this month.

Anyway, I have a few remarks :
- I fully support the no-server option, but IMO it will pose some problems we will have to solve : as we all know, the main limitation and trouble source in MP is the bandwidth. IMO, the low-level method or protocol used to send datas is not the main issue in this regard. The main issue is the amount of datas u send.
So if you want to have an efficient MP code, u first have to define very carefully which datas u send (which obviously wont be all datas) and when u send them. For this purpose, using a server is very effective, cause it can have a global vision of the game and decide accurately which data needs to be sent and which data does not.
If we dont have a server, we have to work to find some effective methods to limit the amount of data we send, knowing that each computer has a very limited vision of the global world.
- I may be wrong, but I think I remember that turns in MP will be simultaneous; and also that army movements will be drawn during the turn and implemented at the end of the turn, showing the results next turn. If this is the case, this will mean that all data exchanges will occur between the end of a turn and the beginning of the next, except for chat and diplomatic exchanges.
- Once again I may be wrong, but it seems to me that one effective method to reduce unneeded data transfers is to use a request model. With this, each computer will tell the others what it needs when it needs it.
If what is above is true (I'm not sure ...), this means that for example in the case of map revelation, each turn a computer will request each other to tell him what movements have occured in a given set a provinces (those provinces this computer knows). The other computers will send an answer, where there will be datas only for the provinces in this set where things have changed since the last turn, and nothing for the other provinces.
All this probably means that we would have a central messenging system for each computer (and no global central messaging system) that would get requests from every part and every model of the game, and would process them toward the other computers, filtering unneeded and redondant data, adressing to the right computer or broadcasting when nessary etc. It could prove efficient to reduce data transfers, and, once again, IMO its a more important issue than the low level transfer protocol.
One last note : if we choose a solution resembling that, we might have to design a high level protocol for the requests and the answers.

In your model, will each computer know all game data or will it be partitioned in some way, so that each computer is responsible for just part of the data?

manurein:

Will each computer have to keep enough knowledge of the other computers to decide for itself which computer to request, or does it need to request it from all computers in turn in order to find the computer that has the data?

How will it know which data it needs to update? Will it have to poll all the other computers for info on e.g. which tiles have changed since last turn, or is there another way?

If Manurein is right in that the choice of low-level protocol won't affect bandwidth consumption much, and that the high-level protocol is much more important, then we should go for the low-level protocol that has the least coding overhead (definitely RMI, IMHO).

I think that every computer Must have at least a Near-Current copy of the whole game available. I talk about what near-current means below.

Each comp would have a chunk of the map and associated civs that it is Responsible for in addition to the rest of the info. Each peer will know what the mapping is between computers responsible for them and map areas & civs. To the extent that most civs will stick to their own areas, the main difficulty is how to re-assemble the area information so everyone starts the turn (or near the start of the turn) with the same data.

My hope is that we can use RMI/request brokering for diplomacy, merchant trade and such. (You wouldn't need brokered requests for map info since each system already will have a current map shortly after turn start) Then there would be a class that covers the movement of military units (which needs to get there before combat can be cranked). Possibly another like this for map changes. Finally, a World or sub-World class for sending all information to each computer, which could be relatively slow. Other than the military/map information we might be able to string along all the other information through the first 2-10 seconds of the turn, taking some of the BW problems away. Also when some players finish their turn earlier than others, the complete information for their sector (possibly excepting military units of civs outside their area of responsibility) can be sent as soon as they are done. Or at least shortly thereafter to allow for any last-minute actions by the AI.

BTW Manu you are right in all your suppositions so far as I can recall.

The problem is as follows : we have no server, so wether each computer knows everything, or each computer knows only its part and relies on other computers to know the rest.

The first case is clearly unhandlable, since it would that we transmit every piece of data to every computer at the end of every turn ...

The second case seems to be the most workable, but it is clear, as I stated and as u confirm by ur questions, that we have to design efficient methods to reduce as much as possible the overhead. And this represents work, real work men!
The example I gave is just an illustartion. It was not really thought out, and I'm sure there are more efficient ways to do the same, based on the same concept.
The advantage of the request model is that no data is sent to a computer that recquires no data. Of course, if we also want a computer to reduce the amount of data it recquires each turn, we have to design a system able to wether forcast some of the datas that wont be needed (and thus that wont be recquired) or give each computer a limited global knowledge.
Here is for example an immediate method for reducing transfered data in the example I gave in the last post :
instead of sending each turn to each computer a list containing all the provinces where updated data is needed, one computer could only send the difference between the set of provinces that was used last turn, and the set of provinces that will be used next turn. This supposes that the provinces where any computer has a look on must be memorized by every computer.
Thus, the first turn each computer tells every other computer every provinces it can know what happens in. Then, the other computers will send it each turn updated datas about those provinces; as long as the computer does not know about other provinces, it wont transfer any list when it recquires the data. When it discovbers a new province in a given turn, it will request updated data to all computers, joining a list containing the new province it knows. Of course it works i both ways, a computer may tell other computers to remove a given province from list it must update.
As u see, what we have to do is to developp many mechanism like this one that will aloow to reduce bandwidth usage for every piece of data that is susceptible to be transfered.
As a general rule, using a "difference" algorithm seems to be a good technique (difference algorithm means that u dont transfer all the data at a given time, u just transfer the differences between the state of the data this turn and the state of the data next turn. This means that a whole set of data is transfered only once, in the initialization phase). Various flavors of this algorithm are widely used in compression algorithms, includind TIFF, fax, MPEG ...

All computers in the game generate a random number. Then they send the number to all the other computers. The numbers are added or whatever to generate the random seed for game creation (thus no one can cheat because you have no idea what numbers they will send you)

All players enter commands and so forth
All commands are sent to all other players, along with a random seed. Nothing besides the commands and the random number is sent, nothing at all. Note that the commands are not sent in a clump at the end of the turn. If they are sent as the player issues them then bandwidth cannot be a problem. It is inconcivible that a human can enter info into a computer faster than a computer can send it. A 14.4 modem would thus be more than sufficient. Time between turns would be .5-1.5 seconds plus processing time as if it were a single player game.

All commands are processed on each computer independantly. Since all computers have exactly the same information and random variables the result on each computer will be indentical. If it is not than the offending player is thrown from the game.

Advantages: low bandwidth and short time between turns, most types of cheats (such as those that give you free money) are impossible

Good ideas! So, using this method each computer would independantly keep the state of the entire game, based on the commands sent by other players, correct? That would allow all players to take turns simultaneously? Definitely better than forcing players to wait for others to play but will this method introduce problems with syncronization? It seems to me that if commands from several players are processed in different orders by each computer the possibility exists for each computers world image to get out of sync. Perhaps there is some way to force all computers to process commands in the same order?

I had an idea that was similar, though not as well developed as yours, but I'll throw it out and see what people think.

At the start of the game everyone starts out with an identical world, perhaps generated from the combined random seed you mention. After that only changes made to the world need to be transmitted to other players. And not ALL changes need to be sent, only those that CAN be known to other players.

I'm right in the middle of coding and can't write long or I'll lose critical track of what I'm doing . so this will be v. brief

Cool idea but...

You need to consider that:
1) we would like to use all the computers for AI
2) all the AI processes will move at different speeds on different computers. So starting with the same seed will not guarantee everything will stay in synch. However with some mods this might be practical.

If the users start a multiplayer game probably the best to collect the system information out from the players machine. After the game can start different AI threads on different machines (the most time consuming on the most powerfull machine). When the system in "idle" status than the system can send the synchronizing messages around the machines of the players.

As for the AI's getting out of sync, just make a limit of how much 'thinking' they can do, the only bad thing is it would take advantage of super fast processors, but you have enough confidence in your AI right mark?

Well I'm off, might come back to message board tommorrow, but I am busy getting linux set up with everything including Java(going to do my programming in Linux).

Please give us at least an outline before you launch into coding, or we could have a real train wreck! I think the actions-sent model is Very good. It fits in quite nicely with the military movement system too. However we need to be sure all the other models are compatible with it too. I've seen things that looked really good up front before that were shown with a little discussion and thought to be pretty lousy... That said this one looks really appealing, Thanks Glak

The actions sent to all peers model does not in any way preclude using All the processing power for the AI. Why do you think it does? It actually supports remote processing very well.

Anyway, why don't you post your vision for what you're going to do with some specifics, and we'll give the group at least a few days to talk it over and make sure all the duke's models are on board. I know you want to get going, so we'll just give a few day comment period, and if it still looks good, cut loose on it .

In Starcraft (where I first figured out how this model works) there are 15 turns per game second (1gsec=1 real sec on speed setting "normal"). If I tell my marine to shoot at your tank first the command passes through my interface. Since your tank is a legal target (I can see your tank) the command is approved by the interface. This is just to make things convinient for the player, this step isn't really needed but it makes the game much better. Let's just say I give this command on turn 10,000 and that we are playing on extra high latency (1.0 second delay). The command is labeled with the time and sent to all computers in the game. It does not reach all computers at the same time. For example it gets to my computer very quickly because it didn't have to go anywhere. It gets to computers at other times. As long as it reaches all the computers within the allotted latency time everything works out, if it doesn't then the game lags to compensate. If people notice the game laggin then they just need to set the latency higher to get rid of the problem. Anyway so the command gets to each player at a different time. This is not a problem however because each command is labeled with the turn that it was issued on. So if my command gets to me at 10,000 and you at 10,005 (a third of a second later) and to laggy lou at 10,013 everything is still ok. All computers wait until 10,015 before executing the command. At turn 10,015 they execute all commands issued on turn 10,000. The commands can even come out of order from various turns without incident. Actually instead of sending individual turns it sends a UDP packet with all the turn's commands bundled together. If on turn 10,015 it has not received the packet from 10,000 the game will wait until it gets it. All players wait too but I am not sure how that works since I have never progammed anything.

Your second paragraph brings up another issue. I think that all computers should always know exactly how things are at all times. Eventually they will have to know. I think that it is much easier to transmit all changes than to update things as need be.

Example: you have a forest and I have seen it before but I don't know anything about its current status. You sent a guy to burn it. (I know this doesn't happen in clash, just an example). You give him the burn forest: all command. He goes about his merry way and destroys your forest. So now you have a whole bunch of burnt forest squares. Later I send someone to check it out. How does my computer suddenly get information about the situation? Either you send it the state of the forest square by square or you just tell it that you used the burn forest commmand. I think that the first case would take more bandwidth, especially if you are sending information on complex things. On the other hand sending the burn command is simpler and can be sent at the time that the player enters it into the interface. Since players will probably be consistent in terms of data entry rate over the course of the game commands will then be sent at a constant rate. If you send information as it is needed it will come in clumps (and you will have to send more I think) which seems inefficient, plus you have to figure out which infomation is needed in the first place. Wow that was long

Mark:

1) You could divide the AI players up by processor strength and then they would play simultaneously with the humans players that they are leeching computer power from. In this way. So say my computer has 3 units of power, Joe has 5 and rick has 6 and there are 10 players in the game. So we add up the numbers and get 14 computer units, divide by 10 and get 1.4 per player. I have 3 so I have to share my computer with one AI guy. (2.8 is close to 3.0), Joe shares with 3 (5.6) and rick shares with 3 (5.6).

The computer would think simultaneously with the human player host. They would send out command packets together etc.. Of course if the human player things slower the AI would divide its commands up in such a way as to minimize bandwidth.

So while you would use all computers to process the AI they would not all work together on the same projects.

This way the AI players become more like human players in terms of how they play the game which I think is a good thing.

2)
For random number generation I guess I don't have any idea how it works. So I'll just propose that it be done a different way.

With every command packet sent out (send at least 10 per second) you could send a random number, if the packet is small you could probably send a bigger number. At the end of the turn if that player hasn't sent enough digits then he has to send the rest all at once. Now the numbers are all added together in order to make some giant super number, like 645983745927301247 but way longer, like more digits than seems reasonable (I assume that this is well within the computer's capabilities). Now each time a random number is needed if can just go grab a few digits off of the super number (except the first since it can't be a 0, so you just drop it). So if I need a random number from 1-100 I just grab the first two numbers 4 and 5 and get 45. Actually instead of using digits you'd probably want to use bits and measure that way but whatever. The only problem that I can forsee is if you need a number from like 1 to 3, but it can't be four. Well I guess you could just reroll (grab more numbers) if you get a 4. If the super number runs out just loop the current number or use the one from last time or something like that. So basically nothing is done using a random number function, you sort of make your own.

Dont worry Mark, it will be impossible for me to program for probably another week.

I'll try to explain it better and if this makes sense then you can probably go back and read the other one for the details.

Basically there are two network models: the peer to peer and the client server. The server model is used in games where there are a lot of people, like quake or everquest. You cannot be fair to all of the people and still get good performance. So instead of keeping the game perfectly synchonized they just accept commands as they come in.

Say we are playing everquest and we both want to shoot arrows at a gnoll. I (using mouse or keyboard) tell my computer that I want to shoot, you do that same thing. Then once my computer knows what I want to do it tells the server. The server lets me shoot as soon as it receives the command (and according to game specific factors). You do the same thing and you do it independantly. The server also lets you shoot as soon as possible. So even if we click the button at exactly the same time it is unlikely that we will shoot at the same time.

While that is the best model to use for games that don't need to be perfectly fair it is not the best for other games. Let's say we turned Starcraft into a turn based game for a moment, and we make a little change to how the server works. Ok so now we both enter commands to our various units. Well let's just say that my commands get to the server first. Now instead of letting them take effect right away now it waits until all players have entered their commands for the turn. Once it receives everyone's command they are all processed simultaneously.

So basically we have a functioning game of SC, though it is slower than normal because each turn takes a minute or so instead of 1/15th of a second.

Now let's make it realtime (yes I know clash will not be realtime but I want to finish the example).

Since it generally takes .5 seconds (or more) for commands to get back and forth from the server to the player we have to introduce something called latency. Since it makes the example easiest to understand we will set the latency to 1.0 seconds (15 game turns on speed setting normal).

Now the game is still turn based however commands are no longer processed on the turn you sent them. Instead turns are held by the server until the proper time, then they are issued. In this example commands are held until 15 turns after they were issued by the client not received by the server. When the commands get to the server isn't important or even kept track of.

So say we take the following actions of the following turns:

t1: I do nothing, you do nothing, the server receives no commands
t2: you issue command A, I do nothing, the server receives no commands (it is far to early for it to get your command, even if you have a great ping)
t3: I issue command B, nothing fo you and server
t4-6: no one does anything
t7: we don't do anything, the server receives command B from me. It does not execute it.
t8-11: nothing
t12: the server receives command A from you.
t13-16: nothing
t17: it is now 15 turns (1.0 seconds) after you entered command A, command A is executed.
t18: command B issued.

Notice how your command was executed before mine, even though mine actually got to the server first. Having a better computer/connection does not help you.

However we are still using a server. Why? Ok now what if we made your computer the server. (you and I still play normally, just the server is using your computer) Does the outcome change? No, since all that matters is when the commands are entered. We could have also made my computer the server with no change to the outcome. If both computers can act as the server with identical results than is a server needed? No, just make both computers think that they are the server and one of the clients, with a second client.

However this only works when cause and effects situations are clearly defined such that both computers given the same input will produce the same output. (no randomness at all). However there are work arounds (because SC does have a token amount of randomness), such as the one that I suggested.

Now how would this work in clash? well first of all it isn't real time so that means you don't have to deal with latency and you probably don't have to use UDP packets either. Also you will want commands to be executed on the turn they are issued, instead of .5-1.0 seconds later. To keep the bandwidth down I suggest breaking the turn up into subturns but only for the purpose of sending information evenly, it will have no effect on the game.

So now we are playing SC with the clash model (back to turn based, no server)

t1a: I issue command A, you issue commands C and B (you are faster with the mouse)
t1b: I issue D, E, F you: G, H, the AI: (who is using my computer to think) X, Y, Z

t1c: me: JK you: LMN AI: TUV

t1 resolution phase: ABCDEFGHJKLTUVXYZ all executed simultaneously.

The commands were sent as they were issued but took effect all at the same time. The AI finished thinking t1b but since it had six commands it divided them equally to minimize bandwidth

At this point turn two begins

Well I hope that cleared everything up for you, and if it didn't then I guess my D in english is justified.

well here it is, the first rough model sketch for the MP object, this lists all the commands for the MP object....

--Multiplayer object's public functions--

-int joinhost(string[])
This command joins the 'string[]' game lobby
error messages = 1=succesful -1=cannot connect
example: mes=joinhost("217.29.174.95"); or mes=joinhost("www.url.com");

-int makehost()
This command simply makes your computer host
1=succesful -1=error making server

-int connected()
This simply returns -1 if the computer is not connected, 1 if it is in
a lobby, or 2 if it is in game play

-int setnick(string[])
This sets the nickname of your computer, this is basically for chat but
other things too.

-int talk(string[]) talk(string[],string[])
The first broadcasts string[] to all players, the second sends a individual
a message and no one else

-string[] gettalk()
the return value of string is "time nick private/broadcast|message"
example: "5000 Osiris broadcast|hello everyone"
I use a | character instead of a space so
if you type something like the following it will work..." spaces behind the text"

-int launchgame()
IF you are the host of the game then you can run this function to
launch the connection over to gameplay mode.

-int sendcommand(string[],string[],string[])
This is a function for any command this player has made OR the AI's on
that computer, string[] = player string[] = category string[] = command

-string getcommand(string[])
this gets the command in a big messy string the string[] = category
so if you want to get a command from government then you do this....
string command;
string=getcommand("government");
The string also has a time stamp on it, so the string would look something
like this...
"5000 military unit100 move south"
all the commands the computer gets is safed in a big buffer, so after
all the players finish's there turns then you can get all the commands.

-int infoendturn(string[])
this returns -1 if string[] player has not ended there turn, returning
1 means the player has ended there turn and is waiting.

-int endturn(string[])
This is to end your turn OR any AI players turn, string[] = player
as usual -1 means error 1 means succesful

Looks good to me... unfortunately I have no idea what I'm doing ;-) I'm trying to think about the existing game models and any changes we need to make now with multiplayer in mind. Unfortunately things are really busy at work and It'll be the weekend before I'll get any chance at all. Soon I will start a thread for the dukes on multiplay effects on their models.

Cya,

===Inside workings of multiplayer(without source code )===

????????????????????????????
| |
| |
Clashnet-------Public functions
| |
| |
| |
Connection |
| | |
| | |
| | |
Read* PlayerRead* |
| | |
[7555] [0,1,2,3,4,5,6,7]

* These class's are used for running threads,
the Read listens on Port 7555, and the PlayerRead access's players sockets 0-7(1-8 for non-progs). Clashnet is the main class, and all the public functions are there, and they also access the 0-7 sockets for writing.

I wish I could start writin this stuff but need a compiler ;

Have you done a web search looking for free trial versions of IDE products? I know there's an old version of visual cafe up for grabs, but it probably doesn't have the most recent jdk in it.

Sun is giving away free beer in form of their
Java Workshop 3.0. It's not the most elegant IDE around, and doesn't include RAD GUI tools (you won't need that for network code, I think ), but it's useful. I use it sometimes when I don't have Emacs around.

Try it out, or find some free IDE you like better, there are several around, as Mark says.