Interview with head of quality assurance at Firaxis

[Bill3000]

Shush. I was just, um, doing stuff.

[VetLegion]

Just curious Boris, what are you basing your estimates of game complexity on?

[Zinegata]

Originally posted by Boris Godunov


And just what are you basing this on? It's been less than a month since Civ4 was released, and the first patch is coming very soon, and by all appearances it's going to be addressing a LOT of issues. You compare that to games that have been out for several years? That's ridiculous.

Precisely why I said that there is the promise of patches from Firaxis. They say they'll do it, but we're not sure how long it'll take them to release a patch nor do we know if they'll keep patching the game many years down the road. We don't know yet what their record will be a few months, and a few years down the road. It is thus a promise and not yet a fact.

Blizzard, on the other hand, is the gold standard when it comes to speed in releasing patches and in terms of maintaining support even years down the road. Diablo 2, for example, has been updated with patches for a longer period of time that Civilization III has been in existence!

In short, Blizzard has already achieved the gold standard. Whether or not Civ IV achieves it is still an open question. However, Civilization already starts in the hole thanks to the installation and performance problems: A good game can be played right out of the box. Quite a number of people are finding Civilization IV won't run out of the box. I myself and a friend have experienced this problem, and this is not something we had to do with many other titles (including Diablo, Starcraft, and Paradox Games). It doesn't matter whose fault it is. A good game should be able to run out of the box if you meet the system requirements written on the box. Violating this can compromise the trust between developer/publisher and customer. A class-A title like Civilization should deserve better QA than this.

In addition, a great game (from a technical standpoint), is a game that can run even if the customer placed the game in a condition that it isn't supposed to be in (i.e. running the game in a machine that is below the minimum specs). Starcraft deserves the honor of being a truly gold standard game as it can run in machines that are even slightly below the "minimum" specs of the game!

You will never piss off a customer if you can make your game run even in a lower spec machine (in fact, they'll be impressed by the team's diligence in making an optimal piece of software!). You will, however, piss of a customer who bought the game, had the recommended specs, and couldn't get the game to run. Even if it's not the fault directly of Firaxis or Take 2, as the owners of the product, it's their job to fix it and people should stop making excuses for them. Complexity of the product is not an excuse. It is a challenge they should have prepared for from the beginning.

EU 2 was released November 15, 2001. The first patch came out December 19, 2001.

Diablo 2 was released in early June 2000, and yes there was an immediate patch, but this was completed before the game even came out. The first patch responding to widespread user issues didn't come until mid-July.

Sure, but both games could run out of the box =). My friend, for example, didn't have to hunt through a dozen forums just to find out how to make his copy of Diablo or Starcraft or Europa Universalis run properly.

So I'd say Civ4 is still on track for meeting the above examples, especially considering it's a significantly more complex game than either of them.

What's ridiculous is saying that Civ is more complex than Europa Universalis or Starcraft, actually, at least from a technical standpoint. Diablo is definitely far simpler. But a real-time strategy game like Europa Universalis or Starcraft being less complex (from a technical standpoint) than Civilization? Hardly!

Real time strategy games are by necessity more complex to program because everything must occur simultaneously. Civilization may have more "playing pieces" (i.e. units) on the board at any given time compared to Starcraft, but it's a lot more difficult (and much more prone to crashing!) to keep track of units all moving around at the same time. Moreover, Civilization is definitely outdone by the complexity of Europa Universalis, which has hundreds of armies and fleets at the board at any given time, plus nearly a hundred different A.I. countries compared to under two dozen for a Civilization game.

In fact, I'm honestly disappointed at how under-optimized, rather than "complex" Civilization games are becoming. Think about it. Turns in Civilization III took abysmally long (especially before patches), and Civ IV turns in the modern era have fairly long lag times as well, to the point the game somtimes CTDs. Rise of Nations, on the other hand, is about as complex as an RTS can get, yet the A.I. can still make decisions instantly and on the fly without having to make the player sit through half a minute watching the screen. "Complexity" does not seem to be an issue here. The failure to optimize, is.

Oh, and another point why Blizzard retains the technical gold standard: Blizzard games rarely suffer from memory leaks or other sub-optimal performance problems. Blizzard games generally produce great graphics using specs lower than other comparable games. This is how one makes a quality product. Civilization IV may eventually meet this standard, but right now it's falling far short.

[CrONoS]

You're argument is fraudulous...

Take any computer chess games.... You'll see how much time a "not complex" games can take to calculate each turn because of the under-optimized chess software.
I maybe wrong, but if we can have smarter move by the a.i. and that's suppose, the programmer have to give more time to the a.i. to calculate what is the best move.... I dont have any trouble with that!!!

Maybe the game is slow... maybe the game doesn't play smooth(in the later age)... I give you the credit for that. And we'll see pretty soon if the game can be optimise.
For me, if any patch can't make the game smoother... I'll buy a new piece of hardware in the next year. But maybe, the game cannot be faster.... It's just strange reviewer doesn't talk about the speed issue.... Maybe this is just because we expected the game to run with the same hardware that run Civ3!!

Originally posted by Zinegata


Precisely why I said that there is the promise of patches from Firaxis. They say they'll do it, but we're not sure how long it'll take them to release a patch nor do we know if they'll keep patching the game many years down the road. We don't know yet what their record will be a few months, and a few years down the road. It is thus a promise and not yet a fact.

Blizzard, on the other hand, is the gold standard when it comes to speed in releasing patches and in terms of maintaining support even years down the road. Diablo 2, for example, has been updated with patches for a longer period of time that Civilization III has been in existence!

In short, Blizzard has already achieved the gold standard. Whether or not Civ IV achieves it is still an open question. However, Civilization already starts in the hole thanks to the installation and performance problems: A good game can be played right out of the box. Quite a number of people are finding Civilization IV won't run out of the box. I myself and a friend have experienced this problem, and this is not something we had to do with many other titles (including Diablo, Starcraft, and Paradox Games). It doesn't matter whose fault it is. A good game should be able to run out of the box if you meet the system requirements written on the box. Violating this can compromise the trust between developer/publisher and customer. A class-A title like Civilization should deserve better QA than this.

In addition, a great game (from a technical standpoint), is a game that can run even if the customer placed the game in a condition that it isn't supposed to be in (i.e. running the game in a machine that is below the minimum specs). Starcraft deserves the honor of being a truly gold standard game as it can run in machines that are even slightly below the "minimum" specs of the game!

You will never piss off a customer if you can make your game run even in a lower spec machine (in fact, they'll be impressed by the team's diligence in making an optimal piece of software!). You will, however, piss of a customer who bought the game, had the recommended specs, and couldn't get the game to run. Even if it's not the fault directly of Firaxis or Take 2, as the owners of the product, it's their job to fix it and people should stop making excuses for them. Complexity of the product is not an excuse. It is a challenge they should have prepared for from the beginning.



Sure, but both games could run out of the box =). My friend, for example, didn't have to hunt through a dozen forums just to find out how to make his copy of Diablo or Starcraft or Europa Universalis run properly.



What's ridiculous is saying that Civ is more complex than Europa Universalis or Starcraft, actually, at least from a technical standpoint. Diablo is definitely far simpler. But a real-time strategy game like Europa Universalis or Starcraft being less complex (from a technical standpoint) than Civilization? Hardly!

Real time strategy games are by necessity more complex to program because everything must occur simultaneously. Civilization may have more "playing pieces" (i.e. units) on the board at any given time compared to Starcraft, but it's a lot more difficult (and much more prone to crashing!) to keep track of units all moving around at the same time. Moreover, Civilization is definitely outdone by the complexity of Europa Universalis, which has hundreds of armies and fleets at the board at any given time, plus nearly a hundred different A.I. countries compared to under two dozen for a Civilization game.

In fact, I'm honestly disappointed at how under-optimized, rather than "complex" Civilization games are becoming. Think about it. Turns in Civilization III took abysmally long (especially before patches), and Civ IV turns in the modern era have fairly long lag times as well, to the point the game somtimes CTDs. Rise of Nations, on the other hand, is about as complex as an RTS can get, yet the A.I. can still make decisions instantly and on the fly without having to make the player sit through half a minute watching the screen. "Complexity" does not seem to be an issue here. The failure to optimize, is.

Oh, and another point why Blizzard retains the technical gold standard: Blizzard games rarely suffer from memory leaks or other sub-optimal performance problems. Blizzard games generally produce great graphics using specs lower than other comparable games. This is how one makes a quality product. Civilization IV may eventually meet this standard, but right now it's falling far short.

[Zinegata]

Originally posted by cronos_qc
You're argument is fraudulous...

Hardly. It's merely the statement of some pretty uncomfortable facts. =)

Take any computer chess games.... You'll see how much time a "not complex" games can take to calculate each turn because of the under-optimized chess software.

Chess is hardly "not complex". There are over a trillion ways of playing the first ten moves alone =).

Now, going back to my original point: Remember that the purpose of a computer game A.I. is not to become smart. It is to become a challenge.

The reason why chess needs such a monstrous A.I. is because you can't cheat in a chess game. A chess-playing A.I. can't suddenly have pawns that can do two moves at once, or queens that are invincible to attack. Becoming smarter is the only way a chess-playing A.I. can be more challenging. It also demonstrates how monstrously expensive making an A.I. smarter is, in terms of the time, money, and computing power needed to make it practical.

In comparison, a Civ designer can simply make his A.I. cheat. Civilization is not bound by the time-honored rules of chess. In fact, in Civilization it is tradition to give cheats to the A.I., especially at higher difficulty levels!

Cheating is a useful tool for optimizing a piece of gaming software. In fact, Civ IV uses it too! =)

I maybe wrong, but if we can have smarter move by the a.i. and that's suppose, the programmer have to give more time to the a.i. to calculate what is the best move.... I dont have any trouble with that!!!

And here's the thing: Does the Civ IV A.I. know how to feint?

A feint is the most basic attack outside of the old tried and tested "gather all units and send them all to the nearest city" attack algorithm. Basically, a feint is sending some forces to one city, while the majority to another. The majority takes the objective, the rest die to distract the enemy.

If the A.I. takes longer to calculate a turn so that it can do some really new stuff (such as feinting), then such delays are justifiable. However, if you're taking longer to calculate stuff yet still use the same algorithm anyway (as in, gather units, and send them all against the nearest city), then it's a waste. Why wait two minutes if you're gonna get about the exact same result if you'd just waited for one?

Maybe the game is slow... maybe the game doesn't play smooth(in the later age)... I give you the credit for that. And we'll see pretty soon if the game can be optimise.

Given that Civ III's speed was improved considerably after released thanks to patches, we already have a demonstration of how Firaxis released a clearly un-optimized product. All indications are that this Civ was in many ways rushed as well, and it has those annoying memory leaks. Thus, I'm fairly sure that Civ IV can be optimized.

The question to ask really, is how much time and effort Firaxis is willing to spend in this endeavour (and why they didn't optimize it before release in the first place).

And no, memory leaks are a developer problem, not a hardware problem like the ATI card issue (which is still the developer/publisher's responsibility)

For me, if any patch can't make the game smoother... I'll buy a new piece of hardware in the next year. But maybe, the game cannot be faster.... It's just strange reviewer doesn't talk about the speed issue.... Maybe this is just because we expected the game to run with the same hardware that run Civ3!!

Hardly. I have a comp that hits the "minimum" end of Civ's requirements. Others have reported being able to play the game with min specs, but only on tiny maps. Fortunately, I had the sense to borrow the game first before buying it.

However, now that you say it, yes, Civ IV ought to run with specs for Civ III. Really, none of the gameplay improvements, save the graphics, make the game exponentially more complex than it was in Civ III. In fact, given that Civ III was in many ways a less complex game than Civ II, the requirements of Civ III itself were already surprisingly high. Many point to the graphics issue, but most player tests have shown that it's not the graphics eating up game performance (video cards that work handle the graphics handily) - it's actually the game engine itself eating up a lot of RAM (and to a large extent this is thanks to memory leaks).

Civ II runs reasonably fast with 8 MB of RAM. Civ III didn't run reasonably well with a system with 8 times the memory. Is Civ III more than eight times more complex than Civ II? I don't think so (at least not until they broke the 7 Civ barrier).

Move on to Civ IV, and we have a game that needs nearly eight times more RAM again. However, is the game complexity again eight times more? And wasn't one of the objectives of Civ IV to reduce the complexity/micro of the game by killing such hoggish tactics like ICS?

Not exactly what I'd call progress, quite frankly.

[LzPrst]

@ zinegata

Word!


civ4, as civ3 are minor improvements to the game that came before. yet they (create vacuum) when it comes to performance and they arent getting much more interesting. my civ4 ran out of the box. my flatmates' versions didnt. 1/3 in my experience have no issues. now of course thats not the case totally, but it does create a feeling that the game isnt all that.

blizzard and paradox are the golden examples of the gaming industry. I got WoW a few days after it released in norway. it worked out of the box and I have had only negligible sporadic issues, and I played nothing else for 6 months. eventually I tired because despite it being a great game, it wasnt complicated enough to hold my interest. but I gladly paid blizzard tons of money. and heres the gist. I am willing to pay good money for a good game. the only reason I'm not screaming at civ is because I bought it cheap. 17£ is an overcomeable loss. and I do consider the money spent a loss. I got a game that held my interest for less than two weeks.

games today are made less and less complicated in order to appeal to bigger and bigger audiences. WoW was fairly simple and fun and thus appealed to a lot of players, but I lost interest as did my friends eventually.
it is the games that are actually challenging that keep me playing. civ isnt.

firaxis is NOT a good example for the gaming industry. civ3 came out bad. civ4 came out poor. patches shouldnt be a requirement to make the games run. released games should not be full of incompletes and rotten design. modders shouldnt be able to make a better game than what was released in the first place.

[LordShiva]

Blizzard

[kittenOFchaos]

World of Warcraft

And whilst we are about it...

Civ IV

Commiserations to those with technical problems.

[CrONoS]

Deleted

[upstart69]

i think these ppl who are calling firaxis and civ a lost cause need to do just what they are advocating. MOVE ON. if you are so utterly dissapointed and feel that firaxis has utterly let you down...then why the hell are you hanging around msg boards dedicated to the products you loathe? are you people masochists? if you feel that the patch is an unfulfilled promise that wont ever materialize....why agonize over it?
this is what i dont understand. the ppl who support the game, the 'fan boys' have a reason to hang around the boards. the people who havent made up their minds, they also have reason to be here, waiting to see if their purchase will be made good on. but the people who thrash the game for everything its worth? why are you people still here? sell your games on ebay and move on. why are you wasting hours of your life dedicated to a cause that makes you upset and foaming at the mouth?
anyways ive been burned by products in the past too. just two examples are the san andreas dvd booklet thing and the star wars dvd set collecting dust on my shelf. each of which i spent good money on. and yet i dont feel the need to bash these products and their makers with every fiber of my being.
but thats just me, i guess..?

[LordShiva]

You miss the point completely.

Nobody here loathes the civ franchise. Nobody here loathes Firaxis. Everybody here was hoping that Firaxis would blow us away with Civ4, and the optimistic among us hope they still will.

If you love the game so much, why are you wasting your time telling other people not to waste their time, instead of just playing the game?

[Zinegata]

Originally posted by LordShiva
You miss the point completely.

Nobody here loathes the civ franchise. Nobody here loathes Firaxis. Everybody here was hoping that Firaxis would blow us away with Civ4, and the optimistic among us hope they still will.

Exactly.

Being a fan is more than just accepting a game in spite of its faults.

[Zinegata]

Originally posted by kittenOFchaos
World of Warcraft

And whilst we are about it...

Civ IV

Commiserations to those with technical problems.

Civilization fans, in recent years, have debated whether the new Civs (III, IV, CTP, etc) are good or bad.

In contrast, the debate among most unbiased MMOG fans (those with no particular loyalty to a game) on World of Warcraft is whether the game is good or great.

And WoW seems to be heading towards selling more copies than all of the other Civs combined (if it hasn't already).

So, sure, give WoW the thumbs down if it makes you feel better, but I don't think anyone can dispute that it is one very good and successful MMOG =).

[LordShiva]

RPGs

But Starcraft, now