Tweet
VERSION 2: The Myth of a Bugless Civ3
Since Markos hijacked the other thread (which was looking to generate some good discussion on an important topic), took away my Moderator title, and called me a troll, I thought that now that I am a free man it would be nice to post this again for those who care to discuss the topic. First, "the article."
________
Written by: Jeff Morris, Programmer / QA manager - Firaxis Games
Published: August 2, 1999
Born in the Los Angeles suburb of Santa Monica, Jeff Morris lived in San Francisco before moving to Austin to work at Origin Systems in June 1995. While at Origin, he worked on such games as ATF, Nato Fighters, USNF '97 and ATF Gold. After leaving Origin in February 1998, Morris moved on to help form Firaxis Games, where he worked on Sid Meier's Gettysburg!. Having been an avid computer gamer for over 15 years, Jeff led the Quality Assurance testing on Alpha Centauri, which involved coordinating professional and public testers. He also programmed the install module, the autorun menu and other miscellaneous portions of the title. So is there such a thing as the bugless game? Morris explores that question for the Adrenaline Vault in his guest editorial.
------------------------
One of the most frequent complaints made against modern computer games is that they are buggy. I don't care what title you think of, there is most likely a group of consumers out there evangelizing to the developers about the need for just one more patch. Try it. Visit the powerful Usenet search engine deja.com, search for any PC game that sold over 10,000 units and add the keyword BUG. I guarantee you'll find at least one heartfelt testimonial about how the customer was cheated by the "unfinished" product and a dire warning to any who might consider purchasing the title.
It's impossible to ignore the fact that games with significant bugs have been released. I strongly recommend that if you feel something you've bought is "unplayable," for whatever reason, you should return it. If the store won't take it back, try the publisher (raise a stink since that almost always works), and if not them, the developer themselves. Massive returns will have a far greater impact on future products than holding onto the title and venting on the Internet. The second option may feel better, but the developer gets to keep your money and, thus, has no real motivation to alter their behavior.
An interesting extrapolation of the perception that "most games released today are buggy" is the inverse statement that "some games released today aren't buggy." Even more interestingly, it seems what the public wants, if not demands, is a 100 percent bug free title. At first, this sounds absolutely plausible, something all developers should strive to achieve and all consumers should expect to get. Unfortunately, reality doesn't quite live up to this ideal. Even worse, it seems all parties concerned -- the developers, the consumers and even the machines -- actively combat any effort to achieve this goal. In short, on the PC platform as it currently stands, there can be no such thing as a bugless game.
Let's approach what at first seems the easiest to believe -- that computer diversity will never allow a bugless game to exist. This statement leads to some interesting questions about what qualifies as a "bug." Now, don't worry. I'm not going in the direction you might think. For the purposes of this discussion, let's say a bug is anything a player considers a bug. For instance, a hardware problem we've encountered on Sid Meier's Alpha Centauri (SMAC) relates to the fact that some customers never selected their monitor type, instead leaving it set to Standard VGA. SMAC, however, requires either an 800x600x8-bit or 1024x768x8-bit screen resolution, so the ceiling of 640x480x8-bit set by this configuration will prevent it from running. Now, is this a bug? Well, maybe.
The programmers could have anticipated this particular problem and displayed a helpful error message suggesting a solution. A great idea, but it is almost impossible to predict every possible incorrect configuration a system can experience (if you could even consider this example incorrectly configured). Even worse, as new hardware becomes available or additional features are added to existing hardware via driver updates, configuration problems that didn't exist during development and testing can create new "bugs." And this problem is only getting more pronounced. The great game testing challenge over the next few years is how to even begin approaching this massive diversity of not only hardware, but also the drivers that make it function. The standard "hardware lab" model of compatibility can't simulate it well, so they are wisely focusing on "off-the-shelf" systems and configurations. This at least gives the developers the knowledge that on a stock Gateway or Compaq, the game will perform as expected. But the innate customization a PC offers ensures it sure won't remain stock for long.
My point is this: The developer cannot realistically tell a customer they haven't experienced a "bug." Other games worked on the consumer's system, and they paid money for the new one to work on their system, but it doesn't work. It is a rare person faced with this prospect who will consider the possibility that his computer may be to blame. Technical support avenues of various flavors can solve the problem, but the fact of the matter is that the customer had problems, and in their minds and the minds of whoever they discusses it with, it will be portrayed as a bug.
Another reason I believe a bugless game will continue to elude reality on the PC is the simple fact that creating and selling computer entertainment is a business, first and foremost. Game developers draw their living wages from this industry and in almost every instance, their checks are cut by a company. And it is well known that companies have a persistent habit of striving for profitability and survival.
Now, I'll admit this is unfortunate and even go as far as saying it was closer to ideal before gaming became a multi-billion dollar industry. I truly wish game development were an altruistic hobby of millionaires, one in which they created vivid interactive entertainment for fans out of sheer generosity. It's a sad reality, though, that genre clones are cranked out because investors like safe bets. Hype machines are sometimes a more reliable method of nailing a sales forecast than the voodoo involved in producing a first class title. And, tragically, some products are rushed out the door because if they don't hit that final milestone, the company will tank and the game will never be released.
This financial angle can sometimes hit the hardest in QA. Requests for additional time by a title's testing staff may be completely justified, but equally impossible to entertain. Extra time required for development typically comes from the "QA padding" at the end of a project, while final deadlines remain unaltered. Unbelievable amounts of money can be spent on securing retail displays, magazine and Internet advertising, and duplication facilities, all of which must be done months ahead of time. Miss your date and not only do you lose the dough, but you have to scramble to secure even the most rudimentary of advertising real estate (read: even more expensive). Salaries are an exceptional way of burning through even a generous advance and a completion bonus can sometimes make the difference between an employee paying their rent or walking out the door, rarely leaving everything in a state of completion. Slip gold by a few weeks, even when it's to fix a critical problem, and the product may not be finished for many months, if ever.
These pressures aren't right or wrong, they're simply reality. If it comes down to a contest between a quasi-repeatable fatal or slipping by four months, laying off a third of your staff and kissing your Christmas bonus goodbye because you missed your window, I know the choice many developers would make. It's an easy choice for the consumer, but a heart wrenching one for those who got into this business to make cool computer games. There's a saying that a title can only be late for awhile, but it can suck forever.
My final and most radical point is that the online gaming community, which most frequently identifies and evangelizes bugs, have strong interests in games not being 100 percent bug free. Let's say a title avoided major hardware conflicts and is produced by a company with the fiscal resources of Midas. The offering is highly anticipated and has a roll out of a quarter of a million units and 20 active fan sites eagerly digesting every bit of news. In this situation, the final product is actually an ending, and those who spent the last year or two following it are faced with saying goodbye to the friends they've made online and moving on to another title.
Why? The reason is that, for a majority of the hardcore, Internet savvy gamers out there, playing a title is secondary to the community around which it forms. This usually gestates well before an offering is available in any playable form, with the seed typically being a news release or trade show unveiling. Fans of previous genre efforts have lots to discuss and even more to hypothesize about. How will it be different from previous titles? What worked and didn't before and why? Here's this crazy idea I had for a game once, what do you think? These vibrant social activities absolutely thrive in a lack of concrete information. Even though the upcoming product is a link that brings these souls together, it is often the least interesting thing to talk about.
As the product nears completion, this community often has existed for a year or more. Mores, legends and leaders (and even development staff pinch hitting as celebrities) are all fulfilling rewarding roles. A genuine Internet society begins to reach a feverish pitch as more and more information about the game is exposed. Finally, the day comes when it hits the shelves. Everyone ejects from their forums/newsgroups/fan pages and plays it for a few hours -- and then guess where they are? Back on the 'net discussing every little nuance. But then, except in rare cases, the fun is over. Unless it's buggy.
Even the best title can't sustain a dialogue as fascinating as that of this society, it being just a computer game. By focusing on negative aspects, real or fabricated, the customer can initiate a dialogue with the developers. While this is a potent and useful tool for improvement, the more gratifying use is to generate news, which in turn fuels the society. In the SMAC forums, there is no easier way to generate a million post thread that for a Firaxis employee to post. This interaction boosts a flagging supply of conversation and delivers what the customer has come to value much more than the game. The fan base as an entity has become something with which they strongly identify and to which they belong. Very few titles can compete with that.
As a heavy consumer of computer entertainment, I expect products to work flawlessly. I want every game for which I cough up 50 bucks the day it's released to install itself, teach me how to play it and then let me win. I expect it to seamlessly deal with whatever witches brew of drivers, hardware and applets I happen to have. As a QA veteran, I know that, regardless of how many 24-hour days or hundreds of thousands of dollars are invested in a game, someone, somewhere, at some point in time will find a "bug." It's inevitable.
This doesn't change a thing. Developers who want to stay in business need to deliver a product that works as advertised and satisfies the customer to the point they tell all their friends to buy it. The computer gaming business in that sense is like any other professional trade. Rushing "buggy" products out the door seriously damages that company's ability to stay viable, and since that is the primary goal of most companies, software or otherwise, it isn't as done as often as it's cited. The goal always remains the same -- to release a bugless game. But even should that occur, it still won't be safe from being labeled buggy.
[
________
Written by: Jeff Morris, Programmer / QA manager - Firaxis Games
Published: August 2, 1999
Born in the Los Angeles suburb of Santa Monica, Jeff Morris lived in San Francisco before moving to Austin to work at Origin Systems in June 1995. While at Origin, he worked on such games as ATF, Nato Fighters, USNF '97 and ATF Gold. After leaving Origin in February 1998, Morris moved on to help form Firaxis Games, where he worked on Sid Meier's Gettysburg!. Having been an avid computer gamer for over 15 years, Jeff led the Quality Assurance testing on Alpha Centauri, which involved coordinating professional and public testers. He also programmed the install module, the autorun menu and other miscellaneous portions of the title. So is there such a thing as the bugless game? Morris explores that question for the Adrenaline Vault in his guest editorial.
------------------------
One of the most frequent complaints made against modern computer games is that they are buggy. I don't care what title you think of, there is most likely a group of consumers out there evangelizing to the developers about the need for just one more patch. Try it. Visit the powerful Usenet search engine deja.com, search for any PC game that sold over 10,000 units and add the keyword BUG. I guarantee you'll find at least one heartfelt testimonial about how the customer was cheated by the "unfinished" product and a dire warning to any who might consider purchasing the title.
It's impossible to ignore the fact that games with significant bugs have been released. I strongly recommend that if you feel something you've bought is "unplayable," for whatever reason, you should return it. If the store won't take it back, try the publisher (raise a stink since that almost always works), and if not them, the developer themselves. Massive returns will have a far greater impact on future products than holding onto the title and venting on the Internet. The second option may feel better, but the developer gets to keep your money and, thus, has no real motivation to alter their behavior.
An interesting extrapolation of the perception that "most games released today are buggy" is the inverse statement that "some games released today aren't buggy." Even more interestingly, it seems what the public wants, if not demands, is a 100 percent bug free title. At first, this sounds absolutely plausible, something all developers should strive to achieve and all consumers should expect to get. Unfortunately, reality doesn't quite live up to this ideal. Even worse, it seems all parties concerned -- the developers, the consumers and even the machines -- actively combat any effort to achieve this goal. In short, on the PC platform as it currently stands, there can be no such thing as a bugless game.
Let's approach what at first seems the easiest to believe -- that computer diversity will never allow a bugless game to exist. This statement leads to some interesting questions about what qualifies as a "bug." Now, don't worry. I'm not going in the direction you might think. For the purposes of this discussion, let's say a bug is anything a player considers a bug. For instance, a hardware problem we've encountered on Sid Meier's Alpha Centauri (SMAC) relates to the fact that some customers never selected their monitor type, instead leaving it set to Standard VGA. SMAC, however, requires either an 800x600x8-bit or 1024x768x8-bit screen resolution, so the ceiling of 640x480x8-bit set by this configuration will prevent it from running. Now, is this a bug? Well, maybe.
The programmers could have anticipated this particular problem and displayed a helpful error message suggesting a solution. A great idea, but it is almost impossible to predict every possible incorrect configuration a system can experience (if you could even consider this example incorrectly configured). Even worse, as new hardware becomes available or additional features are added to existing hardware via driver updates, configuration problems that didn't exist during development and testing can create new "bugs." And this problem is only getting more pronounced. The great game testing challenge over the next few years is how to even begin approaching this massive diversity of not only hardware, but also the drivers that make it function. The standard "hardware lab" model of compatibility can't simulate it well, so they are wisely focusing on "off-the-shelf" systems and configurations. This at least gives the developers the knowledge that on a stock Gateway or Compaq, the game will perform as expected. But the innate customization a PC offers ensures it sure won't remain stock for long.
My point is this: The developer cannot realistically tell a customer they haven't experienced a "bug." Other games worked on the consumer's system, and they paid money for the new one to work on their system, but it doesn't work. It is a rare person faced with this prospect who will consider the possibility that his computer may be to blame. Technical support avenues of various flavors can solve the problem, but the fact of the matter is that the customer had problems, and in their minds and the minds of whoever they discusses it with, it will be portrayed as a bug.
Another reason I believe a bugless game will continue to elude reality on the PC is the simple fact that creating and selling computer entertainment is a business, first and foremost. Game developers draw their living wages from this industry and in almost every instance, their checks are cut by a company. And it is well known that companies have a persistent habit of striving for profitability and survival.
Now, I'll admit this is unfortunate and even go as far as saying it was closer to ideal before gaming became a multi-billion dollar industry. I truly wish game development were an altruistic hobby of millionaires, one in which they created vivid interactive entertainment for fans out of sheer generosity. It's a sad reality, though, that genre clones are cranked out because investors like safe bets. Hype machines are sometimes a more reliable method of nailing a sales forecast than the voodoo involved in producing a first class title. And, tragically, some products are rushed out the door because if they don't hit that final milestone, the company will tank and the game will never be released.
This financial angle can sometimes hit the hardest in QA. Requests for additional time by a title's testing staff may be completely justified, but equally impossible to entertain. Extra time required for development typically comes from the "QA padding" at the end of a project, while final deadlines remain unaltered. Unbelievable amounts of money can be spent on securing retail displays, magazine and Internet advertising, and duplication facilities, all of which must be done months ahead of time. Miss your date and not only do you lose the dough, but you have to scramble to secure even the most rudimentary of advertising real estate (read: even more expensive). Salaries are an exceptional way of burning through even a generous advance and a completion bonus can sometimes make the difference between an employee paying their rent or walking out the door, rarely leaving everything in a state of completion. Slip gold by a few weeks, even when it's to fix a critical problem, and the product may not be finished for many months, if ever.
These pressures aren't right or wrong, they're simply reality. If it comes down to a contest between a quasi-repeatable fatal or slipping by four months, laying off a third of your staff and kissing your Christmas bonus goodbye because you missed your window, I know the choice many developers would make. It's an easy choice for the consumer, but a heart wrenching one for those who got into this business to make cool computer games. There's a saying that a title can only be late for awhile, but it can suck forever.
My final and most radical point is that the online gaming community, which most frequently identifies and evangelizes bugs, have strong interests in games not being 100 percent bug free. Let's say a title avoided major hardware conflicts and is produced by a company with the fiscal resources of Midas. The offering is highly anticipated and has a roll out of a quarter of a million units and 20 active fan sites eagerly digesting every bit of news. In this situation, the final product is actually an ending, and those who spent the last year or two following it are faced with saying goodbye to the friends they've made online and moving on to another title.
Why? The reason is that, for a majority of the hardcore, Internet savvy gamers out there, playing a title is secondary to the community around which it forms. This usually gestates well before an offering is available in any playable form, with the seed typically being a news release or trade show unveiling. Fans of previous genre efforts have lots to discuss and even more to hypothesize about. How will it be different from previous titles? What worked and didn't before and why? Here's this crazy idea I had for a game once, what do you think? These vibrant social activities absolutely thrive in a lack of concrete information. Even though the upcoming product is a link that brings these souls together, it is often the least interesting thing to talk about.
As the product nears completion, this community often has existed for a year or more. Mores, legends and leaders (and even development staff pinch hitting as celebrities) are all fulfilling rewarding roles. A genuine Internet society begins to reach a feverish pitch as more and more information about the game is exposed. Finally, the day comes when it hits the shelves. Everyone ejects from their forums/newsgroups/fan pages and plays it for a few hours -- and then guess where they are? Back on the 'net discussing every little nuance. But then, except in rare cases, the fun is over. Unless it's buggy.
Even the best title can't sustain a dialogue as fascinating as that of this society, it being just a computer game. By focusing on negative aspects, real or fabricated, the customer can initiate a dialogue with the developers. While this is a potent and useful tool for improvement, the more gratifying use is to generate news, which in turn fuels the society. In the SMAC forums, there is no easier way to generate a million post thread that for a Firaxis employee to post. This interaction boosts a flagging supply of conversation and delivers what the customer has come to value much more than the game. The fan base as an entity has become something with which they strongly identify and to which they belong. Very few titles can compete with that.
As a heavy consumer of computer entertainment, I expect products to work flawlessly. I want every game for which I cough up 50 bucks the day it's released to install itself, teach me how to play it and then let me win. I expect it to seamlessly deal with whatever witches brew of drivers, hardware and applets I happen to have. As a QA veteran, I know that, regardless of how many 24-hour days or hundreds of thousands of dollars are invested in a game, someone, somewhere, at some point in time will find a "bug." It's inevitable.
This doesn't change a thing. Developers who want to stay in business need to deliver a product that works as advertised and satisfies the customer to the point they tell all their friends to buy it. The computer gaming business in that sense is like any other professional trade. Rushing "buggy" products out the door seriously damages that company's ability to stay viable, and since that is the primary goal of most companies, software or otherwise, it isn't as done as often as it's cited. The goal always remains the same -- to release a bugless game. But even should that occur, it still won't be safe from being labeled buggy.
[
Comment