Announcement

Collapse
No announcement yet.

My Letter to Brian About A Public Alpha/Beta: Your Thoughts...

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • My Letter to Brian About A Public Alpha/Beta: Your Thoughts...

    May 22, 1999

    Brian,

    When Sid released Civ1, it changed the computer gaming world forever. Now, 2.5 million fans later, the fate of this legacy rests in your hands. But if one thing has become clear to me after pouring over hundreds of posts for the Civ3 list, it is that these 2.5 million fans don’t want Civ2+. They want a new era to dawn on this game.

    I’d like to share an idea that I am convinced will take Civ to a higher level and ensure its future:

    A PUBLIC ALPHA/BETA.

    The advantages to this strategy are numerous and profound:

    ** AN HONEST VIEW OF YOUR PRODUCT:

    If I may be so bold, I believe the reason that you and your team were so surprised at people’s criticism of SMAC’s graphics and theme is that SMAC was primarily an in-house product. Simply, you and your team became used to your own ideas. You became so comfortable with the graphics and technology that you could no longer see the game as an average gamer would. A public alpha ensures this won’t happen again. By putting a Civ3 alpha into gamers’ hands, the relevance and the timeliness of the feedback loop skyrockets exponentially, and there are no surprises for anybody.

    ** QUALITY ASSURANCE:

    This point cannot be stressed enough. This point on its own merits should form the basis of a winning public relations campaign. SMAC had problems. Some of them embarrassing. A public alpha/beta program puts the game on machines very early, at a time when crashes and major bugs are to be expected. This system would allow you to test the wild world of system configurations on a much larger scale than you could ever hope to do at your headquarters. And, I must remind you, Jeff’s insistence that SMAC’s problems were often due to system configurations and not the program itself has, to this day, left a bitter taste in many people’s mouths. No matter how true his statements are, a public alpha/beta would virtually eliminate this need to make excuses.

    ** GENERATING FAN INTEREST:

    You should not rest on Civ’s laurels. As I have said, fans are growing tired of the current Civ formula. As I’m sure you are aware, a number of people simply no longer have faith that any company is capable of taking the risks it will take to move this game to the next level. A few fans are writing their own version as we speak. A public alpha/beta will help squash the counter-momentum. It will give all fans tangible proof that Firaxis doesn’t want to publish ITS game but the GAMERS’ game. The positive hype this will generate among fans is hard to overestimate.

    ** A SHOW OF COMPETENCE AND DARING:

    Few companies have the stomach to put their work out in such an early stage. This is, essentially, vanity and/or a lack of faith. It’s akin to an artist refusing to show his early sketches to the patron. True, if the finished piece is accepted, he’s a hero. If it’s not, he’s looking for work somewhere else. Firaxis is a young company and should set a new paradigm in this industry. Putting your work out early into gamers’ hands shows immeasurable confidence in your own ability to take suggestions. It also shows Firaxis’ commitment to making games for gamers. You simply have no better way of demonstrating these qualities than through a public alpha/beta program. Perhaps "Furby's Mad World of Lawn Bowling" doesn't have fans who care enough to work through rough versions of the game. But Civ, we must admit, is no Furby.

    ** THIS WILL MAKE MONEY:

    Finally, a public alpha/beta program will make you a great deal of money. First, each applicant for the alpha program will pay $10 for the disk (plus shipping and handling). The same for the beta stage. People who participated in the alpha and beta stages qualify to buy the finished game for $10. TOTAL SPENT ON CIV3: $30—a fair price, and you don’t have to wait months to collect it. People who participate in only the alpha or beta stage qualify to buy the finished game for $20. TOTAL SPENT ON CIV3: $30. Again, you get this money without having to wait weeks for these gamers to maybe buy the game. You also get the money from the alpha and beta stages without having to go through the very expensive process of publishing the game. Those who do not participate, of course, pay the full $50 price or wait for the price to naturally fall. But the momentum and positive buzz of the alpha/beta program will no doubt be so positive, and so many “Well, I doubt Firaxis can do this right” questions will have been laid to rest, that the average Civ fan will rush to buy the game, confident that gamer input and quality assurance have been your priority from day one.

    I would suggest opening a password-access web site just for those who participate in the public alpha/beta program. A List room should be set up—similar to the one I’m running on Apolyton but limited to people participating in the program. The feedback on the Alpha becomes the roadmap for the Beta. The feedback on the Beta becomes the roadmap for the final version. The final version becomes the killer game that takes Firaxis and Civ to a whole new level of expertise and admiration.

    Of course, you can do this in-house and risk putting all your eggs in one basket. Or you can put this for the most part into the gamers’ hands—and watch the money and guaranteed success come in.

    I recently read an article in Business Week in which Bill Gates’ right-hand man—I can’t recall his name—talked about the most important thing he learned as he tries to restructure Microsoft and free it from its bureaucracy and bad public image. He said:

    “I have become a leader of leaders.”

    He has learned to let go enough so that his clenched fist becomes an open hand. The people below him are now empowered to make decisions and contribute. Brian, I encourage you to do the same.

    Please let me know your thoughts on this,

    Yin
    (K. McLaughlin)

    ------------------
    CIV3 DEVELOPMENT LIST COORDINATOR

    **(un)Officially Making Lists for Firaxis Since SMAC Enhancement 3!**


    [This message has been edited by yin26 (edited May 22, 1999).]
    I've been on these boards for a long time and I still don't know what to think when it comes to you -- FrantzX, December 21, 2001

    "Yin": Your friendly, neighborhood negative cosmic force.

  • #2
    'Over the top' at the end comes to mind, but hey, sometimes that's good.

    I should say, though, that I don't think a public beta would be the best idea for Firaxis, for piracy reasons. I would certainly be in favor of one for all the reasons you mentioned (as well as for just because I want to play it) but I don't think it would be good for the game or the company.

    But that's certainly no reason not to ask . . . I may just be too pessimistic for my own good . . .
    "In the beginning was the Word. Then came the ******* word processor." -Dan Simmons, Hyperion

    Comment


    • #3
      I love this idea! But...

      1. Wouldn't this seriously screw with the retail/distribution model that game companies go through? I don't think Hasbro would really sign off on something like this. I know that getting more money upfront is good, but not if it messes with their way of business. I don't really think this is Firaxis's call to make. (I know Microsoft charges for their betas, but don't the testers still have to pay normal price for the final?)

      2. Expect thousands of people to be in on this. I haven't done any beta testing, but I assume it gets more confusing when you add on more and more testers. There will be a great deluge of save game files, people throwing out error codes, and a big headache with getting all of the versions straight.
      The product would have to be at the stage of Ultima Online when UO went to beta. (Was anyone here a tester for that?)

      Sorry to be so negative here. If there is a way to make it work, I'm all for it. If not, maybe Brian can give us a public preview of some of the graphics, the interface (with clickable buttons), and details of the design.

      Comment


      • #4
        Bell,

        LOL Yeah, I had some Top Gun music going in the background toward the end! Well, maybe it will inspire the man to let us gamers in on a larger scale. As for the secrets, if they've done the right job from alpha to beta, nobody will doubt that Firaxis is the place for this game. It's just too damn hard to simply rip off these ideas. Game balance in a game like this takes years to learn. Anyway, thanks for the commentary.

        ------------------
        CIV3 DEVELOPMENT LIST COORDINATOR

        **(un)Officially Making Lists for Firaxis Since SMAC Enhancement 3!**
        I've been on these boards for a long time and I still don't know what to think when it comes to you -- FrantzX, December 21, 2001

        "Yin": Your friendly, neighborhood negative cosmic force.

        Comment


        • #5
          russellw,

          Good points. Firaxis would still have to go through all the publishing channels, of course, for the final version. And if anybody were to buy the game, it's no doubt going to be the alpha and beta testers (assuming Firaxis has been actively taking their feedback into consideration). So it's a very good trade off to give them a discount on the final game. The only way Firaxis could lose on this is if they aren't really committed to the player's ideas and just push on with their own stuff. In which case, all the publicity backfires in a BIG way. So they would have to do it right, there's no doubt about that.
          I've been on these boards for a long time and I still don't know what to think when it comes to you -- FrantzX, December 21, 2001

          "Yin": Your friendly, neighborhood negative cosmic force.

          Comment


          • #6
            Good letter.

            But wasn't it written by the same guy who resented the fact that gamers had to pay to beta test?

            Ted If I volunteer to pay and see a sneek preview, that's a whole other story! *Yin*

            Shoot for the stars, maybe we can end up hitting the moon!

            Civ3 n ... Ted S.


            [This message has been edited by yin26 (edited May 22, 1999).]

            Comment


            • #7
              I don't want to rain on anybody's parade, but there are significant problems with this idea.

              Firaxis design process is to have "working prototypes" that iteratively evolve into a game. Now, that is usually a pretty bad ideas from a Software Engineering perspective, which means there are likely to be lots of bugs in their alpha and beta phases. So, instead of having a core group of people who are familiar with: a bug reporting procedure, debugging methodology, and the general state of game development you have any schmoe off the street who can give you ten bucks. This will be more trouble than it is worth for the programmers at Firaxis, because they will get tons of conflicting and confusing reports, and will need to wade through them all, working out which problems are real problems, which are due to innappropriate use, which are known bugs, etc., etc., etc. I've read on a website somewhere that the person who managed the SMAC beta thought it was worthwhile, but found that the biggest problem was that the non-in-house beta testers were unfamiliar with the process. What does this mean from a quality perspective? It means that people are spending their time debugging bug reports instead of debugging software.

              As has already been said, this also undercuts the traditional retail distribution chain. No development house can expect to survive if it undersells both its publisher and the retail market. It can't work from a business perspective. Firaxis does not have the financial or production capacity to be its own publisher (and I doubt that they want to be their own publisher), therefore they can not engage in any shenanigans.

              For the SMAC beta-test, they actively recruited and interviewed people that they thought would be good beta-testers. I remember when they posted their initial call on the SMAC forums, and there were plenty of "dammit, I'm not qualified" posts. Remember, a beta-tester isn't somebody who gets a sneak preview, they are somebody who is working to make the game better.

              I personally dislike the software industry's attitude toward bug-finding. In general, bug-finding for them is an afterthought, done by "lowly" individuals. I personally think that's stupid. The solution to buggy software is a commitment to bug-finding up front, though, not a whole bunch of people pounding on things at the back-end.

              I do not think that a wide public beta test will happen. They will probably repeat what they did for SMAC, which is to gather a small number of people that they think is manageable. There are good reasons for them to do it this way, from their perspective. There are good reasons for them to totally overhaul the way they look at bug-finding as a part of the process of software design, but that is another issue for Firaxis to deal with internally.


              ------------------
              CIV3-THE MASTER LIST-TECHNOLOGY "THREAD MASTER"
              "Can you debate an issue without distorting my statements and the english language?"
              -- berzerker, August 12, 1999 04:17 AM, EDT, in Libertarianism and Coercion

              Comment


              • #8
                In support of Yin's letter, I suggest everyone read the paper: <a target="_blank" href="http://www.tuxedo.org/~esr/writings/cathedral-bazaar/">The Cathedral and the Bazaar</a>.

                Actually I have been thinking about petitioning for Open Source Civ3. But I haven't come up with a business model that could work. Yin, maybe you can after reading that paper.

                ------------------
                The radical invents the views. When he has worn them out the conservative adopts them.
                - Mark Twain


                [This message has been edited by Rong (edited May 22, 1999).]
                The radical invents the views. When he has worn them out the conservative adopts them.
                - Mark Twain

                Comment


                • #9
                  Rong: I think that "OpenSource Civ" is asking them to give away the store and the candy at the same time. "Open Source AI" would be more than enough to put Civ3 on top of the heap for the forseeable future.

                  Yin: I dont think that an "open" alpha / beta testing procedure would provide useful info. Or, more accurately said, the volume of data would be overwhelming. They need to have a *larger* group of alpha & beta testers, but not an unlimited group. As for pay... well your plan changes their cash flow to the positive, so I'm thinking they'll like that. And it *does* lower the testers eventual cost, so there is an reward to the tester.

                  Comment


                  • #10
                    Perhaps an open-alpha test is a bit excessive - so how about only an open-beta test.

                    No, wait, it'd be better the other way around. Have an open-alpha, then a closed-beta. This way, Firaxis will have more time to wade though all the suggestions of the alpha test, PLUS, have a good pool of candidates to choose from for the beta stage. I know that no matter what, I would not want to be Firaxis in the middle of an open-beta test. It'd just be too much.
                    The honorary duty of a human being
                    is to love, I am human and nothing
                    human can be alien to me.

                    -Maya Angelou

                    Comment


                    • #11
                      I've been a tester for both alpha and beta software. An open-public model wouldn't necessarily work for the following reasons. A lot of "alpha" software is just that, the first build of the program. Instead of graphics, lots of numbers, crappy BMP graphics usually in 16 colours, most of the options aren't working, and quite often, it doesn't even run at all!

                      An "alpha" build usually only concentrates on getting the main game engine up and running. I was on an alpha test of a civ-type game (which got scrapped) and there was no AI, only YOU were on the world. All you could build were settlers and militia, no researching, no diplomacy, nothing. All the alpha did was click through the years and move settlers and militia around. No barbarians, no improvements, no wonders, no gold. In fact the years weren't "years" but just turn numbers. There were no "screens" like city status or anything. You couldn't do anything. Would 2.5 million (Yin's figure) accept paying $10 for an alpha that you can't do anything in? AND then have to fill in wads of bug report sheets?

                      But in essence, an open-beta may be successful. Any programmer knows that the best way to bug-check a piece of software is give it to the "dumbest" computer user. By dumbest, I don't mean it literally, but more like computer illiterate. In the public, what better mix do you have? You have (like me) highly qualified computer technicians, software developers, people who don't even know how to use a mouse properlly, quite simply, the entire mix (not meaning any offence ). Also, a beta is usually with most (if not all) the options turned on.

                      So while IMO an open-alpha test would NOT work, an open-beta test may actually provide the developers with good feedback. Plus, in reality it'll shut those naggers who like with CTP, ordered Activision to publish the game before it was complete, cuz they have something to get their hands dirty with.

                      Comment


                      • #12
                        Everyone is making good comments for their opinions. I'm glad you started this post Yin.

                        Firaxis whatever type of testing scheme you use, I have one request: listen to your testers and make sure they have no GRIPES before releasing the game. I remember quite a few post from some of the beta testers for CTP remarking how a lot of their suggestions, concerns and bugs they found were not addressed at all. You don't have to implement any of their suggestions, but you BETTER address all BUGS that they find.

                        Public testers, if it appears Firaxis is giving you guys the run a round, I implore you to hurt them where counts, by posting and posting to forums such as this.

                        Comment


                        • #13
                          I think that a public beta would give Firaxis too much information to be useful. Imagine several hundred thousand people all submitting lenghty e-mails about the game.

                          How about a slightly more limited beta test? Such as just, say, all the thread masters for the suggestion list?

                          [This message has been edited by evil conquerer (edited May 24, 1999).]
                          ------------
                          evil conquerer
                          alphac@flashmail.com
                          Co-webmaster of The Arrival
                          http://ac.strategy-gaming.com
                          "War is the last refuge of the incompetent."
                          Salvor Hardin, "Foundation" by Isaac Asimov

                          -----BEGIN GEEK CODE BLOCK-----
                          Version: 3.12
                          GAT/CS/M/TW d- s:-- a---- C++++ UL&gt;++++ P+&gt;++ L&gt;+++ E W+++&gt;$ N+ o? K- w+ O---- M-- V? PS+ PE Y+ PGP- t++&gt;+++ 5 X- R tv-- b+++&gt;++++ DI+ D G&gt;++ e--&gt;++++ h! !r y?
                          ------END GEEK CODE BLOCK------

                          Wondering what the heck that was? Check out http://www.geekcode.com/.

                          Comment


                          • #14
                            Having been on both ends of the s/w testing stick, I would have to recommend against open alpha testing and allow only limited public beta testing. An organization can only absorb only so much data, and some of it will be facetious or incomplete, e.g. "I don't like the graphics" or "The AI sucks" or "I could have spent a better day fishing than playing this game" - all gripes lacking the necessary details needed to address the concerns. These are all followed by the later "told-ya-so"s. "See, I told them the AI sucked and they didn't fix it!"

                            Comment


                            • #15
                              IMHO we need an public early beta to get feeling about the end product. After we can send back our opinion about the game, so Firaxis can use our feedback to impove the game play. They can send the later betas (which include already the changes) to the beta testers to rid off all the bugs. This part must be closed (Not everybody can or wants to write a correct bug report).

                              Blade Runner
                              Blade

                              Comment

                              Working...
                              X