Only 25 Beta testers

[Grumbold]

Yes I agree vmxa1. They're presumably not looking for these people to try out all strategies to see if they are too weak or powerful, or to exhaustively test every button press or option, but to give their opinion on the playability and fun factor. Its amazing how many dire interfaces or glaring omissions make it to release simply because the developers have lived with it for so long that they have adapted to it. Civ3 being released with no grouping commands or sentry mode springs to mind.

[Xentax]

Originally posted by Nadexander
Actually my opinion comes from the fact that I am a software QA (quality assurance) engineer. Most developers and many engineers who do qa (general engineers hired into qa as opposed to those with qa specialties) are NOT very good at software QA. In fact they are notoriously BAD at it. Now if this is what programming professionals do I simply _cringe_ at what a small random sampling of the game playing public might do. Really.

I'm a software engineer, so I've got 2 cents to add on this issue (naturally). I agree with you, to the extent that I'll say that "your mileage will vary" when it comes to using anyone not specifically trained/experienced in doing QA.

Some people we've had doing system testing, who did NOT come from a QA, or even an engineering background, did a fantastic job testing -- they were thorough, they tried things the implementor didn't think of, etc.

So, random testers, IMHO, aren't "that much" more of a shot in the dark than 25 hand-picked people would be. It's also more likely to get a good cross-section of skill and interest levels. The game won't do well if only diehard fans enjoy it. It should *make* diehard fans of people new to the game, or even new to the genre.

The testers will need direction, to be sure -- they need to know that they should try *literally* anything they can think of to break the code. But you can't always use professionals to test, because it won't just be professionals using the final product. Face it, at the end of the day, people who use computers for a living reach a level of familiarity, at which they stop trying the kinds of things that people new to, or less comfortable with, computers are liable to try.

Things like "Gosh, all these extra files, I bet they don't do anything useful. I'll just delete them." Just how well will MOO3 "fail" when some of it's critical files aren't found? Only one way to find out from our side of the fence...

[Nowhere Man]

What I wonder is with such a limited beta test & only two months left (meaning in practice what has to hit the street on 26 oct. will need to completed and ready for packaging minimum 2 weeks before) how much are they going to catch (or rather not catch).

It's better than Firaxis I guess but still not by much. Getting the reports back - reproducing the errors - sorting out if it's a general or system specific thing (and weeding out all the garbage reports) locating the relevant code, correcting the issue and then testing how it now works out. And then so few testers of which many will be sending in no reports whatsoever.

Actually it sounds more like a PR thing than a real beta-test.

Rather squeezy to say the least & quite minimalistic on top of that meaning the actual beta testers will probably be the first round of buyers and that we will have series of patches following.

Well hopefully we will have a series of patches and hopefully they will continue to patch it until it is pretty much bug-free.

Anyway I really hope we get a great game - though I must say this beta thing here has not exacly lifted my confidence.

Lets hope they just got it right.

[system370]

Can I ask who you think make the best Moo3 Beta Testers?

Game Players? Programmers ? QA testers ?

What about people who have played Moo2 for 6 years and can name 20 bugs that still remain in Moo2? I bet none of the 25 chosen could detail the outstanding bugs in Moo2.

Just last summer a Fellow Moo2 player discovered a bug with Android workers - A new bug or an old one re-discovered I don't know, but IMO , it's those sort of people that should be doing the Beta testing - proven bug finders with a passion for the game.

[danminster]

I think the only thing required to be a decent beta tester is to be a critical "thinker". So long as you are trying to push the game with different approaches, etc., I think you should be able to catch different bugs that pop up. This is an exercise that is close to being able to proofread a manuscript.

I don't believe that it matters if you are a game player, programmer, or a QA tester, just as long as you know what you're doing.

I also think having too much passion for the game can have an adverse impact on one's ability to search through the game for errors.

[kallisti_dk]

Remember, we don't know how many internal testers Infogrames has, and how long they've been testing the game.

[Xentax]

Originally posted by system370
Can I ask who you think make the best Moo3 Beta Testers?

Game Players? Programmers ? QA testers ?

What about people who have played Moo2 for 6 years and can name 20 bugs that still remain in Moo2? I bet none of the 25 chosen could detail the outstanding bugs in Moo2.

Just last summer a Fellow Moo2 player discovered a bug with Android workers - A new bug or an old one re-discovered I don't know, but IMO , it's those sort of people that should be doing the Beta testing - proven bug finders with a passion for the game.

Infogrames has some subset of 100 internal QA people doing the serious bug-hunting. And yes, I'd take professionals, paid to find bugs, over veteran players. I'm not saying they haven't done a thorough job, but they've also had lots of time, and have built upon the play experience of thousands of other players. Finding obscure, hard-to-reproduce bugs in *pre-release* code is a different sort of challenge.

[Nadexander]

NOTHING reveals bugs like a public release. No matter how thourogh the QA team is 1 day in the hands of the end users will no doubt reveal bugs or major design flaws. Game companies know this. What varies is what exactly they decide to do about it.
When I paid $50 for Warcraft 3 I recieved a game that was in a highly refined stage of development, both technically and in terms of game play. I am confident (thanks to my experience with starcraft) that blizzard will continue to refine the game and provide me with updates.
In the case of Civ3, I had the pleasure of paying $70 to be a Pre-Alpha tester. I then declared that certain features that were an advertised part of the game when I pre-ordered (scenario builder, multiplayer) were instead going to be sold as a separate Add-on. Needless to say my confidence in Infogrammes willingness to stand behind the games it releases is less than rock solid.

[Xentax]

Agreed; for computer games, nothing finds bugs and other issues faster than multiplying the user base and spectrum of target hardware configurations by a few orders of magnitude.

I agree that Blizzard has made a good practice of patching (and often) until they feel like they've got it right.

However, I'm a bit more hopeful that QSI and IG will do the right thing when it comes to patches. I'd blame poor patching responsiveness for Civ3 on Firaxis more than IG, personally. QSI has certainly stated that they're committed to getting it right, not just out the door; we'll see how that holds up once the game goes public.

[Grumbold]

I dunno. There is obviously an incentive for both IG and Firaxis to make a new expansion or game to earn more revenue, but I think Firaxis would be quite happy to keep improving the quality of Civ III as long as IG kept paying them to do so. After all, its far more in their interest to have a game that is still being played and spoken of highly years after its release. We can all remember who developed many classic games, but who remembers or cares who published them?

[Xentax]

Well, people remember Sid Meier games as such.

It's one thing to be working on an expansion that adds major features (that were originally to be in the initial release...); it's another to keep patching in a timely fashion in the mean time. They *have* patched Civ3 a few times, but I've heard too many complain that they were slow in coming, or insufficient, etc.

My real beef with Civ3 is that it was *too* far off in the first release; it should never have gone gold with corruption that badly broken, for example. That sort of thing deserves an almost immediate patch, and it was ... weeks, maybe even a month or more, before the *first* patch to deal with corruption came out...

[vmxa1]

I give them some slack as it requires some thought and lots of testing to make those kinds of tweaks. If they tossed it out in 4 weeks, it may have been worse.

[Ozymandous]

Originally posted by Xentax
Well, people remember Sid Meier games as such.

It's one thing to be working on an expansion that adds major features (that were originally to be in the initial release...); it's another to keep patching in a timely fashion in the mean time. They *have* patched Civ3 a few times, but I've heard too many complain that they were slow in coming, or insufficient, etc.

If you've heard too many complain because Civ III wasn't what they wanted then you've probably spent too much time on Apolyton. There were bugs in the initial release to be sure, air superiority as a major one for example, but issues like corruption, etc, were not bugs, IMHO.

My real beef with Civ3 is that it was *too* far off in the first release; it should never have gone gold with corruption that badly broken, for example. That sort of thing deserves an almost immediate patch, and it was ... weeks, maybe even a month or more, before the *first* patch to deal with corruption came out...

A "bug" is when something is broken in the game outside of the players ability to adapt. With air superiority, no matter what the player did it wouldn't work. With corruption, because you can't build 200 cities half way around the game world, doesn't mean it was "broken" merely that people wanted the game to be as the old versions had been, not have to change their play style.

What do you get when you have people who refuse to change how they do things because 'that's how it's always been done'? The general Civ III board here at Apolyton! lol!

When a game company is faced with the dilemna of a feature that works as it is supposed to but some (annoying and very vocal) people don't like, then you should expect a little delay in the company trying to figure out how to 'tweak' the feature so it still does what they want but also quiet's down some of the biggest 'complainers'.

Look at it this way... You may not have liked Civ III as it was initially shipped, but what shows that a game was shipped too early, having a patch come out 2-4 weeks after it hits the shelves or having a patch available as soon as the game hits the market like Blizzard has done in the past?

Oh, by the way, speaking of Blizzard, I guess none of the people who sing their praises ever played Diablo2 and went through all the see-saw cycles when the company released patch after patch that completely re-worked the game mechanics every time and rendered old characters as useless? You want an example of a company that releases stuff fast and then has to constantly fix their mistakes, look no farther than Diablo2 and Blizzard.

Let's just hope that PtW doesn't completely break and unbalance Civ III as LoD did with Diablo 2.

[vmxa1]

I agree corruption was not broken, I think it did what they intended. That some found it unplatable, is not to say it was broken.

[player1]

exactly