Beta test

[Sir Og]

Is there any news on a beta test like they did with Civ4?

[CanuckSoldier]

Nothing official yet, no.

CS

[Sir Og]

I doubt that they can make a good game without extensive testing. Especially with so many important changes being made.
How long before the release of civ 4 did they start the beta test? They don't have much time left if they want to do this thing right and release this year.

[Krill]

Firaxis rarely do open beta tests. They usually hand pick community members that they feel would be productive testers. They don;t just ask Joe Blogs to help them out.

[wodan11]

Truth, and a shame. It results with inbred testing in a closed community that already thinks along the same lines. Guaranteed to produce substandard testing and less than comprehensive evaluation.

[Krill]

Erm, I'd refute that, but I can't really offer evidence without getting sued.

[wodan11]

The proof's in the pudding. Every version of Civ has had glaring game balance issues and major poor game design decisions.

Anyway your statement "they hand pick community members" doesn't mesh with a refute of my statement. I was pretty harsh but that doesn't mean there isn't an element of truth there.

Unless your rebuttal would have been (if you had offered it) that the testers don't, in fact, think along the same lines. Which may be true, and I don't doubt there are some heated differences of opinion.

I suppose my point is that a test sample which is statistically small, when compared to the size of the actual target market, is, by definition, insufficient testing. The very term "hand picked" defines the testing as inadequate.

[Solver]

Doesn't seem like you entirely understand the testing process.

First of all, any testing is going to be done with a group that's very small compared to the game's market. Civ games are among games that sell really well. When you have a million players, then any test group is small compared to that, not even half a percent, of course.

Next, there's no single test group. Testing is done in several groups, always. It includes internal testing by the developers, testing done by the publisher and also the group that's picked from the community. Developers will also typically try and do some testing with people who haven't played a similar game before, to see how easy or hard it seems to learn.

Having been a member of the testing group for Civ4 and expansions, I can guarantee you that testers don't think along the same lines. Those testers were all hand-picked and united by a passion for Civ but they still differed greatly. Some had a singleplayer-only mindset, others were hardcore MP players, others yet were in-between. There were builders and warmongers, there were average players and top-notch strategy experts. During discussions within that group, the disagreements were often very significant.

Also, what you see as a non-tester is biased in the end. You see balance issues. Yes, they are there, but what you see in the end is only part of it. For every balance issue you see in the final game, there were several at least as bad that were caught and changed in testing. Believe me, this is one thing that external QA did very effectively for Civ4. I should not be going into details but some of the issues that got caught by testers really outshadow the ones that make it to release.

[CanuckSoldier]

Solver is completely correct, although all testers are dedicated to see Civ games be the best they can be, their opinions on what makes that happen vary considerably. And we and the game are all better for that discussion in the end.

CS

[Dale]

Solver is completely correct, although all testers are dedicated to see Civ games be the best they can be, their opinions on what makes that happen vary considerably. And we and the game are all better for that discussion in the end.

CS

Agreed. Nearly all of the new concepts in Civ4 resulted from a mass-debate between us the community testers, the developers and 2k-games participants. Some of those threads got to 30-40 pages long.

May I remind the testers of Civ4 about the lengths Sirian would go to, to get his point across?

[vulture]

I'm still sulking about the Civ IV beta test. I was invited to join the test team, signed the NDA and sent it back, and pretty much the day the disk arrived a whole bunch of things started cropping up one after the other (including adding a new daughter to the family) that meant I suddenly had no time at all to play civ for what turned out to be the next three or more years. Only really finding the time to play regularly again in the last 6 months or so.

So I joined the beta team, and then completely blew the chance to contribute to even the smallest degree

[Diadem]

Erm, I'd refute that, but I can't really offer evidence without getting sued.

So is firaxis still looking for beta testers?

[Krill]

Wouldn't surprise if they are, but it's going to be silent. Most likely need to be a high visibility poster in this community and/or CFC, or maybe a few of the more varied and twisted sites (ie RB). You need to be contributing to the community, and know what you are talking about game mechanics wise.

Looking at the back of the CIV manual, they seem to want less than a hundred beta testers...even if they have upped the number to help stress test, I'm doubtful they want much more than 150. Not great odds of getting the job.

[Diadem]

Well I've been inactive for a while, but I have been a staff member here at Apolyton. And I've betatested an expansion before

I understand it's silent. I'm just wondering if there's somewhere I can apply

[DaShi]

They've said that there won't be a beta test because they've found in the past that people involved are just too negative.