Declaring War on Reviewers: One company did it...should Firaxis?

[yin26]

So the question is: Should Firaxis target to whom they give review copies? This is what Shrapnel has done (and, of course, is being criticized for by looking like somebody only wanting good reviews...they say: We only want to send copies to sites that give 'fair' reviews).

[Frugal_Gourmet]

To answer the last question...

Should Firaxis give copies to only those sure to give favorable reviews? Probably not. I'm sure the bottom line is they'll do whatever they think will make them the most money. And since Civ III is a high profile, larger budget game, they almost certainly want and need space in the more commonly read gaming magazines and sites out there. This means sending out lots and lots of preview copies to reviewers.

And I'll bet Firaxis *thinks* they'll get good reviews from almost any source anyway.

As far as should MP be included in the review... I say absolutely. If it the reviewer thinks MP would make for an interesting feature and misses it, he should include this is the review. After all, reviews are nothing more than official published opinions.

[yin26]

Here's a letter one guy sent to them criticizing this 'war.'

I don't like the idea of you guys at Shrapnel selecting who gets review articles. Although I know it's not there, it stinks of cronyism, and a playground attitude of "If you don't want to play my way, than I'm taking my marbles home." I understand you reason for limiting the review list, and I agree. Some sites are dedicated to a different player market than others. Instead of selecting which sites to keep based on who gives you good reviews, why not do a user poll. Personally, I basically only check reviews off of GamesDomain, a site that often gives your games glowing reviews. Although I discovered you originally off of Malfador's site directly. I'd bet that lots of other game players limit themselves to two or three sites that also understand the different nature of the Shrapnel Games market. So why not find out who is popular with your players?

Response:

Another intelligent letter! And a fair question. First, I'll comment on your comments:

I don't like the idea of you guys at Shrapnel selecting who gets review articles.

All I can say is "Tough!" We pay for the games that go out to each e-zine / magazine / reviewer to the tune of $20 -$25 per copy. Sometimes I think the reviewers forget this fact. I would say that if you don't like it, please feel free to buy additional products and send them to any reviewers you wish. Of course, the reviewers, be it persons or 'zines are also free to buy copies for review purposes.

And yes, our attitude is just as you stated - we are taking our marbles and going home. No harm in that, in fact, in the military you need to know when to fight and when to withdraw. You can't win every battle and when you press a bad situation, you lose.

Instead of selecting which sites to keep based on who gives you good reviews...

Hold it right there. We are not selecting sites on who give us good reviews - we are selecting sites on who gives us fair reviews.

why not do a user poll.

That would be fun, wouldn't it? However I am not in charge of user polls. I will however talk to those who are in charge and see what can be done. No promises though...

I'd bet that lots of other game players limit themselves to two or three sites that also understand the different nature of the Shrapnel Games market. So why not find out who is popular with your players?

Finally, the question. We already know which sites refer the most gamers to our site. These are statistics we get every month - intelligence is a good thing.

But, and here is the real catch. We are not after those gamers who already follow and support Shrapnel Games. These are our loyal fans and they, for the most part, don't listen to these reviews anyway. For us and our developers to prosper and stay around for a long time, bringing you games that you love to play, we need to find and bring in new gamers. And with sites reviewing our games and spending so very much of their energies on speaking to graphics rather than GAME PLAY (these are games after all and not works of art) we will have, and do have, a huge uphill battle in increasing our market share and assisting our developers in making the monies necessary to stay in business and prosper.

That last highlighted part is particularly interesting to me. Is Firaxis NOT in that position of having to get new gamers? I certainly heard the argument many times that Civ3 'needs to attract the casual gamer' ... so, is the 'war' on?

[Frugal_Gourmet]

Wow.

So if companies start pulling their expensive advertisements from magazines and other publications that DON'T give them the best possible reviews... shouldn't we at some point begin questioning the validity and integrity of the reviews themselves? This seems dangerous to me, somehow.

[Mahdimael]

The problem with game reviews is that for a lot of aspects that they examine, there are no quantifiable values. Even for those that there are, such as graphics or sound, the reviewer can be subjective and/or objective. I've always believed that games should be reviewed based on how well they do what they're trying to do. For Civ3, for example, I wouldn't take points off due to a lack of multiplayer. It wasn't included, and while that should be noted in the review, it shouldn't cause it to lose points. If the game had multiplayer that crashed all the time, then you make marks against it. Similarly, if hills and plains look like mountains in the game, you can mark against that, but you can't deduct points because it doesn't have a fully 3d engine.

Too many reviews do compare against other games of the genre, and while in some cases (first person twitch games) this is valid because differences between games are sometimes cosmetic at best, in other genres (TBS strategy, rpg) games can vary widely. You wouldn't compare Civ3 to Starcraft, but it would be valid to compare Civ3 to Ctp2. Even so, a game must be judged on it's own merits, not how it stacks up to similar games.

[Ozymandous]

Bam! Frugal hit the nail on the head.

The biggest problem with game reviews is this:

They are inherently biased towards those companies that advertise in their magazines.

If you don't believe it ask yourself when was the last magazine to give a game the crap rating it deserved if that developer/publisher also spent a lot of money advertising that, and a lot more games in that magazine?

Black & White and Blizzard 2: Lord of Destruction are two games that come to mind when I think of how biased reviewers are. Black & White was hailed as the best thing ever, yet once people had played the game for more than a week or two it plummeted like a rock because other than the creature management the game stunk like old fish. In D2's X-pack the case was the X-pack essentially broke the old game. Not only did BLizzard essentially change how D2 classic played but they even broke vital elements of the game itself with the expansion. And then it only took them a month to release a patch to fix all the numerous broken elements.

If you want a fair review give the game to someone who doesn't have their ass hanging out needing advertising dollars from the game publisher and who also isn't devoted to that type of game but who will at least look at the game with a neutral eye.

Until this happens reviews aren't typically worth the paper they are on for anything else than placing in the bottom of a bird cage, litter box or maybe to get some of the oil from when you change the filter in your car.

[Ozymandous]

Originally posted by Mahdimael
The problem with game reviews is that for a lot of aspects that they examine, there are no quantifiable values. Even for those that there are, such as graphics or sound, the reviewer can be subjective and/or objective. I've always believed that games should be reviewed based on how well they do what they're trying to do. For Civ3, for example, I wouldn't take points off due to a lack of multiplayer. It wasn't included, and while that should be noted in the review, it shouldn't cause it to lose points. If the game had multiplayer that crashed all the time, then you make marks against it. Similarly, if hills and plains look like mountains in the game, you can mark against that, but you can't deduct points because it doesn't have a fully 3d engine.

Too many reviews do compare against other games of the genre, and while in some cases (first person twitch games) this is valid because differences between games are sometimes cosmetic at best, in other genres (TBS strategy, rpg) games can vary widely. You wouldn't compare Civ3 to Starcraft, but it would be valid to compare Civ3 to Ctp2. Even so, a game must be judged on it's own merits, not how it stacks up to similar games.

Question then. If other games in that genre DO have MP and a 3-D engine then wouldn't it be fair to compare the games? If they are directly competing then YES it is valid to mark off points if a game doesn't have what the other does, all things being equal.

Does EU have MP? Is it in competition to Civ3? If so, then a reviewer would have every right to mark off the fact that Civ3 doesn't have MP when it's direct competition does have MP. Same goes for graphics. While all games will not have the budget to hire 20+ artists there is NO excuse not to make games that don't look at least as well as other games in their genre. Why? Because graphic programs are cheap and I have seen people make mod's that look better than the original game.

[Taliseian]

Declaring War on Reviews and Mags

After reading thru this thread, I'm concerned that such actions could produce a dangerous precident. I have to agree with Frugal_Gormet, that if this starts to happen that we would need to question the value of the reviews.

If this happens, doesn't it mean that basically the value of the reviews is near worthless?

Honestly, reviews are opinions of whoever plays the game. I'll be the first to admit that there are alot of reviewers out there who probably play the game for no more than an hour or two and type up a 500+ word review on the entire game. But to only give review copies (and a chance to review before the games hit the shelves) to those who are basically nothing more than "company mouthpieces" doesn't sit well with me.

I want to hear from people what they think. The exchange of ideas (either in print, or in places such as this forum), is important in any aspect, be it a game review or other more important topics.

If the graphics aren't up to the reviewer's liking, it's their job to say so....same with MP or any other facet of the game. I say, let them print their thoughts....it's still up to me if I want to spend my money on the game or not.

For the most part, after reading several different reviews, I'm able to tell who are the ones who actually play the game for a while and put an honest and objective opinion and those who are the "letsplaythisgamequicklyandtypeupareview" type.

I hope that Firaxis lets anyone and everyone who wants to review Civ3 for a print or online mag do so.


Taliseian

[Boris Godunov]

While there are some valid points made in the game company's rant, there are some things I take issue with in addition to what others have mentioned.

The examples they cited, IMO, are from very unknown game sources with little exposure to the public as compared to a site like Gamespot or Avault. They seem to be selectively taking potshots at the guys who did use silly criteria to somehow impugn all game reviewers. I do rely on game reviews for feedback before buying most products, but I never rely on just one. I check as many reviews as I can, both from professionals and consumers. If the balance is strong, I will get it. If not, I will pass it by. I initially was really looking forward to Pool of Radiance II. But the majority of reviews from critics AND players were negative, so I missed that one. Civ3 is the first game I've bought in a long time by preorder.

Second, I am very dubious of how this company will wage their "war." It looks to me like they are threatening all critics with a vague "If we don't like your reviews, we will not send you anymore games to review." Isn't this blackmail? How can we trust this company to not get upset at ANY negative review, no matter how valid it may be, and then pull this childish stunt? IMO, the are just looking for excuses to get their half-assed products on the shelves while bypassing the critics.

Cheers.

PS--I for one think Graphics are integral to computer games and should be as polished as possible, within reason. Why else play a game on the computer? Why not just use a text-based game with moving dots if it's just functional graphics you want? You don't have to spend even close to $8 mil. to have nice graphics.

[Ludwig]

I think the people who are defending the reviewers in this situation are missing the point.

While no one wants to encourage blackmail of reviewers by game developers, I think it is fairly obvious that there are types of games that in general aren't given a fair shake by most reviewers.

CTP2 was a flawed game. But I saw reviews of CTP2 that said, essentially, "TBS games suck and are boring. I want to play Command and Conquer again. I wish I was playing Command and Conquer instead of sitting here stuck writing this review of a TBS game."

If CIV III didn't have Sid Meier's name on it, I can guarantee you that reviews would appear that would say, "Ick. There's too much to keep track of here. I can't play this game with my joystick, and that stinks. Why am I being given a history lesson when I just want to shoot things?"

It really isn't just a matter of "eye candy" vs. "gameplay". A considerable portion of the reviewing community [as these developers point out, mainly the younger generation] actually actively dislike the things that make for good strategy gameplay. They moan about how they get bogged down in micromanagement, they moan about the pace of the game, they moan about "cumbersome" attempts to attain a basic measure of historical realism. Basically it seems to me that a lot of game reviewers are the guys who years ago would have spent their time playing "Sorry!" while I was playing "Squad Leader". And it just isn't right that these guys get to make or break a game.

End of rant.

[Mahdimael]

Originally posted by Ozymandous


Question then. If other games in that genre DO have MP and a 3-D engine then wouldn't it be fair to compare the games? If they are directly competing then YES it is valid to mark off points if a game doesn't have what the other does, all things being equal.

Does EU have MP? Is it in competition to Civ3? If so, then a reviewer would have every right to mark off the fact that Civ3 doesn't have MP when it's direct competition does have MP. Same goes for graphics. While all games will not have the budget to hire 20+ artists there is NO excuse not to make games that don't look at least as well as other games in their genre. Why? Because graphic programs are cheap and I have seen people make mod's that look better than the original game.

Sorry, Ozy, but I disagree. The differences between EU and Civ3 are many. The games themselves play differently. Why can we compare Quake 3 and Unreal Tournament. Because at their core, they're both about running around and shooting people. The differences between the two games are skin deep.

Let's say Ctp3 comes out and it does feature a beautiful 3d world and the best multiplayer options, but the gameplay is the same. In this case, I believe that it would be acceptable to compare CtP3 with Civ3. At the same time, it needs to be examined for it's own merits. The differences should be noted, but not used as a condemnation. If the AI in Ctp3 has fewer options than Civ3, it shouldn't matter. What matters is how well the AI works in the context of CtP3.

[Dinner]

Originally posted by Stuie
The only unfair comments Civ3 will receive will be that it does not include Multiplayer. Guess what: it is not (currently) being marketed as Multiplayer, nor does the box make any mention of multiplayer, so taking ANY points off for lack thereof is tacitly unfair.

My two cents.

It is entirely fair for a reviewer to knock civ3 for not having MP. As you yourself pointed out games must compete with each other for the consumer's cash. If Civ3 doesn't have a feature and the competetion does then the consumer has a right to know. There is nothing unfair about that; it is simply free market economics...

[jackshot]

I agree with Ludwig, reviews suck for the most part, and he pretty much said why. Everything seems to come from the point of view of what's in it for the FPS player. If Shrapnel and other companies can make reviews better by these methods, more power to them.

But the game companies have to separate reviews from marketing. I'm sure Shrapnel feels the sting of not being able to independently market itself, just like it would rather have better graphics except for the little problem of $$$, but it has to realize that a reviewing industry that is seen as manipulated by game companies WILL NOT help them market themselves.

As it is, reviewers have to separate themselves MORE from being mouthpieces to increase readership and the faith of that readership they do have, and, at the same time, be in a stronger position to help companies like Shrapnel. Maybe this is a useful surgical strike for Shrapnel, but still you play the game their way, Shrapnel loses due to lack of clout. Wait until the MOO3 guys only give copies to those who badmouth SE IV, then what will they do?

So I don't get the point. My personal opinion has always been that Shrapnel's games needed better graphics anyway.

[Spatzimaus]

I don't think it's fair at all for the game companies to pick and choose reviewers based on whether past reviews were favorable. A reviewer shouldn't have to base his analysis on whether the company making the game will be happy with it.
If a site/magazine is so ridiculously biased that a "fair" review is impossible, the readers usually know that already. There are several game sites I go to that I know never give fair reviews of certain types of games; guess what? I don't go there for reviews of those games. Find a web site that only reviews one specific type of game, and use that for the "definitive" review.
There have been many really really lousy games released. We depend on review sites and magazines to help us avoid the worst of these, and if producers took it upon themselves to guarantee that only favorable reviews were printed, what happens?

That being said, most of the problems people have had with "unfair" reviews seem to have been when a game was reviewed by someone who either didn't like the genre or only liked one specific game in that genre. This isn't usually a problem for sites/magazines with large numbers of reviewers, since there'll always be at least one person who loves that type of game as a whole.
For example, I generally don't like first-person shooters. But, in college I played a LOT of Doom and Doom 2. I'd be a bad choice to review Unreal Tournament, since I'd continually compare it to the Doom games and complain about atmosphere, while ignoring the other advances made. It's hard to compete with nostalgia. On the other hand, I play a lot of turn-based games, so if I worked for a reviewer I could probably make a reasonably fair review.

Oh, and I do think lack of multiplayer SHOULD hurt Civ 3's reviews, somewhat. It's a feature most competitors have, after all. If the game was entirely text-based, would it be unfair to give it less points than an identical game with a nice graphical interface? Of course not. Multiplayer is considered by most reviewers to be a standard feature in just about every genre these days, and its absence is a detriment. The fact that I'm far more interested in the single-player game is irrelevant.

[jackshot]

Originally posted by Spatzimaus
Oh, and I do think lack of multiplayer SHOULD hurt Civ 3's reviews, somewhat. It's a feature most competitors have, after all. If the game was entirely text-based, would it be unfair to give it less points than an identical game with a nice graphical interface? Of course not. Multiplayer is considered by most reviewers to be a standard feature in just about every genre these days, and its absence is a detriment. The fact that I'm far more interested in the single-player game is irrelevant.

Not sure I agree with this. I like the standard of only expecting the game to do what it is designed to do.

All games have a presentation, so the issue of how well it presents (ie, graphics) applies to every game. But multiplayer is a *way* to play not a part of the gameplay itself. The capability for multiplayer should be noted as information, but the game itself should be allowed to be judged as itself. A game is not worse because it lacks multiplayer, it just isn't a multiplayer game.

Otherwise its like critising a VHS tape for not playing in your DVD player. Sure I wish the movie was on DVD, but that's not really germane as to how good a tape it is.