The Reverse Racism Effect

[Kidlicious]

You're the one repeating nonsense. MOST research studies aren't done twice. MOST research studies don't do "do overs". MOST studies are done right the first time.
IF this study attracts the resources and money to be done right on a national basis, then we will see some results that will have some reliability. Until then, there is no significance to the study. But it's no surprise that you would find a flawed study like this and try to use it to push your agenda.

This is NOT how research is normally done.
Try again...

La la la la

[Ming]

La la la la

Gee... finally something that fits you.

[Kidlicious]

Don't know if you're an actual researcher or not Ming.

Do you have actual evidence that research is not normally done over again?

[rah]

Generally in market research, we only repeat a study if it's a tracking study.
Otherwise, do it once, do it right because someone is paying for it.

[Kidlicious]

Generally in market research, we only repeat a study if it's a tracking study.
Otherwise, do it once, do it right because someone is paying for it.

There is a range in quality of market research isn't there? How much will people spend?

[Kidlicious]

Isn't it true that market research often fails? So market researchers don't have to prove for example that people will like New Coke, or whatever?

[rah]

It used to be (mail or phone) vs internet since internet was 10x cheaper.
Before well before that it was (door to door or mall intercept) vs (mail or phone) since (mail or phone) was 10x cheaper.

Needless to say the cheapness of internet has almost virtually eliminated all other methods. Cost is king.

Sample size used to also be a major cost factor but now if you're going internet it's immaterial except for the data collection expenses (cleaning and review)
Making sure it's representative is the most important and difficult aspect.

Demonstration or process research is still quite expensive and Sample size is always seriously considered.

But with the proper selection methodology you'd be surprised how small of a sample you can use.
(but in the case of this study the didn't really do any)

The biggest problem with internet studies is the natural bias of the internet.
Similar to when phone surveys first became popular and you had to adjust for the fact that few blacks had phones. (1960s)
Now that internet surveys have been around for awhile, we're learning better how to weight the returns and supplement them with alternate methodologies to offset the biases. BUT having said that, there are still inherent problems relying solely on the internet. But as I said, since its so much cheaper than any other method, clients DEMAND it.

Proper Questionnaire design can also be done cheap or expensive but since its only done once, not that big of an overall expense on a project.
Which is why I'm always disappointed when I see a crappy questionnaire.

Needless to say, scientific behavioral studies can be quite expensive in comparison. Every minute adds to the cost so many cut corners. Which is the main reason the replication is important.
Which was my biggest surprise on this on. It was obviously an expensive study to administer. I'm sure that was the major consideration in how small the sample size was. When that is the case, you spend a few extra bucks making sure you have a representative sample because when the sample size is so small, weighting is not always effective.

Obviously it's more complex than this but these are just some of the general concepts.

[rah]

Isn't it true that market research often fails? So market researchers don't have to prove for example that people will like New Coke, or whatever?

Yes it has been know to fail, but in most cases, it's the result of crappy research. Remember, a lot of market research goes out for open bids. Cheap may win the proposal but it doesn't guarantee quality.

[Kidlicious]

It used to be (mail or phone) vs internet since internet was 10x cheaper.
Before well before that it was (door to door or mall intercept) vs (mail or phone) since (mail or phone) was 10x cheaper.

Needless to say the cheapness of internet has almost virtually eliminated all other methods. Cost is king.

Sample size used to also be a major cost factor but now if you're going internet it's immaterial except for the data collection expenses (cleaning and review)
Making sure it's representative is the most important and difficult aspect.

Demonstration or process research is still quite expensive and Sample size is always seriously considered.

But with the proper selection methodology you'd be surprised how small of a sample you can use.
(but in the case of this study the didn't really do any)

The biggest problem with internet studies is the natural bias of the internet.
Similar to when phone surveys first became popular and you had to adjust for the fact that few blacks had phones. (1960s)
Now that internet surveys have been around for awhile, we're learning better how to weight the returns and supplement them with alternate methodologies to offset the biases. BUT having said that, there are still inherent problems relying solely on the internet. But as I said, since its so much cheaper than any other method, clients DEMAND it.

Proper Questionnaire design can also be done cheap or expensive but since its only done once, not that big of an overall expense on a project.
Which is why I'm always disappointed when I see a crappy questionnaire.

Needless to say, scientific behavioral studies can be quite expensive in comparison. Every minute adds to the cost so many cut corners. Which is the main reason the replication is important.
Which was my biggest surprise on this on. It was obviously an expensive study to administer. I'm sure that was the major consideration in how small the sample size was. When that is the case, you spend a few extra bucks making sure you have a representative sample because when the sample size is so small, weighting is not always effective.

Obviously it's more complex than this but these are just some of the general concepts.

Good post. Thanks.

I just have one problem. You're misrepresenting the intention of the researcher. The small sample size was planned to save money, not to prove conclusively. The plan was, from the beginning, to get funding for a bigger research project.

I've already said that. I hope I don't have to repeat it again.

At any rate, so market research is often done cheap, right?

[Kidlicious]

Yes it has been know to fail, but in most cases, it's the result of crappy research. Remember, a lot of market research goes out for open bids. Cheap may win the proposal but it doesn't guarantee quality.

So when you do this "cheap" research everyone knows it may give the wrong conclusion, right? Yet the client still spends money on the new product. Why?

[rah]

People don't always understand value. Why do people buy cheap knockoffs of quality products and then get pissed when it breaks?

And sometimes people don't know they're buying cheap research. They may be paying but getting cheated.

The plan was, from the beginning, to get funding for a bigger research project.

The best way to get funding for a larger project is to produce quality results.
I would not fund anything based on their methodology.
To me, a non representative sample invalidates any results.
And if you ask around, almost every researcher would agree with me.

Yet the world is awash with crappy internet research. You can find an online poll to prove almost anything. It doesn't make them true.

[Kidlicious]

What do you think is wrong with their methodology? I'm trying to explain the reason for the small sample size, but you aren't getting it.

When your clients choose low quality research do you expect them to do cost/benefit analysis to choose different costs? Or do you think the most expensive research is always best for them?

[rah]

The small UNREPRESENTATIVE sample is the problem with their methodology. It invalidates EVERYTHING.
How many times do I have to say this?

To me it's like finding two vegetarians on the street and after talking to them making the claim that people hate meat and then go home hoping it will interest people and lead to a bigger meat hating research grant.

Got it now?

[rah]

When your clients choose low quality research do you expect them to do cost/benefit analysis to choose different costs? Or do you think the most expensive research is always best for them?

The Government or Companies will generally put out an RFP (request for proposal) that has the requirements for the study.
Research Companies will write a proposal and say how they will meet those requirements and what it will cost.
Sometimes companies will list in their proposal different levels of service for the price but in general have to meet the requirements in the original rfp
The entities seeking the research will then select which company/price wins.

How they determine that is up to them. They sometime will try to bargain and say "we really liked your proposal but thought it was too much money" or "you know you were the most expensive" (sometimes legal requirements dictate no back and forth and either reject or accept) And sometimes it's just a question of who knows who and it's kind of rigged, regardless of the perceived quality involved.
Then we have the choice of dropping the price or changing the proposal. But they get the final choice. Sometimes they're idiots or get blinded by the lowest cost.

[ZEE]

Rah, you are defeated.

You might want to dust off that Parcheesi board or start a game of Minesweeper because Kidicious just PWNed your ass. Nobody will think less of you if you leave Apolyton for a few weeks because of this.

Kidicious simply is unbeatable.