Gelman's Monkey Cage Post « Political Science Rumors

8 years - 302 posts - Latest - RSS
OP: 7 Goods vs 10 No Goods
Thread: 2310 Goods vs 1334 No Goods

Political Science Rumors » Political Science » Political Science Discussion

Gelman's Monkey Cage Post

« Previous 1…456…16 Next »

Frederica

^but he is either a plagiarist or a liar. Either he posted this, in which case he knew of the fraud 5 months ago and his tale of how he was put on to ML's fraud is bulls**t or he copied it wholesale from a psr poster. You could say that he came to the same conclusion on his own, but that would not explain the striking similarities between the language the anonymous poster uses here (not one in X) and the Broockman, Kalla, Aronow report. It is simply not credible that the two critics (one anonymous, one Broockman) used precisely the same phrase to describe the shocking finding that these data are not credible.

This is all true, but Stanford will not give a single fu#k.

8 years ago # QUOTE 5 YEA 2 NAY !
Anise

Broockman f**ks around on PSR and still is still this productive--impressive!
Broockman is a secret Stata monkey--not impressive!

8 years ago # QUOTE 9 YEA 3 NAY !
Jerry

So what if he suspected five months ago and didn't act until now?

8 years ago # QUOTE 3 YEA 2 NAY !
Davin

^but the report is false. That is a big deal. Stanford may not care, but the discipline as a whole will. DB has a LOT of explaining to do given this post.

If DB does not react to this within 10 days I will go public and put my name to this as someone who read the post and is flabbergasted by a blatant case either of stealing someone else's idea and not giving credit or lying blatantly about a timeline put forward as part of discovering the fraud. No one forced DB to put the timeline out. He did so to make sure we all thought is motives and methods pristine. If that is not true, and if he lied in the timeline, that is a very big deal. It may be grounds for legal action by ML, even if ML is indeed guilty of fraud.

8 years ago # QUOTE 12 YEA 22 NAY !
Davin

BECAUSE...read the timeline he put out as part of his report. There is no problem except that he claimed in his timeline that he came to this while trying just to replicate and extend. That can't be true in light of this thread. It also can't be true that the first indications of fraud were what he claimed them to be. Instead, it'd mean that he downloaded the data and smelled a rat. But if that is true then the B, K, and A report's timeline is a rank lie.

So what if he suspected five months ago and didn't act until now?

8 years ago # QUOTE 4 YEA 7 NAY !
Modestine

^but the report is false. That is a big deal. Stanford may not care, but the discipline as a whole will. DB has a LOT of explaining to do given this post.
If DB does not react to this within 10 days I will go public and put my name to this as someone who read the post and is flabbergasted by a blatant case either of stealing someone else's idea and not giving credit or lying blatantly about a timeline put forward as part of discovering the fraud. No one forced DB to put the timeline out. He did so to make sure we all thought is motives and methods pristine. If that is not true, and if he lied in the timeline, that is a very big deal. It may be grounds for legal action by ML, even if ML is indeed guilty of fraud.

Or they found the same thing independently? Posting something on PSR doesn't really give anyone a strong claim to ownership of an idea...

8 years ago # QUOTE 22 YEA 8 NAY !
Codie

"The authors would like to thank Judith (PSR) for her excellent research assistance"

8 years ago # QUOTE 58 YEA 1 NAY !
Quincey

^ No one cares if DB suspected 5 months ago and didn't act til now when Qualtrics confirmed the fraud, ML. Stop posting on here.

8 years ago # QUOTE 18 YEA 5 NAY !
Carine

Sticky the thread?

8 years ago # QUOTE 3 YEA 1 NAY !
Anise

"The authors would like to thank Judith (PSR) for her excellent research assistance"

holy s**t lol!

8 years ago # QUOTE 3 YEA 2 NAY !
Deangelo

It is highly unlikely that Broockman was the anonymous poster, for at least two reasons:

1) If he had discovered this and intended to publish a report, he would not have leaked the findings on PSR, and waited 5 months.
2) Unlike LaCour, he hasn't earned a Masters in statistics. His work mainly uses simple techniques such as bar charts, not simulations etc. It all seems obvious now (hindsight bias), but at the time many were skeptical, and not even Green himself, or the editors and reviewers at APSR and Science could detect the abnormal data patterns. It must have been someone with statistical expertise far exceeding Broockman's.

bros, either the anonymous poster was Broockman or Broockman practically plagiarized from this guy. "Not one out of X" repeatedly is the trope. So, David, which is it? Are you the PSR white knight who broke this story, or are you taking your research cues from PSR??

8 years ago # QUOTE 7 YEA 19 NAY !
Jerry

The report and timeline mentions doubts about the data in January. Five months ago. I really don't see the issue.

8 years ago # QUOTE 5 YEA 5 NAY !
Shayla

Could it have been one of the other coauthors who wrote the post?

8 years ago # QUOTE 6 YEA 2 NAY !
Davin

look at the fiecking language similarities. It is not possible that those phrases appeared here and in the B, K, A report by coincidence. Try paying closer attention. It will make you a better scholar.

^but the report is false. That is a big deal. Stanford may not care, but the discipline as a whole will. DB has a LOT of explaining to do given this post.
If DB does not react to this within 10 days I will go public and put my name to this as someone who read the post and is flabbergasted by a blatant case either of stealing someone else's idea and not giving credit or lying blatantly about a timeline put forward as part of discovering the fraud. No one forced DB to put the timeline out. He did so to make sure we all thought is motives and methods pristine. If that is not true, and if he lied in the timeline, that is a very big deal. It may be grounds for legal action by ML, even if ML is indeed guilty of fraud.

Or they found the same thing independently? Posting something on PSR doesn't really give anyone a strong claim to ownership of an idea...

8 years ago # QUOTE 5 YEA 11 NAY !
Anise

It is highly unlikely that Broockman was the anonymous poster, for at least two reasons:
2) Unlike LaCour, he hasn't earned a Masters in statistics. His work mainly uses simple techniques such as bar charts, not simulations etc.

You're not being serious right now.

8 years ago # QUOTE 12 YEA 3 NAY !
Hallam

desperate distraction tactics by ML

8 years ago # QUOTE 11 YEA 4 NAY !
Madonna

This site tracks IP address. If that person used a fixed ip, he/she could repost under the same username

Just took a look at the data. Kind of weird.... Check out this crosstab.
. insheet using science-data.txt, comma clear
. reshape wide therm_level ssm_level, i(panelist_id) j(wave)
. tab ssm_level1 ssm_level2
1 | 2 ssm_level
ssm_level | 1 2 3 4 5 | Total
-----------+-------------------------------------------------------+----------
1 | 2,754 623 25 2 3 | 3,407
2 | 182 779 208 11 1 | 1,181
3 | 4 148 657 185 11 | 1,005
4 | 0 13 226 1,044 307 | 1,590
5 | 0 0 24 526 2,864 | 3,414
-----------+-------------------------------------------------------+----------
Total | 2,940 1,563 1,140 1,768 3,186 | 10,597
. corr ssm_level1 ssm_level2
(obs=10597)
| ssm_le~1 ssm_le~2
-------------+------------------
ssm_level1 | 1.0000
ssm_level2 | 0.9519 1.0000
That amount of intertemporal stability is not believable to me. Not one of the 2,940 people who answered at 1 at time 2 answered at 4 or 5 at time 1? Is that consistent with other online panels?
Same thing in the feeling thermometer...
. destring therm_level*, replace force
. corr therm_level1 therm_level2
(obs=10597)
| therm_~1 therm_~2
-------------+------------------
therm_level1 | 1.0000
therm_level2 | 0.9882 1.0000
.988? That's nuts. Even if the measurements were a week apart.
Don't know how to reproduce this on PSR but check out this plot too on your own computers...
. scatter therm_level1 therm_level2
The noise is weirdly normally distributed, with *NOT ONE* respondent out of 10597 deviating by more than 25 in a negative direction or 38 in a positive direction. That complete lack of survey error in a sample so large is bizarre. All it takes is one person to misinterpret the feeling thermometer, think it's for a different question, etc.
Here's ANES 2012 vs. 2013 just as a rough benchmark, realizing that the time difference is way more and mode is different etc.:
. corr gayft2012 gayft2013
(obs=1492)
| gay~2012 gay~2013
-------------+------------------
gayft2012 | 1.0000
gayft2013 | 0.7106 1.0000
The amount of instability is *24x* lower in this study than in the ANES. Not saying that couldn't happen, but it is weird. Anecdotally, I've seen party ID within survey have a much lower correlation.
It's consistent with ML having a fixed sample size and a fixed believable effect size and having to work backward to what would be statistically significant (if there was less stability, the study would be less well-powered).
DEFINITELY not saying this nails anything, just that digging deeper into this data probably should be done...

8 years ago # QUOTE 1 YEA 2 NAY !
Beverley

Does anyone have a copy of ML's dataset (as opposed to the Science dataset)?

8 years ago # QUOTE 0 YEA 2 NAY !
Davin

I now see that the language in the report is intentionally vague and misleading:

January - April, 2015. Broockman and Kalla were impressed by LaCour and Green (2014) and wanted
to extend the article’s methodological and substantive discoveries. We began to plan an extension. We
sought to form our priors about several design parameters based on the patterns in the original data
on which the paper was based, LaCour (2014). As we examined the study’s data in planning our own
studies, two features surprised us: voters’ survey responses exhibit much higher test-retest reliabilities
than we have observed in any other panel survey data, and the response and reinterview rates of the
panel survey were significantly higher than we expected. We set aside our doubts about the study and
awaited the launch of our pilot extension to see if we could manage the same parameters. LaCour and
Green were both responsive to requests for advice about design details when queried.

Translated, in light of what we now know, what this means is "We downloaded the data because we found the results incredible. We rapidly discovered irregularities in the data that strongly suggested to us that something was seriously amiss. We then forged ahead with our replication and extension." WHAT?

8 years ago # QUOTE 4 YEA 2 NAY !
Gabby

they wanted to find the smoking gun on the field. but it turned out that they found it before when they called the polling firm.

8 years ago # QUOTE 4 YEA 1 NAY !

« Previous 1…456…16 Next »

Gelman's Monkey Cage Post

Reply »