Jakana Thomas' IO article on "Wolves in Sheep's Clothing" and replication data

2 years - 234 posts - Latest - RSS
OP: 11 Goods vs 4 No Goods
Thread: 764 Goods vs 427 No Goods

Political Science Rumors » Political Science » Research / Journals

Jakana Thomas' IO article on "Wolves in Sheep's Clothing" and replication data

12…12 Next »

Merla

APSA's Guide to Professional Ethics in Political Science, https://www.apsanet.org/portals/54/Files/Publications/APSAEthicsGuide2012.pdf

Page 9, "5.6 When statements that are challenged are based on reproducible data authors are obliged to facilitate replication."

Don't delete this thread until the results in Thomas' IO article match her replication data.

Table 2 on page 19 in that article can't be replicated with her replication data.

Free access to Her forthcoming IO article:

https://www.cambridge.org/core/journals/international-organization/article/wolves-in-sheeps-clothing-assessing-the-effect-of-gender-norms-on-the-lethality-of-female-suicide-terrorism/D9BFE04ECEDC0DC9E96818C0FBA6DE8B

Free access to her replication data:

https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/YQ40LH

2 years ago # QUOTE 11 YEA 4 NAY !
Manuel

just give us the punchline, is it trivial non-replication? broad interpretations change?

2 years ago # QUOTE 1 YEA 1 NAY !
Merla

just give us the punchline, is it trivial non-replication? broad interpretations change?

The latter. My replicated coefficients are NOT significant.

2 years ago # QUOTE 3 YEA 0 NAY !
Heidi

Why don't you write an article about it, OP?

2 years ago # QUOTE 8 YEA 5 NAY !
Sawyer

Why don't you write an article about it, OP?

why are you posting about it here? write a response paper or email her and ask for a correction. this thread will just become a way to bully the author.

2 years ago # QUOTE 24 YEA 3 NAY !
Jodie

NOT significant.

You need a stats class

2 years ago # QUOTE 1 YEA 5 NAY !
Danny

Why don't you write an article about it, OP?

why are you posting about it here? write a response paper or email her and ask for a correction. this thread will just become a way to bully the author.

we've recommended this to OP 100x. s\he wants to bully

2 years ago # QUOTE 3 YEA 1 NAY !
Merla

NOT significant.

You need a stats class

Model 1

Negative binomial regression Number of obs = 2,398
Wald chi2(14) = 884.99
Dispersion = mean Prob > chi2 = 0.0000
Log pseudolikelihood = -7559.2006 Pseudo R2 = 0.0235

(Std. Err. adjusted for 27 clusters in campaign_number)

Robust
number_killed Coef. Std. Err. z P>z [95% Conf. Interval]

female0_1 .0921835 .2252278 0.41 0.682 -.3492548 .5336218
numwoundmonth_lag -.0004275 .0008411 -0.51 0.611 -.002076 .001221
lnbd -.0020845 .0438223 -0.05 0.962 -.0879746 .0838057
number_terrorists_year .0122675 .0164681 0.74 0.456 -.0200093 .0445444
numkillmonth_lag .0024239 .0020701 1.17 0.242 -.0016335 .0064813
religion_impt .0027452 .0096134 0.29 0.775 -.0160968 .0215871
many_attackers_cat -.6574785 .3344121 -1.97 0.049 -1.312914 -.0020427
weapon0_1 .0060996 .0370542 0.16 0.869 -.0665252 .0787244
assassination0_1 -.2165313 .1235204 -1.75 0.080 -.4586269 .0255642
political -.2149538 .0939054 -2.29 0.022 -.399005 -.0309026
security -.8816511 .0919077 -9.59 0.000 -1.061787 -.7015153
t1 .0090545 .0132741 0.68 0.495 -.0169623 .0350713
t2 -.000077 .0000812 -0.95 0.343 -.0002362 .0000822
t3 1.36e-07 1.30e-07 1.04 0.296 -1.19e-07 3.90e-07
_cons 3.460524 .7422251 4.66 0.000 2.005789 4.915259

/lnalpha .5598168 .1334456 .2982682 .8213655

alpha 1.750352 .2335768 1.347523 2.273602

Model 2

Negative binomial regression Number of obs = 2,398
Wald chi2(15) = .
Dispersion = mean Prob > chi2 = .
Log pseudolikelihood = -7542.227 Pseudo R2 = 0.0257

(Std. Err. adjusted for 27 clusters in campaign_number)

Robust
number_killed Coef. Std. Err. z P>z [95% Conf. Interval]

1.female0_1 .8713702 .5772283 1.51 0.131 -.2599764 2.002717<...See full post
br /> v2x_gencs 1.674458 1.136496 1.47 0.141 -.5530326 3.901949

female0_1#c.v2x_gencs
1 -1.437427 1.285307 -1.12 0.263 -3.956581 1.081728

numwoundmonth_lag -.0003947 .0008564 -0.46 0.645 -.0020732 .0012838
lnbd -.0057017 .0411733 -0.14 0.890 -.0863999 .0749964
number_terrorists_year .0123497 .0143653 0.86 0.390 -.0158057 .0405052
numkillmonth_lag .0023283 .0020228 1.15 0.250 -.0016362 .0062929
religion_impt .0086122 .0085485 1.01 0.314 -.0081425 .025367
many_attackers_cat -.4856774 .3676681 -1.32 0.187 -1.206294 .2349389
weapon0_1 .0119349 .03483 0.34 0.732 -.0563307 .0802004
assassination0_1 -.2104922 .1264849 -1.66 0.096 -.4583981 .0374136
political -.1226906 .0972053 -1.26 0.207 -.3132095 .0678282
security -.8006774 .0775152 -10.33 0.000 -.9526045 -.6487503
t1 .004448 .0108953 0.41 0.683 -.0169064 .0258024
t2 -.0000363 .0000611 -0.59 0.552 -.0001561 .0000834
t3 6.47e-08 9.26e-08 0.70 0.485 -1.17e-07 2.46e-07
_cons 1.57673 .9727694 1.62 0.105 -.3298632 3.483323

/lnalpha .5453489 .1286941 .293113 .7975848

alpha 1.72521 .2220244 1.340594 2.220172

......and go on......

2 years ago # QUOTE 1 YEA 5 NAY !
Heidi

Tables don't post well here. If you are so smart, why don't you write a rebuttal to it? Maybe you can finally publish your first solo article!

2 years ago # QUOTE 17 YEA 4 NAY !
Merla

Tables don't post well here. If you are so smart, why don't you write a rebuttal to it? Maybe you can finally publish your first solo article!

In Political Science, people can publish anything close to garbage (I'm not saying Thomas did it). However, it is extremely rare to publish a rebuttal unless you're Gary King.

2 years ago # QUOTE 5 YEA 2 NAY !
Bert

Conspiracy theory: Marla is actually Jakana, posting her own errors here to preemptively smear any potential replicators who try to take her down publicly.

2 years ago # QUOTE 8 YEA 0 NAY !
Annalisa

I ran the replication file.

I got slightly difference results (coefficients) for all models. The n is slightly different in the published versus the replication files as ran. ~20-30 less observations in the published paper than the replication files.

The non-significance for model 2 and 4 is due to her reporting 1 tailed test and Stata defaulting to 2 tailed test. So she is doing a somewhat slight of hand in reporting.

Reported model 1 coef: 0.222. Replication: 0.092
Reported model 2 coef: 0.697. Replication: 0.871
Reported model 3 coef: 0.658. Replication: 0.683
Reported model 4 coef: 0.467. Replication: 0.452

2 years ago # QUOTE 8 YEA 2 NAY !
Annalisa

There are also differences in the expanded and non-expanded datasets. If you use the non-expanded dataset, you can replicate the results perfectly to model 2 but you can't estimate table 1. If you use the expanded dataset, you can replicate table 1 but the number for table 2 are slightly difference as outlined above. Didn't read the article or appendix so I don't know what the differences are (lol).

Either way, seems like the biggest issue here is reporting 1 tailed test and someone replicating and seeing 2 tailed test and freaking out.

2 years ago # QUOTE 7 YEA 3 NAY !
Kameron

There are also differences in the expanded and non-expanded datasets. If you use the non-expanded dataset, you can replicate the results perfectly to model 2 but you can't estimate table 1. If you use the expanded dataset, you can replicate table 1 but the number for table 2 are slightly difference as outlined above. Didn't read the article or appendix so I don't know what the differences are (lol).
Either way, seems like the biggest issue here is reporting 1 tailed test and someone replicating and seeing 2 tailed test and freaking out.

hi Ben!

2 years ago # QUOTE 4 YEA 1 NAY !
Roni

There are also differences in the expanded and non-expanded datasets. If you use the non-expanded dataset, you can replicate the results perfectly to model 2 but you can't estimate table 1. If you use the expanded dataset, you can replicate table 1 but the number for table 2 are slightly difference as outlined above. Didn't read the article or appendix so I don't know what the differences are (lol).
Either way, seems like the biggest issue here is reporting 1 tailed test and someone replicating and seeing 2 tailed test and freaking out.

Um... so we can replicate all of her findings with the data she provides? Sounds like a real problem. Way to uphold the standards with your typically aplomb, PSR.

Did she report one-tailed findings in the article? If so, there's no problem here at all.

2 years ago # QUOTE 9 YEA 2 NAY !
Alain

However, it is extremely rare to publish a rebuttal unless you're Gary King.

Doesn't mean it has to be. We should do this more as a discipline. That would give people like OP a healthier way to debate findings than turning to an anonymous forum and smearing the author. OP needs to be subject to peer review as well.

2 years ago # QUOTE 4 YEA 0 NAY !
Candice

nerdz

2 years ago # QUOTE 4 YEA 2 NAY !
Roni

However, it is extremely rare to publish a rebuttal unless you're Gary King.

Doesn't mean it has to be. We should do this more as a discipline. That would give people like OP a healthier way to debate findings than turning to an anonymous forum and smearing the author. OP needs to be subject to peer review as well.

GTFO. So it's the discipline's fault that OP went to an anonymous forum and didn't email the author? They have correspondence information for just that purpose.

How about OP getting it wrong? That the discipline's fault, too?

2 years ago # QUOTE 3 YEA 5 NAY !
Candice

^ hahahahahahahahahahaha. whenever we talk about inane stuff on here, this board is "for the discipline". Then someone posts actual poli sci stuff and suddenly it becomes son "anonymous forum".

Make up your minds.

2 years ago # QUOTE 7 YEA 1 NAY !
Merla

I ran the replication file.
I got slightly difference results (coefficients) for all models. The n is slightly different in the published versus the replication files as ran. ~20-30 less observations in the published paper than the replication files.
The non-significance for model 2 and 4 is due to her reporting 1 tailed test and Stata defaulting to 2 tailed test. So she is doing a somewhat slight of hand in reporting.
Reported model 1 coef: 0.222. Replication: 0.092
Reported model 2 coef: 0.697. Replication: 0.871
Reported model 3 coef: 0.658. Replication: 0.683
Reported model 4 coef: 0.467. Replication: 0.452

The information looks misleading. Coefficients mean nothing unless p-value is reported together.

p-value for the first replicated coefficient: 0.682
The second: 0.131
The third: 0.027
The fourth: 0.142

2 years ago # QUOTE 1 YEA 2 NAY !

12…12 Next »

Jakana Thomas' IO article on "Wolves in Sheep's Clothing" and replication data

Reply »