R & R question « Political Science Rumors

7 years - 15 posts - Latest - RSS
OP: 2 Goods vs 2 No Goods
Thread: 48 Goods vs 23 No Goods

Political Science Rumors » Political Science » Research / Journals

R & R question

Karolyn

Reviewer tells me to include 2 new variables in my model and says a data source has the necessary information to create these variables. I look and I find that the information is actually incomplete to capture the intended effects. Should I:

1. Respond thoughtfully about why I didn't include the variables in my model in the body of the paper, but provide new specifications with the flawed variables in an appendix.

2. Respond thoughtfully about why I didn't include them and move on (no appendix).

2. Include the variables that I know to be flawed in the model and try to incorporate a justification into my paper.

It's an R & R with minor revisions. I'm new at this and just want to be smart. Thanks.

7 years ago # QUOTE 2 YEA 2 NAY !
Ezekiel

1.

End of story.

If the new specifications in the appendix alter your primary result, you'll need to explain why we should still prefer your model (likely because the new specifications reduce the n dramatically). If they don't alter the primary result, just note that alongside your thoughtful response indicating why you didn't include them in the main model.

Anyone who tells you differently is an idiot.

7 years ago # QUOTE 11 YEA 3 NAY !
Barrett

I don't know if I'd sign on to Ezekiel's rather stark (and unnecessarily controntational) concluding statement, but I think the rest of his comment is accurate and good advice for someone in your situation, and I would advise you to take his suggestion to heart.

Best of luck with your revisions.

7 years ago # QUOTE 11 YEA 2 NAY !
mcclurg
Rep: 38

I'd opt for #1. Its responsible but honest. Good luck!

mcclurg

7 years ago # QUOTE 6 YEA 3 NAY !
Camellia

You should definitely run the models with the s**t data but I don't think these should go in an appendix that will be publicly available. This isn't a robustness test its a new specification on a different dataset. Any results, negative or positive, could be a result of bias introduced by the now lowerN. These are junk findings and publishing them, even in an appendix, just adds to the noise around your subject.

Run the bulls**t models and include them in the memo only with a full outline explaining why it is worthless and potentially damaging to publish such nonsense.

7 years ago # QUOTE 2 YEA 3 NAY !
McGarry

#1. Good luck

7 years ago # QUOTE 0 YEA 1 NAY !
Primula

1, always choose tbe safest possible route in this situation.

7 years ago # QUOTE 1 YEA 2 NAY !
Coline

Run the additional models; include in the memo, but not the paper and not the appendix.

7 years ago # QUOTE 2 YEA 1 NAY !
Dom

Email Dan and ask.

7 years ago # QUOTE 1 YEA 2 NAY !
Gordy
Rep: 6

I'd probably actually agree with Coline. Run the additional models, include them in the memo, and put a footnote in the paper at most.

7 years ago # QUOTE 1 YEA 0 NAY !
Ezekiel

I'll retract my earlier "stark (and unnecessarily controntational)" opinion, and suggest including the models in the appendix if there is any reasonable justification for running them. If you believe the reviewer is just plain wrong and cannot see any value to his suggestion, you can just place them in the memo and add a footnote to the paper, with the additional caveat that you offer to the editor to put them in an appendix if they would like.

I'll also note that "put them in a memo" wasn't one of the OP's provided options, so you're all a bunch of cheating douchebags.

7 years ago # QUOTE 6 YEA 2 NAY !
Oberon

Ran into a similar situation but didn't know "put them in the memo" was an option, so I put them in the appendix in the r&r version. Now suppose the paper is accepted, is it appropriate to ask the editor for permission to remove them from the appendix, but add a footnote in the paper?

7 years ago # QUOTE 1 YEA 0 NAY !
Monique

Yea, because political science is just a multiple choice exam.

I'll retract my earlier "stark (and unnecessarily controntational)" opinion, and suggest including the models in the appendix if there is any reasonable justification for running them. If you believe the reviewer is just plain wrong and cannot see any value to his suggestion, you can just place them in the memo and add a footnote to the paper, with the additional caveat that you offer to the editor to put them in an appendix if they would like.
I'll also note that "put them in a memo" wasn't one of the OP's provided options, so you're all a bunch of cheating douchebags.

7 years ago # QUOTE 1 YEA 1 NAY !
Oceane

OP is not clear about what "incomplete" means, but if it means missing data, consider this:

Original model: Y = b0 + b1X1 + b2X2
N=200

New Model: Y = a0 + a1X1 + a2X2 + a3X3 + a4X4
N=150

Suppose as a result that b1 does not equal a1 and/or b2 does not equal a2. Just showing both models doesn't tell us why the change happened because there are two differences between the models: 1) Specification, 2) Sample.

Re-run the original model on just the 150 cases available for the new model. Now you can report the cause of the change and make a more informed choice. If missing data is the problem, you could also consider multiple imputation.

7 years ago # QUOTE 8 YEA 0 NAY !
Carry

^if the se went up but point estimate didn't change much then no need to do this or show this but otherwise a good idea.

7 years ago # QUOTE 0 YEA 1 NAY !

R & R question

Reply