[JoPP-Public] Re : Re: Improving peer review for JoPP
Toni Prug
toni.prug at gmail.com
Wed Mar 28 19:56:12 CEST 2012
publishing all submitted pieces in a separate queue would be a bold
move, i wish all journals did that. Who wouldn't want to see what the
"highest ranked" journals in each field reject? Or, how they "guide"
authors to change pieces? Such openness would make political ideology of
journals a lot more visible. It would open up information to try to hold
journals more accountable in the long run, and more scientific in a
positive sense. As it stands, ideological rejection of "unsuitable"
texts, as well as submissions of terrible quality by authors, happens a
lot over the place.
This is what was argued in my piece Open Process Publishing couple of
years ago:
a) increase reputation cost by making every single submissions visible
in a separate, clearly marked, queue.
"In the current system, with externally invisible submissions, the
reputation cost of submission for authors it too low: they can submit
any rubbish without adjusting it to the journal’s guidelines. The only
people who see these disrespectful acts (towards work of editors,
especially volunteer work), and who associate it with author’s name, are
editors. If submissions were openly visible, the cost of submitting
random, unadjusted, low quality, undeveloped papers would be far higher,
since such disrespectful behaviour would be publicly linked to the author."
Here's what a journal that already does that said:
"public peer review and interactive discussion deter authors from
submitting low-quality manuscripts, and thus relieve editors and
reviewers from spending too much time on deficient submissions. [..] The
deterrent is particularly important, because reviewing capacities are
the most limited resource in the publication process." (Koop 2006)
A full open process model would make:
b) all subsequent comments, reviews, versions, should be open too
"To summarise, open-process academic publishing would amount to the
following being open: initial submission, editorial collective and
individual comments, peer reviews, further peer comments, author
comments back to reviewers, all the subsequent drafts, and the final
published or rejected text."
I support any proposals that go a step closer to any of the above.
Of course, there are valid arguments for a an in-between model too, for
partial openness of some of the publishing steps.
http://hackthestate.org/2009/12/16/open-process-academic-publishing-v1-2/
On 28/03/2012 16:17, Felix Stalder wrote:
>
>
> On 03/28/2012 03:01 PM, Mathieu ONeil wrote:
>> Hi Felix, Athina, all
>>
>> I think this is a very good idea and I'd like to add to it. My first
>> suggestion is non-controversial:
>> -if the article is submitted somewhere else and published [and we find
>> out about it] we could publish a link to that versione
>
> Sure, but we want to publish original stuff, so that should not be an issue.
>
>> My second suggestion may be a deterrent for some authors:
>> -we could publish alongside titles, abstracts, outcome (did not publish,
>> etc) etc the reviews?
>
> Rather not. Reviews without the full article don't make that much sense.
>
>
>> Another question is where this would appear on the site. There would
>> need to be a specific page / groupe of pages. What would we call it?
>
> I would do a page called "submission queue" where we could list
> Title, Author, Abstract, Date of Submission, and state (under review,
> published, rejected, withdrawn) and I would order this reverse
> chronologically based on date of submission.
>
> Felix
>
>
>
>> cheers
>>
>> Mathieu
>>
>> Le 03/28/12, *Athina Karatzogianni *<athina.k at gmail.com> a écrit :
>>> Hi All,
>>>
>>> I wonder whether we can do what Felix suggests but publish in a
>>> separate place inside our site papers that didnt make it and have an
>>> archive that way of everything which was ever submitted to us, which
>>> is transparent. There might turn out to be lets say 30 papers over
>>> five years (i doubt there d be more than that) or whatever and we ll
>>> have a live record of this, if the authors who submitted them give
>>> their consent. We d have to get consent when they submit to us in the
>>> first place (in case this doesnt work out can we publish in the didnt
>>> make it archive?) Its one way to do it and we might be the first
>>> journal ever to do so...
>>> why not? theres plenty of space online to do stuff like that
>>>
>>> Athina
>>>
>>> On Wed, Mar 28, 2012 at 12:57 PM, Felix Stalder<felix at openflows.com
>>> <mailto:felix at openflows.com>> wrote:
>>>
>>>
>>> I'm not sure if this is a good way, since we might end up
>>> publishing the
>>> crappy version, whereas others get the good one.
>>>
>>> But there could be an intermediary step. For example, one might
>>> publish
>>> a list with all submissions (Name, Title, Abstract, and date of
>>> submission) so it gets at least transparent if people are submitting
>>> here first, and the take it somewhere else.
>>>
>>> Felix
>>>
>>>
>>>
>>>
>>> On 03/27/2012 02:29 PM, Mathieu ONeil wrote:
>>> > ps. I'll answer my own question: a way to do that would be to
>>> explicitly
>>> > state that a version of all submissions will be published, so
>>> that by
>>> > the act of submitting to us authors are in fact agreeing for us to
>>> > publish something.
>>> > However in this scenario:
>>> > -we may end up publishing more (duly signaled as such) crappy
>>> articles
>>> > than we would wish, and
>>> > -we would be limiting the freedom of authors
>>> >
>>> > On 03/27/12, *Mathieu ONeil *<mathieu.oneil at anu.edu.au
>>> <mailto:mathieu.oneil at anu.edu.au>> wrote:
>>> >> Hi Christian, all
>>> >>
>>> >> Thanks for commenting. I can't respond address your points straight
>>> >> away but I am curious about how you propose to implement this
>>> part of
>>> >> your proposal : "((publishing all versions of a paper from the
>>> first
>>> >> submitted one (or, at least, the last negotiated version of each
>>> >> paper))) __without allowing the authors to pull out.__ "?
>>> >> How do we stop people from pulling out? Sign a blood oath over the
>>> >> Internet? ;-)
>>> >>
>>> >> cheers
>>> >>
>>> >> Mathieu
>>> >>
>>> >> On 03/27/12, *Christian Siefkes *<christian at siefkes.net
>>> <mailto:christian at siefkes.net>> wrote:
>>> >>> Hi Mathieu and all,
>>> >>>
>>> >>> On 03/26/2012 04:17 PM, Mathieu ONeil wrote:
>>> >>> > Openness undoubtedly has great virtues, but in the case of
>>> academic
>>> >>> > publishing it can also generate some bad side-effects.
>>> >>> >
>>> >>> > For this issue of JoPP five papers were sent out for review.
>>> Three
>>> >>> of the
>>> >>> > papers will be published with reviews and signals. Two other
>>> papers
>>> >>> were not
>>> >>> > great. Reviewers worked long and hard to address
>>> shortcomings and make
>>> >>> > suggestions.
>>> >>> >
>>> >>> > One author decided that it would not be possible to make these
>>> >>> adjustments
>>> >>> > though much time kept being added.
>>> >>> >
>>> >>> > The other agreed to make changes but then used the time
>>> excuse as
>>> >>> well as
>>> >>> > sickness.
>>> >>> >
>>> >>> > There is nothing preventing either author from now
>>> submitting their
>>> >>> > much-improved papers to another journal...
>>> >>> >
>>> >>> > In my view, we should try to address this obvious waste of
>>> reviewer
>>> >>> (and
>>> >>> > editorial) work/energy.
>>> >>>
>>> >>> hmm, isn't this a problem of being (maybe) not open enough
>>> instead of
>>> >>> being
>>> >>> too open? In the experience from my own academic this, this is
>>> a quite
>>> >>> possible scenario in the traditional peer review process:
>>> reviewers send
>>> >>> criticism and suggestions, the author might then revise the
>>> paper and
>>> >>> send
>>> >>> back a revised version, or submit the revised version elsewhere.
>>> >>> Especially
>>> >>> if a paper is re-submitted by multiple journals (after being
>>> refused
>>> >>> -- with
>>> >>> reviewer feedback -- by each of them), it would cause
>>> reviewers a lot of
>>> >>> work. (Say if there are 3 reviewers per paper and you submit it
>>> >>> sequentially
>>> >>> to 4 journals, you would already occupy a dozen reviewers,
>>> while none of
>>> >>> them would benefit of the work already done by others, since they
>>> >>> don't know
>>> >>> about it.) Also, if you re-submit a text sufficiently often, it
>>> >>> becomes more
>>> >>> and more likely to be accepted somewhere by pure chance, almost
>>> >>> regardless
>>> >>> of the quality of the paper, I would presume.
>>> >>>
>>> >>> The only chance to avoid that would be more openness, not
>>> less, i.e.
>>> >>> publishing all versions of a paper from the first submitted
>>> one (or, at
>>> >>> least, the last negotiated version of each paper), without
>>> allowing the
>>> >>> authors to pull out. Not sure if we want to go this way, but
>>> blaming
>>> >>> "openness" for the shortcomings of the current approach
>>> strikes my as
>>> >>> definitively wrong.
>>> >>>
>>> >>> Best regards
>>> >>> Christian
>>> >>>
>>> >>> --
>>> >>> |------- Dr. Christian Siefkes ------- christian at siefkes.net
>>> <mailto:christian at siefkes.net> -------
>>> >>> | Homepage: http://www.siefkes.net/ | Blog:
>>> http://www.keimform.de/
>>> >>> | Peer Production Everywhere: http://peerconomy.org/wiki/
>>> >>> |---------------------------------- OpenPGP Key ID: 0x346452D8 --
>>> >>> UNIX was not designed to stop you from doing stupid things,
>>> because that
>>> >>> would also stop you from doing clever things.
>>> >>> -- Doug Gwyn
>>> >>>
>>> >> --
>>> >> ****
>>> >> Dr Mathieu O'Neil
>>> >> Adjunct Research Fellow
>>> >> Australian Demographic and Social Research Institute
>>> >> College of Arts and Social Science
>>> >> The Australian National University
>>> >> email: mathieu.oneil[at]anu.edu.au<http://anu.edu.au>
>>> >> web: http://adsri.anu.edu.au/people/visitors/mathieu.php
>>> > --
>>> > ****
>>> > Dr Mathieu O'Neil
>>> > Adjunct Research Fellow
>>> > Australian Demographic and Social Research Institute
>>> > College of Arts and Social Science
>>> > The Australian National University
>>> > email: mathieu.oneil[at]anu.edu.au<http://anu.edu.au>
>>> > web: http://adsri.anu.edu.au/people/visitors/mathieu.php
>>> >
>>> >
>>> > _______________________________________________
>>> > JoPP-Public mailing list
>>> > JoPP-Public at lists.ourproject.org
>>> <mailto:JoPP-Public at lists.ourproject.org>
>>> > https://lists.ourproject.org/cgi-bin/mailman/listinfo/jopp-public
>>>
>>> --
>>>
>>> --- http://felix.openflows.com ----------------------- books out now:
>>> *|Deep Search. The Politics of Searching Beyond Google. Studien. 2009
>>> *|Mediale Kunst/Media Arts Zurich.13 Positions.Scheidegger&Spiess2008
>>> *|Manuel Castells and the Theory of the Network Society. Polity, 2006
>>> *|Open Cultures and the Nature of Networks. Ed. Futura/Revolver, 2005
>>>
>>>
>>> _______________________________________________
>>> JoPP-Public mailing list
>>> JoPP-Public at lists.ourproject.org
>>> <mailto:JoPP-Public at lists.ourproject.org>
>>> https://lists.ourproject.org/cgi-bin/mailman/listinfo/jopp-public
>>>
>>>
>>>
>>>
>>> --
>>> <http://www.routledge.com/books/search/keywords/karatzogianni/>
>>> Dr Athina Karatzogianni
>>> <http://www2.hull.ac.uk/FASS/humanities/media,_culture_and_society/staff/karatzogianni,_dr_athina.aspx>
>>> Senior Lecturer in New Media and Political Communication
>>> Faculty of Arts and Social Sciences
>>> The University of Hull
>>> United Kingdom
>>> HU6 7RX
>>> T: ++44 (0) 1482 46 5790
>>> F: ++44 (0) 1482 466107
>>> E: a.karatzogianni at hull.ac.uk<mailto:a.karatzogianni at hull.ac.uk>
>>>
>>> Download my work for free here:
>>> http://works.bepress.com/athina_karatzogianni/
>>>
>>>
>>>
>> --
>> ****
>> Dr Mathieu O'Neil
>> Adjunct Research Fellow
>> Australian Demographic and Social Research Institute
>> College of Arts and Social Science
>> The Australian National University
>> email: mathieu.oneil[at]anu.edu.au
>> web: http://adsri.anu.edu.au/people/visitors/mathieu.php
>>
>>
>> _______________________________________________
>> JoPP-Public mailing list
>> JoPP-Public at lists.ourproject.org
>> https://lists.ourproject.org/cgi-bin/mailman/listinfo/jopp-public
>
More information about the JoPP-Public
mailing list