[JoPP-Public] Re : Re: Improving peer review for JoPP
Mathieu ONeil
mathieu.oneil at anu.edu.au
Wed Mar 28 22:41:07 CEST 2012
Hi all
@Toni: publishing first drafts as a matter of course would be bold - and might seriously dry up the pool of subs. I'm not dead against it but feel we might need to get a few more issues out to build our rep more first. We already publish first drafts of published pieces and (AFAIK) are the only journal to do that... There's a point to be made about editorial comments which hitherto remain invisible contrary to reviews.. They vary in importance, sometimes just a passing-on-the-review type message, sometimes more substantial. Something to think about for sure.
@Felix: OK, sounds good. We could have a top menu called "peer review" or whatever which would have as sub-menus a) a description of the process (current "peer review" page) and b) the "submission queue". Btw should there be "rejected" as a category - anything is publishable with signals...?
Question: do we include the previously mentioned two papers who flaked out or do we give them a pass - they were not warned about this?
@Franco: The site is down right now so can't access it, is there a problem? Thanks for advising.
Also is there a tool to make tables with the following categories:
Title, Author, Abstract, Date of Submission, and state (under review, published, rejected, withdrawn)
cheers,
Mathieu
On 03/28/12, Toni Prug <toni.prug at gmail.com> wrote:
> publishing all submitted pieces in a separate queue would be a bold
> move, i wish all journals did that. Who wouldn't want to see what the
> "highest ranked" journals in each field reject? Or, how they "guide"
> authors to change pieces? Such openness would make political ideology of
> journals a lot more visible. It would open up information to try to hold
> journals more accountable in the long run, and more scientific in a
> positive sense. As it stands, ideological rejection of "unsuitable"
> texts, as well as submissions of terrible quality by authors, happens a
> lot over the place.
>
> This is what was argued in my piece Open Process Publishing couple of
> years ago:
>
> a) increase reputation cost by making every single submissions visible
> in a separate, clearly marked, queue.
>
> "In the current system, with externally invisible submissions, the
> reputation cost of submission for authors it too low: they can submit
> any rubbish without adjusting it to the journal’s guidelines. The only
> people who see these disrespectful acts (towards work of editors,
> especially volunteer work), and who associate it with author’s name, are
> editors. If submissions were openly visible, the cost of submitting
> random, unadjusted, low quality, undeveloped papers would be far higher,
> since such disrespectful behaviour would be publicly linked to the author."
>
> Here's what a journal that already does that said:
>
> "public peer review and interactive discussion deter authors from
> submitting low-quality manuscripts, and thus relieve editors and
> reviewers from spending too much time on deficient submissions. [..] The
> deterrent is particularly important, because reviewing capacities are
> the most limited resource in the publication process." (Koop 2006)
>
> A full open process model would make:
>
> b) all subsequent comments, reviews, versions, should be open too
>
> "To summarise, open-process academic publishing would amount to the
> following being open: initial submission, editorial collective and
> individual comments, peer reviews, further peer comments, author
> comments back to reviewers, all the subsequent drafts, and the final
> published or rejected text."
>
> I support any proposals that go a step closer to any of the above.
>
> Of course, there are valid arguments for a an in-between model too, for
> partial openness of some of the publishing steps.
>
> http://hackthestate.org/2009/12/16/open-process-academic-publishing-v1-2/
>
>
>
> On 28/03/2012 16:17, Felix Stalder wrote:
> >
> >
> > On 03/28/2012 03:01 PM, Mathieu ONeil wrote:
> >> Hi Felix, Athina, all
> >>
> >> I think this is a very good idea and I'd like to add to it. My first
> >> suggestion is non-controversial:
> >> -if the article is submitted somewhere else and published [and we find
> >> out about it] we could publish a link to that versione
> >
> > Sure, but we want to publish original stuff, so that should not be an issue.
> >
> >> My second suggestion may be a deterrent for some authors:
> >> -we could publish alongside titles, abstracts, outcome (did not publish,
> >> etc) etc the reviews?
> >
> > Rather not. Reviews without the full article don't make that much sense.
> >
> >
> >> Another question is where this would appear on the site. There would
> >> need to be a specific page / groupe of pages. What would we call it?
> >
> > I would do a page called "submission queue" where we could list
> > Title, Author, Abstract, Date of Submission, and state (under review,
> > published, rejected, withdrawn) and I would order this reverse
> > chronologically based on date of submission.
> >
> > Felix
> >
> >
> >
> >> cheers
> >>
> >> Mathieu
> >>
> >> Le 03/28/12, *Athina Karatzogianni *<athina.k at gmail.com> a écrit :
> >>> Hi All,
> >>>
> >>> I wonder whether we can do what Felix suggests but publish in a
> >>> separate place inside our site papers that didnt make it and have an
> >>> archive that way of everything which was ever submitted to us, which
> >>> is transparent. There might turn out to be lets say 30 papers over
> >>> five years (i doubt there d be more than that) or whatever and we ll
> >>> have a live record of this, if the authors who submitted them give
> >>> their consent. We d have to get consent when they submit to us in the
> >>> first place (in case this doesnt work out can we publish in the didnt
> >>> make it archive?) Its one way to do it and we might be the first
> >>> journal ever to do so...
> >>> why not? theres plenty of space online to do stuff like that
> >>>
> >>> Athina
> >>>
> >>> On Wed, Mar 28, 2012 at 12:57 PM, Felix Stalder<felix at openflows.com
> >>> <mailto:felix at openflows.com <felix at openflows.com>>> wrote:
> >>>
> >>>
> >>> I'm not sure if this is a good way, since we might end up
> >>> publishing the
> >>> crappy version, whereas others get the good one.
> >>>
> >>> But there could be an intermediary step. For example, one might
> >>> publish
> >>> a list with all submissions (Name, Title, Abstract, and date of
> >>> submission) so it gets at least transparent if people are submitting
> >>> here first, and the take it somewhere else.
> >>>
> >>> Felix
> >>>
> >>>
> >>>
> >>>
> >>> On 03/27/2012 02:29 PM, Mathieu ONeil wrote:
> >>> > ps. I'll answer my own question: a way to do that would be to
> >>> explicitly
> >>> > state that a version of all submissions will be published, so
> >>> that by
> >>> > the act of submitting to us authors are in fact agreeing for us to
> >>> > publish something.
> >>> > However in this scenario:
> >>> > -we may end up publishing more (duly signaled as such) crappy
> >>> articles
> >>> > than we would wish, and
> >>> > -we would be limiting the freedom of authors
> >>> >
> >>> > On 03/27/12, *Mathieu ONeil *<mathieu.oneil at anu.edu.au
> >>> <mailto:mathieu.oneil at anu.edu.au <mathieu.oneil at anu.edu.au>>> wrote:
> >>> >> Hi Christian, all
> >>> >>
> >>> >> Thanks for commenting. I can't respond address your points straight
> >>> >> away but I am curious about how you propose to implement this
> >>> part of
> >>> >> your proposal : "((publishing all versions of a paper from the
> >>> first
> >>> >> submitted one (or, at least, the last negotiated version of each
> >>> >> paper))) __without allowing the authors to pull out.__ "?
> >>> >> How do we stop people from pulling out? Sign a blood oath over the
> >>> >> Internet? ;-)
> >>> >>
> >>> >> cheers
> >>> >>
> >>> >> Mathieu
> >>> >>
> >>> >> On 03/27/12, *Christian Siefkes *<christian at siefkes.net
> >>> <mailto:christian at siefkes.net <christian at siefkes.net>>> wrote:
> >>> >>> Hi Mathieu and all,
> >>> >>>
> >>> >>> On 03/26/2012 04:17 PM, Mathieu ONeil wrote:
> >>> >>> > Openness undoubtedly has great virtues, but in the case of
> >>> academic
> >>> >>> > publishing it can also generate some bad side-effects.
> >>> >>> >
> >>> >>> > For this issue of JoPP five papers were sent out for review.
> >>> Three
> >>> >>> of the
> >>> >>> > papers will be published with reviews and signals. Two other
> >>> papers
> >>> >>> were not
> >>> >>> > great. Reviewers worked long and hard to address
> >>> shortcomings and make
> >>> >>> > suggestions.
> >>> >>> >
> >>> >>> > One author decided that it would not be possible to make these
> >>> >>> adjustments
> >>> >>> > though much time kept being added.
> >>> >>> >
> >>> >>> > The other agreed to make changes but then used the time
> >>> excuse as
> >>> >>> well as
> >>> >>> > sickness.
> >>> >>> >
> >>> >>> > There is nothing preventing either author from now
> >>> submitting their
> >>> >>> > much-improved papers to another journal...
> >>> >>> >
> >>> >>> > In my view, we should try to address this obvious waste of
> >>> reviewer
> >>> >>> (and
> >>> >>> > editorial) work/energy.
> >>> >>>
> >>> >>> hmm, isn't this a problem of being (maybe) not open enough
> >>> instead of
> >>> >>> being
> >>> >>> too open? In the experience from my own academic this, this is
> >>> a quite
> >>> >>> possible scenario in the traditional peer review process:
> >>> reviewers send
> >>> >>> criticism and suggestions, the author might then revise the
> >>> paper and
> >>> >>> send
> >>> >>> back a revised version, or submit the revised version elsewhere.
> >>> >>> Especially
> >>> >>> if a paper is re-submitted by multiple journals (after being
> >>> refused
> >>> >>> -- with
> >>> >>> reviewer feedback -- by each of them), it would cause
> >>> reviewers a lot of
> >>> >>> work. (Say if there are 3 reviewers per paper and you submit it
> >>> >>> sequentially
> >>> >>> to 4 journals, you would already occupy a dozen reviewers,
> >>> while none of
> >>> >>> them would benefit of the work already done by others, since they
> >>> >>> don't know
> >>> >>> about it.) Also, if you re-submit a text sufficiently often, it
> >>> >>> becomes more
> >>> >>> and more likely to be accepted somewhere by pure chance, almost
> >>> >>> regardless
> >>> >>> of the quality of the paper, I would presume.
> >>> >>>
> >>> >>> The only chance to avoid that would be more openness, not
> >>> less, i.e.
> >>> >>> publishing all versions of a paper from the first submitted
> >>> one (or, at
> >>> >>> least, the last negotiated version of each paper), without
> >>> allowing the
> >>> >>> authors to pull out. Not sure if we want to go this way, but
> >>> blaming
> >>> >>> "openness" for the shortcomings of the current approach
> >>> strikes my as
> >>> >>> definitively wrong.
> >>> >>>
> >>> >>> Best regards
> >>> >>> Christian
> >>> >>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ourproject.org/pipermail/jopp-public/attachments/20120328/de7882ff/attachment-0001.htm
More information about the JoPP-Public
mailing list