[JoPP-Public] Re : Re: Improving peer review for JoPP

Mathieu ONeil mathieu.oneil at anu.edu.au
Wed Mar 28 22:41:07 CEST 2012


Hi all

@Toni: publishing first drafts as a matter of course would be bold - and might seriously dry up the pool of subs. I'm not dead against it but feel we might need to get a few more issues out to build our rep more first. We already publish first drafts of published pieces and (AFAIK) are the only journal to do that... There's a point to be made about editorial comments which hitherto remain invisible contrary to reviews.. They vary in importance, sometimes just a passing-on-the-review type message, sometimes more substantial. Something to think about for sure.

@Felix: OK, sounds good. We could have a top menu called "peer review" or whatever which would have as sub-menus a) a description of the process (current "peer review" page) and b) the "submission queue". Btw should there be "rejected" as a category - anything is publishable with signals...?

Question: do we include the previously mentioned two papers who flaked out or do we give them a pass - they were not warned about this?

@Franco: The site is down right now so can't access it, is there a problem? Thanks for advising.
Also is there a tool to make tables with the following categories:
Title, Author, Abstract, Date of Submission, and state (under review, published, rejected, withdrawn)

cheers,

Mathieu


On 03/28/12, Toni Prug  <toni.prug at gmail.com> wrote:
> publishing all submitted pieces in a separate queue would be a bold 
> move, i wish all journals did that. Who wouldn't want to see what the 
> "highest ranked" journals in each field reject? Or, how they "guide" 
> authors to change pieces? Such openness would make political ideology of 
> journals a lot more visible. It would open up information to try to hold 
> journals more accountable in the long run, and more scientific in a 
> positive sense. As it stands, ideological rejection of "unsuitable" 
> texts, as well as submissions of terrible quality by authors, happens a 
> lot over the place.
> 
> This is what was argued in my piece Open Process Publishing couple of 
> years ago:
> 
> a) increase reputation cost by making every single submissions visible 
> in a separate, clearly marked, queue.
> 
> "In the current system, with externally invisible submissions, the 
> reputation cost of submission for authors it too low: they can submit 
> any rubbish without adjusting it to the journal’s guidelines. The only 
> people who see these disrespectful acts (towards work of editors, 
> especially volunteer work), and who associate it with author’s name, are 
> editors. If submissions were openly visible, the cost of submitting 
> random, unadjusted, low quality, undeveloped papers would be far higher, 
> since such disrespectful behaviour would be publicly linked to the author."
> 
> Here's what a journal that already does that said:
> 
> "public peer review and interactive discussion deter authors from 
> submitting low-quality manuscripts, and thus relieve editors and 
> reviewers from spending too much time on deficient submissions. [..] The 
> deterrent is particularly important, because reviewing capacities are 
> the most limited resource in the publication process." (Koop 2006)
> 
> A full open process model would make:
> 
> b) all subsequent comments, reviews, versions, should be open too
> 
> "To summarise, open-process academic publishing would amount to the 
> following being open: initial submission, editorial collective and 
> individual comments, peer reviews, further peer comments, author 
> comments back to reviewers, all the subsequent drafts, and the final 
> published or rejected text."
> 
> I support any proposals that go a step closer to any of the above.
> 
> Of course, there are valid arguments for a an in-between model too, for 
> partial openness of some of the publishing steps.
> 
> http://hackthestate.org/2009/12/16/open-process-academic-publishing-v1-2/
> 
> 
> 
> On 28/03/2012 16:17, Felix Stalder wrote:
> >
> >
> > On 03/28/2012 03:01 PM, Mathieu ONeil wrote:
> >> Hi Felix, Athina, all
> >>
> >> I think this is a very good idea and I'd like to add to it. My first
> >> suggestion is non-controversial:
> >> -if the article is submitted somewhere else and published [and we find
> >> out about it] we could publish a link to that versione
> >
> > Sure, but we want to publish original stuff, so that should not be an issue.
> >
> >> My second suggestion may be a deterrent for some authors:
> >> -we could publish alongside titles, abstracts, outcome (did not publish,
> >> etc) etc the reviews?
> >
> > Rather not. Reviews without the full article don't make that much sense.
> >
> >
> >> Another question is where this would appear on the site. There would
> >> need to be a specific page / groupe of pages. What would we call it?
> >
> > I would do a page called "submission queue" where we could list
> > Title, Author, Abstract, Date of Submission, and state (under review,
> > published, rejected, withdrawn) and I would order this reverse
> > chronologically based on date of submission.
> >
> > Felix
> >
> >
> >
> >> cheers
> >>
> >> Mathieu
> >>
> >> Le 03/28/12, *Athina Karatzogianni *<athina.k at gmail.com>  a écrit :
> >>> Hi All,
> >>>
> >>> I wonder whether we can do what Felix suggests but publish in a
> >>> separate place inside our site papers that didnt make it and have an
> >>> archive that way of everything which was ever submitted to us, which
> >>> is transparent. There might turn out to be lets say 30 papers over
> >>> five years (i doubt there d be more than that) or whatever and we ll
> >>> have a live record of this, if the authors who submitted them give
> >>> their consent. We d have to get consent when they submit to us in the
> >>> first place (in case this doesnt work out can we publish in the didnt
> >>> make it archive?) Its one way to do it and we might be the first
> >>> journal ever to do so...
> >>> why not? theres plenty of space online to do stuff like that
> >>>
> >>> Athina
> >>>
> >>> On Wed, Mar 28, 2012 at 12:57 PM, Felix Stalder<felix at openflows.com
> >>> <mailto:felix at openflows.com <felix at openflows.com>>>  wrote:
> >>>
> >>>
> >>>      I'm not sure if this is a good way, since we might end up
> >>>      publishing the
> >>>      crappy version, whereas others get the good one.
> >>>
> >>>      But there could be an intermediary step. For example, one might
> >>>      publish
> >>>      a list with all submissions (Name, Title, Abstract, and date of
> >>>      submission) so it gets at least transparent if people are submitting
> >>>      here first, and the take it somewhere else.
> >>>
> >>>      Felix
> >>>
> >>>
> >>>
> >>>
> >>>      On 03/27/2012 02:29 PM, Mathieu ONeil wrote:
> >>>      >  ps. I'll answer my own question: a way to do that would be to
> >>>      explicitly
> >>>      >  state that a version of all submissions will be published, so
> >>>      that by
> >>>      >  the act of submitting to us authors are in fact agreeing for us to
> >>>      >  publish something.
> >>>      >  However in this scenario:
> >>>      >  -we may end up publishing more (duly signaled as such) crappy
> >>>      articles
> >>>      >  than we would wish, and
> >>>      >  -we would be limiting the freedom of authors
> >>>      >
> >>>      >  On 03/27/12, *Mathieu ONeil *<mathieu.oneil at anu.edu.au
> >>>      <mailto:mathieu.oneil at anu.edu.au <mathieu.oneil at anu.edu.au>>>  wrote:
> >>>      >>  Hi Christian, all
> >>>      >>
> >>>      >>  Thanks for commenting. I can't respond address your points straight
> >>>      >>  away but I am curious about how you propose to implement this
> >>>      part of
> >>>      >>  your proposal : "((publishing all versions of a paper from the
> >>>      first
> >>>      >>  submitted one (or, at least, the last negotiated version of each
> >>>      >>  paper))) __without allowing the authors to pull out.__ "?
> >>>      >>  How do we stop people from pulling out? Sign a blood oath over the
> >>>      >>  Internet? ;-)
> >>>      >>
> >>>      >>  cheers
> >>>      >>
> >>>      >>  Mathieu
> >>>      >>
> >>>      >>  On 03/27/12, *Christian Siefkes *<christian at siefkes.net
> >>>      <mailto:christian at siefkes.net <christian at siefkes.net>>>  wrote:
> >>>      >>>  Hi Mathieu and all,
> >>>      >>>
> >>>      >>>  On 03/26/2012 04:17 PM, Mathieu ONeil wrote:
> >>>      >>>  >  Openness undoubtedly has great virtues, but in the case of
> >>>      academic
> >>>      >>>  >  publishing it can also generate some bad side-effects.
> >>>      >>>  >
> >>>      >>>  >  For this issue of JoPP five papers were sent out for review.
> >>>      Three
> >>>      >>>  of the
> >>>      >>>  >  papers will be published with reviews and signals. Two other
> >>>      papers
> >>>      >>>  were not
> >>>      >>>  >  great. Reviewers worked long and hard to address
> >>>      shortcomings and make
> >>>      >>>  >  suggestions.
> >>>      >>>  >
> >>>      >>>  >  One author decided that it would not be possible to make these
> >>>      >>>  adjustments
> >>>      >>>  >  though much time kept being added.
> >>>      >>>  >
> >>>      >>>  >  The other agreed to make changes but then used the time
> >>>      excuse as
> >>>      >>>  well as
> >>>      >>>  >  sickness.
> >>>      >>>  >
> >>>      >>>  >  There is nothing preventing either author from now
> >>>      submitting their
> >>>      >>>  >  much-improved papers to another journal...
> >>>      >>>  >
> >>>      >>>  >  In my view, we should try to address this obvious waste of
> >>>      reviewer
> >>>      >>>  (and
> >>>      >>>  >  editorial) work/energy.
> >>>      >>>
> >>>      >>>  hmm, isn't this a problem of being (maybe) not open enough
> >>>      instead of
> >>>      >>>  being
> >>>      >>>  too open? In the experience from my own academic this, this is
> >>>      a quite
> >>>      >>>  possible scenario in the traditional peer review process:
> >>>      reviewers send
> >>>      >>>  criticism and suggestions, the author might then revise the
> >>>      paper and
> >>>      >>>  send
> >>>      >>>  back a revised version, or submit the revised version elsewhere.
> >>>      >>>  Especially
> >>>      >>>  if a paper is re-submitted by multiple journals (after being
> >>>      refused
> >>>      >>>  -- with
> >>>      >>>  reviewer feedback -- by each of them), it would cause
> >>>      reviewers a lot of
> >>>      >>>  work. (Say if there are 3 reviewers per paper and you submit it
> >>>      >>>  sequentially
> >>>      >>>  to 4 journals, you would already occupy a dozen reviewers,
> >>>      while none of
> >>>      >>>  them would benefit of the work already done by others, since they
> >>>      >>>  don't know
> >>>      >>>  about it.) Also, if you re-submit a text sufficiently often, it
> >>>      >>>  becomes more
> >>>      >>>  and more likely to be accepted somewhere by pure chance, almost
> >>>      >>>  regardless
> >>>      >>>  of the quality of the paper, I would presume.
> >>>      >>>
> >>>      >>>  The only chance to avoid that would be more openness, not
> >>>      less, i.e.
> >>>      >>>  publishing all versions of a paper from the first submitted
> >>>      one (or, at
> >>>      >>>  least, the last negotiated version of each paper), without
> >>>      allowing the
> >>>      >>>  authors to pull out. Not sure if we want to go this way, but
> >>>      blaming
> >>>      >>>  "openness" for the shortcomings of the current approach
> >>>      strikes my as
> >>>      >>>  definitively wrong.
> >>>      >>>
> >>>      >>>  Best regards
> >>>      >>>  Christian
> >>>      >>>
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ourproject.org/pipermail/jopp-public/attachments/20120328/de7882ff/attachment-0001.htm 


More information about the JoPP-Public mailing list