2009/05/27 0:04 CEST ARTICLES and PAPERS

The Catch 22 of Academic Publishing

Publish or perish“. You’ve heard the phrase, right? Well, apparently, getting published in the first place is not as easy as it seems, and the peer-review process may not as objective and unbiased as you may think. If you’re in (that is belonging to the right academic circles, and thus worthy of being published), you’re in, almost no matter what you write, but if you’re not in, finding someone willing to take you in is practically impossible, or is it?

It helps to be “next of kin”

Not long ago I came across an interesting blog post on interorganisational.org, titled Questions of science, science and progress,  referring to Ring a Ring o Roses, an article published in Journal of Management Studies that examined the gamesmanship of publishing in quality journals. And fair enough, it does appear that the some “established” authors seem to get published a lot more than other younger authors. That article made me think about my own efforts at getting published.

Periodical usefulness

Actually, this is not the first time that I have stumbled across this topic on interorganisational.org. In their post Ranking Academic Journals, they directed the reader towards Periodical Usefulness, a paper reporting the findings of an international survey of logistics academics on their views of journals used for research, teaching and outreach. But for now, let’s look at the ‘Ring a Ring o Roses’ article.

The Catch 22

Albeit I am working in a different field,  I do believe that some of their observations are found even in most fields of study, even in my own, Logistics and Supply Chain Management, as reported in the Periodical Usefulness article. As to Management Studies, the authors state that

A paper in one of the quality journals of Management Studies is much more important as a unit of measurement than as a contribution to knowledge. It measures academic performance and determines much academic funding. There is consequently some pressure to publish in quality journals. But quality journals are defined in terms that are themselves defined in terms of quality journals – a circularity that explains both the paper’s title and the frustration of those who do not mix in these circles.

Here’s the Catch 22: In order to be published in a quality journal, you need to belong to the right circle, but in order to be admitted to the circle you need have been published in a journal related to the circle in the first place. An impossible circular logic…

What makes a quality journal?

Are the number of publications an academic (or institution) has in quality journals a true measure of research output? Or is it that the interest in the measure exceeds the interest in the research itself? And what makes a quality journal anyway? Are these true indicators of quality:

  • the number of publications from top business schools?
  • the number of citations in other journals/publications?
  • the rejection rate amongst submissions?

On occasion, I act as a reviewer for journal submissions. Am I contributing to upholding quality?

Publishing: Fame and fortune, just fame, or just fortune?

Many academic departments are rewarded with funding according to the number of publications depending on where they are published. This may lead to gaming, because

the indicator supplants the indicated

and now everybody plays the game,

  • universities
    • by encouraging their their staff to publish
  • authors
    • by churning out paper after paper
  • editors
    • by asking for more papers for ‘special editions’

In addition, in order to get published, it is indeed advisable to write something a particular journal might favor:

Select your journals
Determine consensus in area of research
Do research
Extract from research the bits that fit each journal
Write paper and submit to journal

Is that how it’s done? Simply by playing the game, by submitting to the same “quality” journals that were quality journals decades ago, because that is where “quality” sits?

There is a profusion these days of printed, and more recently, online journals…an “academic bubble” perhaps? But all new journals cannot be quality journals, can they? Are they just the result of frustrated academics who could not find other research outlets? Or are they the the result of editors eager to further promote already existing and well-known (‘quality’) authors?

Should I laugh or cry at this kind of game? Maybe I shouldn’t take this whole publishing thing so serious. After all, as the authors put it

Publication in quality journals should not be held in such awe.

Why? Because

more than half of all academic papers are never cited anywhere, and the majority of academics never receive as many as three citations in a lifetime.

I find that last statement hard to believe. Are there actually articles around that are never cited by anyone? But then again, failure to be cited has probably more to do with utility than quality, and consequently, when writing academic articles one should think usefulness, not quantity  nor funding.

Maybe it is what you do that should matter, not what you publish? This article in The Economist, referenced in this blog, had an interesting twist on the matter: Publish and be wrong.

 

References

  • Gibson, B., Hanna, J., & Menachof, D. (2004). Periodical usefulness: an international perspective International Journal of Logistics, 7 (3), 297-311 DOI: 10.1080/13675560412331298482
    A working paper version can be downloaded at SSRN
  • Macdonald, S., & Kam, J. (2007). Ring a Ring o? Roses: Quality Journals and Gamesmanship in Management Studies Journal of Management Studies, 44 (4), 640-655 DOI: 10.1111/j.1467-6486.2007.00704.x

Related

  • http://nasw.org/users/mslong Michael Scott Long

    The general “catch 22″ is also seen in research funding, issuing of awards, job promotions, and throughout life in general, presumably as a consequence of egos and malicious competition.

    • Jan Husdal

      You may be on to something here. There’s always some “in”-crowd somewhere that seems to be getting all the good stuff, while most of us have to settle for scraping the breadcrumbs of their leftovers.

  • Pingback: How to Write a Really Good Research Paper « Successful Researcher

  • http://aclinks.wordpress.com/ Successful Researcher: How to Become One

    Great post! I linked to it in my paper writing tips.

    • Jan Husdal

      Thanks! I’m glad you enjoyed it.

  • http://www.laughingspider.com Peter Charlot

    Thomas Kuhn (1922 – 1996) is the grand master of this insight. His, “The Structure of Scientific Revolutions” goes into great detail on just how science is manufactured. My own insight is that neuroscientists reflect the process they are looking for.

    • http://www.husdal.com Jan Husdal

      Kuhn…the grand master of paradigms…you have a point here. Maybe we need a shift in paradigms as to the value of academic publishing?

  • http://blog.lib.umn.edu/levin031/transportationist/ David Levinson

    ISI and the like are pseudo-metrics. Journals have to have them, but everyone is fooling themselves to believe that they indicate anything real.

    Quality cannot be directly assessed numerically by citations, the best you can hope is that it is accepted by a peer-reviewed journal (which hopefully means the reviewers did not see any obvious flaws). My sense is that for most academic peer-reviewed journals in transportation and planning, the average quality of papers does not differ too much, but higher ranked journals may attract slightly more original work (and more submissions overall). Higher ranked journals probably have a somewhat higher rejection rate as well, but this is highly stochastic process, depending on the luck of the draw and randomness of reviewers, with a slight bias to accepting better papers and rejecting bad papers.

    Utility may have a hope being assessed by citations, but really it just depends on if you are in a hot field or sub-field, and who you are (early papers in a particular academic branch will be cited, as will papers by famous people and editors and people likely to be reviewers).

    At any rate, I like the philosophy of PLOS One
    “Rigorous Peer-Review
    Too often a journal’s decision to publish a paper is dominated by what the Editor/s think is interesting and will gain greater readership — both of which are subjective judgments and lead to decisions which are frustrating and delay the publication of your work. PLoS ONE will rigorously peer-review your submissions and publish all papers that are judged to be technically sound. Judgments about the importance of any particular paper are then made after publication by the readership (who are the most qualified to determine what is of interest to them).”

    • Jan Husdal

      Hello David,
      I was wondering when you would turn up here…and thank you for an insightful comment.

      Yes, you may be right that the average quality of papers doesn’t differ too much from paper to paper, as least as far as I have seen. And being “hot” or “cold” definitely matters, and sometimes luck plays a role, too: I still remember during the early days of the war in Afghanistan, soon after 9/11, when the only expert on Afghanistan CNN could come up with was some History professor who had studied the British invasion in the 19th century…I’m sure he never thought he ever would be on prime time TV given his obscure field.

      As to PLoS ONE, that sounds like a well thought-out attitude. If it is technically sound, the public IS the best judge. If it used, it is good; if not, it will soon be forgotten.

Selected partner

Incapsula

Selected advertiser

Selected books from amazon.com