Browse by title

Research output is evaluated by a combination of quantity (the number of research articles published) and quality, where quality is typically measured by the status of the journal in which the research is published. The citation-based approach (also called the citational approach and bibliometric method) has become the dominant method used to rank journals, reflecting in part the increasing availability of citation data. For all practical purposes, citation analysis began with the publication of the Science Citation Index (SCI) in 1961. Measuring journal quality by citation indices is questioned on the grounds that citations do not necessarily reflect impact. Although a long list of arguments can be presented against the use of citations, the citation approach is less subjective than other approaches to journal ranking.

POP has been found to be detrimental to the health and well-being, and a threat to the job security and livelihood, of academics. In response to the challenge posed by POP, a tendency has arisen to indulge in research misconduct that takes various shapes and forms. Misconduct includes misappropriation of ideas, plagiarism, self-plagiarism, impropriety of authorship, failure to comply with legislative and regulatory requirements, violation of generally accepted research practices, falsification and fabrication of data, failure to support validation of research, and inappropriate behaviour in relation to suspected misconduct. One indicator of the rising incidence of misconduct is the high frequency of paper retraction.

A consequence of the POP culture has been the proliferation of published research at a rate that is disproportional to the advancement of human knowledge. However, most of the published work goes unnoticed even by fellow academics. The POP culture has adverse consequences for the quality of published research, and it impedes the discovery process. Furthermore, the POP culture Slows down the dissemination of knowledge, drives a wedge between published research and reality, makes research findings unreliable and biased, and introduces bias against research from developing and non-English speaking countries and against non-article publications. POP also has an adverse effect on non-research activities, including teaching and community service.

The POP culture has led to a rapid growth in the journal industry, which has become an oligopolistic market, and changed authorship patterns away from single-author papers. The proliferation of scholarly journals, resulting from the POP culture, is disproportional to the growth in human knowledge. The rise of predatory journals is also a consequence of POP. The proliferation of journals, predatory or otherwise, has led to a rapidly declining quality of published research. Other related consequences of the POP culture are the growth of parasitic activities, such as the organization of low-quality conferences, and the rise of elitism and class structure in academia. POP has brought with it the fractional author, as papers with a large number of authors have become the norm.

Journal ranking lists may be constructed by government bodies (such as the Australian Research Council), joint ventures (such as the Australian Business Deans Council and the British Chartered Association of Business Schools), as well as societies, universities and even departments within universities. The construction of these lists is costly while they are potentially harmful. Three explanations can be put forward for why the production of journal ranking lists is a thriving industry, despite the problems associated with these lists: (1) the administration of journal lists and the pressure put on academics to comply create jobs for people who would otherwise have no jobs; (2) it is easier to check each item against a list to reach the conclusion that the output is good or bad, than evaluating research output on its own merits by reading the material; and (3) any harmful enterprise has its own beneficiaries who want to maintain the status quo.

Alternatives to the citation-based approach to journal ranking include the opinion-based approach, the frequency of downloads approach, and the subscription-based approach (market-based ranking). The opinion-based approach can be distinguished from other approaches because the other three provide quantifiable measures of journal quality or impact and because it is least objective and most subjective. It is also the most expensive because administering a survey is costly when the data required for journal ranking by citations, downloads and subscriptions are available free of charge. The debate seems to be about how journals are ranked when it should be about whether or not journals should be ranked. At least three problems make journal ranking a hazardous endeavour.

Peer review, which determines what does and does not get published, is so problematical that alternatives are being sought. Because of the problems associated with the process, journal ranking is too problematical to be useful. Peer review, which can be described as a stochastic process, has so many shortcomings, including methodological and ideological bias, bias against new ideas, confirmation bias, obsession with finding faults, reckless and dishonest practices, referee incompetence, lack of scrutiny, and delays. Several alternatives to the current per review practices have been suggested, including the cascading and portable peer review models. For all the problems associated with the practice, looking for alternatives to peer review is a matter that is taken seriously.

The journal ranking craze has become international, spreading worldwide in the spirit of globalization. Journal ranking is the means to a big end, that of ranking universities, departments and disciplines for the purpose of allocating scarce funds. The link between research evaluation and journal ranking is that journal ranking is central to the evaluation of research. According to the OECD, 13 countries have been identified as using systems of ex post research output evaluation for the purpose of determining the distribution of government funding. These countries have been following the lead of the U.K., where research evaluation started in 1986 under the auspices of the Thatcher government. The overall range of indicators used by other countries is similar, but various combinations and weightings are employed.