When papers are submitted to journals, editors/editorial boards often make the first selection, based on topic, originality, presentation etc.The second step in evaluation is the peer review by experts in the field. Peer review is usually "blind" in that the expert reviewer remains anonymous and does not know who the author is. The reviewers make recommendations for improvements, if any, or suggest rejection.
Although there have been many suggestions for the modification of the peer review process due to its well documented shortcomings (e.g., intentional delay for personal gains, bias toward the author), there is general agreement that peer review provides quality control.
Peer review is compatible with Open Access (OA). The Directory of Open Access Journals (DOAJ) seeks to include only journals that apply some form of quality control - either at the editorial level and/or through a peer-review process. (DOAJ, About http://www.doaj.org/doaj?func=loadTempl&templ=about)
The impact factor, which is based on the citation indexes in the Web of Science and published by Thomson Corporation, is a measure of identifying the predominance or lack thereof of specific journals in particular disciplines. The impact factor is calculated by dividing the number of citations in one particular year to articles published in two previous years within one specific journal by the number of articles published in the same previous two years of the same journal. The impact factor of journals, whether open access or not, can be determined provided they are indexed in Web of Science, and can assist authors in deciding in what journal within their discipline to publish.
A study by McVeigh for Thomson Scientific in 2004 found that open access journals had competitive numbers in terms of impact factor ratings. Another study by Testa and McVeigh for Thomson Scientific in the same year found similar results.
It should be noted that the impact factor is a quantitative measure and cannot be equated with the quality of individual articles and that not every journal published is indexed in the Web of Science databases. Furthermore, studies have recently questioned the validity of data used to calculated the impact factor since the number of citations refer to a different set of articles as the number of articles to which the citation compare (see Rossner, M., Van Epps, H. and Hill, E. "Show me the data ", Journal of Cell Biology (2007): 179: 1091-1092 http://www.jcb.org/cgi/content/full/179/6/1091).
Over the years, there has been a growing concern over the exclusive significance of a journal's impact factor in situations where promotion and tenure is decided. Since this element is likely to remain an important factor for hiring and promotion, numerous open access journals are measured in terms of their impact factor and compare favourably to for-profit and not-for-profit journals of the same caliber.
With changes in scholarly communication the traditional process of tenure and promotion has received greater attention and there have been efforts to recommend different forms of research output in evaluating promotion and tenure, including digital scholarship and other forms of new media.
- Report from the Association of Universities and Colleges of Canada (2006)
- Modern Language Association Report on Evaluating Scholarship for Tenure and Promotion (2007)
- Create Change Canada: Rewards of Scholarship