© 2003 American Thoracic Society
Impact Factor and the JournalThe Journal's impact factor rose by 0.611 units in 2002 to reach 6.567 (Figures 1 and 2) . This is the largest ever one-year increase for AJRCCM. What does it mean?
The Institute for Scientific Information has published impact factors of biomedical journals since 1974 (1, 2). Despite serious limitations, this citation metric has become the most widely used index for ranking the overall quality of biomedical journals. Why are editors so attentive to impact factor (3–12)? They see science as a collective and cooperative enterprise: an edifice of knowledge built by contributions from successive generations of researchers (13). Each new article is based on the preceding research that the author cites, and a published article is of little significance unless other scientists cite it and build on it. (Between 22% and 46% of medical research articles attract not even one citation [14, 15].) When authors cite an article, it usually means they found it helpful when pursuing their own research. Editors recognize that researchers are in effect casting votes on the importance of a journal every time they cite it (16). Listing references at the end of an article is a trademark of scholarship. Within a year of its launch in 1665, authors were placing references in the margins of the Philosophical Transactions (17). Footnotes first appeared in 1683 and remained the preferred method of referencing until the numbers grew too large. Endnotes first appeared in 1876 and are now the standard (17). Along the way, the number of references per article increased from less than 5 before 1700 to over 40 in 1950. Investigators cite the work of previous investigators for several reasons: motivating new research, according credit for ideas, providing greater detail on methodology, highlighting errors in previous research, and persuading readers of the legitimacy of a new claim (18). In a survey of 26 authors on the motivation behind more than 900 citations, the significant reasons were to accord positive credit and to demonstrate that the author was up to date in a research discipline (19). But the dominant reason by far was persuasiveness: an author wanting to establish legitimacy for a new research claim. How well a reader is persuaded, however, will depend on the reputation of the scientist being cited and peer-review standards of the journal in which the research was published. Because it is notoriously difficult to measure scientific quality, governments and funding agencies—especially outside the United States—rely heavily on impact factor. In Germany and Italy, agencies decide departmental funding by plugging impact factors into formulae (20). The Spanish parliament has sanctioned this approach (21, 22), and it is claimed to have led to improved scientific rigor and productivity (21, 22). An equivalent system is used in Quebec (23). In Finland, $7,000 is added to departmental funding for a unit increase in impact factor of the journal in which each article is published (20). We can rail against the foolishness of such a reward system, but we also must see that it carries major implications for the future of research in our field. The impact factors of pulmonary and critical care journals (Figure 2) lie considerably below those for leading journals of other internal medicine subspecialties: Journal of the National Cancer Institute (14.500), Gastroenterology (13.440), Circulation (10.255), Blood (9.631), Diabetes (8.256), and Arthritis and Rheumatism (7.379), to list a few. Simple arithmetic predicts that funding for pulmonary and critical care research may fall considerably below that of other clinical disciplines in European countries. When confronted by criticism, advocates of a rewards system based on impact factor counter that it is concrete as opposed to being anecdotal (20, 23–25). Promotion committees may not use the term impact factor explicitly, but they are using it implicitly whenever they base advancement on number of papers published and journals in which they appeared, rather than a reading of each paper. Even less defensible is the practice of basing academic promotion on the number of dollars a researcher received in funding, without checking whether the money yielded publications of any importance (26). On a list of journals within a particular research field, advocates of bibliometric indexes contend that a journal's impact factor will correlate well with the quality of research it publishes (20). Recent research supports the contention. When 264 physicians (about evenly divided between researchers and practitioners) were asked to rate the quality of nine general medical journals, impact factor explained 82% of the variation in ratings (r = 0.91) (16). The high correlation may reflect not only the reputation of a journal but may truly reflect scientific quality. In another study of 243 original research articles from 30 journals, an objective measure of scientific quality was more closely related to the journal's citation rate and impact factor than to any other variable (27). Impact factor is measured as a ratio. The numerator of the impact factor for AJRCCM in 2002 was the number of citations in all journals in 2002 to articles that had been published in AJRCCM in 2000 and 2001 (8,708). The denominator was the number of articles published in AJRCCM in 2000 and 2001 (757 + 569 = 1,326). Although pleased that the Journal's impact factor increased by 0.513 in 2001 and by 0.611 in 2002 (Figure 1), both numbers were brought down by the unusually large number of articles published in 2000. In that year, we published 26% more articles than in 1999 so as to shorten our publication lag—then headed for 10 months. (Our publication lag is now less than 3 months.) In contrast to the impact factors for 2001 and 2002, the denominator next year will not include the articles of 2000. The formula for impact factor captures three main qualities of a journal (28). One, the relevance of the journal to researchers in an active field of inquiry. Two, the ability of editors to discriminate between submissions of the highest quality and the next tier. Three, the speed that the journal publishes articles after acceptance. Only the first two relate to scientific quality. Nevertheless, speedy publication of accepted articles is also important for the advance of science. If all pulmonary and critical care journals were to achieve a publication lag of three months or less, the impact factor of every journal in our disciplines would increase. In 2000, AJRCCM experienced an increase in submissions of almost 300—equal to the total annual submissions of many pulmonary journals. Faced with a simultaneously growing publication lag, the Journal necessarily tightened its standards. This step was greatly aided by the introduction of online submissions, enabling us to make the selection of reviewers more scientific. Through use of descriptor numbers and an online database of over 6,000 reviewers, we now maximize the match between manuscript content and reviewer expertise. As I have emphasized on several occasions (13, 29), peer review is the primary instrument for ensuring quality control at AJRCCM (30). And the expectations and demands of scientists who serve as reviewers are the major determinants of the standing of AJRCCM among other scientific journals (31). The limitations of impact factor have been discussed by many (20, 23, 32). It is ironic that science has not produced a more scientific tool for measuring scientific quality. But deficiency in assessing excellence is not limited to science. "We pretend that so many courses, so many credits, so many books read add up to an education. The same is true of research. We seem immensely satisfied with the outer husk of the enterprise—the number of dollars spent, the number of publications. Why do we grasp so desperately at externals? Partly because we are more superficial than we would like to admit," lamented John Gardner (33) (Secretary of Health, Education, and Welfare in the Johnson administration). How true. Much more important than impact factor, we were pleased to learn from the recent survey of the American Thoracic Society that members considered AJRCCM as the first among all services provided by the Society (34–36). FOOTNOTES Conflict of Interest Statement: M.J.T. is editor of AJRCCM and is responsible for ensuring its quality. He receives a fixed stipend from the American Thoracic Society and is not affected financially by the impact factor of the Journal. REFERENCES
This article has been cited by other articles:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
