Impactology, Impactitis, Impactotherapy

In 2004 the impact factor (IF) of Revista Española de Cardiología reached 1.802.¹ This figure for the popular bibliometric indicator was nearly double our previous IF, and represented a significant achievement not only for our journal, but for all medical journals in Spanish.² We editors thus feel that this is a magnificent opportunity to review the underlying foundations, implications and limitations of the bibliometric indicators that are now used most widely, and to offer some reflections on our editorial procedures and policies.

Calculating Bibliometric Indicators

The challenge of performing a balanced assessment of the merits of research remains unresolved. Methods presumed to be objective, e.g., citation rates and the IF, have been proposed to evaluate the quality of research in biomedical journals.^3-9 Data for citations are obtained from a database maintained by the Institute for Scientific Information (ISI) which now forms part of a private institution (Thomson Scientific) with its headquarters in Philadelphia. The ISI continuously computes and classifies all citations in the reference sections of articles published in an extensive list of science journals from around the world. These references are processed in their databases to determine how many times a given article has been cited during a specific period, and by whom. Since 1963 these data have been compiled in the Science Citation Index (SCI), and more recently in SCI-Expanded. Lists of publications by different authors make it possible to calculate how many citations a researcher has garnered in a given year, whereas citation rates for science journals can be measured as the IF, which calculates the mean percentage of citations received, divided by the total number of articles published in a given journal. Since 1975 journal IFs published annually in the SCI's Journal Citation Reports (JCR) have provided a widely accepted way to objectively compare the scientific quality of journals.^1-6 In principle the IF was developed by ISI simply as an internal indicator of the relative quality of scientific publications to help them decide which journals to include in their database.¹⁰ Later the popularity of the indicator spread because of its simplicity, and was subsequently consolidated when it was shown that the IF correlated to an acceptable degree with several parameters of quality in biomedical journals.¹¹ As we discuss below, although the IF is currently considered one of the most widely accepted indicators of a publication's visibility and prestige, it is not without important limitations.^7-10

The ISI indexes almost 8500 journals in 200 different knowledge areas.¹ In the area of biomedicine, 16 of the indexed journals are in Spanish, and 11 of them are published in Spain. Although it was Gross et al¹² in 1927 who first suggested the usefulness of counting references in science articles, the IF itself was "invented" 50 years ago by Eugene Garfield, founder of ISI, as a simple method to compare different journals regardless of their size.^3-6 This was an attempt to correct for the increase in citations to a journal simply as a result of the journal publishing more articles. The journal's IF for a given year is calculated as the number of citations published in that year (in other journals indexed by ISI) to articles published by the target journal during the 2 preceding years (numerator), divided by the total number of citable articles published in the target journal during the 2 preceding years (denominator).^3-6 As noted, the ISI is an independent organism and it is frequently hard to establish accurately how many items are considered citable (substantive articles, source items, citable items). Moreover, the IF tells us only how frequently articles in a given journal are cited in the short term.^3-6

Another interesting bibliometric indicator provided by the ISI is the cited half-life. This indicator reflects the number of years (counting backward in time) that comprise 50% of the citations received by the journal during a given year. In practical terms, this indicator expresses the time during which articles published in the journal are cited and are assumed to be useful to the scientific literature.^1-6 A similar indicator­the inverse of citation half-life, in a sense­is the half-life of articles that are cited by the journal of interest (the citing half-life).

The immediacy index, a measure of citations received during 1 year by articles published in that year divided by the total number of citable articles in that year, provides an approximate idea of the journal's editorial dynamics, i.e., how current the journal's coverage of the discipline is.

Another parameter is the absolute citation count. This is the total number of citations a journal receives in a given year for any article published previously. This approach partly counteracts the problems arising from the short-term view typical of the IF, although it has its own limitations in that it favors older journals and journals that publish larger numbers of articles.⁵

Limitations and Abuses of Bibliometric Indicators

The main problems that arise from the use of bibliometric indicators can be classified into 3 groups: 1) limitations intrinsic to the indicators; 2) inappropriate use of the indicators; and 3) editorial implications. Per O. Seglen⁷ provided an excellent summary of the fundamental problems that result from the use of the IF of biomedical journals to evaluate scientific research (Table 1).

Limitations Inherent to Bibliometric Indicators

Citations received. The citations received by a journal depend critically on the number of journals in a given subject category that are included in the ISI database.^3-10,13 This is the case because only citations from this select club of journals are counted. The fact that this database includes a relatively small number of journals whose language of publication is not English accounts for the bias that clearly favors English-language publications.^7-10,14-19

Indeed, there have been proposals to calculate a modified IF for journals that are published in languages other than English, so as to consider citations from journals not included in the ISI database but that are published in the same language as the target journal.^20,21 The tendency toward self-citation among researchers in the USA has also been noted--a tendency that further increases the number of citations these scientists receive.^7,22-24

Basic science journals usually receive large numbers of citations.^3-10 These publications are centered on recent original research, are often present in large numbers in the ISI database, and are cited not only in basic research articles but also in practitioner-oriented articles. It is important to realize that citation habits differ between areas of knowledge, and that in some disciplines the rate of growth and development of knowledge is such that long citation half-lives are the norm. Other fields where growth is more dynamic and the literature quickly becomes obsolete benefit clearly from bibliometric indicators like the IF, which reward short-term citations.^3-10 In response to these differences in growth dynamics, proposals have been made to calculate the IF for periods of 5 or even 10 years rather than 2 years.^3-6 Moreover, it is readily seen that clinicians might read important articles in practice-oriented journals that lead them to significantly change their daily practice, but that they will never cite in new publications. In comparison to specialty journals, general medical journals also tend to benefit from the IF.^7-10 Aside from the advantages of their wide field of influence, these publications can change tack to better cover topics that currently attract the most interest. Review journals are also highly favored by the IF, as they compile large numbers of citations and their articles are frequently cited as a single, global source of information on a topic.^3-10 For these reasons is has been suggested that it may be worthwhile to correct the IF within subject areas so that comparisons can be made across disciplines.^25,26

Journals that publish more articles have more chances of being cited, and in this sense the IF is useful since it "standardizes" the factor across journals of different sizes.^3-6 However, it has also been shown that long articles, those with many authors, items on interdisciplinary topics and collaborative studies involving authors from different centers are also highly cited.^3-6,27 Publishing 2 articles on the same topic in the same issue of the journal also increases the chances that they will be cited.⁶ Some journals produce spotlight or monographic issues on highly timely aspects that attract more citations than regular issues.^7-9

Self-citation of articles published in the same journal as the citing article can also raise specific problems, as this factor is not taken into account (i.e., self-citations are treated the same way as citations to items that appeared in other journals) when the IF is calculated.^3,10 The trend toward higher self-citation rates can be considered justifiable under certain circumstances, as in the case of national journals--especially those not published in English--and in the case of journals that deal with very narrow topics or areas that few other publications cover.^7,28 Garfield²⁹ accepted that editors might favor self-citation as long as the citations were scientifically relevant, as a way to encourage readers to compare the published article with other easily accessible studies in an equivalent context. Self-citation may also be favored in journals that are owned by a scientific society, although this practice eventually devaluates the IF that is attained.^29-32 To avoid this problem, some researchers have suggested calculating a corrected IF that omits self-citations from the numerator.^10,22,25

A final consideration is that journals that are more widely disseminated receive more citations.^3,10 Moreover, publications with agile editorial practices not only attract authors by offering the possibility of publishing their results rapidly, but also benefit from the process in that it allows fewer citations (that count toward the IF) to expire while the manuscript is undergoing review or during the period between acceptance and publication.^3-9,33 These considerations show that straightforward technical factors have a substantial bearing on how different bibliometric indicators are calculated.^6-10

Published articles and citable articles. It is important to note that not all articles count as "citable articles" in the calculations performed by the ISI.^3-9 Editorials, letters to the editor and congress abstracts do not count as citable items in the denominator of the IF; however, both citations to these documents and the articles they cite are considered valid references for the IF. Keeping a journal "dynamic" by publishing frequent editorials and devoting substantial space to correspondence has direct benefits not only on the immediacy index, but also on the IF.^3-10 Paradoxically, research letters can have the opposite effect: they count as citable items in the denominator, but because they are short and relatively uninteresting, they tend to generate few citations. In fact, this is what seems to have caused a decline in the IF of The Lancet^34,35; nevertheless, the editors stood by the editorial policy they had implemented to foment this type of correspondence. To avoid these problems, Garfield⁶ proposed a somewhat more complex method (Journal Performance Indicators) which made it possible to track and evaluate all citations to "citable" articles over time. This information can be now obtained on the Internet by subscribing to the ISI's Web of Knowledge.¹⁷

Reducing the total number of manuscripts that are published (i.e., reducing the denominator) is another--obviously risky--way to increase or sustain the publication's IF. One interesting study showed that in the 5 most prestigious medical journals, the IF correlated inversely with the number of articles published.³⁵ This is why some editors pay heed to the advice not to publish too much, and to chose what they publish wisely.

Inappropriate use of Bibliometric Indicators

It should be recalled that the IF applies only to journals, and that articles and authors are not said to have a certain IF but to have received a certain number of citations. On the assumption that the journal is representative of its articles, one proposal has been to estimate an author's scientific achievements by adding the IF of all journals in which he or she has published.^36-40 Because some institutions use the IF of the journals in which scientific articles are published to reward research (especially in Europe),^39,40 many authors have made efforts to publish in journals with the highest IFs. This completes the vicious circle that rewards journals with higher IFs by attracting more researchers, and punishes journals with a lower IF despite their broad dissemination and excellent acceptance by readers.

However, it has been shown that publishing an article in a journal with a high IF does not guarantee that the article will be highly cited.⁷ Because the distribution of article IFs is neither homogeneous nor gaussian and the numbers vary widely, a journal's IF is not representative of the IFs of all its articles: the most cited 15% of the articles yield 50% of all citations received.⁷ Although researchers recognize that the distribution of citations is asymmetrical, citations are still evaluated on the basis of estimates of the mean (such as the IF) instead of the median.^7,41

The number of citations a research article receives does not necessarily indicate its importance in intellectual terms. Of the 25 journals with the highest IF, 60% of them publish only reviews.¹⁰ Why then is their impact so high if they do not publish novel research?¹⁰ Many original contributions are quickly incorporated by the scientific community into the stable body of knowledge, and the original reference may lose interest before it has been heavily cited. However, merely technical or methodological contributions within a specific area may become obligatory references and remain so for many years.⁷ Meanwhile other truly pioneering studies may take years to be accepted by the international scientific community, and may need to "ripen" before they attain the recognition they deserve. It is only in the long term that such studies are judged appropriately by bibliometric indicators. Unfortunately, all the corrections proposed thus far to offset the limitations of the IF undermine its simplicity and have not taken hold within the scientific community.^13,25,26

We should recall that the ultimate goal of medical research is to improve the health of the populations we serve. Although the bibliometric indicators described above are useful to evaluate the quality of scientific study, they can not estimate the impact it will have on health.⁴² Therefore a number of countries^42,43 are looking into the development of new indicators that would shed light on the so-called "social impact" of research and determine the actual benefits of research for health.

The limitations noted above should reassure researchers and provide further arguments they can use to defend their work before those who, in our kafkaesque world,³⁸ prefer to count rather than read the articles they judge.^36,37,41-44 What really matters is an article's scientific content--its original contribution to knowledge--and nothing can replace reading and assessment, ideally by a panel of experts.^36,37,41-44 However, both authors and institutions are now frequently judged and evaluated (not only for the purpose of awarding research grants) on the basis of their publications in high-IF journals.

Bibliometric Indicators: Editorial Implications

From a publisher's point of view the IF can be considered a suitable currency for comparing the scientific quality of different journals^11,41 and a useful aid to both librarians and publishers.⁴⁴ Although the IF is not a perfect instrument there is currently no better alternative, and despite its important limitations as mentioned above, the IF is widely accepted by the scientific community. In fact, most of the problems arise from its misuse rather than from defects inherent in this parameter. Since a publication's international success is closely linked to its IF (which can be considered the journal's calling card), it would seem desirable to try to optimize this bibliometric indicator with reasonable editorial measures. Otherwise many worthy national journals might be condemned to ostracism despite their critical importance to readers in intellectual terms. Moreover, the IF can be used by editors as a means rather than as an end in itself, since attracting high-quality articles can help improve their journal's scientific status.

The best ways to increase a journal's IF are to improve the scientific quality of the articles and facilitate the journal's dissemination.^2,45,46 Proactive decisions aimed at broadening the journal' area of interest and influence can also capture a wider audience, and eventually lead to improvements in the IF.⁴⁵ Reducing the time needed for peer review and the lag between acceptance and publication, encouraging review articles or items covering recent methodological advances, and favoring self-citation are additional tools the editor can use to enhance the journal's IF.^3,10 However, editorial maneuvers aimed at raising the IF at any price are not in the least justifiable.^29-32

When the IF improves, editors editorialize (this article is in fact an excellent example), whereas when the IF declines, they generally remain silent.⁴⁷ Interestingly, it is easy to find examples of editorials that contain substantial numbers of references to articles published recently in the editor's own journal.^8,33,48,49 In fact, this extensive Editor's Page article with its 5 appropriate self-citations might raise our IF for the year 2005 by as much as 0.02, assuming that the number of citable articles remains stable.

Citations Versus Reading Biomedical Journals

Trying to determine whether journal articles are actually read is a challenge. This issue can be approached by analyzing the number of visits or full article downloads (in html or pdf format) from the journal's website (www.revespcardiol.org)² via search engines and databases, or via PubMed.⁸ Facilitating electronic connectivity is a key measure for improving knowledge dissemination, and the Web Impact Factor⁵⁰ has been defined for this purpose as the number of links that provide access to the journal's website. Moreover, the exponential increase in electronic publications means that we will soon be able to measure and compare data for visits to e-journal websites, and this in turn will soon make it possible to create access factors for individual articles as well as for journals.⁸ Paradoxically, we may be surprised to find that many widely cited articles are not read as often as their citation rates would suggest, and that a whole mythology has sprung up around authors able to cite much more than they read. It has been shown that availability on the Internet of the full contents of the journal online increases dissemination and eventually increases the IF.⁵¹ Nevertheless, an interesting development is that electronic search tools are acting as levelers in terms of the journals that are being located, and that researchers currently obtain the articles they are most interested in directly from the Internet, regardless of whether they were published in high-IF journals or elsewhere.⁵²

Other researchers have found that studying download rates for specific articles can be considered equivalent to the interest factor,⁴⁹ a number which makes it possible to identify immediately which articles the readers are most interested in. However, it should be recalled that these documents are often downloaded simply on the basis of their title, the authors' names, or the key words, before the abstract has been read.⁴⁹ Moreover, this approach does not take into account readers of the printed version of the journal. Another possibility is that readers may not find the downloaded and saved document of interest; conversely, it is hard to imagine that a document that was not felt to be interesting or at least controversial would be used as a reference. It has even been suggested that authors might fraudulently access their own articles many times so that they appear to attract many visits. Despite these limitations, it is clear that this new form of bibliometrics (webometrics) is here to stay; in this connection it is interesting to see once again that review articles and articles in special issues are consulted most frequently.⁴⁹ An immediate analysis of the original articles that have attracted the most interest might help editors to orient their journal toward those topics which their readers find most attractive. Indeed, many journals already point their readers toward their own "best sellers."

Where Does Revista Española de Cardiología Stand?

The IF achieved for the year 2004 consolidates Revista Española de Cardiología¹ as a high-quality international science journal. In 2004 we received a total of 997 citations, of which 427 were recent citations (2002-2003). Currently, Revista Española de Cardiología has the highest IF of all medical journals in Spanish. Moreover, our journal is ranked 28th by IF among the 71 prestigious journals included by the ISI in the cardiovascular subject category of the JCR. It is also significant that this notable improvement occurred while the percentage of self-citations remained constant in comparison to previous years. This confirms that the number of citations Revista Española de Cardiología receives from other publications continues to grow. In the future we should aim to maintain the favorable trend in our IF, the slight ("sawtooth") yearly variations notwithstanding,^53,54 but despite editorial strategies intended to enhance the quality and visibility of our publication,^{2,14,44-46,55} it seems unrealistic to expect further increases on the scale of the most recent ones. Once a journal of our size (publishing a mean of 100 citable items per year) has reached stability, we should expect variations in the IF no larger than ±25%.^27,51

Although the 570 other citations received during 2004 were older and did not boost our IF, they are undoubtedly a valuable indication that articles published in Revista Española de Cardiología continue to be considered useful, and thus continue to be cited in the longer term.

As noted above, the immediacy index provides a window on the journal's citation dynamics. This parameter has also improved clearly (1.023, with 88 citations), and our journal now ranks an impressive sixth in the immediacy index ranking among the 71 journals in the ISI's cardiovascular subject category.

Table 2 lists the editorial strategies that have been implemented in our publication over the years, and which have undoubtedly been key factors in our current achievements.^{2,14,44-46,55}

Issues Related to Editorial Policies

We are proud of the way the IF of Revista Española de Cardiología has climbed steadily to its current level of international competitiveness. Our journal is apparently not only widely valued and read,² but is also cited increasingly often.¹ However, a journal's fundamental mission should not consist solely of serving as an efficient means of communication among researchers in a specific area (which is something the IF measures well), but should also comprise the goal of providing accurate, high-quality information that clinicians find of interest for their daily practice. Although some studies have shown that the quality of a publication as perceived by researchers and clinicians correlates clearly with the IF, the correlation is much better from the researcher's standpoint.⁵⁶ In addition, other studies have emphasized that the IF is poor at accurately identifying articles that will eventually be included as references in evidence-based clinical practice guidelines.^55,57 Accordingly, and although we are pleased that we can offer our authors and researchers an attractive IF, it is clear that the quality and prestige attained by Revista Española de Cardiología do not depend upon a mere number. Thus our editorial policies will not be aimed at achieving this objective. We are convinced that the substantial increase in our IF will not cause impactitis, but that it will--like a gently applied therapy (impactotherapy)--allow us to open new doors and further improve the scientific content of our journal. Our mission as editors will continue to be to foment the overall development of our publication to achieve a harmonious balance between its attractiveness for researchers and usefulness to practitioners. The latter can continue to look forward to stimulation and support in Revista Española de Cardiología for the complex processes of decision-making they face in their day-to-day encounters with patients.

---

Correspondence:
Revista Española de Cardiología. Sociedad Española de Cardiología.
Nuestra Señora de Guadalupe, 5-7. 28028 Madrid. España.
E-mail: [e-mail]