Keywords
EVIDENCE BASED DECISION MAKING: ROLE OF SECONDARY SOURCES OF INFORMATION
In pediatric cardiology, as with any other area of medicine, we are constantly faced with the challenge of providing quality health care in situations that give rise to doubts concerning the different aspects of our clinical practice. Such aspects include the value of a new drug, the importance of a different diagnostic test, the harmful effects of a treatment, or the prognosis of a particular disease. Traditionally, we have attempted to resolve these doubts with the aid of books and journals, and by consulting colleagues with greater experience in the relevant field. However, these practices have several important limitations:
- Because of the inherent delay in publication of text books, these often contain information that is obsolete by the time the book comes off the press, especially concerning diagnostic techniques and treatment strategies, though not so much so when the book deals with pathophysiological or etiological aspects. At other times the books include subjective evaluations, with no proven, underlying scientific basis.
- The large number of biomedical journals available provide too much information, and we do not have sufficient time to consult them all. Currently, over 2 million articles are published each year in some 20 000 journals worldwide. The quality of these articles is often very varied, or they contain methodological errors that compromise the results, or the results are presented in such a way as to limit their correct interpretation.
- Clinical practice has relied on diagnostic and therapeutic procedures whose validity has not been proven in scientific studies. Accordingly, both our experience and the opinions of our colleagues may lead to us taking less than the best decision.
This model can be considered the traditional method used for the process of decision-making. However, due to the inherent limitations, a new paradigm has arisen: evidence based decision-making, which is related with the new paradigm of scientific thinking known as evidence based medicine (EBM), or, as it is known in Spanish, medicine based on proof.1
Evidence based medicine is a new conceptual framework for solving clinical problems. It does so by means of making the results of clinical research more accessible to medical practice. It arose as a way for health care professionals to better confront the challenges of present-day medicine and which affect us very closely. These include the presence of an enormous amount of continuously evolving scientific information, the requirement to provide health care of the highest quality, and the limitation in health care resources.2-4 Evidence based medicine consists of the systematic search for the best scientific evidence published in the biomedical literature, its critical evaluation, and the application of the findings of research in clinical decision-making. What varies is the degree of the association that health care professionals wish to establish with EBM; there are basically 2 main groups1: a) an active, more costly group. This involves producing EBM (supported by the teachings of such bodies as the Evidence Based Medicine Working Group, the Colaboración Cochrane Iberoamericana, or the Spanish CASP group). This is the level it would be desirable to attain, and whose international forum collaborates in the process of systematic reviews for the Cochrane Collaboration, as well as the drawing up of clinical practice guidelines (CPG); b) another more passive, less costly group, involving EBM consumers. Here the physician searches sources of published information for scientific evidence elaborated by others (mainly the so-called secondary sources of information), and attempts to apply the results to clinical practice, individualizing them to the particular circumstances of the patient involved.
The efficient search for biomedical information is therefore one of the key aspects in the practice of evidence based decision-making within the scientific paradigm of EBM. Evidence based decision-making relies on the best evidence obtained by exhaustive review of various sources of biomedical information.1,5,6 The physician has ceased to be an accumulator of information to become a searcher of sources of information, such that EBM is a possible solution when faced with the current excess of medical information. The problem is both quantitative (to access and review in depth all publications about a particular topic is impossible) and qualitative (critical analysis of current scientific evidence and evaluating the usefulness of new information as compared with what was previously known is difficult). From the didactic point of view, sources of reference information are divided into 2 large groups, in accordance with the paradigm of EBM: secondary sources of information, which usually imply critical evaluation of the documents, and primary or "traditional" sources of information, in which it is necessary to undertake a critical evaluation of the articles, and to analyze their scientific validity, clinical importance, and applicability in practice6 (Table 1).
Improving the care of our cardiologic patients requires basing our clinical practice on the best scientific evidence and performing evidence based decision-making, based on the steps proposed by Muir Gray in his book Evidence based healthcare7: is this the best research method to answer our structured clinical question?, is the research of suitable scientific quality?, what is the clinical importance of the beneficial and harmful effects found?, are the research results generalizable to the general population from which the study sample was taken?, are the results applicable to our population?, are the results relevant for my patient?
The systematic reviews of the Cochrane Collaboration answer all these questions with quality criteria. It is thus a very useful reference tool to enable us to base our decision-making on the best scientific evidence. The Cochrane Collaboration is considered the prototype of secondary sources of information and one of the most important reference sources in terms of scientific value and clinical importance. As can be seen in Table 1, the systematic reviews of the Cochrane Collaboration aim to bring together and synthesize exhaustively all the information available about a particular clinical problem. They occupy a primordial, intermediate position among the other sources of secondary information.
THE COCHRANE COLLABORATION AND SYSTEMATIC REVIEWS
The Cochrane Collaboration defines itself as an international, non-profit organization whose mission is to help in decision-making in matters of health, providing the best available information. The aim of the Cochrane Collaboration is to analyze, maintain, and divulge systematic reviews of the effects of health care by means of controlled clinical trials (and in the absence of clinical trials, reviews of the most reliable evidence taken from other sources). These mainly consist of reference summaries, generally undertaken by more than 1 person, following a methodology that is structured (defined at different stages), explicit (it defines the different types of design in each phase), and systematic (it aims to access all available information).8-10 The elaboration of a systematic review follows a well-defined methodology8,11,12: to define the aim, identify fully the methodology used to search for the information, select the data with defined inclusion and exclusion criteria, evaluate the quality and validity of the studies, synthesize the information, analyze the results, and make conclusions based on the review data. The aim is to make the process as exhaustive, rigorous, and objective as possible, and thus reproducible (that is, if the process were undertaken by other authors, these would ideally obtain equivalent results) and which differentiate it from a narrative or author review10 (Table 2).
The systematic reviews undertaken with the Cochrane Collaboration method follow a peer review process supervised by an editorial team (coordinator and several editors), in accordance with a previously agreed protocol and with a defined structure (Table 3). Occasionally, the systematic reviews provide not only qualitative but also quantitative conclusions: in these cases we speak about a meta-analysis. A meta-analysis applies statistical procedures that enable results to be grouped together and overall, numerical estimates to be obtained.13-15 It should be borne in mind that a meta-analysis is only applicable when the studies included in it provide numerical results that make sense when combined both clinically and statistically, as they deal with homogenous concepts.16
In essence, then, the Cochrane Collaboration aims to synthesize and disseminate reliable information concerning matters of health. It is, in fact, a vast project which, despite all its limitations, 1 author has even compared to the human genome project.17 Indeed, as in the human genome project, the Cochrane Collaboration requires the international cooperation of many persons with different skills, and its extremely extensive scope and requirements never stop growing, bearing in mind the exponential growth in biomedical information.
The declaration of principles governing the work of the Cochrane Collaboration establishes the following: build on individual enthusiasm, encourage collaboration, avoid bias and duplication of effort, keep up to date, and ensure maximum access to the information. In the latest documents edited by the managerial team, the Cochrane Collaboration also expresses interest in directing its efforts towards the topics that concern people, maintaining and raising the quality of its reports, accepting and incorporating criticism, and ensuring the continuity of its aims.
All the studies and information collected and synthesized by the Cochrane Collaboration are to be found in the Cochrane Library: full texts currently available via internet. The development and expansion of the Cochrane Collaboration has coincided with the advance and explosion of internet, such that its databases, which were originally published on CD ROMs, have, since 1998, been available on internet. This represented a historic landmark, as not only did it facilitate potential access to many more people (during the initial phases after payment of a fee), but more especially it facilitated the interaction of the reviewers with the authors of studies and the readers, as well as with other relevant parties. Later, the possibility of free access in different countries through institutional agreements led to the even greater spread of information. In Spain, the Cochrane Collaboration has taken on increased importance since 2003, when an agreement with the Spanish Ministry of Health resulted in the free access via internet to the whole content of the library. The pharmaceutical industry has also collaborated by providing funds for the works to be translated into Spanish.
ORGANIZATIONAL STRUCTURE AND CURRENT STATUS OF THE COCHRANE COLLABORATION
Figure 1 provides a graphic representation of the organization of the Cochrane Collaboration. Surrounding the steering group are the following elements:
Fig. 1. Chart of the collaboration Cochrane.
1. A fundamental part of its organization is the establishment of Collaborative Review Groups (CRG). Each reviewer in the Cochrane Collaboration is a member of the CRG, which is composed of professionals from different disciplines but who share a common, specific interest in one particular topic. These CRG do not necessarily coincide with the traditional medical specialties, but rather they are directed towards specific problems or groups of diseases. Each one of the CRG periodically chooses an editorial committee to act as a peer reviewer of the systematic reviews that are elaborated: this committee not only judges, it also supports and helps with the drawing up of the review, trying to prevent overlapping and encouraging the establishment of links between reviewers from the various parts of the world, in order to stimulate further collaboration.
Currently there are 50 CRG. Those that at the present time (Issue 2, 2005) have most reviews are related with perineonatology, Cochrane Pregnancy and Childbirth (with 273 systematic reviews and 76 protocols) and Cochrane Neonatal Group (with 198 systematic reviews and 61 protocols). These are far ahead of the remainder of the CRG (Table 4). One of the CRG is aimed specifically at cardiological matters (Cochrane Heart Group: www.cochrane.org/cochrane/revabstract/VASCAAbstractIndex.htm); other CRG are concerned with questions that are closely related with cardiology, such as hypertension (Cochrane Hypertension Group: www.cochrane.org/cochrane/revabstract/HTNAbstractIndex.htm) and peripheral vascular diseases (Cochrane Peripheral Vascular Diseases Group: www.cochrane.org/cochrane/revabstract/PVDAbstractIndex.htm).
2. A series of centers covering different cultural and geographical areas has been set up in order to intensify and promote the activities of the Cochrane Collaboration. To date, there are 10 active centers, in Germany, Australia, Brazil, Canada, China, Denmark, Latin America, Italy, Scandinavia, and South Africa. For us in Spain, the Latin America Cochrane Center was formed as an enlargement of the functions of the Spanish Cochrane Center, which was set up in 1997. Its present headquarters are at the Hospital de la Santa Creu i de Sant Pau, Barcelona, Spain, and it operates under the management of X. Bonfill. One of the most important tasks of this Center, as well as the diffusion of information, is methodological support for future reviewers and for groups of physicians or researchers. The Center provides training for the reviewers and also offers support to 2 CRG that are coordinated within the Center's area of influence (Lung Cancer Group and Movement Disorders Group). It also coordinates the translation of the material from the Cochrane Collaboration, especially that relating to the Spanish edition of the Cochrane Library (Cochrane Library Plus) and the manual search of scientific journals.
3. Another element is known as Fields. Its job is to organize the information, facilitate coordination between groups, and promote research in other wider areas. Currently, there are 10 fields and each has its own web page: field of health care of the elderly, health promotion field, field of rehabilitation and associated therapies, child and adolescent health field, vaccines field, primary health care field, complementary medicine field, Cancer network, Neurological network, and the Cochrane pharmaceutical field (possible).
4. The strict synthesis of health care information is always a challenge. The Cochrane Collaboration understands that the methodology of the systematic reviews is not a fully resolved process. Accordingly, it has established Methods Groups, which are working groups involved in different methodological aspects, ranging from statistics to the treatment of bias, and including the synthesis of diagnostic studies: placebo effects, statistical methods, quality of life associated with health, screening and diagnostic tests, information about bias, meta-analysis of individual patient data, prospective meta-analysis, information recovery, health economics, empirical studies, information studies, methods for use and recommendations, non-randomized studies, training and support (possible), qualitative research (possible), and drug safety (possible).
5. The Cochrane Collaboration also has a network of health care consumers. This is to encourage any health care user to voice their opinions, needs, and concerns. The idea is to develop the systematic reviews with a patient-oriented focus. The network has members in 50 countries and includes an international coordinating group that works mainly via internet.
THE COCHRANE LIBRARY DATABASES AND PEDIATRIC CARDIOLOGY
This is the set of databases that can be consulted via internet and which include both the main sources and the results of the research of the Cochrane Collaboration. The web page on internet is the most interesting part of the Cochrane Collaboration. It is where the results of all the work are to be seen and where we can have access to all the information. A small summary of the contents of the main databases included is given below (Table 5):
1. The Cochrane Database of Systematic Reviews (CDSR).This is the main database of the library. It contains the full text of the systematic reviews prepared and updated by the CRG of the Cochrane Collaboration. It is updated every 3 months and it has 2 sections: full reviews and protocols. The latter explain the previously agreed, complete methodology for the elaboration of the systematic reviews that are in the preparation phase.
In Issue 2, 2005 of Cochrane Library, the Cochrane Heart Group published a total of 44 systematic reviews and 59 protocols. Of the 44 systematic reviews, 6 can be encompassed within the context of pediatric cardiology,18-23 although some of the studies were not carried out exclusively in children, but rather included patients of mixed ages (children and adults).18,19,22,23
In Issue 2, 2005 of Cochrane Library, the Cochrane Hypertension Group published a total of 9 systematic reviews and 29 protocols. None of the 9 systematic reviews could be considered to include pediatric cardiology.
In Issue 2, 2005 of Cochrane Library, the Cochrane Peripheral Vascular Diseases Group published a total of 42 systematic reviews and 27 protocols. Only 1 of the 42 systematic reviews could be considered to include pediatric cardiology.24
Besides the above, systematic reviews dealing with neonatal cardiology have also been published by the Cochrane Neonatal Group. In Issue 2, 2005 of Cochrane Library, the group published a total of 198 systematic reviews and 61 protocols. Of the 198 systematic reviews, 13 could be included within the context of pediatric cardiology,25-37 with 2 interesting subgroups: the treatment of patent ductus arteriosus25-32 and the treatment of low blood pressure in premature infants.33-36
Table 6 summarizes the systematic reviews of the Cochrane Collaboration dealing with pediatric cardiology detected in the various CRG, together with the characteristics of the studies included, the main results, and the comments on their applicability.
In the study we undertook of the references of the Cochrane Neonatal Group, we detected some areas that have received a lot of attention (respiratory and gastroenterology-nutrition), whereas few systematic reviews have been undertaken in other areas that are also prevalent in neonatal clinical practice, such as cardiovascular, infectious, and neurological problems; we have already remarked on this in previous studies.38-41 An interesting aspect is that almost all the study areas of the Cochrane Collaboration include systematic reviews on therapeutic interventions and preventive measures, but very few systematic reviews relating to diagnostic tests. This represents a future challenge for the Cochrane Collaboration.38-41
The results of the bibliometric analysis of the systematic reviews of the Cochrane Collaboration should be considered orientative when deciding on projects for future systematic reviews, as can be appreciated in the Cochrane Collaboration Protocols on pediatric cardiology that are currently in progress: "Corticosteroids for hypotension in preterm infants" and "Intravenous indomethacin for symptomatic patent ductus arteriosus in preterm infants" by the Cochrane Neonatal Group, and "Antibiotics for brain abscesses in people with cyanotic congenital heart disease" by the Cochrane Heart Group.
At the present time, and based on the systematic reviews in pediatric cardiology by the various CRG of the Cochrane Collaboration, we can undertake suitable decision-making for the treatment of patent ductus arteriosus and hypotension in the premature infant, and also for the treatment of children with Kawasaki disease (Table 7). The results of the remaining systematic reviews in pediatric cardiology are not conclusive: the systematic reviews on rheumatic fever because they are either based on old clinical trials18 or these are of poor quality19; the systematic reviews on interventions in infant obesity because of the difficulty generalizing the results based on the heterogeneity of the clinical trials20,21; and in the other study areas because no clinical trials were found in the pediatric population. 22,23,37
Only 2 systematic reviews on cardiology have been undertaken by Spanish authors,42,43 though neither were related with to pediatric cardiology.
2. The Database of Abstracts of Reviews of Effectiveness (DARE). This database contains structured summaries and references of systematic reviews published in the Cochrane Collaboration or in other biomedical journals, and that have been critically reviewed.
3. The Health Technology Assessment Database (HTA).1 This database contains summaries of different health technology evaluation agencies. It includes the full reports, as well as those projects that are still being elaborated. It is kept up to date with the collaboration of the INAHTA (International Network of Agencies for Health Technology Assessment).
4. NHS Economic Evaluation Database (NHS-EED). This database contains summaries of the economic evaluations of the health services.
These 3 databases (DARE, HTA, and NSH-EED) can also be consulted in the York Centre for Reviews & Dissemination, using a specific search engine. The databases also include articles of interest concerning pediatric cardiology over and above the efficacy of health care interventions, particularly related with effectiveness and efficiency.44-50
5. The Cochrane Central Register of Controlled Trials (CENTRAL). This is the main database of clinical trials currently in existence, and it is included in the computer searches with Medline and Embase, for example, as well as in manual searches of journals and the "gray" literature, such as books on lectures and communications, or theses, undertaken by volunteers around the world on behalf of the Cochrane Collaboration.
6. The Cochrane Review Methodology Database (CRMD). This database contains references of books and articles, summaries and references about methodological questions of the critical reviews and the synthesis of studies, relevant to summarize the evidence in health care. The database is continually updated and currently contains over 6000 references.
7. About the Cochrane Collaboration: bibliographic references concerning the concepts and methodology upon which the Cochrane Collaboration is based, as well as the specifications of collaborating groups and centers in the different parts of the world.
THE COMPLEMENTARITY OF THE CLINICAL PRACTICE GUIDELINES IN PEDIATRIC CARDIOLOGY
After performing the bibliometric analysis in the topics involved in the study areas of the Cochrane Collaboration, we are in a position to state that, at the present time, few systematic reviews exist on pediatric cardiology and more than one third of these lack sufficient scientific evidence to permit evidence based decision-making.38,39,41 Many therapeutic interventions in pediatric cardiology still lack the support of scientific evidence, and we can currently only undertake adequate decision-making in clinical practice for the treatment of patent ductus arteriosus and hypotension in the premature infant, and for Kawasaki disease (Table 7). We detected important gaps in the study of cardiologic topics of great interest in pediatrics, such as heart failure, shock, arrhythmia, hypertension, or congenital heart disease.
Clinical practice guidelines are a secondary source of information that complement the systematic reviews, and that go even further by bringing together and synthesizing the exhaustive information available not about a specific clinical problem (like the systematic reviews), but rather as a complete clinical process.51,52 The CPG are recommendations that are developed systematically to help physicians and their patients decide about the most suitable health care in a particular clinical situation, and that contribute to lessen the variability in clinical practice. They are documents drawn up at the initiative of health care organizations and official institutions, who nominate a working group (experts in methodology, health care personnel, administrators, user representatives). They are based on a wide review and critical evaluation of the medical literature available about a particular health care topic: they occasionally use previously elaborated systematic reviews, but at other times they are elaborated ad hoc.51-56
The criteria which good and efficient CPG should fulfill are52: a) ensure the correct evidence is being managed (scientific and technical information that has been evaluated and contrasted); b) consider the most frequent conditions for use in clinical practice (for CPG to be effective, they must be perceived by physicians as locally relevant, not just internationally relevant); c) consider the factors that influence the adoption of new technologies.
Three methods exist for the development of a CPG (experts'opinions, consensus methods, and evidence-based methods), although the best method is a combination of all 3 systems.52 It should not be forgotten that the most desirable development model for a CPG is an evidence-based procedure, to which consensus methods are added, and in which experts play an important role.
The structure of a CPG is as follows: introduction and justification for the CPG, report of the systematic reviews (sometimes in a separate section, depending on its length), detailed description of the discussion and initial "evidence-based" recommendations, report on the response given by the professionals (pilot response), final, piloted recommendations (main content of the guidelines).
The development of a CPG can be useful for patients, physicians, administrators, and politicians. It can also lead to such benefits as a reduction in iatrogenic disease, an improvement in efficiency, medical behavior based on scientific rationality that can be used as a defense in case of legal claims, and it can facilitate the decision-making process. Although it is difficult to adapt clinical variability to schematic algorithms, we are nevertheless approaching a key conviction: clinical variability does not justify clinical arbitrariness. The aim of a CPG should never be to impose criteria, and it is difficult to accept that the sum of all the individual uncertainties that the experts have can give rise to a final proposal satisfactory to all. Above all, a CPG should be a model of restraint, although this does not impede the recommendation of the "obviously" recommendable or the rejection of the "obviously" rejectable.
In an attempt to determine the current situation of CPG in pediatric cardiology, we undertook a search of the main international information centers (the National Guidelines Clearinghouse: www.guidelines.gov/index.asp and the Canadian Medical Association Infobase: www.cma.ca/cpgs/index.asp) and national information centers (the Guía Salud: www.guiasalud.es and Directorio de Guías Clínicas en Español from the Fisterra gateway: www.fisterra. com/recursos_web/castellano/c_guias_clinicas.asp).
In the international clearing houses, we found a few guidelines related with pediatric cardiology, dealing with the cardiovascular evaluation and management of patients,57 electrocardiographic monitoring,58 echocardiography,59 invasive cardiac procedures,60 implantation of cardiac pacemakers,61 the management of growth in patients with congenital heart disease,62 and the management of Kawasaki disease.63
In the national clearing houses, the most validated information was found in the Guía Salud, as the Fisterra directory considers as CPG certain documents that are in fact only protocols. From the 2 directories, we obtained texts dealing with such topics as dyslipidemia, hypertension, and ischemic heart disease that were not applicable to children. The Guía Salud is a recent project whose aim is to offer a catalog of CPG that have been drawn up and are used by professionals of the Spanish National Health Service System. To date (16 May 2005), a total of 322 documents have been reviewed, of which just 20 fulfilled the strict inclusion criteria to be considered as true guidelines; of these, only 3 correspond to CPG related with cardiology (2 about hypertension and 1 about hypercholesterolemia). Using the search engine of the Revista Española de Cardiología, and applying the key word "clinical guidelines," we found 10 documents of public interest in this journal, which functions as the official publication of the Spanish Society of Cardiology, some of which were useful in pediatric cardiology.64-67
Great variability exists in decision-making among physicians when faced with the same disease process, and in the same physician when faced with different patients all having the same disease. This uncertainty regarding observations, perceptions, reasoning, interventions, and practice styles is referred to as variability in clinical practice. Variability in clinical practice is not appropriate when there exists important scientific evidence, but variability is expected, and even desirable, in the presence of just weak scientific evidence. We are not only interested in clinical trials and systematic reviews, which explains the importance of the Cochrane Collaboration, but also other secondary sources of information (mainly CPG), as well as comparative studies (benchmarking) that analyze variations in the results of health care units which apply different guidelines.68,69
Correspondence: Dr. J. González de Dios.
Prof. Manuel Sala, 6, 3.o A. 03003 Alicante. España.
E-mail: gonzalez_jav@gva.es