The overall aim of the project was to update the 2000 Kidney Disease Outcomes Quality Initiative (KDOQI) Clinical Practice Guidelines on Hemodialysis and Peritoneal Dialysis Adequacy, and Vascular Access. The Work Group sought to update the guidelines using an evidence-based approach. After topics and relevant clinical questions were identified for the updates, the available scientific literature on those topics was systematically searched and summarized.
Update of the guidelines required many concurrent steps to:
Separate Work Groups were created for each subject area: hemodialysis adequacy, peritoneal dialysis adequacy, and vascular access. The 3 groups worked in parallel to create the guidelines. The Work Group Chairs conferred regarding overlapping topics across guidelines. The Evidence Review Team, comprised of experts in systematic review and guideline development, guided the Work Groups in all methods and aspects of guideline development.
Creation of Groups
The KDOQI Advisory Board selected the Work Group Chairs and the Director of the Evidence Review Team then assembled groups to be responsible for the development of the updates. These Work Groups and the Evidence Review Team collaborated closely throughout the project.
The Work Groups consisted of domain experts, including individuals with expertise in nephrology, surgery, radiology, pediatrics, nursing and nutrition. For each guideline update, the first task of the Work Group members was to define the overall topics and goals of the updates. They then further developed and refined each topic, literature search strategies, and data extraction forms (described below). The Work Group members were the principal reviewers of the literature, and from their reviews and detailed data extractions, they summarized the available evidence and took the primary roles of writing the guidelines and rationale statements. Completed data extractions were posted on a National Kidney Foundation (NKF) website for direct access by Work Group members.
The Evidence Review Team consisted of nephrologists (1 senior nephrologist and 2 nephrology fellows), methodologists, and research assistants from Tufts-New England Medical Center with expertise in systematic review of the medical literature. They instructed the Work Group members in all steps of systematic review and critical literature appraisal. The Evidence Review Team also coordinated the methodological and analytical process of the report, defined and standardized the methodology of performing literature searches, of data extraction, and of summarizing the evidence in summary tables. They organized abstract and article screening, created forms to extract relevant data from articles, organized Work Group member data extraction, and tabulated results. Throughout the project the Evidence Review Team led discussions on systematic review, literature searches, data extraction, assessment of quality and applicability of articles, evidence synthesis, and grading of the quality of the body of evidence and the strength of guideline recommendations.
Refinement of Update Topics and Development of Materials
The Work Group reviewed the 1995 Dialysis Outcomes Quality Initiative (DOQI) Clinical Practice Guidelines and the 2000 KDOQI updates and decided which of the guideline recommendations required updates and which should remain unchanged. These assessments were based primarily on expert opinion regarding the currency of the previous guidelines and the likelihood of availability of new evidence. Preliminary literature searches were made to inform this process. To allow for timely review, it was determined that each set of guidelines would be able to have systematic reviews on only a limited number of topics. After literature review, the experts decided which recommendations would be supported by evidence or by opinion. As described below, recommendations based on adequate evidence were categorized as Guidelines (CPGs), while opinion-based statements were categorized as Clinical Practice Recommendations (CPRs).
The Work Groups and Evidence Review Team developed: a) draft guideline statements; b) draft rationale statements that summarized the expected pertinent evidence; and c) data extraction forms containing the data elements to be retrieved from the primary articles. The topic refinement process began prior to literature retrieval and continued through the process of reviewing individual articles.
Literature Search
Based on the draft guideline statements, the Work Group members agreed on topics that would be systematically reviewed and formulated questions defining predictors, interventions, comparators, and outcomes of interest. Search strategies were developed based on these questions and topics, in addition to the study designs and years of publications of interest to the Work Group. Articles of interest were identified through MEDLINE searches of English language literature of human studies in May through July 2004. Broad search terms were used to avoid missing potentially pertinent articles. The searches were supplemented by articles identified by Work Group members through June 2005.
Only full journal articles of original data were included. The searches were limited to studies published since January 1997 since earlier publications were reviewed in the previous DOQI guidelines. Editorials, letters, abstracts, and unpublished reports were not included. Selected review articles, however, were included for background material. No systematic process was followed to obtain review articles.
Abstracts and titles from the MEDLINE search results were prescreened by members of the Evidence Review Team for general relevance. A second round of screening was performed on the abstracts by Work Group members for relevance using predefined eligibility criteria, described below. Articles were retrieved by the Evidence Review Team and then rescreened by Work Group members and/or the Evidence Review Team. Eligible studies were extracted using standardized extraction forms. Domain experts made the final decisions regarding the eligibility of all articles.
Generation of Data Extraction Forms
Data extraction forms were designed to capture information on various aspects of the primary articles. Forms for all topics included study setting and demographics, eligibility criteria, causes of kidney disease, numbers of subjects, study design, study funding source, dialysis characteristics, comorbid conditions, descriptions of relevant risk factors or interventions, description of outcomes, statistical methods, results, study quality (based on criteria appropriate for each study design (see below), study applicability (see below), and sections for comments and assessment of biases. Training of the Work Group members to extract data from primary articles occurred by emails and teleconferences. Work Group members were assigned the task of data extraction of articles.
Generation of Evidence Tables
The Evidence Review Team condensed the information from the data extraction forms into evidence tables, which summarized individual studies. These tables were created for the Work Group members to assist them with review of the evidence and are not included in the guidelines. All Work Group members (within each Update) received copies of all extracted articles and all evidence tables. During the development of the evidence tables, the Evidence Review Team checked the data extraction for accuracy and re-screened the accepted articles to verify that each of them met the initial screening criteria determined by the Work Group. If the criteria were not met, the article was rejected, in consultation with the Work Group.
Format for Summary Tables
Summary Tables describe the studies according to the following dimensions: study size and follow-up duration, applicability or generalizability, results, and methodological quality. Within each table, the studies are first grouped by outcome type.
Data entered into Summary Tables were derived from the data extraction forms, evidence tables, and/or the articles by the Evidence Review Team. All Summary Tables were reviewed by the Work Group members.
Within each outcome, studies are ordered first by methodological quality (best to worst), then by applicability (most to least), and then by study size (largest to smallest). When relevant, outcome thresholds (eg, of access flow measurement) are included. Results are presented by using the appropriate metric or summary symbols, as defined in the table footnotes.
Systematic Review Topics, Study Eligibility Criteria, and Studies Evaluated
The topics for each Update were selected by the respective Work Group members for systematic review (Table 1, Table 2, Table 3). The eligibility criteria were defined by the Work Group members of each Update in conjunction with the Evidence Review Team.
Literature Yield for Hemodialysis Adequacy (Table 4)
A total of 2,526 citations were screened, of which 319 were review articles and 14 were added by Work Group members. There were 223 articles (191 studies in adults and 32 in children) that were potentially relevant. These articles were retrieved for full review. Of these, 87 adult articles were accepted for full data extraction by the Work Group members. Eight articles in children were formally data extracted by a pediatric nephrologist on the Work Group. Articles in adults were randomly assigned to individual Work Group members for data extraction. Of these, 23 studies answered questions pertinent to topics chosen for systematic listing in Summary Tables.
Literature Yield for Peritoneal Dialysis Adequacy (Table 4)
A total of 2,307 citations were screened and 7 were added by Work Group members. There were 293 articles (263 studies in adults and 30 in children) that were potentially relevant. These articles were retrieved for full review. Of these, 101 adult articles were accepted for full data extraction by the Work Group members. Nine articles in children were formally data extracted by a pediatric nephrologist on the Work Group. Articles in adults were randomly assigned to individual Work Group members for data extraction. Of these, 27 studies answered questions pertinent to topics chosen for systematic listing in Summary Tables.
Literature Yield for Vascular Access (Table 4)
A total of 2,892 citations were screened, of which 388 were review articles. There were 112 articles (89 studies in adults, 13 in children, 10 review articles) that were potentially relevant. These articles were retrieved for full review. Of these, 58 articles were accepted for full data extraction by the Work Group members. Because of small sample sizes, articles in children were not formally data extracted but reviewed in detail by the 2 pediatric nephrologists on the Work Group and used to write the narrative summary in the pediatric section. Articles in adults were randomly assigned to individual Work Group members for data extraction. Five additional articles were added by Work Group experts and the Evidence Review Team. Finally, 24 studies answered questions pertinent to topics chosen for systematic listing in Summary Tables.
Search terms for all updates are shown in Appendix 2.
Grading of Individual Studies
Study Size and Duration
The study (sample) size is used as a measure of the weight of the evidence. In general, large studies provide more precise estimates of prevalence and associations. In addition, large studies are more likely to be generalizable; however, large size alone, does not guarantee applicability. A study that enrolled a large number of selected patients may be less generalizable than several smaller studies that included a broad spectrum of patient populations. Similarly, longer duration studies may be of better quality and more applicable, depending on other factors.
Applicability
Applicability (also known as generalizability or external validity) addresses the issue of whether the study population is sufficiently broad so that the results can be generalized to the population of interest at large. The study population is typically defined primarily by the inclusion and exclusion criteria. The target population was defined to include patients with kidney failure, specifically those on dialysis. A designation for applicability was assigned to each article, according to a three-level scale. In making this assessment, sociodemographic characteristics were considered, as well as comorbid conditions and prior treatments. Applicability is graded in reference to the population of interest as defined in the clinical question. For example for the question of treatment of catheter-related infections the reference population is that of HD patients with infected cuffed tunneled HD catheters.
Sample is representative of the target population, or results are definitely applicable to the target population irrespective of study sample.
Sample is representative of a relevant sub-group of the target population. For example, sample is only representative of people with virgin arteriovenous fistulas, or only a specific relevant subgroup, such as elderly individuals or incident dialysis patients.
Sample is representative of a narrow subgroup of patients only, and not well generalizable to other subgroups. For example, the study includes only a small number of patients or patients with a rare disease or virgin fistulas with no access dysfunction. Studies of such narrow subgroups may be extremely valuable for demonstrating exceptions to the rule.
Results
The type of results available in each study is determined by the study design, the purpose of the study, and the question(s) being asked. The Work Group decided on the eligibility criteria and outcomes of interest (see Tables 1-3).
Diagnostic Test Studies
For studies of diagnostic tests, sensitivity and specificity data or area under the curve were included when reported. When necessary, sensitivity and specificity data were calculated from the reported data. Diagnostic tests were evaluated according to a hierarchy of diagnostic tests.* Each test was assessed according to diagnostic technical capacity, accuracy, diagnostic and therapeutic impact, and patient outcome. This ultimately affected the overall strength of a recommendation regarding a diagnostic test.
*Fineberg HV, Bauman R, Sosman M: Computerized cranial tomography. Effect on diagnostic and therapeutic plans. JAMA 238:224-227, 1977
Methodological Quality
Methodological quality (or internal validity) refers to the design, conduct, and reporting of the clinical study. Because studies with a variety of types of design were evaluated, a 3-level classification of study quality was devised:
Least bias; results are valid. A study that mostly adheres to the commonly held concepts of high quality, including the following: a formal study; clear description of the population and setting; clear description of an appropriate reference standard; proper measurement techniques; appropriate statistical and analytical methods; no reporting errors; and no obvious bias. Not retrospective studies or case series.
Susceptible to some bias, but not sufficient to invalidate the results. A study that does not meet all the criteria in the category above. It has some deficiencies but none likely to cause major bias.
Significant bias that may invalidate the results. A study with serious errors in design or reporting. These studies may have large amounts of missing information or discrepancies in reporting.
Summarizing Reviews and Selected Original Articles
Work Group members had wide latitude in summarizing reviews and selected original articles for topics that were determined not to require a systemic review of the literature.
Guideline Format
The format for each guideline chapter is outlined in Table 5. Each guideline contains 1 or more specific “guideline statements” that represent recommendations to the target audience. Each guideline contains background information, which is generally sufficient to interpret the guideline. The rationale for each guideline describes the evidence upon which each guideline recommendation is based. The guideline concludes with a discussion of limitations of the evidence review and a brief discussion of clinical applications, and implementation issues regarding the topic. Research recommendations for each guideline update are summarized in a separate section at the end of each guideline update.
Rating the Strength of Recommendations
After literature review, the experts decided which recommendations were supported by evidence and which were supported by consensus of Work Group opinion. Evidence-based guideline recommendations were graded as strong (A) or moderate (B). Recommendations based on weak evidence (C) and/or consensus of expert opinion were labeled as Clinical Practice Recommendations (CPRs). An “A” rating indicates “it is strongly recommended that clinicians routinely follow the guideline for eligible patients. There is strong evidence that the practice improves health outcomes, and benefits substantially outweigh harm.” The “B” rating indicates “it is recommended that clinicians routinely follow the guideline for eligible patients. There is moderately strong evidence that the practice improves health outcomes.” A “CPR” rating indicates “it is recommended that clinicians consider following the guideline for eligible patients. This recommendation is predominantly based on consensus of opinions of the Work Group and reviewers that the practice might improve health outcomes.” (See Table 6).
The strength of each guideline recommendation is based on the quality of the supporting evidence as well as additional considerations. Additional considerations, such as cost, feasibility, and incremental benefit were implicitly considered. The quality of evidence was not explicitly graded. It was implicitly assessed according to the criteria outlined in Table 7, and considered: i) the methodological quality of the studies; ii) whether or not the studies were carried out in the target population, ie, patients on dialysis, or in other populations; and iii) whether the studies examined health outcomes directly, or examined surrogate measures for those outcomes, eg, blood flow instead of access survival.
Limitations of Approach
While the literature searches were intended to be comprehensive, they were not exhaustive. MEDLINE was the only database searched, and searches were limited to English language publications. Hand searches of journals were not performed, and review articles and textbook chapters were not systematically searched. However, important studies known to the domain experts that were missed by the literature search were included in the review.
Because of resource limitations and other practical considerations, there were several deviations from the original protocol for several of the update topics. These primarily resulted in nephrologists in the Evidence Review Team, rather than Work Group members, performing the primary article screening and the data extraction for articles included in several Summary Tables. However, all articles that met criteria for all topics, all completed data extraction forms, and all Summary Tables were distributed to relevant Work Group members for critical review and incorporation into guidelines.
