WorkComp Needs Quality Measures for Clinical Practice Guidelines

The U.S. Department of Health and Human Services’ Agency for Healthcare Research and Quality (AHRQ) has announced it will no longer fund its National Guideline Clearinghouse™ (NGC). With nearly 4,000 published clinical practice guidelines (CPG) from 39 countries in existence, poorly developed content is widely available, and unfortunately widely used. Workers’ Compensation needs quality measures to ensure the standards of care used to treat injured workers are reliable and trustworthy.

A recent article announcing AHRQ’s decision, quoted an industry leader describing the NGC as a convenient place to find a large number of medical treatment guidelines. He continued by stating, “Is it mission critical to the work comp system? I think not.” He concluded by suggesting that whether someone steps in to take over NGC’s work – and how quickly they do so – will be an indication of its importance. Respectfully, but emphatically, I disagree.

Why did AHRQ establish the NGC?

The NGC’s importance cannot be validated by someone continuing the work. Its importance was established in 2008 by U.S. Congress when they asked the National Academies of Sciences Engineering and Medicine (NASEM), formerly the Institute of Medicine (IOM), to conduct a study to define best practices used in developing CPGs.1 Congress expressed concern in the vast amount and quality-deficient information that guideline users, including medical practitioners, were inundated with.

The NGC’s purpose was two-fold:

  • To be a mechanism used to immediately and objectively identify high quality, trustworthy CPGs to help inform improved health-related decision making. In doing so, patients will benefit from enhanced health care quality and outcomes; and
  • To help developers create high quality guidelines that adhere to established standards.

Congress’ concerns were echoed by the NASEM in their Report Brief stating, “Most guidelines used today suffer from shortcomings in development. Dubious trust in guidelines is the result of many factors, including failure to represent a variety of disciplines in guideline development groups, lack of transparency in how recommendations are derived and rated, and omission of a thorough external review process.” 1

The NGC is Relevant 10-Years Later

A decade later, a set of well-known guidelines in workers’ compensation was evaluated for technical quality by RAND.2 Appraisers from RAND evaluated stakeholder involvement, including whether the review group included at least one methodology expert. The publisher of the guidelines introduced RAND to content chapter leads who had relevant clinical expertise, but lacked evidence of substantial methodological expertise pertinent to conducting literature searches and developing guidelines.

Following this determination, RAND noted that the publisher revised the reviewed guidelines to state that two methodologist were involved in development but failed to provide information on names, titles, roles in development of the guideline, or relevant expertise. The score in this area was reduced as a result and left RAND with uncertainty about the involvement of methodological experts in the development of the publisher’s content.2

Not surprisingly, the lack of methodological expertise fostered other critical weaknesses in the content. This included, among other things, limited information regarding the process by which the publisher’s chapter development teams identify, select, evaluate, and synthesize evidence. 2

Although RAND’s study, which was funded by iCare Insurance & Care New South Wales, Australia, ultimately provided overall favorable ratings on the content, the NGC removed it entirely from its database for its material deficiencies in development methodology and process.

Concerns regarding the quality of information that guideline users are inundated with should be as prevalent today as it was in the U.S. Congress in 2008. Meeting IOM criteria will be even more important if the NGC doesn’t find another home. Without appropriate oversight in the development of CPGs, anything might be called “quality” imperiling the well-being of injured workers.

The sentiment reflected in the comments about the NGC’s importance is a clear indication that the workers’ compensation community has not made the connection between quality CPGs and measurable improved health outcomes. It is also a glaring indicator that the need for quality measures for CPGs is very real and of utmost urgency.



  1. Report Brief”. The National Academies of Sciences Engineering and Medicine 2011. Web.
  2. Nuckols, Teryl K., Kanaka Shetty, Laura Raaen, and Dmitry Khodyakov. “Technical Quality and Clinical Acceptability of a Utilization Review Guideline for Occupational Conditions: ODG® Treatment Guidelines by the Work Loss Data Institute”. Santa Monica, CA: RAND Corporation, 2017. Web.

Categories: Evidence-Based Medicine (EBM), State Workers' Compensation Standards, Uncategorized

Tags: , , , , , , , , , , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: