Every now and then I still hear my 4th grade home-room teacher’s voice echoing in my head saying, “Mr. Luna, you MUST show your work.” She was referring to every grade school student’s rite of passage – mastering long division and multiplication problems while clearly showing the route taken to arrive at the solution. In the world of patient-centric Evidence-Based Medicine (EBM), showing one’s work is a critical qualifier of the outcome being classified as EBM.

Similar to grade school motives, EBM researchers and developers must clearly show the route taken to arrive at the solution to ensure the content is transparent, reproducible, defensible, and is conducive to external review. Absent of these traits, content is simply not EBM; documenting the work done behind the scenes ensures the product isfree of bias and is trustworthy.

The Institute of Medicine (now known as The Health and Medicine Division (HMD);https://www.nationalacademies.org/hmd/), a private, not-for-profit institution that provides independent, objective analysis to help inform public policy decisions related to science, technology, and medicine, published eight standards for developing trustworthy clinical practice guidelines:

1. Transparent and Reproducible Process
2. Management of Conflict of Interest
3. Guideline Development Group Composition
4. Guideline Systematic Review
5. Establishing Evidence Foundations and Rating Strength of Recommendations
6. Articulation of Recommendations
7. External Reviews
8. Updating Content Process

(.PDF is here:http://www.nationalacademies.org/hmd/~/media/Files/Report%20Files/2011/Clinical-Practice-Guidelines-We-Can-Trust/Clinical%20Practice%20Guidelines%202011%20Insert.pdf)

These standards provide an objective list of considerations to ensure the content developed is free of bias, conflicts of interest, and complies with the cornerstone principles of EBM.

Reviewing the two commercially available guidelines for Workers’ Compensation injuries/illnesses (ACOEM/MDGuidelines Practice Guidelines and ODG Treatment Guidelines) one may surmise that the two content sources have some things in common. Although, when measured against IOM’s eight standards, the two could not be more different:

• ACOEM/MDGuidelines adheres to all eight IOM standards
• ODG adheres to three IOM standards, lacking (1) Transparent and Reproducible Processes, (2) Management of Conflict of Interest, (4) Guideline Systematic Reviews, (5) Establishing Evidence Foundations and Rating Strength of Recommendations, (6) External Reviews

The National Guideline Clearinghouse (NGC) (https://www.guideline.gov/), run by the federal Agency for Healthcare Research and Quality (AHRQ), a division of the U.S. Department of Health and Human Services, announced that it has removed ODG from its listing due the guideline’s inability to meet EBM standards. Some examples of the unmet standards were provided by NGC in an article published in WorkCompCentral (https://www.workcompcentral.com/) on Thursday June 23rd, 2016 included(1):

• Explaining how they selected studies for their evidence-based review
• Including the number of studies identified
• Including the number of studies evaluated
• Providing a summary of inclusion and exclusion criteria

According to the Health Science Administrator of the AHRQ, evidence from the selected studies must be synthesized in a detailed description or evidence table; ODG did not meet these requirements for all of its treatment recommendations.

Without having the ability to reproduce the research, following the documented development methodology, how can the content be vetted for appropriateness, accuracy, and relevance?

Without transparency, what assurance is there that the content is free of bias and is actually based on the best available scientific evidence?

The original definitions of evidence-based medicine (EBM) focused on the care of individual patients, using evidence to improve outcomes. In a 1996 editorial in the British Medical Journal, Sackett defined EBM as “…the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.”(2) It was further noted that the practice “…means integrating individual clinical expertise with the best available external clinical evidence from systematic research.”(2) In 2005, EBM was defined as “… a set of principles and methods intended to ensure that to the greatest extent possible, medical decisions, guidelines, and other types of policies are based on and consistent with good evidence of effectiveness and benefit.”(3) Properly done, the use of EBM is a process which entails identification of high-quality scientific evidence, as defined by rigorous criteria, and synthesis of the entire body of evidence applicable to a given condition to guide (not dictate) medical practice.

I’d like to believe that the Workers’ Compensation industry will take heed of NGC’s de-listing of ODG. Our injured workers deserve the best treatment possible. We should not tolerate any treatment recommendation or drug prescription from a source that does not meet the strictest criteria for EBM. Lowering our standards and accepting the status quo because it’s easy, or because it’s difficult to change is a path our industry must avoid at all costs. This is an opportunity for our industry to stop saying all the right things, and actually do the right thing.

(1) Goodman, Elaine. “National Guideline Clearinghouse Drops ODG.” WorkcompCentral, June 2016. Web. 23 June. 2016.

https://www.workcompcentral.com/news/story/id/2f4e88cd49a5de5d36558a987274ec469ad26582

(2) Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. Br Med J. 1996;312(7023):71-2.

(3) Eddy DM. Evidence-based medicine: a unified approach. Health Aff 2005;24(1):9-17.