The State of Health Care Quality Measurement in the Era of COVID-19: The Importance of Doing Better | Health Care Reform | JAMA | JAMA Network
[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
Purchase Options:
[Skip to Content Landing]
COVID-19: Beyond Tomorrow
June 25, 2020

The State of Health Care Quality Measurement in the Era of COVID-19: The Importance of Doing Better

Author Affiliations
  • 1Johns Hopkins Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine, Baltimore, Maryland
  • 2Department of Anesthesiology and Critical Care Medicine, Johns Hopkins Medicine, Baltimore, Maryland
  • 3Department of Medicine, Johns Hopkins Medicine, Baltimore, Maryland
JAMA. 2020;324(4):333-334. doi:10.1001/jama.2020.11461

The coronavirus disease 2019 (COVID-19) pandemic has highlighted a number of strengths and weaknesses in the US health care system. One strength is the incredible efforts and dedication of health care workers focused on providing the best care possible to patients, even in the most challenging of working conditions. But the data needed to understand the quality of the care being delivered to patients during this pandemic have proven difficult to obtain. As a result, there is a lack of information that would help clinicians improve care delivery in the moment and learn for the future. This situation highlights how the current approach to quality and safety measurement remains too labor intensive, contains significant data lags, and lacks sufficient standardization that allows for rapid sharing of data.

As preparations for the pandemic increased, it became readily apparent that many approaches for measuring quality across the US health care system were not going to be sustainable. In March 2020, the Centers for Medicare & Medicaid Services (CMS) announced the granting of wide and sweeping exceptions for the collection and submission of data for Medicare quality programs, citing the need for hospitals and clinicians to focus on preparation for a potential surge of patients. For example, CMS indicated that data from the first 6 months of 2020 will not be used in any of the current hospital-based performance or payment programs.1 The announcement by CMS was shortly followed by similar announcements from other quality reporting organizations.2

These decisions were undoubtedly correct, given the urgent need to focus all resources on surge preparation. But these decisions to suspend quality reporting requirements would not have been required if the approach to quality measurement in the US did not rely on so much manual abstraction and human intervention. To date, health care has approached quality measurement too much as an ancillary matter, treating it as a double-check after care is delivered, rather than information evaluated simultaneously with the provision of care. The current approach, therefore, is simply not resilient enough when attention and resources need to be focused elsewhere, such as during the current pandemic.

Another deficiency in the current approach to quality measurement is the substantial time lag between the provision of care and reporting of measurements on the quality of that care. For example, many claims-based quality measures, such as the Patient Safety Indicators, have a 12-month lag between the end of the care delivery period and the reporting to hospitals on their performance. Even non–claims-based measures, such as hospital-acquired infection measures, have long lags. Feedback loops are a vital component of any robust quality improvement framework, and they only work when the feedback is timely.

In the best of circumstances, hospitals and clinicians find delayed quality data to be less meaningful and less actionable for improvement.3,4 During a health crisis, the effect of the data lag is magnified, as it translates to an absence of timely information about how the health system is performing. For example, validated data are still lacking on whether hospitalized patients with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection developed venous thromboembolism at a different rate than those without infection or whether treatment with corticosteroids made a difference in outcomes for patients with COVID-19. By the time those data are available, significant opportunities for learning and improvement will have been missed.

The current approach to quality measurement is also challenged by the insufficient standardization of data for purposes of data sharing. When health systems provide care to patients with new types of clinical problems, or provide care in new ways for which there are no standards, it can be helpful for health systems to be able compare their performance with the performance of other health systems. This is especially important for care for which optimal performance is not fully understood and a comparison with care at other health systems would be informative, if not necessary. For example, despite the substantial increase in telehealth during this pandemic, defined performance measures that could help to understand how telehealth has affected the quality of care delivered to patients are lacking. Given the relative newness of telehealth, it is not clear what constitutes optimal quality for care delivered this way. This issue can be even more acutely apparent in the intensive care setting. For example, whether placing patients with SARS-CoV-2 infection in a prone position made a difference in oxygenation and mortality could be ascertained much more quickly if hospitals had a standardized method of collecting and sharing data. Having a rapid way to identify and disseminate a standardized way of collecting data about new types of clinical problems or new ways of delivering care, that all health systems could use, could allow a health system to understand and improve performance in relation to the performance of their peer institutions.

Lessons from this pandemic make clear a number of recommendations for improvement. First, approaches to reduce the measurement burden are needed. Priority areas include the development of data capture systems that collect data from hospitals, other health care centers, and clinicians without requiring extra effort on their part, along with data abstraction that does not require manual chart review. This would include building new data capture systems designed with quality measurement in mind and abstraction systems that can maximize the use of information technology. The government and other quality reporting organizations can play a key role in pushing the market in this direction, by insisting that all new measures be electronic and even setting a timeline for phasing out manually abstracted measures.

Second, reducing the lag in reporting quality data will require approaches to better capture and report quality simultaneously with the provision of clinical care. Measures should be less reliant on claims data, which by nature have a time lag, and focus on measures that can be generated from the electronic health record (EHR). This shift will require the development of more rapid and more efficient methods of data validation and possibly alterations in the workflows of how care is delivered. The full capabilities of the EHR will need to be harnessed to accomplish this goal, which will require more formal engagement with leading quality measurement organizations and EHR vendors.

Third, to address the lack of data standardization, a series of nimble national measurement committees or task forces should be established to quickly define measures and help guide how those measures will be abstracted. This could allow health systems, at least in the short term, to use standardized methods to better understand their performance in relation to other systems. These measurement committees could work with the EHR vendors to understand what changes are needed to EHR systems to accommodate these new measures, and then the EHR vendors could implement software changes as needed.

Quality measurement is essential during both times of stability and times of crisis. During a crisis, health care is still being delivered, and the need to understand the quality and safety of that care becomes more important as the care processes continue to rapidly change. The challenges highlighted in the current approach to quality measurement were known prior to the COVID-19 pandemic. This crisis has further demonstrated how problematic these challenges can be. Implementing the recommendations for improvement will require a higher level of planning and coordination than in the past around quality measurement, but the risk in not following these recommendations is too high. The health care system should prevent being in a situation with a poor understanding of the quality of health care being delivered, regardless of whether there is a public health crisis.

Back to top
Article Information

Corresponding Author: J. Matthew Austin, PhD, Department of Anesthesiology and Critical Care Medicine, Johns Hopkins Medicine, Johns Hopkins Armstrong Institute for Patient Safety and Quality, 750 E Pratt St, 15th Floor, Baltimore, MD 21202 (

Published Online: June 25, 2020. doi:10.1001/jama.2020.11461

Conflict of Interest Disclosures: None reported.

CMS announces relief for clinicians, providers, hospitals and facilities participating in quality reporting programs in response to COVID-19. Centers for Medicare & Medicaid Services. Published March 22, 2020. Accessed June 4, 2020.
Leapfrog Group. The Leapfrog Group’s response to coronavirus (updated March 30). Published March 30, 2020. Accessed June 4, 2020.
Borgert  M, Binnekade  J, Paulus  F, Goossens  A, Vroom  M, Dongelmans  D.  Timely individual audit and feedback significantly improves transfusion bundle compliance—a comparative study.   Int J Qual Health Care. 2016;28(5):601-607. doi:10.1093/intqhc/mzw071PubMedGoogle ScholarCrossref
Clark  RC, Carter  KF, Jackson  J, Hodges  D.  Audit and feedback: a quality improvement study to increase pneumococcal vaccination rates.   J Nurs Care Qual. 2018;33(3):291-296. doi:10.1097/NCQ.0000000000000289PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words