Do Quality Measures Really Indentify Quality Care?

Do Quality Measures Really Identify Quality Care?
Despite efforts to refine them, QMs still have their problems


The Centers for Medicare and Medicaid Services’ (CMS) Quality Measures (QMs) are publicly reported measures documenting outcomes for long- and short-stay nursing home residents. Although they were designed to provide fair measurement of facility quality across variations in facility case-mix, long-term care staff would be well-advised to understand the QMs’ strengths and limitations. For a variety of reasons explained in this article, QMs may not always reflect excellent care, and other measures might well identify quality of care more accurately.
QMs are grouped into chronic care measures and post-acute measures (Table). The QMs are similar to the CMS Quality Indicators (QIs), in that they define quality of care in terms of the nursing home’s clinical/functional outcomes. Like QIs, they do not evaluate directly the specific processes of care in the nursing home. QMs are based on Minimum Data Set (MDS) assessments conducted over a three- or six-month period and represent outcomes that are (1) amenable to quality improvement efforts and (2) of interest to stakeholders (e.g., consumers, families, providers, insurers, etc.).

The six chronic care QMs are intended to measure conditions germane to long-stay residents of long-term care facilities. The proportion of residents with a particular condition allegedly comments on the quality of care provided in the facility. Because excluding admission assessments lessens the impact of pre-existing chronic conditions on QM reports, the 90-day (quarterly) assessment is considered “ideal” for the chronic care population. The practice of excluding 5-day Medicare assessments has the same effect for post-acute measures. In this case, 14-day Medicare assessments are targeted for analysis (although this is not the case for all post-acute measures).

As a practical matter, excluding admission assessments (required by both OBRA and Medicare) does exclude many pre-existing conditions. Pressure ulcers, delirium, and pain upon admission are not considered in the calculation of QMs, for example. However, the very nature of a chronic condition can result in pre-existing conditions being included in QM calculations. Consider, for instance, a resident admitted with a stage III pressure ulcer. It is important to recognize that most facilities will not “cure” such an ulcer within 90 days. Indeed, debridement may make a wound appear to have worsened; other appropriate “best practices” might cause the ulcer to reduce to stage II. In either case, the pressure ulcer will be reflected in the facility’s QM score once it is coded on the “postadmission” MDS. Our firm, LTCQ, found empirical evidence of this. We recently studied 3,700 residents from a large national nursing home chain. The study found that 362 residents (9.8%) had triggered the pressure ulcer QM, but more than half of those residents (54.4%) had a pressure ulcer upon admission. Thus, for more than half of all residents who triggered the pressure ulcer QM, facility care or lack thereof was not necessarily responsible for the development of the ulcer.

Although the intention of using the QMs’ postadmission MDS is to target Medicare 14-day and OBRA 90-day assessments, other assessments not anticipated but mandated when a resident change occurs (e.g., significant change and corrections assessments) also are included in QM results. These significant change or correction assessments can cause pre-existing conditions to trigger a QM as soon as within the first 10 days of residency. In another study of a large multistate chain, LTCQ identified 2,379 residents who, over a one-year period, had been discharged from long-term care “with a return anticipated” and readmitted with a “significant change” assessment. Of these residents, 540 (22.7%) developed a new pressure ulcer while in the hospital. These new pressure ulcers were included in the facilities’ pressure ulcer QM.

CMS uses two risk adjustment practices that attempt to “level the playing field” when calculating QMs: exclusions and covariates. Each practice has merit and, when used together, they come closer to accurate assessment and more fair comparison.

Exclusions remove select residents from the QM analysis-for example, as mentioned above, not counting admission assessments is one form of exclusion. Additionally, few would argue that there is value in measuring a resident’s “improvement in walking” when he or she is near death. Most QMs have identified exclusions, and providers incorporating QMs into their quality improvement activities should be versed in these technicalities. CMS provides many excellent resources for understanding the subtle nuances of the QMs (see “Historical Background).

While exclusions remove many “inappropriate” residents from QM calculations, in practice CMS’ approach may not be enough. Consider residents with end-stage diseases. They are excluded from the calculation of the ADL decline, walking improvement, and delirium QMs, and yet the end-stage MDS item itself is often undercoded. Checking “end-stage” on the MDS requires a physician to document “six months or fewer” life expectancy, and this requirement has created a disincentive to code for end-stage disease. Many physicians hesitate to make such predictions and, as a result, often refuse to complete the required documentation. Close examination of MDS data illustrates this tendency to undercode “end of life.” Using a large dataset containing 13,000 MDS assessments, LTCQ found that 358 (2.7%) of the residents had an MDS indication of end-stage disease. A validated MDS-based death index that included weight loss, shortness of breath, very low body mass index, swallowing problems, congestive heart failure, and advanced age identified 938 (7.2%) residents who were likely to die within one year.1 However, among the residents identified by the death index, the MDS item identified only 7% as having an end-stage disease. Given its poor agreement with a validated measure of proximity to death, the validity of the MDS end-stage disease item must be called into question. With “end-stage disease” being undercoded, residents who clinically should be excluded from QM calculations are included.

Resident-level covariates are functional or clinical conditions that should not exclude the resident from QM consideration, yet warrant different treatment. An example would be the pain QM for chronic care. The resident-level covariate for this QM is an MDS item indicating cognitive impairment, with the rationale being that residents with greater cognitive impairment are less likely to report pain. Therefore, to account for the fact that residents with impaired cognition will report less pain, the pain QM is adjusted to account for the level of cognitive impairment. In this way, a facility with a higher-than-average resident population with intact cognition (and therefore more likely to be able to report pain) will not be “dinged” unfairly in comparison with facilities that have many more cognitively impaired residents. A resident-level covariate is also employed when calculating the post-acute delirium QM.

Table. The Quality Measures.

Type of
1.Residents who had an
unexpected loss of
function in some basic
daily activities
Incidence basedNone
2.Residents with infectionsPrevalence basedNone
3.Residents with pressure
Prevalence basedNone
4.Residents with pressure
ulcers (FAP*)
Prevalence basedFacility level
5.Residents with painPrevalence basedResident level
6.Residents in physical
Prevalence basedNone
7.Residents with deliriumPrevalence basedResident level
8.Residents with delirium
Prevalence basedResident level
Facility level
9.Residents with painPrevalence basedNone
10.Improvement in walkingIncidence basedFacility level
Key:áPost-acute QMs
*FAP: Facility Admission Profile
The Facility Admission Profile (FAP) is a facility-level covariate. The purpose of the FAP is to adjust QM calculations for facilities that admit specific populations relevant to the QM (e.g., a special care unit for pressure ulcers). Few QMs risk-adjust at the facility level-specifically, chronic care pressure ulcers, and post-acute walking and delirium. Although the purpose of the FAP is justifiable, it may not have a strong impact on the reported outcomes. LTCQ found that, within a large multistate chain, 36% of facilities had no difference between their adjusted (FAP) and unadjusted QMs for pressure ulcers, and 35% of facilities had a difference of only 1 percentage point. More than two-thirds, (71%), of facilities’ FAP adjusted and unadjusted QMs differed by one percentage point or less. An additional 17% of facilities had a difference of 2 percentage points. While these differences are statistically valid, they may not be particularly meaningful to consumers making decisions about care or placement.

Risk adjustment is an attempt to account for facility case-mix. Providers should be aware of how their case-mix impacts their QMs (and QIs). A facility’s mission statement can result in its attracting a unique concentration of select residents and generating potentially less desirable QMs. If one’s mission is, for example, to serve a traditionally under-served population-poor inner-city elders, residents with head trauma, or residents with multiple sclerosis-the facility will most likely have QMs differing from more traditional nursing facilities. Specialty care units impact case-mix, as well. Although the term specialty care unit isn’t used uniformly, use of the term creates a perception that often contributes to the facility being a “magnet” for select types of residents. Facility location also impacts case-mix, as do referring relationships and sources.

While it is unsound to use QM reports without additional insights and analysis for purposes of litigation and risk appraisal, this may occur nevertheless. Allegations of poor care should be evaluated through superior risk adjustment and incidence-based metrics, which would include appropriate regional and national benchmarks to reflect local and regional variances (e.g., staffing numbers and patterns). These adjustments and comparisons sift through the impact of case-mix and regional differences and shed light on quality of care. Because the QM approach to analysis appears to prevail, facility management should strive to be clear about their quality status in addressing the survey process, public relations materials, and other forums. This means presenting specific information about your improvement and cure rates-ideally compared to similar facilities-to offset the negative tenor of the QMs.

Despite these limitations, it would do providers and consumers a disservice to write off the QMs as fatally flawed. An educated eye reviewing them can spot trends in performance over several months. Aberrant behavior that cannot be attributed to case mix should be investigated, beginning with a review of MDS data integrity related to items used in QM calculation and ending with an analysis of the resident’s care plan and the facility’s relevant policies and procedures. CMS is committed to continually improving these measures and the associated Quality Improvement initiative. That is a commendable journey, and we must take it together. NH

Historical Background

The goals for the Quality Measures (QMs) are to enable consumers to compare facilities with others in their state or locality, and to give nursing homes useful measures for internal quality improvement programs. In April 2002, CMS, under the auspices of the Department of Health and Human Services, published QMs from its six-state pilot study, both in local newspaper advertisements and on the Nursing Home Compare Web site ( Based on this pilot experience, CMS selected a final set of 10 measures for national implementation in November 2002. For more information about the QM project, its stakeholders, and the process of QM development, go to

Steven B. Littlehale, MS, APRN, BC, is chief clinical officer at LTCQ, Inc., and a board-certified advanced practice nurse in gerontology. Sam Simon, MA, is LTCQ’s research analyst. For further information, phone (781) 674-9600 or e-mail To comment on this article, please send e-mail to

1. Flacker JM, Kiely DK. Mortality-related factors and 1-year survival in nursing home residents. J Am Geriatr Soc 2003;51:213-21.

Topics: Articles , Facility management , Regulatory Compliance