Statistics don’t lie, but…

Statisticians—that's another story. And when politicians use statistics to achieve their particular policy goals—well, watch out!

That observation wouldn't surprise many students of the Washington scene. Let's face it; statistics are simply one of the many tools used by policy practitioners to document not only the need for policy change, but the policy change desired. And when the policy topic is as volatile as nursing home quality, statistics fly fast and furious.

I write this just one day after the conclusion of the first wave of congressional hearings on the involvement (and repercussions) of private equity's foray into nursing homes. But the issue, in my mind, is not the pros or cons of investor-owned nursing homes (not that I don't have opinions in that regard. See my article in Nursing Homes/Long Term Care Management, “When It Comes to Quality, Wall Street Might Be a Problem,” May 2005, p. 18.) Rather, it is the use of data to support policy goals that really gets my goat. Here's just one example:

At the November hearings before the Health Subcommittee of Ways and Means, the Service Employees International Union (SEIU) witness compared the number of nursing home survey deficiencies from the annual inspection prior to being bought by private equity against the number of deficiencies found during the most recent annual inspection. The results, he argued, were “distressing.” SEIU found a 29.4% increase in violations of federal resident care standards during the most recent inspections of Mariner facilities after their being acquired by National Senior Care—more than double the 11.9% increase in violations in the other homes in the states in which Mariner operated. Ergo, keep private equity out of the nursing home business.

The above-cited hearings are only the most recent reflection of data selection supporting policy biases. Certainly, as far back as the summer of 1998, increasing attention has been paid to the quality (or lack of it) in America's nursing facilities. Beginning with hearings held in July of that year (by the profession's long-standing nemesis, the Senate Committee on Aging), nursing homes have occupied a central position in the healthcare policy arena. Those events were the initial salvos in what I have referred to previously as a decade-long national “crusade against nursing facilities.” To listen to Senator Charles Grassley (R-Iowa), then chair of the Senate committee, and Nancy-Ann Min DeParle, then administrator of the Health Care Financing Administration (now CMS), nursing home performance was on a downward slope in 1999, with the survey process legislated by OBRA seeming only to document, not reverse, that trend—based, of course, on survey results.

Here are some of the details reflecting HCFA survey data showing differences between 1998 and 1999—and supporting, obviously, the need for more rigorous federal oversight:

• “Poor quality of care” continued to increase, as cited in 21% of facilities surveyed.

• Citations for pressure sores increased from 17.1% to 18% of facilities reviewed.

• “Failure to preserve dignity” rose from ninth place in frequency to sixth.

• Inadequate or inaccurate care plans increased in frequency from 15.2% to 16.1% of homes surveyed.

But, wait just a minute. Shortly thereafter (in 2001), the Institute of Medicine (IoM) unveiled its own report on the quality of care in the nation's long-term care facilities. The Institute set up a committee to assess long-term care services in the United States, including nursing homes, assisted living facilities, and home health services. This was the first report on long-term care the IoM had issued since its 1986 landmark study that spurred creation of the federal nursing facility survey and certification system. In its 2001 report, “Improving the Quality of Long Term Care,” the IoM concluded that the quality of care provided by nursing facilities had actually improved over the decade since OBRA, even though nursing homes were then serving a more seriously ill population.

The IoM review focused on outcomes in such areas as restraint use, the prevalence of pressure sores, malnutrition, continence care, and pain management. In the area of physical restraints, for example, the report pointed to a 41% decline in their use from the early 1990s. While improvement in other areas of care was not as pronounced, the report certainly didn't paint the bleak picture presented by CMS. So, what's going on here?

Well, what's going on is an example of the self-fulfilling prophecy. CMS, of course, assumes that the survey system actually measures quality of care. As part of the campaign initiated as far back as 1998, it emphasized more rigorous surveys, especially in states with traditionally lower levels of citations. The result was, of course, higher levels of citations. CMS's conclusion: Those higher levels of deficiencies were conclusive proof that quality had declined.

And it gets worse. Actually, policy-makers castigate the system when it suits their purpose—and use it when it supports their underlying policy goals. Sen. Grassley, in a letter just a couple of years ago to the administrator of the CMS: “…it is apparent from our review that the survey and certification process upon which we rely for accurate, objective and independent data on the operation and activities of facilities, is just plain broke. It has been corrupted by unscrupulous individuals and we need to restore the integrity of the system in every state and locale.” Yet, it is that same “corrupted” system that the senator (like SEIU) relies on for supporting data when arguing in a recent letter (October 2, 2007) to the comptroller general against the ownership of nursing homes by private equity companies.

But let's let the numbers speak for themselves. Accurate or not, the fact that citations do not consistently measure quality is apparent in CMS's own data over the course of at least the past 10 years. In 1999, it showed that in two states (New Jersey and New Hampshire), 48% and 43% percent of all facilities, respectively, were deficiency-free. In two other states (California and Michigan), only 2% and 4% of facilities had survived their annual surveys without citations.

“Substantial compliance,” had the same results: Four percent of California's facilities and 59% of New Jersey's were deemed to be operating fundamentally in accord with federal standards.

CMS would argue then, as it does now, that the surveys in New Jersey, New Hampshire, and Louisiana were flawed, in that the process followed was excessively lenient toward the facilities surveyed. Look, for example, at Grassley's contention that the system is corrupted in favor of providers (except, of course, when he uses it for his own purposes.) More responsible statisticians would reply that it is equally likely that the reverse might be true. Especially since the variation in survey results is not just a geographic but a temporal phenomenon. In 1994, Idaho and Louisiana saw 2.6% and 2.8% of all facilities cited, respectively, for substandard quality of care. By 1998 that percentage had increased to 11.8% in Idaho but had actually dropped to 2.1% in Louisiana! Similar results in Nevada and Arizona (from 9.5% to 12.2% in Nevada and from 9.7% to 1.8% in Arizona.)

Can the quality of care in nursing facilities really differ that much over time and space? Not very likely. Let's face it, the survey and certification process has, over the years, been noted for its inconsistency. Its results are as much a function of surveyor judgment (and bias) as they are of consistent standards. Look, for example, at CMS's own examples of declining quality, “failure to preserve dignity” being one. Failure to preserve dignity is as much in the eye of the beholder as it is a reflection of obvious transgressions that might actually be harmful. Failure to knock before entering a resident's room can easily rise from an unfortunate (perhaps even unavoidable) oversight to an official citation of actual “harm” (to use survey parlance) when the pressure is on to reach one's “quota” of citations.

Subjectivity based on surveyor perceptions continues to be the most noticeable hallmark of the system. A report by the Department of Health and Human Services’ own inspector general pointed out that the proportion of deficiency-free nursing homes in 2001 ranged from 33.5% in Virginia to 0 in Nevada. Nationally, the average deficiency rate for nursing homes surveyed that year was 6.2 per nursing home, ranging from 2.9 deficiencies per nursing home in Vermont to 11.2 per home in California.

CMS's own data (published in 2002) showed the percentage of facilities posing “immediate jeopardy” to its residents ranging from a low of 0.5% in CMS's region three to a high of 5.9% in region six. These disparities are not much different when it comes to state-specific citations. Take tag number F324 (prevention of accidents) as an example: While the average state showed about 17% of all facilities out of compliance with that particular requirement, the states themselves ranged from 0 to almost 40% incidence. Are we really to believe that potential hazards are that much more prevalent in Michigan and Oregon than they are in Oklahoma and Utah?

Well, thank God for the Institute of Medicine. It certainly can't be accused of being in the pockets of the nursing home industry. Its 1986 study, for example, was a product that captured the attention and respect of most parties to the nursing home quality debate, provider and consumer groups alike. Thankfully, its focus then (and now) was less politics than it was policy. While the IoM often works for the Congress, it apparently sees no need to pander to it (nor, certainly, to the executive branch.)

The IoM clearly recognizes the difficulty of measuring quality by focusing on proxy measures—that is, not on the actual care delivered, but on the processes by which it is delivered. “At the present,” the report noted, “most measurable standards of quality do not recognize excellence in care with criteria based on positive outcomes.” The report goes on to point out that, over the years, “nursing home quality has been measured by structural variables,” even though “opinions in the literature and in testimony given to the [IoM] committee, and even among the committee, vary widely about the empirical evidence linking specific structural features or processes of care to the outcomes in question.” That said, “when such standards are applied, the quality of care is considered lower if problems are found than if they are not found,” the lack of any proven relationship to outcomes notwithstanding.

I've personally seen it happen. A facility with which I am very familiar has prided itself over the years on its exemplary survey results. Indeed, over the past decade it had amassed a stellar run of deficiency-free surveys. And the way it did so was to provide stellar care. But deficiency-free surveys are suspect in this crusade-driven era, with some assuming that a deficiency-free survey is more likely a result of political pressure than of quality care. Well, this month's survey results for the facility I question reflected political pressure from the opposite side: Immediate jeopardy! Threats of closure! “How could this happen?” I asked the owner. “How many deficiencies did you get hit with? In what areas? What's changed?” “One deficiency,” he replied (and a procedural one, at that.) Not one resident was harmed—far from it. But what did change was the survey system, certainly as applied to his facility. Incompetence (by surveyors, not providers,) coupled with personal bias, finally caught up with him. As did a system under pressure to produce “results.” And those results are more and more deficiencies, and more stringent enforcement.

The IoM is clearly aware of the continuing problems with quality in America's nursing facilities. But it is also willing to concede the progress achieved. And it's not likely to equate inadequate quality with profitability or the nature of the investment itself. While it recognizes that there are still operators whose primary interest does not appear to be patient well being, it points also to the shared responsibility of the administration, Congress and the states.

It is aware of the chronic understaffing of America's nursing facilities. And it recognizes staffing as the root cause of many of the problems still documented (as, for example, in the prevalence of pressure sores.) But it is equally cognizant of state and federal responsibilities to make available the requisite resources. “Over the course of this study,” the IoM's 2001 report concluded, “the committee became increasingly persuaded that the amounts and ways we pay for long-term care are probably inadequate to support a work force sufficient in numbers, skills, stability, and commitment to provide adequate clinical and personal services for the increasingly frail or complex populations using long-term care.”

We don't see that same candor in CMS or congressional reports or testimony. But how could we expect to? After all, it was those branches of the federal government, in concert with the states, that supported repeal just 10 years ago of the Boren amendment, the statutory requirement that Medicaid pay the costs of the “efficient and economically operated facility.” I would assume that includes the cost of adequate personnel, which account for two-thirds of nursing home costs. What duplicity, now, for the administration and Congress to condemn the quality of nursing home care and the inadequacy of staffing, without acknowledging their own shared culpability for that sad state of affairs.

But recrimination achieves little. Perhaps the current congressional hearings might just serve as a catalyst for change (no matter how inauspicious their beginnings.) Maybe they will focus on the fact that we all share responsibility for the problem—and that, perhaps, we can all share in its resolution. Criticism based on spurious data is not likely to achieve real resolution. Failure on the parts of CMS and the Congress to recognize that most providers are trying their best won't help. Failure on the part of industry to admit that still too many aren't doing so will be equally shortsighted. Let's move from recrimination to true resolution. And let's use data to support, not undermine, good public policy.

To send your comments to Dr. Willging and the editors, e-mail willging1207@nursinghomesmagazine.com

Sidebar

Paul R. Willging, PhD, was involved in long-term care policy development at the highest levels for more than 20 years. Dr. Willging served for 16 years as president/CEO of the American Health Care Association, was President/CEO of the Assisted Living Federation of America, and later went on to cofound the successful Johns Hopkins Seniors Housing and Care postgraduate program (cosponsored by the National Investment Center for the Seniors Housing & Care Industry). He is currently Associate Director of both the Johns Hopkins Medical School's Division of Geriatric Medicine and Gerontology as well as the Johns Hopkins Center on Aging and Health. He has enjoyed an equally long-lived reputation for offering outspoken, often provocative views on long-term care.