Making (Up) the Grade: What You Need to Know About Nursing Home "Report Cards"

by Charles Crecelius, MD, PhD, CMD, with Kathleen M. Wilson

Consumers face a daunting task when placing their loved one in a nursing home. Although some printed guides are available, assessing the quality of care in a facility and making a good decision still requires physical visits and a significant investment of time. Web sites that use analyses of federally derived data to arrive at simplified rating systems have been promoted to help consumers quickly determine nursing home quality.

Although these sites may have merit at first glance, the data on which they base ratings--OSCAR reports and survey findings available on the Medicare Web site (www.medicare.gov)--were not designed for use by prospective residents and their families. Therefore, such sites are limited in their ability to provide a complete and accurate picture of the current status of the nursing home. Worse, they may give misinformation and deter families from considering worthy facilities.

Long-term care providers should understand these rating systems to help the consumer determine their use and potential misuse. This is especially important now, since the Centers for Medicare and Medicaid Services will begin to use certain quality measures to produce nursing home "report cards" in October (see "Cultural Change Key to Nursing Home Quality Initiative" in the March 2002 issue of Caring, and the follow-up article "Mixed Reaction to CMS' Release of Nursing Home Quality Data" in this issue).

Popular Rating Systems

The Medicare site offers Nursing Home Compare, an analysis prototype from CMS that allows consumers to search for facilities by state, town, zip code, or name. No specific rating is offered, but data are presented in the context of state and national averages. Information provided includes the latest standard and complaint survey deficiencies with scope-and-severity notations, selected OSCAR data, and staffing information.

The OSCAR data include selected quality indicators: bowel and bladder incontinence, dependency in eating, unplanned weight loss, use of physical restraints, bed bound residents, pressure sores, and behavioral symptoms. Staffing data include hours per resident per day for RNs, licensed staff, and CNAs.

HealthGrades (www.healthgrades.com) offers analyses of federally available information on any health care facility, including nursing homes. The Web site made headlines several months ago when it presented data on the best and worst cities for nursing home care (based on the percentage of homes with actual harm citations). Whether the data really represented quality of care or variability in survey consistency was debatable, but this was not obvious to the consumer reading the article. HealthGrades' standard nursing home report uses a three-tiered global rating (top 30%, middle 40%, bottom 30%) that is available free of charge. A more intense analysis--including the last four years of standard and complaint survey results, with weighting factors determined by an expert panel to arrive at simplified ratings--is available for a fee. The rating is unique in showing repeat citations as evidence of possible system problems.

Premier Care (www.premiercare.com) offers a one-to-five-star rating--based on both current and historical data, and compared with state and national standards--for free. Similar to Health Grades, a more complete analysis is available for a price. This site also reports the Nursing Home Compare OSCAR data and staffing levels, and weighs data using an expert panel to arrive at a simplified grade.

There are several other sites, such as Nursing Home Reporter (www.seniorcarehelp.com) and Search Nursing Homes (www.thebestisyet.net), which only offer information for a fee. These sites claim to give more complex, expert-weighted analysis using federally available data. All of these non-governmental Web sites provide variable information on utilizing the data, suggestions on visiting and questions to ask the facility staff, and general information on types of homes, finances, and surveys.

Problems in Public Presentation of Federal Data

As noted earlier, federal data gathered through the Minimum Data Set and survey process were not originally designed to be utilized by prospective residents and their families. The information was derived to facilitate analysis of nursing home characteristics, identification of possible quality concerns, and substantiation of compliance with federal regulations. Although they are important concepts and tools, their usefulness in assisting the consumer is questionable. It is difficult enough to assimilate and consolidate reams of federal law and regulations provided in the State Operations Manual into a user-friendly form for the professional, much less for the resident and family.

It is unclear if an "expert panel," such as those claimed to be used by HealthGrades and Premier Care, can resolve these and other concerns to arrive at a meaningful, simplified "grade."

For example, survey results may not reflect a year's worth of poor quality care, but only isolated instances. Even presenting several years of survey results can be misleading, since data from past years may be irrelevant to current issues and quality concerns.

Surveys are subject to surveyor variability and individual interpretation. They are often not indicative of process problems and do not identify root causes or underlying system problems. More importantly, they provide no information on plans of correction. Does the nursing home have the ability to accurately assess and correct problems, or do they merely find "band-aid" solutions?

Can concepts such as potential versus actual harm and its relationship to immediate jeopardy be adequately explained or understood? If not, can the consumer really understand severity determinations?

OSCAR data have several drawbacks when used to present quality indicators without adequate explanation to consumers. There are too many quality indicators to present in a concise form to consumers. The indicators used may not reflect issues of concern to the consumer. For example, since many quality indicators are prevalence-based, they don't address how facilities take care of inevitable problems such as weight loss, pressure ulcers, and symptoms of depression. In most instances, the quality indicators are so difficult to interpret that they can't simply tell consumers if their loved ones will be cared for.

These and other considerations--for example, not differentiating between acute and chronic residents--contributed to the Nursing Home Quality Initiative pilot program, which attempts to provide consumers with comparative data on facility quality measures. However, in the six-state pilot project, the data were not risk adjusted for the six chronic-care measures and three post-acute measures (see the box Quality Measures in CMS Pilot Project included in the article "Mixed Reaction to CMS' Release of Nursing Home Quality Data"in this issue).

Without these adjustments, comparing nursing homes may be difficult. For example, nursing homes that specialize in wound care may look bad in pressure ulcers prevalence. Both quality measures and quality indicators have several confounding factors that reduce their usefulness to consumers. There can be significant inter-rater reliability problems in certain categories, for example, with up to 10% to 15% disagreement among relatively seasoned raters. Also, most data represent a single point in time and do not show trends that may be more representative of quality than intrinsic resident acuity characteristics. There is no method to consider how facilities use the data to guide effective quality-assurance programs, a better measure of effective management.

Furthermore, staffing numbers do not adequately reflect real-world care issues. There is no consideration of staff turnover. Quality and frequency of training aren't represented, nor is use of agency nursing. Experience, teamwork, leadership, efficiency, and morale are examples of characteristics that cannot be reduced to numbers, but are far more important than the absolute number of staff in the building. Acuity of residents and the projected stable and trained staff required to meet their needs would be a more accurate method of determining staffing's relationship to resident care quality, but there currently are no means of presenting these data.

What Can Long-Term Care Providers Do?

There are several ways consumers can be helped to understand the limits of report cards while assisting them to truly understand facility quality (see the box at right).

Providers should understand recent OSCAR data, survey results, and plans of correction. They should be able to relate the acuity of the facility's population and trends in the data to current quality-improvement efforts. Providers should also be able to explain their roles in the quality-assurance process, staff training, oversight of medical staff, and in ensuring resident care quality.

A "response statement" can be attached to the latest survey or report card to explain the facility's interpretations and current efforts to improve quality. Nursing homes can provide their own "quality-scoring sheet," containing a display of the positive aspects of the home, instructions on how to appropriately rate a nursing home, and a checklist of things to look for (there is an abundance of material on how families can perform a "walk- through" inspection both in print and on the Web).

Facilities should collect completed (anonymously) checklists for use as a quality-assessment tool. They should provide additional blank checklists for use in assessing other nursing homes the family may wish to investigate. Current residents and families can be given a similar tool to complete, so that staff can better understand consumer concerns related to quality indicators and measures. Repeat polling can help determine whether quality-improvement goals are met.

Looking Ahead

The national presentation of quality measures in October will provide the consumer with a more simplified, readily available "report card" compared with those currently available on Web sites. Whether the CMS initiative will meet its goal of providing data that reflect facility care--and how consumers will use such data--remain to be seen.

Administrators, medical directors, and directors of nursing are in a unique position to understand relevant data such as resident characteristics, staff quality, and care-related survey results, and should be able to present a true report of the facility's capabilities.

Physicians, in particular, still maintain a unique level of respect with residents and families, and their independent opinions should be as important as any other "report card."

Whether provided through Web site analyses or quality measures, simple "grades" cannot adequately explain the complexities of nursing home quality. Such "grades" can be more informative when accompanied by a thorough interpretation and explanation of the strengths and limitations of the data from the facility.

Dr. Crecelius is President of the Missouri Association of Long-term Care Physicians and Medical Director of Delmar Gardens in St. Louis, MO. Kathleen M. Wilson is AMDA's Former Director of Government Affairs and a Contributing Writer to Caring.

back to top