Survey Tips

Benchmarking surveys can be a daunting and confusing process. This
page will try to make it more manageable and useful to you.

Take Small Bites

Have you ever heard the advice of nutritionists to take small bites of your food and chew thoroughly to aid in good digestion? Well, this advice might apply equally well to benchmarking! All too often, massive surveys are completed that take enormous amounts of time and produce such overwhelming reports that the resultant data is completely ignored. It might be best to identify just a few benchmarks to survey at a time. Then allow your organization to thoroughly “chew up” the resultant data and comfortably “digest” it before taking another bite. In that manner, the information can be comfortably absorbed and your organization won’t develop a case of “in-data-gestion”! OK, enough of the eating metaphor. It is helpful to identify the most pressing issues for benchmarking purposes. Perhaps your organization is addressing significant financial challenges, or your Board has issued certain performance mandates. Maybe your strategic plan calls for special attention to certain issues. Allow your leadership priorities guide your choice of benchmark selections. Avoid those that present curiosity value only. Once the data is obtained, actively work it throughout the system. Then, perhaps a few months later, acquire additional benchmark data and repeat the process. In this way, you can get the best value for your benchmarking dollar.

Don’t Jump to Conclusions

Data is intended to generate hypotheses…not provide definitive conclusions. Error factors are present in all forms of data---this is a reality. The data should not provide a final destination in your thinking, but rather establish a road towards a reasonable set of deductions. The first set of factors to consider in interpreting benchmarking figures is to ask if your organization’s data collection methods were sound. Is your data providing a faithful representation of your organization’s performance? Also, if trended data is not available, it should be asked if there anything that might make that particular sample different from other periods, for example seasonal issues. Finally, differences between the population served by the organization and the national norms may be another factor to consider. At the same time, the purpose of benchmarking is to call the question raised by the data. It is sometimes helpful to submit the findings to the “What If” test. That is, ask the following question: What if that particular finding is accurate? What else would we expect to see within this organization? Let us assume, for example, that your organization’s re-admission rate to the inpatient unit significantly exceeds the established norms. You then ask yourself “If that is true, what else might be present, either as causes, effects, or corroborating pieces of evidence”? Factors such as poor follow-through with outpatient care, inpatient lengths of stay that are too brief, inpatient unit understaffing, and clinical outcome variables might be explored. Upon examining those factors, you might end up establishing what statisticians refer to as a “nomological net”. That is, a set of inter-related factors that support one another’s validity. If your high re-admission rate is accompanied by recent reductions in length of stay, staffing reductions, poor outpatient follow-up, and unfavorable clinical outcomes, you may have identified a very meaningful set of realities for your organization to address.

Be Open to Something You Don’t Want to Know

Let’s face it. We all like good news and hate bad news. Data that tells wonderful stories is eagerly and warmly accepted by organizations and their leadership. The half-life of most negative data is miniscule. All organizations fall prey to an “emperor’s new clothes” stance when they are asked to consider negative data. The temptation to dismiss data as invalid because it is inconsistent with widely held beliefs or sacred convictions is almost too overpowering. Yet it is those dissonant pieces of information that can prove to be most helpful in the long run. Accept data with an open mind and recognize that embracing opportunities for improvement is the highest form of expression of an organization’s commitment and respect for those served.

When interpreting benchmarking information, it is important to keep those points in mind.