Benchmarking surveys can be a daunting and confusing process. This
page will try
to make it more manageable and useful to you.
Take Small Bites
Have you ever
heard the advice of nutritionists to take small bites of your food and chew
thoroughly to aid in good digestion? Well, this advice might apply equally well
to benchmarking! All too often, massive surveys are completed that take enormous
amounts of time and produce such overwhelming reports that the resultant data is
completely ignored. It might be best to identify just a few benchmarks to survey
at a time. Then allow your organization to thoroughly “chew up” the resultant
data and comfortably “digest” it before taking another bite. In that manner, the
information can be comfortably absorbed and your organization won’t develop a
case of “in-data-gestion”! OK, enough of the eating metaphor. It is helpful to
identify the most pressing issues for benchmarking purposes. Perhaps your
organization is addressing significant financial challenges, or your Board has
issued certain performance mandates. Maybe your strategic plan calls for special
attention to certain issues. Allow your leadership priorities guide your choice
of benchmark selections. Avoid those that present curiosity value only. Once the
data is obtained, actively work it throughout the system. Then, perhaps a few
months later, acquire additional benchmark data and repeat the process. In this
way, you can get the best value for your benchmarking dollar.
Don’t Jump to
Data is intended to generate hypotheses…not provide definitive
conclusions. Error factors are present in all forms of data---this is a reality.
The data should not provide a final destination in your thinking, but rather
establish a road towards a reasonable set of deductions. The first set of
factors to consider in interpreting benchmarking figures is to ask if your
organization’s data collection methods were sound. Is your data providing a
faithful representation of your organization’s performance? Also, if trended
data is not available, it should be asked if there anything that might make that
particular sample different from other periods, for example seasonal issues.
Finally, differences between the population served by the organization and the
national norms may be another factor to consider. At the same time, the purpose
of benchmarking is to call the question raised by the data. It is sometimes
helpful to submit the findings to the “What If” test. That is, ask the following
question: What if that particular finding is accurate? What else would we expect
to see within this organization? Let us assume, for example, that your
organization’s re-admission rate to the inpatient unit significantly exceeds the
established norms. You then ask yourself “If that is true, what else might be
present, either as causes, effects, or corroborating pieces of evidence”?
Factors such as poor follow-through with outpatient care, inpatient lengths of
stay that are too brief, inpatient unit understaffing, and clinical outcome
variables might be explored. Upon examining those factors, you might end up
establishing what statisticians refer to as a “nomological net”. That is, a set
of inter-related factors that support one another’s validity. If your high
re-admission rate is accompanied by recent reductions in length of stay,
staffing reductions, poor outpatient follow-up, and unfavorable clinical
outcomes, you may have identified a very meaningful set of realities for your
organization to address.
Be Open to Something You Don’t Want to Know
it. We all like good news and hate bad news. Data that tells wonderful stories
is eagerly and warmly accepted by organizations and their leadership. The
half-life of most negative data is miniscule. All organizations fall prey to an
“emperor’s new clothes” stance when they are asked to consider negative data.
The temptation to dismiss data as invalid because it is inconsistent with widely
held beliefs or sacred convictions is almost too overpowering. Yet it is those
dissonant pieces of information that can prove to be most helpful in the long
run. Accept data with an open mind and recognize that embracing opportunities
for improvement is the highest form of expression of an organization’s
commitment and respect for those served.
When interpreting benchmarking
information, it is important to keep those points in mind.