What’s Wrong-And Right-With The Baldrige Awards

Many chief executives respond to the Malcolm Baldrige National Quality Awards in one of two rather off-the-mark ways.

Some embrace it with an almost religious fervor. They see it as the salvation of American business and pursue an award as if it were the Holy Grail itself.

Others-perhaps fueled by negative press coverage of Baldrige-rattle off a litany of criticisms: It costs too much to apply for the award, the prize is not a reliable predictor of financial performance, companies with poor product quality have won.

The criticisms usually are bolstered by a recital of the faults of previous Baldrige winners-of which admittedly there are some-and ultimately imply the entire award program is a load of bunk. In fact, in a recent survey by a Chicago-based accounting and management consulting firm, Grant Thornton, 28 percent of executive respondents said the “awards are a waste of time and money and should be abandoned.” Both of these bipolar views are unrealistic. The Baldrige, meaning the prize itself, is not the be-all, end-all that some claim it is; neither is it the empty, invalid exercise portrayed by its detractors. Instead, Baldrige should be seen as an effective tool for launching and assessing specific total quality management efforts and for judging organizational performance in a more general sense.

I believe the perception problem stems from focusing only on the award itself and ignoring the Baldrige process: the powerful concepts and practices represented in the Baldrige application and guidelines. Curt Reimann, who directs the Baldrige award program, observes, “I suspect there is still a lack of understanding of the [Baldrige] system.”

And it’s the system, or process, of using this painstakingly developed, and internationally recognized and accepted, set of criteria (see graphic) to assess and improve operations that can help any company better position itself to become a viable, competitive, long-term player in the marketplace. These benefits are available to all companies, most of which probably will never apply for the award.

THE CRITICAL SIDE

The Baldrige has had many positive ramifications. For example, it has helped individual companies, business schools, and American business in general to develop a standardized “language of quality,” while increasing awareness of quality-improvement methods. Yet the Baldrige has been controversial almost from the moment its creation was announced in August 1987. The lion’s share of Baldrige-related press coverage has gone toward enumerating the award’s shortcomings. To he sure, they exist. But as any American senior executive knows, the press often emphasizes bad news over good, cynicism overpraise.

Here’s a rundown of some of the items on the laundry list of criticisms. Most have some basis in fact, but many-like reports of Mark Twain’s death-have been greatly exaggerated.

There is no relationship between Baldrige scores and financial performance or long-term planning.

This complaint stems largely from the fact that several winners-Memphis, TN-based Federal Express; Houston pipe-andvalve distributor Wallace Co.; and Detroit’s Cadillac Motor Car Co. among them-suffered poor financial results after winning the award. (Wallace, in fact, filed for protection from its creditors less than 18 months after winning the Baldrige.)

It is true that the first edition of the Baldrige judging criteria gave short shrift to financial indicators. But the criteria have been revised over time (consistent with that basic quality concept, “continuous improvement”) and now require applicants to include more financial data. However, the fact remains that Baldrige examiners never will be able to predict a company’s financial future with precision. Performance is subject to many uncontrollable factors, including difficulties in the international marketplace (as in the case of Federal Express, for example) and unforeseeable economic and competitive changes. For this reason, it is shortsighted to point to had financials from a single quarter or a single year and claim that a particular Baldrige award was not merited. Surely the organizational strengths implied in a good Baldrige score or a Baldrige “win” are likely to translate into higher chances of strong financial performance over the long term. This point was supported by a General Accounting Office study of 24 Baldrige finalists, whose key operating and financial indicators were proven to be consistent with sustained success and future competitiveness-though, of course, there are no guarantees.

Moreover, the Baldrige process in many cases has shown a direct correlation with financial performance. Armonk, NY-based IBM, for example, administers an internal competition using the Baldrige judging methodology. In 1991, the company compared the performance of its top-scoring and lowest-scoring divisions on several measures. The top divisions performed significantly better in such areas as customer satisfaction, employee satisfaction, market share, revenue growth, and profitability.

Finally, the criticism about Baldrige’s failure to consider long-term planning issues was valid during the award’s first years. As with the new criteria regarding financial data, however, the Baldrige application has been expanded to contain more questions about both short- and long-term planning, including questions such as, “How [are] such elements as key quality requirements…integrated into overall business planning[?]”

The Baldrige application is bard to read, understand, and complete, making it too costly to apply for the award.

Baldrige believers surely would respond to this one by citing the case of Globe Metallurgical in Beverly, OH, one of the first award winners. The company’s vice president of operations reportedly wrote the application in a single weekend, with the help of one assistant.

Clearly it must have been a hectic weekend! Anyone who has seen the Baldrige application knows the questions are numerous, lengthy, and complex, akin to a college entrance examination. But, despite their number, they are not especially difficult to understand: Every item is explained in detail, with examples of what kinds of data to include and what programs to describe. (See sidebar for an excerpt of one section of the Baldrige application.)

Still, it can be a formidable task to gather all the necessary information-and properly so. Quality improvement is not a simple process. If the application were less thorough, the evaluation and assessment process would be far less effective and obviously less valid. There are no shortcuts in quality improvement, and no shortcuts in evaluating a company-wide quality process. In contrast to the near-legendary story of Globe Metallurgical, other companies have doled out tidy sums to complete the Baldrige application. Stamford, CT-based Xerox, for example, reportedly spent $800,000 and paid 20 workers full-time for over a year to prepare its winning Baldrige application. Those who spent similar amounts without winning an award are probably the most vocal in making this second criticism. Xerox, however, believes that its money was well-spent-and not because it won the award, but because the Baldrige process served as a catalyst for its improvement efforts. A former Xerox vice president said, “Compared to the benefits we received, the investment of $800,000 and the time of those people was very small.”

Perhaps most important, Xerox waged a full-scale effort to win the award and reaped the benefits from the improvements it made internally before it could compete successfully. Most companies should follow Xerox’s example and concentrate on using the Baldrige methodology internally for evaluation and improvement purposes rather than aiming at winning an award.

Baldrige judges the nature of a company’s 7QM and quality improvement processes, not the quality of its products or services.

Critics raised this objection when Cadillac won a Baldrige award. For one thing, its products failed to garner the highest ratings in some automobile-quality surveys the year the company won. In addition, conventional wisdom has long held that the quality of American cars is inferior, and that GM in particular is a problem-ridden organization.

Baldrige examiners don’t have a crystal ball any more than CEOs do: Public reaction to a product occasionally may seem out of synch with Baldrige scores. And the judging criteria do stress a company’s deployment of leading-edge quality processes as much as they stress the end product or service.

Just the same, Cadillac could not have earned a decent score, let alone won, if it did not have an efficient, cost-effective approach to producing high-quality products. Its automobiles were rated high on customer satisfaction (and what better single quality measure exists?), placing Cadillac on a par with its primary competition, companies such as Japan’s Lexus and Germany’s Mercedes-Benz.

It is also important to note that Cadillac’s Seville was named Motor Trend’s Car of the Year for 1992-the year in which the company’s TQM efforts truly reached fruition. One Baldrige examiner observed, “Most people’s view of a company comes from hindsight [looking at what a product has been], but the Baldrige is looking forward [i.e., at what Cadillac’s new processes were designed to, and ultimately did, achieve].” In short, if the Baldrige is intended to recognize world-class quality, it achieved that goal in conferring an award on Cadillac, despite the company’s historically poor image.

Finally, few people would criticize the excellent quality of most other Baldrige winners’ products and services (e.g., those of Xerox, IBM, and Federal Express). Most of the sound and fury from this third Baldrige criticism has been aimed at one winner out of 17.

Baldrige represents an additional workload for executives already straining to coordinate a host of other improvement initiatives.

This beef reflects a misunderstanding not just of Baldrige but of quality-improvement programs in general. Properly conceived, a quality process serves as an integrating “umbrella” for all of a company’s operations and planning. A Baldrige-based process is the best available tool for this integration, providing a detailed road map for performance improvement.

Not long ago, I met with the senior executive team at a hospitality chain to discuss Baldrige and its uses. The chief financial officer told me Baldrige actually would be dangerous to his firm. With all the other important initiatives the company was working on, he said, it would be irresponsible to think about adding an award application to executives’ sky-high “in” boxes. “We’re already overloaded, and people in the trenches are gagging,” he added. “We just can’t take on one more thing right now.” The others in the room voiced their agreement. I asked the executives what initiatives were in progress, and they reeled off a long, helter-skelter list: developing new programs and structures for employee empowerment, management compensation, customer retention, a new MIS system, training, etc. (and, inevitably, quality improvement, though this was way down on the list).

As in any case of “initiative overload,” something has to give-and usually, it’s a little bit of everything. Few of these programs were getting the time, attention, and resources they needed, and employees were (understandably) skeptical about management’s commitment to any of them.

Given my familiarity with Baldrige, I saw that each item the executive team mentioned fit into the Baldrige framework, and that the goals these programs were designed to achieve dovetailed with Baldrige goals. I explained how Baldrige, rather than being another onerous-and separate-responsibility, could be the company’s salvation in sorting out, prioritizing, and implementing its now-scattershot initiatives.

After walking through selected sections of the award criteria and seeing how the process could be used internally, the executives did a turnaround. They saw that by conducting a Baldrige assessment of their company’s strengths and weaknesses, and translating the results into a strategic quality plan, they could control their initiative overload and organize their various programs. Using the Baldrige this way makes a company concentrate resources where (and when) they should go for dramatic performance gains.

A DIFFERENT PERSPECTIVE

What many critics fail to recognize is that the primary value of Baldrige lies in the process-the road map provided in the award guidelines-rather than in the handful of awards conferred each year.

Those who have dismissed Baldrige after reading a few critical articles in the business press are, sadly, missing out. The truth is that most companies can’t dream of winning the prize right now, given the current state of their quality processes, but they still stand to win by using the Baldrige judging standards and methods for their internal assessment and strategic planning.

How a company uses Baldrige depends on many things, including the sophistication and extent of its quality efforts. A fifth-grader can’t just plop down and take an SAT exam from start to finish. Likewise, a company whose quality efforts are still fledgling naturally would be overwhelmed if it tried to take the whole Baldrige “test,” which includes gathering all the data required for a formal award application, writing extensive reports about programs in place and their qualitative and quantitative results, and detailing a long-term strategic quality plan.

But even the business equivalent of a fifth-grader can use portions of Baldrigeformally or informally, internally and/or with the help of quality professionals.

OPTIONS FOR SELF-ASSESSMENT

The simplest use of Baldrige is as a discussion vehicle. The application guidelines can be distributed in whole or in part to all employees, selected employee groups or divisions, or just senior management. At the very least, this will help staff members master new quality concepts and identify weak areas they would like to strengthen. An executive can glean many insights, and often a surprising number of suggestions for immediate improvement, just by organizing a group (or groups) to discuss how the judging criteria apply to the company and where obvious quality gaps exist.

Another way to use Baldrige is as a survey instrument. It can be applied rigorously (with data collection) or informally (as a way of gathering opinions); it can be applied to individual departments, functions, geographic regions, or to the entire organization; it can be applied in its entirety or a section at a time (for example, Leadership, Human Resources, or Customer Focus, the overarching judging criteria depicted in graphic).

To begin, executive, managerial, and employee groups can be asked to respond to Baldrige questions on “areas to address”: Where does the company stand on education and training? What are the company’s strengths and its largest problem areas? (Baldrige numbers about 100 areas to address, and management may wish to “collapse” these into a smaller number, both to make the survey easier to answer and to highlight certain issues.)

Although this kind of survey yields only opinions, not facts, it can produce eye-opening results. For one thing, mind-boggling gaps often exist between what executives say about their quality focus and what lower-level employees think the company does about quality. For another, this is a fast way to identify areas that everyone agrees are out of control.

I have found that some departments in an organization are far ahead of others in their quality “mind-set” and processes. Such departments benefit greatly from using the entire Baldrige framework to assess their functions and pinpoint areas for further improvement. At one company I’ve worked with, the marketing department took this tack; at another, human resources did. Later, these groups can shepherd other parts of the organization through the Baldrige process.

Ultimately, of course, as a company moves from the fifth-grade to college-entrance level, the Baldrige is of optimum benefit when used to assess the entire organization and to monitor the development of long-term strategic plans. This kind of assessment requires extensive data collection, analysis of findings, and report writing. As a result, it usually is performed with the help of experts from outside the company. This is a good route to take when performance improvement is a high priority, when the time of company executives is at a premium, and/or when internal politics might bias the results of a self-assessment.

While a full assessment-preparing a formal Baldrige application without applying for an award-can be costly, there are creative ways to clamp a lid on costs. For example, in a corporation with more than one business unit, assessors can be “borrowed” from another division to gain the objectivity of a consultant without a consultant’s fees. Some companies also have tapped representatives of their major suppliers and even customers for their assessment teams. Keep in mind, however, that the presence of a key customer can greatly bias the process; few companies want all their “warts” exposed to customers. Most often, this type of assessment is performed with a combination of internal people and quality consultants.

Wherever they come from, outside assessors help ensure that the Baldrige assessment will be objective and that results will have credibility throughout the organization. Outsiders are more likely to ask candid questions and receive candid answers about politically sensitive issues, especially about the quality of a company’s leadership team. Additionally, outside experts with extensive Baldrige experience are familiar with common quality problems and solutions; they are more likely than insiders to recognize patterns in the assessment findings and help an organization determine what to do with those findings.

When a company has reached a point at which a full-scale assessment has been performed or is being considered, it may be ready to think about applying for a Baldrige Award.

I have maintained here that the award itself is not the main benefit of Baldrige. But benefits can be gained from going through the work of submitting a formal application, even if the company is still figuratively in, say, 11th grade.

The Baldrige examiners provide detailed feedback (free consulting) to all applicants about areas they most need to work on. The examiners draw attention to programs that “sound good” but which may not be yielding measurable results such as reducing employee turnover or boosting customer satisfaction. They also emphasize the need to streamline operations and bureaucracy for faster cycle time and to foster innovation.

Those companies lucky enough (and good enough) to become finalists earn site visits from examiners, who provide even more in-depth feedback (and whose additional questions, posed on-site, can be a valuable learning experience).

 ASSESSING VALUE

The upside of all the criticism leveled at the Baldrige Awards is that it has helped those who administer the award program fix its shortcomings and continually improve the process. The downside is that it has caused more than a few executives to write off the awards as meaningless, preventing them from looking at what the Baldrige methodology has to offer.

Whether or not a company applies for the award, going through a Baldrige-based assessment forces executives to learn more about their companies; to come to grips with their leadership abilities; and to develop workable, prioritized strategies for future improvement. Sure, it’s possible to achieve these goals without using the Baldrige road map, but it’s a lot harder. Why spend a fortune hiring quality experts to design an assessment methodology-one that inevitably would look a lot like Baldrige assessment techniques anyway? In the Baldrige application and guidelines, every company has a ready-made framework, designed by hundreds of top quality experts in industry and education from around the globe. Better still, it’s free.

This is one instance in which government truly has provided business with an invaluable service.

Christopher W.L. Hart is president of The TOM Group, a Boston-based management consulting and executive education firm that works with clients to develop and implement breakthrough strategies for achieving extraordinary levels of service performance. He has written extensively on quality issues. One of Hart’s most recent books is “The Baldrige: What It Is, How It’s Won, How to Use It to Improve Quality in Your Company.”