BALDRIGE APPLICATION REVIEW AND SCORING PROCESS
The Baldrige Award Evaluation Process
The Baldrige evaluation process is rigorous, and designed to be objective and tamper proof
against political pressures. In the first stage, each application is thoroughly reviewed by
approximately 7 examiners chosen from among leading quality professionals in business,
education, healthcare, and nonprofits (all of whom are volunteers). Examiners evaluate the
applicant’s response to each examination item, listing major “strengths” and “opportunities
for improvement” relative to the criteria. Strengths demonstrate an effective and positive
response to the criteria. Opportunities for improvement do not prescribe specific practices or
examiners’ opinions on what the company should be doing, but rather deficiencies in
responding to the criteria. Based on these comments, a percentage score from 0 to 100 in
increments of 10 is given to each item. Each examination item is evaluated on
approach/deployment or results.
Approach refers to the methods the company uses to achieve the requirements addressed
in each category. The factors used to evaluate approaches include:
1. The appropriateness of the methods to the requirements
2. The effectiveness of methods, namely, the degree to which the approach is
repeatable, integrated, and consistently applied; the degree to which the
approach embodies evaluation/improvement/learning cycles; and is based on
reliable information and data
3. Alignment with organizational needs
4. Evidence of innovation
Deployment refers to the extent to which the approaches are applied to all requirements
of the item. The factors used to evaluate deployment include:
1. Use of the approach in addressing item requirements relevant to the
organization
2. Use of the approach by all appropriate work units
Results refers to outcomes in achieving the purposes given in the item. The factors used to
evaluate results include:
1. Current performance levels
2. Performance levels relative to appropriate comparisons and benchmarks
3. Rate, breadth and importance of performance improvements
4. Linkage of results to key customer, market, process, and action plan
performance requirements identified in the approach/deployment items and
other important factors to the organization
The scoring guidelines by which an application is evaluated can be found in the
criteria document on the CD-rom. Basically, each item is scored from 0 to 100 percent in
increments of 10 percent to reflect the strength in addressing the evaluation factors described
above. The scoring guidelines are constructed so that 50 percent represents a very solid and
effective approach; few organizations – including winners – receive item scores exceeding 70
or 80. Numerical scores for each examination item are then computed by multiplying the
examiner’s score by the maximum point value that can be earned. For example, the Senior
Leadership item is worth 70 points. Thus, a score of 60 percent would result in 42 points
toward the maximum possible point total of 1000.
The examiner team conducts a consensus process in which they discuss variations in
individual scores and arrive at consensus scores and comments for each item. The Panel of
Judges then reviews the scores and selects the highest scoring applicants that they believe
have the potential to be a recipient for site visits. At this point, an examiner team visits the
company for up to a week to verify information contained in the written application and
resolve issues that are unclear. Final contenders each receive more than 1,000 hours of
evaluation. The judges use the site visit reports to recommend award recipients to the
Secretary of Commerce. All information is kept strictly confidential, and examiners are
bound by conflict of interest rules and a code of conduct.
All applicants receive a feedback report that critically evaluates the company’s strengths
and areas for improvement relative to the award criteria. The feedback report, frequently 30
or more pages in length, contains the evaluation team’s response to the written application. It
includes a distribution of numerical scores of all applicants and a scoring summary of the
individual applicant. This feedback is one of the most valuable aspects of applying for the
award.
Independent Review
Following the receipt of the Award applications, the first step of the Award process review
cycle (shown in Figure 1) begins with the Independent Review, in which members of the
Board of Examiners are assigned to each of the applications. Assignments are made
according to the Examiners’ areas of expertise and to avoid potential conflicts of interest.
Each application is evaluated independently by Examiners, who write observations relating
to the Scoring System described in the Criteria for Performance Excellence.
Consensus Review
Beginning in 2007, all applicants are benefiting from Consensus Review. A team of
Examiners, led by a Senior Examiner, conducts a series of reviews, first managed virtually
through a secure Web site and eventually concluded through a focused conference call. The
purpose of this series of reviews is for the team to reach consensus on comments and scores
that capture the team’s collective view of the applicant’s strengths and opportunities for
improvement. The team documents its comments and scores in a consensus scorebook. The
Consensus Review process is shown in Figure 2.
Site Visit Review
After the Consensus Review process, the Panel of Judges selects applicants to receive site
visits based upon the scoring profiles. If an applicant is not selected for Site Visit Review,
one of the Examiners on the Consensus Team edits the final consensus report, which
becomes the feedback report.
Site visits are conducted for the highest-scoring applicants to clarify any uncertainty
or confusion the Examiners may have regarding the written application and to verify that the
information in the application is correct. After the site visit is completed, the team of
Examiners prepares a final site visit scorebook. The Site Visit Review process is shown in
Figure 3.
Applications, consensus scorebooks, and site visit scorebooks for all applicants
receiving site visits are forwarded to the Panel of Judges. The Judges make final
recommendations on which applicants should receive the Award. The Judges discuss
applications in each of the six Award categories separately, and then they vote to keep or
eliminate each applicant. Next, the Judges decide whether each of the top applicants should
be recommended as an Award recipient based on an “absolute” standard: the overall
excellence and the appropriateness of the applicant as a national role model. The process is
repeated for each Award category. The Judges’ review process is shown in Figure 4.
Judges do not participate in discussions or vote on applications in which they have a
competing or conflicting interest or in which they have a private or special interest, such
as an employment or a client relationship, a financial interest, or a personal or family
relationship. All conflicts are reviewed and discussed so that Judges are aware of their own
and others’ limitations on access to information and participation in discussions and voting.
Following the Judges’ review and recommendations of Award recipients, the Site Visit Team
Leader edits the final site visit scorebook, which becomes the feedback report.
Award Process Review Cycle
Feedback Report
to Applicant
Judges Meet
Mid-November
Judges Recommend Award
Recipients to NIST Director/
Secretary of Commerce
Not Selected
Applications Due
CD: Mid-May
Paper: Late May
Independent Review
& Consensus
Review
June–August
Figure 1—Award Process Review Cycle
6
Step 1
Step 2
Step 3
Step 4
Consensus Planning:
• Clarify the
timeline for the
team to complete
its work
• Assign
Virtual Consensus:
• Review all
Independent
Review
evaluations—
draft consensus
Consensus Calls:
• Discuss a limited
number of issues
related to specific
comments or
scores, and
Post-Consensus Call
Activities:
• Revise comments
and scores to
reflect consensus
decisions
Figure 2—Consensus Review Process
Step 2
Site visit:
• Make/receive
presentations
Figure 3—Site Visit Review Process
7
Step 2
Evaluation by Category:
• Manufacturing
• Service
Figure 4—Judges’ Review Process
SCORING
The scoring system used to score each Item is designed to differentiate the applicants in
the various stages of review and to facilitate feedback. The Scoring Guidelines shown in
Figure 5 (for healthcare – similar guidelines are published for business and education) are
based on (1) evidence that a performance excellence system is in place; (2) the maturity of
the applicant’s processes as demonstrated by Approach (A), Deployment (D), Learning (L),
and Integration (I); and (3) the results the applicant is achieving.
8
SCORE
PROCESS (For Use With Categories 1–6)
0% or 5%
§ No systematic approach to Item requirements is evident; information is anecdotal. (A)
§ Little or no deployment of any systematic approach is evident. (D)
§ An improvement orientation is not evident; improvement is achieved through reacting to
problems. (L)
§ No organizational alignment is evident; individual areas or work units operate
independently. (I)
10%, 15%,
20%, or 25%
§ The beginning of a systematic approach to the basic requirements of the Item is evident.
(A)
§ The approach is in the early stages of deployment in most areas or work units,
inhibiting progress in achieving the basic requirements of the Item. (D)
30%, 35%,
40%, or 45%
§ An effective, systematic approach, responsive to the basic requirements of the Item,
is evident. (A)
§ The approach is deployed, although some areas or work units are in early stages of
deployment. (D)
§ The beginning of a systematic approach to evaluation and improvement of key processes
is evident. (L)
§ The approach is in the early stages of alignment with your basic organizational needs
identified in response to the Organizational Profile and other Process Items. (I)
50%, 55%,
60%, or 65%
§ An effective, systematic approach, responsive to the overall requirements of the Item,
is evident. (A)
§ The approach is well deployed, although deployment may vary in some areas or work
units. (D)
70%, 75%,
80%, or 85%
organizational–level analysis and sharing. (L)
§ An effective, systematic approach, responsive to the multiple requirements of the Item,
is evident. (A)
§ The approach is well deployed, with no significant gaps. (D)
§ Fact–based, systematic evaluation and improvement and organizational learning are key
90%, 95%, or
100%
the Organizational Profile and other Process Items. (I)
§ An effective, systematic approach, fully responsive to the multiple requirements of the
Item, is evident. (A)
§ The approach is fully deployed without significant weaknesses or gaps in any areas or
work units. (D)
Figure 5—Scoring Guidelines for the Health Care Criteria
9
SCORE
RESULTS (For Use With Category 7)
0% or 5%
§ There are no organizational performance results or poor results in areas reported.
§ Trend data are either not reported or show mainly adverse trends.
§ Comparative information is not reported.
§ Results are not reported for any areas of importance to your organization’s key mission
or
organizational requirements.
10%, 15%,
20%, or 25%
§ A few organizational performance results are reported; there are some improvements
and/or early good performance levels in a few areas.
§ Little or no trend data are reported or many of the trends shown are adverse.
30%, 35%,
40%, or 45%
§ Improvements and/or good performance levels are reported in many areas addressed in
the Item requirements.
§ Early stages of developing trends are evident.
§ Early stages of obtaining comparative information are evident.
§ Results are reported for many areas of importance to your key mission or organizational
requirements.
50%, 55%,
60%, or 65%
§ Some trends and/or current performance levels—evaluated against relevant comparisons
§ Organizational performance results address most key patient and other customer, market,
§ Improvement trends and/or good performance levels are reported for most areas
addressed in the Item requirements.
§ No pattern of adverse trends and no poor performance levels are evident in areas of
importance to your key mission or organizational requirements.
70%, 75%,
80%, or 85%
§ Current performance levels are good to excellent in most areas of importance to the Item
requirements.
§ Most improvement trends and/or current performance levels have been sustained over
time.
§ Many to most reported trends and/or current performance levels—evaluated against
relevant comparisons and/or benchmarks—show areas of leadership and very good
relative performance.
§ Organizational performance results address most key patient and other customer, market,
process, and action plan requirements.
90%, 95%,
§ Evidence of health care sector and benchmark leadership is demonstrated in many areas.
§ Organizational performance results fully address key patient and other customer, market,
process, and action plan requirements.
§ Current performance levels are excellent in most areas of importance to the Item
requirements.
Figure 5—Scoring Guidelines for the Health Care Criteria (continued)
§ Little or no comparative information is reported.
§ Results are reported for a few areas of importance to your key mission or organizational
requirements.
10
2007 Scoring Band Descriptors
Band Band % Applicants Descriptors
Number in Band1
0–275 1 11% The organization demonstrates the early stages of developing and implementing approaches to Item
requirements, with deployment lagging and inhibiting progress. Improvement efforts focus on problem
solving. A few important results are reported, but they generally lack trend and comparative data.
276–375 2 27% The organization demonstrates effective, systematic approaches responsive to the basic requirements of
the Items, but some areas or work units are in the early stages of deployment. The organization has
developed a general improvement orientation that is forward looking. The organization obtains results
stemming from its approaches, with some improvements and good performance. The use of
comparative and trend data is in the early stages.
376–475 3 31% The organization demonstrates effective, systematic approaches responsive to the basic requirements of
most Items, although there are still areas or work units in the early stages of deployment. Key processes
are beginning to be systematically evaluated and improved. Results address many areas of importance
to the organization’s key requirements, with improvements and/or good performance being achieved.
Comparative and trend data are available for some of these important results areas.
776–875 7 0 The organization demonstrates refined approaches responsive to the multiple requirements of the Items.
It also demonstrates innovation, excellent deployment, and good–to–excellent performance levels in
most areas. Good–to–excellent integration is evident, with organizational analysis, learning, and sharing
of best practices as key management strategies. Industry2 leadership and some benchmark leadership
are demonstrated in results that address most key customer/stakeholder, market, process, and action
plan requirements.
Figure 6—Scoring Band Descriptors