Chapter Nine: Appraisal and Research in the Practice of
Clinical Mental Health Counseling
Chapter Overview:
Appraisal and research are foundational skills for the counseling professions. CACREP
identifies these as two of the eight core areas of study for all entry-level programs. These are
precision and quantify a sample of an individual’s behavior. As such, the results of appraisal can
provide relatively objective data upon which clinical and programmatic decisions can be based.
Clinical mental health counselors must insure that the appraisal techniques used are both valid
and reliable. Without such assurance, risk of harm to clients can incur due to basing treatment
decisions on inaccurate data and misleading conclusions. The importance of test standardization
approaches to sampling, validity, reliability, and constructing operational definitions. Focus,
then, moves to an exploration of specific models of research design. These include: direct
observation; survey and interview; correlation; experiment; within-subject design; qualitative
methods; and meta-analysis. The chapter concludes with a discussion of ethical principles in
research. The role of human subjects review boards and implications of conducting research over
the internet are explored.
Learning Objectives
By the end of this chapter, students should be able to:
• Explain and describe the considerations of ethical practice in research
Key Words/Terms
• Tests- measurement devices used to quantify a sample of an individual’s behavior
• Classical true-score theory– assumes that any observed score (x) on a given test consists
of two components: (a) the true score (T), and (b) the error score €, and is stated by the
measure
• Content validity- the extent to which a test measures the skills or subject matter that it is
supposed to measure
• Table of specification– a two-dimensional chart that assists in the development of the test
by listing the major domains to be covered in the test and how many items should be
and then multiplying by 100
• Zone of proximal development- the range of tasks currently beyond the child’s
capabilities that can be mastered through the verbal guidance and modeling of a more
skilled child or adult
• Personality- the relatively stable and distinctive characteristics of behavior that reflects
resolving problems, and gaining greater understanding into the nature of specific
phenomena and processes
• Scientific method- a research method consisting of six steps: statement of the problem or
research question is constructed, hypothesis formed, plan to answer question is
developed, data collected, data analyzed and interpreted, and conclusions are drawn
• External validity- the extent to which independent researchers using the same methods
can obtain the same or similar results
• External reliability- extent to which independent researchers using the same methods can
obtain the same or similar results
• Operational definitions- used in research and appraisal to safeguard against threats to the
records the ongoing behavior of participants, but does not seek to exert influence on the
situation or manipulate behavior in any way
• Naturalistic observation- data collection occurs in the existing context with no
manipulation of the environment
• Participant-observer research- when the researcher joins and participates in the group
under study
• Effect size- a common statistic shared by each of the compared studies that reflects the
relative strength of the intervention
Lecture Outline for Chapter Nine
Appraisal and Research in the Practice of Clinical Mental Health Counseling
I. Introduction
A. CACREP has assessment and research and program evaluation as two of the eight
core areas of all graduate-level counseling programs
B. Appear on certification and licensure exams
C. Reasons for use:
2.New therapeutic techniques and preventive psychoeducation programs are
discussed in the literature
3.Use skills to wade through accurate research reports
II. Appraisal
A. The Use of Tests
2. Results of specific tests can provide focused, objectively based
information on which conclusions can be made
B. Key Concepts and Principles in Appraisal
1.Classical true-score theory– assumes that nay observed score (X) on a
2. Standard error of measurement– how much an observed score varies from
the true score
3.Reliability– refers to the degrees to which scores obtained on tests are
consistent, dependable, and repeatable
4.Validity– the extent to which an instrument accurately measures what it is
supposed to measure
5. Test Standardization- the process of gathering data on the performance of
a sample of person who are representative of those with whom the test will
eventually be used
C. Categories of Appraisal Techniques
1.Intelligence Tests
a. Intelligence- one’s capacity to learn and one’s ability to reason,
judge, and effectively adapt to the environment
2. Achievement Tests
a. Definition: appraisal instruments that measure a person’s degree of
learning in a specific subject or task
b. Boston Public School system began substituting written for oral
3. Aptitude Tests and Interest inventories
4. Personality Tests
a. Personality- the relatively stable and distinctive characteristics of
behavior that reflects the person’s reactions to the environment and
unique adjustment to life
5.Self-Report Clinical Scales
III. Neuropsychological screening and assessment
A. Clinical mental health counselors expected to having working knowledge of the
1.Makes use of real-time displays of electroencephalography (EEG) to
2.States of neural over-or underarousal can be compared to client’s
concurrent reports of symptoms
3.EEG used alongside other techniques
IV. Ethical Practice in Appraisal
A. Task 1: Test Selection
2. Be sensitive to the specific needs and characteristics of the client
B. Task 2: Test Administration
C. Task 3: Test Interpretation
1. Conclusions never based solely on test results
D. Task 4: Test Reporting
2. Communicate results clearly and directly
E. Test Storage- Confidentiality
V. Research
A. Research– a systematic process of obtaining data for the purpose of answering
questions, resolving problems, and gaining greater understanding into the nature
of specific phenomena and processes
B. Scientific Method
C. Working knowledge of research methods is important
1.Primary means to determine counselor/agency cost effectiveness
3.Necessary to understand and assess empirical foundations of theories and
techniques applied in professional practice
D. Key Concepts and Principles in Research
1.Sampling- the techniques used to gather data from a subset of the entire
population or set of behaviors in such a way that the data gathered are
representative of the problem
2. Validity
a. Internal validity- the extent to which the results of a study can be
3. Reliability
a. Internal reliability- the extent to which methods of data collection,
4. Operational Definitions– used in research and appraisal to safeguard
against threats to the validity and reliability by defining concepts and
variables in terms of the specific operations in which they can be
measured
E. Specific Models of Research Design
1. Research designs
a. Longitudinal design- collection of data over time and at specific
2.Direct Observation
a. Observational research- refers to methods in which the researcher
observes and records the ongoing behavior of participants, but does
3.Survey Methods
4.Correlational Methods
5. Experimental Methods
a. Researcher deliberately manipulates or varies one or more
6. Within-Subject Designs
a. Investigates individual under both experimental and non-
7.Qualitative Methods
a. Use words rather than numbers to describe phenomena
8. Meta-analysis
a. Sophisticated statistical approach used in conducting literature
reviews
b. A way of conducting research about previously conducted research
c. Relies on the concept of error variance
d. Effect size: common statistic shared by each of the studies
VI. Ethical Practice in Research
A. Most basic responsibility is to respect the dignity of participants and promote their
welfare
B. Human Subjects Review Board is mandated to insure the protection of
participants of research conducted at the respective institution
C. Informed consent
D. Increased use of internet for surveys and experiments- requirement of informed
consent
VII. Conclusion
A. Research important to improve intervention efforts
B. Important to comprehend the current, existent literature on empirically validated
therapies
C. Need to evaluate own practice
References/Sources for Chapter Nine
Aiken, L. R. (2003). Psychological testing and assessment (11th ed.). Boston, MA: Allyn and Bacon.
American Counseling Association. (2014). 2014 ACA Code of Ethics. Alexandria, VA: Author.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders – fifth edition.
Arlington, VA: Author.
Ainsworth, M. D., Blehar, M. C., Waters, E., & Wall, S. (1978). Patterns of attachment. Hillsdale, NJ: Erlbaum.
American Mental Health Counselors Association. (2015). AMHCA Code of Ethics. Alexandria, VA: Author.
Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). Upper Saddle River, NJ: Prentice Hall.
Anderson, N., Schueter, J. E., & Carlson, J. F. (2016). Tests in print IX. Lincoln, NE: University of Nebraska Press.
Drummond, R. J., Sheperis, C. J., & Jones, K. D. (2016). Assessment procedures for counselors and helping
professionals (8th ed.). Upper Saddle River, NJ: Pearson Education.
Durlak, J. A., Meerson, I., & Foster, C. J. E. (2003). Meta–analysis. In J. C. Thomas, & M. Hersen (Eds.),
Understanding research in clinical and counseling psychology (pp. 243-267). Mahwah, NJ: Lawrence
Erlbaum Associates.
Field, T. A., Beeson, E., T., & Jomes, L. K. (2015). The new ABCs: A practitioner’s guide to neuroscience-informed
cognitive-behavior therapy. Journal of Mental Health Counseling, 37, 206-220.
Gibson, R. L., & Mitchell, M. H. (2003). Introduction to counseling and guidance (6th ed.). Upper Saddle River,
NJ: Merrill Prentice Hall.
Gottman, J. M., Murray, J. D., Swanson, C. C., Tyson, R., & Swanson, K. R. (2002). The mathematics of marriage:
Dynamic nonlinear models. Cambridge, MA: MIT.
Hadley, R. G., & Mitchell, L. K. (1995). Counseling research and program evaluation. Pacific Grove, CA:
Brooks/Cole.
Hays, D. G., & Singh, A. A (2012). Qualitative inquiry in clinical and educational settings. New York, NY:
Guildford.
Heppner, P. P., Wampold, B. E., & Kivlighan, D. M. (2008). Research design in counseling (3rd ed.). Belmont, CA:
Thomson Brooks/Cole.
Hogan, T. P. (2010). Bare bones R: A brief introductory guide. Thousand Oaks, CA: Sage.
Kaplan, R. M., & Saccuzzo, D. P. (2013). Psychological testing: Principles, applications, and issues (8th ed.).
Belmont, CA: Wadsworth.
Weir, K. (2015, October). Share and share alike. Monitor in psychology, 46(9), 37-39.
Whiston, S. C. (2013). Principles and applications of assessment in counseling (4th ed.). Belmont, CA:
Brooks/Cole.
Wilkinson, G. S., & Robertson, G. J. (2007). Wide Range Achievement Test 4. Lutz, FL: Psychological Assessment
Resources.