"Reflective Professionals Building Learning Communities"

COURSE TITLE

COUN 404 Evaluation Techniques

INSTRUCTOR INFORMATION

Name: Ken Currier, Ph.D.
Office: CLS 4097
Phone: Office: x5576; Home: 773.588.1829; Cell: 773.213.4114
E-mail: Office: K-Currier@neiu.edu ; Home: ken.currier@comcast.net (Preferred)

COURSE DESCRIPTION

This course focuses on principles of test construction, basic statistical concepts, study of commonly used standardized tests, and other evaluative procedures. Interpretation of test data is an integral part of this course. This course contributes to the Assessment component of the program and is required of all students.

PREREQUISITE: Graduate standing and admission to the NEIU Counseling Program.

REQUIRED TEXTS and READINGS

Drummond, R.J. (2004). Appraisal procedures for counselors and helping professionals (5th ed.). Upper Saddle River, NJ: Prentice-Hall.

Counseling Dictionary (included with the purchase of the text)

Goldman, L. (1992). Qualitative assessment: An approach for counselors. Journal of Counseling & Development, 70, 616-621.

Ovid. (January 2005). Mental measuremests yearbook, (Vols. 9-16). Available online through the Ronald Williams Library under “Databases” at: http://www.neiu.edu/~neiulib/ (or, go to: http://www.neiu.edu and click on “Information Resources”(

REQUIRED ASSESSMENT INSTRUMENT PACKET

COPSystem Career Measurement – Self-scoring: A packet of three assessment instruments and a self-interpretation career profile folder. Available through Beck’s Bookstore in the Student Union (approximately $8.00). This includes an interest survey, an ability profile, work values inventory, and a career profile workbook

REQUIRED SUPPLIES

Calculator: Bring your calculator to class every meeting from class week 2-5. Your calculator must be able to compute the square of a number and the square root. Usually this is termed a “scientific calculator”. You may purchase one at many outlets, including the bookstore and Office Max, Wal-Mart, Staples, etc. It should not cost more than about $15 unless you choose to buy the Cadillac of calculators. Further, it is recommended that you keep a three-ring notebook to gather hand-outs, sample tests, and other materials that may be useful as an assessment handbook for later use.

TECHNOLOGY

This is a Web-enhanced assisted course. Students must have a working email account and access to the World Wide Web (Internet). See instructor if this is a challenge. The instructor uses Microsoft Office 2000 or XP products for word processing and presentations (MS WORD and PowerPoint).

Blackboard is a web-based instructional software that allows instructors to teach online or to supplement site-based instruction. Course notes, scanned articles, handouts, and other printed materials will be posted on the Blackboard site for COUN 404-Evaluation Techniques in Counseling. The instructor will attempt to post weekly class notes (outline version of PowerPoint presentations) prior to class meetings so that students may print off class notes prior to attending the class. This is NOT met to replace class attendance and is offered as a supplement to facilitate processing of information. Missing three or more class meetings is problematic and may have significant adverse effects on the student’s final grade per university policy.

Access to Blackboard:

Go Online using an Internet browser such as Internet Explorer
Go to: http://www.neiu.edu
Click on: Information Resources link
Click on: Northeastern’s Blackboard Bb

From NEIU’s Blackboard Bb. Click on: Create an Account Fill in all appropriate boxes. You have control of your username and password. A suggestion: keep username and password consistent with other university information resources or internet sites – it is much easier to keep track and remember.
Exit or return to Internet Explorer or other browser
Return to Blackboard window (Steps 1-4)

From NEIU’s Blackboard Bb main page:
Click on: Course Catalog
Click on: College of Education
Click on: Counselor Education
Click on: Evaluation Techniques
Click on: Enroll (icon at far right)
Fill in your username and password that you established above

TO ACCESS BLACKBOARD REGULARLY AND OBTAIN COURSE DOCUMENTS, DISCUSSION BOARD, ANNOUNCEMENTS, EXTERNAL LINKS, ETC.:
Enter course Blackboard website
From course website: Click on: Course Documents
Here you will find the posted documents for the course. By double clicking on the item, your computer should automatically open up the document in Word or other appropriate format. Note: some items may be posted in .pdf format (portable document format). Adobe Acrobat software is used to open these type of documents. You may obtain Adobe Acrobat Reader software free online at http://www.adobe.com . This free software enables you to only read and print .pdf (portable document format) documents, NOT create them (you need additional software for this that usually has a cost). Adobe Acrobat Reader has its own toolbar from which to print, save, etc. These tool icons are located below those of your main computer or browser toolbar. You may also go to the course Blackboard site and click on Announcements then click on the “View All” tab to access Adobe Reader link and instructions to download such. Other elements of the Blackboard site may be utilized such as Announcements but your main resource will be Course Documents.

MODE OF INSTRUCTION

Lecture/discussion supplemented with individual exploration

COURSE OBJECTIVES

The objectives of the course support the College of Education Performance Standards #4 Learning Styles, #9 Assessment, and #13 Diversity. (See Appendix A for table of standards.) In addition, these objectives are designed to help students to

1. Understand the field of educational and psychological appraisal, including the theoretical bases for appraisal techniques and methods of interpreting appraisal data;

2. Distinguish among the methods of establishing validity, including content, construct, and criterion-related validity;

3. Understand the methods of establishing reliability, including equivalence and internal reliability;

4. Identify major categories of test instruments, including intelligence tests, achievement tests, aptitude tests, personality tests, career inventories, sociometric measures, behavioral observations, and computer related methods;

5. Master basic descriptive statistics, including types of standard scores, measures of central tendency, indices of variability, standard error, and correlation;

6. Know the principles for selecting, administering, and interpreting appraisal data in counseling, including educational, psychological, and career counseling;

7. Understand ethical and legal issues in the use of appraisal data in counseling;

8. Consider the use of tests with minorities, persons with disabilities, and other special populations; and

9. Enable the student to evaluate selected test instruments that are appropriate to his/her counseling area.

REQUIRED PROJECTS

Students are required to evaluate two test instruments that are appropriate to their work as counselors and present their findings in a written report called a Test Critique (see p. 12 of syllabus). This project satisfies the performance-based assessment component of the course. A form to guide your written report will be provided. The textbook also provides direction on selecting tests or assessment instruments. APA style and format is required for these and all written reports in this course and within the counseling profession. See the Blackboard site for a Writing and Documentation Guide on writing in APA style/format. Students are required to practice test administration, scoring, interpretation, and reporting through a self-assessment or pseudo-client assessment using the COPSystem Career Measurement battery, interpreting and reporting the results through the submission of a 1-2 page summary psychological report. Guidelines and instruction will be provided. See Blackboard site for a PowerPoint presentation on Writing and Documentation, a PowerPoint presentation on behavioral-oriented report writing, and Chapter 17 in the textbook.

STUDENT EVALUATION

Each student will be required to complete a midterm and a final examination covering the material in the text, other assigned readings, and the lecture material presented in class. Grades will be assigned on the basis of the student's performance on the exams and the written test report.

GRADING POLICY

Midterm exam 100 points
Final exam (take home) 100 points
Two (2) instrument reviews @50pts each 100 points
COPSystem administration, scoring, interpretation report 100 points
Blackboard Discussion Board/ activities @ 10 pts each 100 points
Total 400 points

A = 90-99% of Total Points D = 60-69% of Total Points
B = 80-89% of Total Points F = below 60% of Total Points
C = 70-79% of Total Points

ASSESSMENT OF LEARNING (FEEDBACK LOOP)

Assessment of learning begins in the initial class session and is on
going throughout the course. Data from the specific assessment tools (midterm exam, final exam, test critiques, and written test report) will be analyzed and the results used to improve instruction and facilitate increased student learning. This feedback may be used to make adjustments in the course as it progresses and to implement future changes.

STUDENT ACCOMMODATIONS

In accordance with the Americans with Disabilities Act of 1990, Northeastern Illinois University does not discriminate against employees or students on the basis of disability. In addition, the University provides reasonable accommodations for both employees and students with disabilities. Students seeking reasonable accommodations in the classroom should contact the Accessibility Center/HELP Office, A-118 (phone 773/442-5495, 5496, or 5497; TDD 773/442-5499).

The University policy on services for students with disabilities may be found at
http://www.neiu.edu/%7EDeanSt/survival/disabilities.pdf . Students should feel free to meet with the instructor at any time to discuss any reasonable accommodations or removal of barriers that may hinder full inclusion and participation in the course.

STUDENT RESPONSIBILITIES

Students are expected to attend all class sessions, read assigned materials in advance, participate in class and/or Blackboard discussions, submit a written test report, and take midterm and final examinations. Missing 3 or more class meetings may have a significant effect on the final course grade.

COURSE TOPICS AND SCHEDULE

Week
Date
Topic
Assignment Due
Activity on Bb Discussion Board

Week 1
May 25
Introduction, Overview
Overview, syllabus
Bb Exercises

Week 2
June 1
Historical, Philosophical Foundations
Legal and Ethical Issues
Anxiety Survey
Chapter 1, 2, 3
Bb Exercises

Week 3
June 8
Statistical Concepts
Measurement concepts
The “normal curve”
Chapters 3, 4, 5
Bb Exercises/Practice

Week 4
June 15
Qualitative Assessment
Online article: Goldman article
Bb Exercises

Week 5
June 22
Ability & Intelligence, Aptitude & Achievement
Chapter 6, 7, 8, 17
Test Critique #1 Due
Bb Exercises

Week 6
June 29
Midterm examination
Chapter 1-8
Bb Exercises

Week 7
July 6
Ability & Intelligence, Aptitude & Achievement
Career, Employment
Test interpretation
Chapters 6, 8, 9
COPS test due – bring to class
Bb Exercises

Week 8
July 13
Personality, Clinical
10, 11
COPS Report Due

Week 9
July 20
Assessment of Development.
Environmental Assessment.
12, 13
Test Critique #2 Due
Bb Exercises

Week 10
July 27
Computer Assisted Testing
Diversity
14, 15, 16
Final Exam available

Week 11
August 3: Final Exam Due - Chapters 9-17


References
Aiken, L. R. (1996). Rating scales and checklists: Evaluating behavior, personality, and attitudes. New York: John Wiley.
Aiken, L.R. (1998). Tests and examinations: Measuring abilities and performance. New York: J. Wiley.
Allen, M. J. & Yen, W. M. (2001). Introduction to measurement theory. Waveland Press.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. [Highly recommended that students and future counselors purchase this!]
Anastasi, A. (1988). Psychological testing (6th ed.). New York: Macmillan.
Anastasi, A. (1992). What counselors should know about the use and interpretation of psychological tests. Journal of Counseling & Development, 70, 610-612.
Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). New York: Macmillan.
Berven, N. (1984). Assessment practices in rehabilitation counseling. Journal of Applied Rehabilitation Counseling, 15, (3), 9-14, 47.
Cohen, R. J. & Swerdlik, M. E. (2001). Psychology and assessment: An introduction to tests and measurement (5th Ed.). New York: McGraw-Hill.
Crocker, L. & Algina, J. (1986). Introduction to classical and modern test theory. Belmont, CA: Wadsworth.
Cronbach, L.J. (1990). Essentials of psychological testing (5th ed.). New York: Harper Collins
Dana, R.H. (1993). Multicultural assessment perspectives for professional psychology. Boston: Allyn and Bacon.
Fink, A. (1995). Evaluation for education and psychology. Thousand Oaks, CA: Sage.
Goldman, L. (1992). Qualitative assessment: An approach for counselors. Journal of Counseling & Development, 70, 616-621.
Groth-Marnat, G. (2003). Handbook of psychological assessment (4th ed.). New York: John Wiley.
Hood, A.B., & Johnson, R.W. (2002). Assessment in counseling: A guide to the use of psychological assessment procedures. Washington, DC: American Psychological Association.
Hopkins, K.D. & Stanley, J.C. (1998). Educational and psychological measurement and evaluation (8th ed.). Englewood Cliffs, NJ: Prentice-Hall.
Kaplan, R. & Saccuzzo, D. (2000). Psychological tests: Principles, applications, and issues (5th ed.). Belmont, CA: Wadsworth.
Keyser, D.J. & Sweetland, R.C. (Eds.). (1984-current). Test critiques (vols. I-VI). Kansas City, MO: Test Corporation of America.
Keyser, D.J., & Sweetland, R.C. (Eds.). (1984-1994). Test critiques. Kansas City, MO: Test Corporation of America.
Kline, P. (1998). The new psychometrics: Science, psychology, and measurement. New York: Routledge.
Kubiszyn, T. & Borich, G. D. (2002). Educational testing and measurement (7th ed..). New York: John Wiley.
Loewenthal, K. M. (2001). An introduction to psychological tests and scales (2nd ed.). Psychology Press.
Meier, S. T. (2002). Bridging case conceptualization, assessment, and intervention. Thousand Oaks, CA: Sage Publications.
Murphy, L.L., Conoley, J.C., & Impara, J.C. (Eds.). (1994). Tests in print IV: An index to test, test reviews, and the literature on specific tests. Lincoln, NE: Buros Institute of Mental Measurements, University of Nebraska-Lincoln, University of Nebraska Press.
Murphy, L.L., Impara, J.C., & Plake, B.S. (Eds.). (1999). Tests in print V: An index to tests, test reviews, and the literature on specific tests. Lincoln, NE: Buros Institute of Mental Measurements, University of Nebraska-Lincoln, University of Nebraska Press.
Special Issue. (November 1998). Multicultural assessment. Counseling Psychologist, 26(6).
Sprinthall, R.C. (2002). Basic statistical analysis (7th ed.). Needham Heights, MA: Allyn & Bacon.
Stiggins, R.J. (2002). Assessment Crisis: The Absence Of Assessment FOR Learning [Electronic version]. Phi Delta Kappan, 83(10), 758-765. Available online at: http://www.pdkintl.org/kappan/k0206sti.htm
Suzuki, L.A., Ponterotto, J.G., & Meller, P.J. (Eds.). (2001). Handbook of multicultural assessment: Clinical, psychological, and educational applications. San Francisco: Jossey-Bass.
Urbina, S. (2004). Essentials of psychological testing. Indianapolis: Wiley.
Walsh, W. B. & Betz, N. E. (2000). Tests and assessments (4th ed.). Englewood Cliffs, NJ: Prentice-Hall.

Appendix A: Table of Standards


The course objectives for Evaluation Techniques are linked to the following standards. (COE = College of Education; CACREP = Council for Accreditation of Counseling and Related Educational
Programs; CORE = Council on Rehabilitation Education.)

COE Performance Standards #4 Learning Styles, #9 Assessment, and #13 Diversity
CACREP II.K.7 = Assessment
CORE C.7 = Assessment

1. Understand the field of educational and psychological assessment, including the theoretical bases for appraisal techniques and methods of interpreting appraisal data;

COE Performance Standard #9
CACREP II.K.7.a
CORE C.7.3

2. Distinguish among the methods of establishing validity, including content, construct, and criterion-related validity;

COE Performance Standard #9
CACREP II.K.7.e
CORE C.7.3

3. Understand the methods of establishing reliability, including equivalence and internal reliability;

COE Performance Standard #9
CACREP II.K.7.d
CORE C.7.3

4. Identify major categories of test instruments, including intelligence tests, achievement tests, aptitude tests, personality tests, career inventories, sociometric measures, behavioral observations, and computer
related methods;

COE Performance Standard #9
CACREP II.K.7.b and II.K.7.g
CORE C.7.6 and 7.9

5. Master basic descriptive statistics, including types of standard scores, measures of central tendency, indices of variability, standard error, and correlation;

COE Performance Standard #9
CACREP II.K.7.c
CORE C.7.3

6. Know the principles for selecting, administering, and interpreting appraisal data in counseling, including educational, psychological, and career counseling;

COE Performance Standard #9
CACREP II.K.7.g
CORE C.7.10 and c.7.11

7. Understand ethical and legal issues in the use of appraisal data in counseling;

COE Performance Standard #4
CACREP II.K.7.i
CORE C.7.5

8. Consider the use of tests with minorities, persons with disabilities, and other special populations; and

COE Performance Standard #13
CACREP II.K.7.f
CORE C.7.3 and C.7.4

9. Enable the student to evaluate selected test instruments that are appropriate to his/her counseling area.

COE Performance Standard #9
CACREP II.K.7.
CORE C.7.12



Criteria for Reviewing and Evaluating Assessment Measures (Tests)

See page 90-93 in the text. Note: Not all criteria are appropriate to a specific test.
For a list of possible tests to review see textbook under specific domain chapters or appendices A & B pp. 415-430. Two references are required: 1) MMY, 2) an article or book chapter that refers to the instrument in some manner such as its use in a study, critique of properties, or as a counseling methodology.

• Purpose (what do the results yield?)”
• Age group:
• Group or Individual? Or both?
• Timed or not timed?
• Cost:
• Qualifications for administration/scoring/interpretation:
• Uses and population for which the test is intended.
• For what uses are the norming, sample size and test validation appropriate? For example, is there sufficient sample size in the norming to be generalized to the Latino/Latina client?
• Reliability: To what degree can you be confident of the results? Is it dependable? The Standard Error of Measurement (SEM) is a function of the reliability.
• Predictive validity: To what degree does the measure predict performance? Applicable more to ability tests.
• Construct validity: To what degree does the test measure what you intend for it to measure? For example, is it measuring anxiety, depression, confidence, self-efficacy or some indirect method of determining this? How is it correlated with criterion measures?
• Administration: Are the directions clear and detailed? To what degree is standardization important? Is behavioral observation and documentation important?
• Reporting: How are the scores and interpretation reported? e.g. Raw scores, percentiles, grade equivalents, stanines?
• Test bias? e.g. gender, race, ethnicity, cultural, linguistics (language), geography, socio-economic status or ‘class’, educational level. Is this test available in alternative languages? If a standardized test, is the norming sample representative of the individual or group that you are appraising?
• Necessary characteristics of the examinee: reading level, culture, language, mental health (reality testing)
• Qualifications necessary and appropriate for the examiner? May a technician administer the test and then a trained psychologist interprets results? Many test publishers require that the purchaser pre-qualify by educational, licensure/certification, and/or training level.