DEVELOPMENT OF AN ASSESSMENT PLAN
I.
Introduction
A.
“Nine
Principles of Good Practice for Assessing Student Learning” (Astin, A, Banta, T., Cross, P., El-Khawas,
E., Ewell, P., Hutchings, P., Marchese,
T., McClenney, K., Mentkowski,
M., Miller, M., Moran, E., and Wright, B. (2003), American Association for Higher
Education Assessment Forum, AAHE,
1.
The assessment of student learning begins with
educational values.
a.
assessment is a vehicle
for educational improvement.
b.
assessment is driven
by what we most value for students to learn and gain from their experience with
us.
2.
Assessment is most effective when it reflects an
understanding of learning as multidimensional, integrated, and revealed in
performance over time.
a.
learning entails what students know and what they
can do with what they know
b.
use of diverse methods for assessment
c.
use of measurements over time to reveal change and
growth
3.
Assessment works best when the programs it seeks to
improve have clear, explicitly stated purposes.
a.
assessment is a
goal-oriented process.
b.
clear, shared goals and goals that can be
implemented are the cornerstone of assessment that is focused and useful
4.
Assessment requires attention to outcomes but also
and equally to the experiences that lead to those outcomes.
a.
we need to know where students end up, but also how
they develop along the way
b.
assessment helps us see how students learn best
5.
Assessment works best when it is ongoing, not
episodic.
a.
assessment is a process whose power is cumulative
b.
monitor progress toward intended goals in a spirit
of continuous improvement
6.
Assessment fosters wider improvement when
representatives from across the educational community are involved.
7.
Assessment makes a difference when it begins with
issues of use and illuminates questions that people really care about.
a.
assessment results in
evidence that is relevant and people will find the results to be credible,
suggestive, and applicable to decisions that need to be made.
b.
it is a
process that starts with the questions of decision-makers, that involves them
in the gathering and interpreting of data, and that informs and helps guide
continuous improvement.
8.
Assessment is most likely to lead to improvement
when it is part of a larger set of conditions that promote change.
a.
when the campus values continuous improvement,
assessment results will be sought out
9.
Through assessment, educators meet responsibilities
to students and to the public.
a.
we have an obligation to the public to improve
b.
the public has an obligation to support our efforts
to improve
B.
Elements of
Good Assessment
1.
Asks important questions
2.
Reflects the departmental/institutional mission
3.
Reflects identified learning outcomes
4.
Is linked to a plan for decision-making
5.
Encourages involvement
6.
Contains relevant assessment techniques
7.
Is shareable and leads to reflection
C.
Assessment’s
Relation to A Program’s Outcomes: (Bresciani, M.J., NetResults,
1.
What are we trying to do and why are we doing it?
2.
What do we want the student to learn or know as a
result of our program or interaction with our department?
3.
How well are we doing?
4.
How do we know?
5.
How do we use this information to improve?
6.
Does it work?
D.
Potential
Benefits of Assessment: (Bresciani, M.J., NetResults,
1.
reinforce or emphasize the mission of the unit or
department
2.
modify, shape, and improve programs and/or
performance (formative assessment)
3.
critique a program’s quality or value compared to
the program’s previously defined principles (summative assessment)
4.
inform planning
5.
inform decision-making
6.
evaluate programs, not personnel
7.
support the request for additional funds from the
university and external community, and
8.
assist in meeting
accreditation requirements, models of best practices, and national benchmarks.
E.
Beginning
the Discussion
1.
What role do you think assessment has in our
organization?
2.
Describe what you think assessment should
accomplish in our organization.
3.
What role do you think you would like to have in
providing assessment in our organization?
4.
Do you have any concerns or questions about the
assessment process that should be addressed as we begin to develop a strategic
plan for assessment?
5.
What support structures do you feel you would need
to actively participate in conducting assessment projects?
II.
Developing
a Strategic Plan for Assessment
A.
Assessment
begins with knowing what you want
B.
Define your
assessment vision: what do you hope to accomplish?
1.
Where are we going?
2.
What have we accomplished so far?
3.
Define the vision in a short statement that
inspires and motivates others
4. Have a wide-ranging discussion that includes how this assessment vision fits into the larger vision of Student Affairs and the University
C.
Consult
with the Institutional Review Board (IRB) of the University
1. Dr. Sandra Holmes of the Psychology Department is chair of the IRB
2. She will provide guidelines for ensuring protection for participants in research
D.
Review the
type of resources that exist
E.
Decide on a
guiding model, such as the Context, Input, Process, and Product model (The CIPP
Model of Daniel Stufflebeam) of
planning and evaluation (From: Stufflebeam, D. 2001, Evaluation
Models; and Rodgers, R., 1979, “A Student Affairs Application of the CIPP
Evaluation Model”, in Kuh, G. Evaluation in
Student Affairs, 1979).
1.
Examples of other evaluation models
a.
objectives based studies
b.
objective testing programs
c.
outcome evaluation as value-added assessment
d.
performance testing
e.
experimental studies
f.
management information systems
g.
benefit-cost analysis approach
h.
case study evaluations
i.
accreditation/certification approach
F.
Operationalize your plan: assign
responsibilities and roles
III.
Initiating
the Process (Upcraft
and Schuh, 1996, Assessment in Student Affairs)
A.
Do not do a
study that no one wants
B.
Determine
confidentiality
C.
Determine
who should be involved
D.
Conduct a
sound study
E.
Understand
what kinds of information you will need to provide in your report.
IV.
Defining
Your Assessment Vision
A.
Determine
who will be the audience for the results
B.
Determine
the appropriate format(s) of the study
C.
Involve the
key stakeholders in the planning of the study and allow them to review the
instrument drafts
D.
Example of
an assessment vision from
V.
Identifying
Resources
A.
Is money
available for your assessment needs?
B.
Who is
available to analyze your data?
C.
Do you have
the cooperation of department and other key stakeholders?
D.
Do you have
people to assist with data collection?
E.
Can you
offer incentives?
F.
Is your
organization supportive of your efforts?
VI.
Developing
an Assessment Model Tied To Your Planning Model
A.
Describe
your departmental mission, vision, priorities, and values
1.
a.
needs to be concise
b.
what does the department stand for?
c.
what does the department do for our students?
d.
what impact do you have on your students vs. what
programs do you offer
2.
Vision (Example: (“Changing Lives at UWSP”)
a.
compelling
b.
inspirational
c.
calls people to change
d.
offers a “sound bite”
3.
Values
a. what
does your department value?
4.
Priorities
a.
based on your department’s vision, mission, and
values, what are your priorities?
b.
Are these priorities connected to budget decisions?
5.
Goals: general description of ultimate intended
benefits or results or desired outcomes for the program (what core issues and
values does the program address?)
6.
Objectives: measurable statements about specific
intended outcomes that a particular program
activity or service is expected to accomplish in a given time period.
a.
has a target group (who)
b.
has what is to be done (program)
c.
has a time frame (when)
d.
has a target performance (how much)
e.
has a measurement (how it will be measured)
7.
Objectives that are useful should:
a.
tell who
b.
is going to be doing what
c.
when
d.
how much, and
e.
how we will measure it
8.
Checklist for Evaluating Written Objectives (
a.
uses action verbs that specify definite, observable
behaviors
b.
uses simple language
c.
describes student rather than staff behavior
d.
describes a learning outcome rather than a learning
process
e.
indicates a single outcome per objective
f.
can be assessed by one or more indicators (methods)
g.
is clearly linked to a goal
h.
is realistic and attainable
i.
is not simple when complexity is needed
j.
is clear to people outside the department
k.
is validated by departmental colleagues
B.
Example of
A Goal and Objective (Anti-Smoking Program) (From: Grayson, T., “Constructing
Logic Models”, 2000,
1.
Goal: The life-expectancy of all Americans will
increase to 76 years of age by the year 2020 without creating any economic
downturns in the Nation’s economy.
2.
Objective: To reduce the number of all teenage
smokers by implementing a Nationally funded anti-smoking initiative starting in
the year 2000 and continuing until the year 2020, by 98%, as measured by a
stratified random sampling of teenagers each year, beginning in the year 2000.
C.
Example of
Goal and Objective (Get Ready Program) (From: Grayson, T., “Constructing Logic Models”, 2000,
1.
Goal: Ensure that all individuals with disabilities
acquire self-determination skills necessary for gainful employment or
post-secondary schooling after graduation from high school.
2.
Objective: To increase the number of high school
graduates with disabilities securing gainful employment or entering
post-secondary schools within 6 months after graduation by developing and
implementing the Get Ready Program in all secondary schools in
D.
Evolution
of a Good Objective (From: Grayson, T., “Constructing Logic Models”, 2000,
1.
Stage 1: To increase the reading skills of at risk
students (customer and expected result)
2.
Stage 2: To increase the reading skills of at risk
students ages 14 to 18 (specific target)
3.
Stage 3: To increase the reading skills of at risk
students, ages 14 to 18, through tutoring (the program)
4.
Stage 4: To increase the reading skills of at risk
students, ages 14 to 18, through tutoring, as measured by the performance on
the school district’s reading comprehension test (means of measuring results)
5.
Stage 5: to increase the reading skills of at risk
students, ages 14 to 18, through tutoring, as measured by performance on the
school district’s reading comprehension test to be administered before and
after the program (when the results are expected)
6.
Stage 6: To increase the reading skill of at risk
students, ages 14 to 18, through tutoring, and as measured by an average
increase of five percent on the school district’s reading comprehension test to
be administered before and after the program (the standard of success)
7.
Stage 7: To increase the reading skills of 25 at
risk students, ages 14 to 18, through tutoring, and as measured by an average
increase of five percent on the school district’s reading comprehension test to
be administered before and after the program (the number of program recipients)
E.
Describe
your strategic objectives
1.
Learning outcomes
2.
Service outcomes
3.
Program outcomes
4.
Behavioral indicators
5.
Key performance indicators
F.
Describe
the activity or
program that is designed to impact students
1.
A program is an intentional use of resources to
support specific strategies or activities to produce defined results to address
strategic problems to achieve the department’s mission
2.
Activities such as training, outreach, maintenance,
and management are major strategies.
3.
What is to be done to achieve our intended goals
and mission?
4.
Direct products of the program activities could
include number of individuals served or the number of training sessions served
5.
How much do you do and for how many individuals?
G.
Describe
how strategic objectives will be measured
H.
Describe
how reporting of results and feedback from stakeholders will occur
I.
Objectives
Vs. Outcomes
1.
Objectives: intended results or consequences of
instruction, curricula, programs, or activities
a.
they specify what is expected and describe what
should be assessed
2.
Outcomes: achieved results or consequences of what
was learned- evidence that some learning took place
a.
outcomes are behaviors and products generated by
students after the program is delivered and are the object of the assessment.
b.
outcomes should be realistic, achievable, and
directly related to the activities of the program
c.
outcomes can be immediate, short-term, or
long-term.
J.
Types of
Outcomes
1.
Program Outcomes
a.
illustrate what you want your program to do
b.
present measurable and meaningful statements of
what you want the program to accomplish
2.
Service Outcomes
a.
related to program outcomes
b.
Example: Financial Aid- develop a financial aid
package that ensures that students have enough money to enroll and stay
enrolled in college
c.
Example:
3.
Learning Outcomes
a.
illustrate the learning that you want to occur
b.
assess cognitive abilities, such as critical
thinking skills
4.
Developmental Outcomes
a.
assess affective dimensions or attitudes
b.
examples would include being sensitive to the
values of others, becoming aware of one’s own talents and abilities, and
developing an appreciation for life-long learning
K.
Creation of
Learning Outcomes
1.
Learning:
a.
is affected by the educational climate
b.
is an active search for meaning
c.
is developmental- involving the whole person
d.
is done by individuals who are tied to others as
social beings
e.
is enhanced by taking place in the context of a
compelling situation
f.
is fundamentally about making and maintaining
connections
g.
is grounded in particular contexts and individual
experiences