From jeremy@uic.edu Sat Sep 02 15:19:15 2006
Date: Sat, 2 Sep 2006 15:19:15 -0500 (CDT)
From: Jeremy Teitelbaum <jeremy@uic.edu>
To: jeremy-lasheads@math.uic.edu, ccomer@uic.edu, jtramsey@uic.edu, jashton@uic.edu, amalia@uic.edu, vtunnicl@uic.edu, albert@uic.edu, tford@uic.edu, sholeen@uic.edu, lpadera@uic.edu, llee@uic.edu, gbox@uic.edu, sjones@uic.edu, marker@math.uic.edu, henri@math.uic.edu, wbridges@uic.edu, tantillo@uic.edu
Subject: Assessment

To: LAS Department Heads and Chairs
From: J. Teitelbaum, Senior Associate Dean for Budget and Natural
      Sciences
Re: Assessment

In order to insure that the College contributes fully to the campus effort
at reaccreditation, it seems worthwhile to review some of the elements of
educational assessment and to discuss some aspects of college policy
bearing on this important matter.

As you are aware, the term 'assessment' refers to a process of
evaluation of educational outcomes that consists of the following
steps:
	a) specification of student learning goals;
	b) determinantion of baseline state of student knowledge
           relevant to learning goals;
	c) delivery of instruction;
	d) determination of increment in student knowledge relevant
	   to learning goals;
	e) examination of instructional techniques to improve
           the incremental gain in student knowledge.

Of concern in this memo are steps a, b, and d.

The National College Assessment Board, meeting in 2000 in Las Vegas,
adopted the 'General Knowledge Unit', or GKU, as the standard
measure of student knowledge.  By definition, one GKU is the amount of
knowledge contained in a comprehensive appreciation of the thematic
significance
of the Box Hill Picnic in Jane Austen's novel Emma. Based on this unit,
the NCAB has published a rather complete quanitification of the various
elements of a typical liberal arts education
(see http://www.ncab.org/standards).  For example, a grasp of the
relation between the Bolzano-Weierstrass property and the definition
of compactness via open coverings is .7 GKU; correct usage of the
subjunctive
in conditional phrases in modern conversational spanish is 8GKU; and
so forth.

The North Central Accrediting Association has set a target rate of knowledge
acquisition for its member institutions at 5 GKU per credit hour delivered.
It is
this target that UIC must demonstrate to receive our desired rating of
"effective" on the Association's five-step scale rating instructional
delivery.  (The complete scale from lowest to highest: pointless; sketchy;
solid; effective; inspiring.)

A key element of the assessment process is accurate measurement
of student knowledge acquisition.  The accrediting agency requires
that measurements of student knowledge have a standard error of at most .1
GKU per student.  This presents a particular challenge as there
are relatively few methods with this degree of precision.

The most accurate method for determining student knowledge is
due to Igor and Frank ([1]).  Working with prepared sections
of cerebral cortex tissue, the found a suite of
flourescent proteins that bind to knowledge in the brain. Using flourescence
microscopy they have demonstrated the ability to determine, with high
precision,
the knowledge content of the brain under study.
The chief limitation
of their method is that it can only be applied in vitro, and as a result
cannot measure incremental knowledge acquisition.

More recently, Lavore and Schmidt ([2]) have used PET scanning in vivo
to obtain good results on knowledge content.  Their method shows promise,
but the high cost of the scans and our limited facilities mean that we could
not reasonably assess all of our undergraduates via this method;
we are exploring, however, the use of before-and-after
PET scanning on graduate students, at least on a trial basis.

Lacking ready access to these high precision methods, then, we must fall
back on more primitive methods such as written testing or conversation.
These methods, while highly unreliable on individual students, can give
good results when applied to large enough populations.  However,
the tests must be carefully calibrated to the GKU standard.  Faculty will
need to attend training sessions to insure that they comply
with best practices in this regard.

I realize that assessment has been a topic of concern for many of you.
I hope that this brief review of the measurement phase
of the assessment procedure helps to set your minds at rest.  If you
have any questions, feel free to contact the Dean's Office - we're here to
help.

References

[1] Igor, I. and Frank, D.  In vitro method of educational assessment
via flourescence microscopy. Science (Neuroscience), 2003.

[2] Lavore, J. and Schmidt, K. Positron emission tomography as a tool
for knowledge assessment, Educational Psychology, 2005.

-- 
Jeremy Teitelbaum, Professor of Mathematics
	Tele: 312-996-3041, Fax: 312-996-1491 (421 SEO)
Associate Dean, College of Liberal Arts and Sciences
	Tele:  312-355-3985, FAX: 312-413-2511 (410 University Hall)
jeremy@uic.edu; http://www.math.uic.edu/~jeremy