Where criterion-referenced assessment is used, pre- and post-measurement of the skills that are critical to optimal performance is the only concrete way of assessing whether instructional or training programs are making a difference. It is also important for learners to be able to track their own progress. In both program evaluation and learner feedback, the difficulty is always in finding a systematic way to examine key indicators that are valued in optimal performance.
Curriculum-Based Measurement (CBM). Because norm-referenced tests, and sometimes criterion-referenced measures, are intended for broad use, there is still considerable concern over the mismatch between content of such tests and the content of training or instructional programs. It is important to note, however, that standardized tests make no pretense of fitting training or instructional curricula precisely. What kinds of assessment might have a better fit in this respect?
Curriculum-based measurements (CBMs) are repeated measurements of an adult learner's performance on a single global task across time. They offer multiple assessments of adult learner progress toward a long-term course goal specific to mastering a task. CBMs use the specific curriculum provided in a training or instructional setting as the basis for tracking the adult learner's performance on specific desired outcomes. For example, at the beginning of a train-the-trainer program, the adult learner is given a set of questions pertaining to the personal attributes of a good trainer. Using the provided CBM rating scale, they determine their personal understanding of principles critical to the train-the-trainer learning process. At the end of the train-the-trainer course, the adult learner again is assessed using this same CBM. Of critical importance is how well the rating scale tracks post program changes in the degree of individual understanding of train-the-trainer "best training practices."
It is important in CBMs that learners review their performance before each assessment. The goal line, for example, helps learners to monitor their own progress. In addition, progress toward the estimated goal helps instructors make decisions about modifying instruction, changing the level of reading passages, or modifying the goal line.
A CBM that could be applied to a management development program would be to monitor and assess effective communication within a work team by using a two-minute reading/response passage for each participant. This reading passage might be used on a weekly basis and the accuracy of responses calculated.
Participants' progress (accuracy of responses) is graphed. An optimal performance, or goal line, might be determined by using the average of participants' performances on two baseline scenarios and adding a percentage of accurate responses to the score as a target for improvement. It is again important in CBMs that learners review their performance before each assessment to monitor their own progress.
Several factors must be taken into consideration when designing CBMs:
- Select tasks that will represent those the adult learner is expected to perform in class/training sessions. Such tasks can be taken directly from sample curriculum materials being used.
- Remember that CBMs are timed. Keep the measurement short, for example, one to five minutes. The purpose of the CBM is to track learner performance in multiple tasks over time.
- CBMs are most useful when they are given frequently, for example, every other day, or twice a week, depending upon the length of the class/training session. Useful measurement occurs when the measurement task looks and functions like any other class/training session task. In other words learners should not feel like they are taking a test.
- CBMs can be normed. Give the same measurement to a sample of five to ten learners identified by the supervisor/ employer as average peers of the learner. Or, if the trainer/ employer wants to compare the learner's performance against optimally desired performance, the five to ten learners might be selected from among top-rated on-the-job per-formers. The performance of this sample of five to ten average or optimal learners sets the baseline for comparison of the individual learner's performance against the norm. 5. The most effective way to represent learner growth is graphically. Norm performance, or goal-related performance, and the learner's actual performance can be charted on a line graph.
Better assessments are built upon three key assumptions. First, assessment is seen as integral to learning. The one-shot notion of assessment celebrated in standardized tests does not fit an emerging developmental and constructivist view of knowledge and understanding. The major purpose of assessment should be to aid learning. Assessment cannot be conceived as a series of discrete milestones but as part of a continuous and coherent learning process.
Second, assessment is linked to outcomes. Learning and understanding go beyond what one knows to what one does with that knowledge. Thus it is important that assessments directly relate to instructional and training objectives. Assessments that reflect specific and explicit criteria for performance, continuous feedback, and self-assessment are more effective in capturing performance of valued outcomes over time.
Third, abilities must be developed and assessed in multiple modes and contexts. Most abilities and the real-life situations in which they are tested are multidimensional and complex. Assessment should provide learners with repeated opportunities to experience, practice, and assess their performance in varied contexts and at varied levels of mastery.
