Outcomes-Based Task for Assessment

Participants might be asked to select an action, taken by an influential person in the company, that had important consequences for that company. They are asked to determine the characteristics of the decision that had to be identified by this person before such an action was taken. What dilemmas did the person face? What alternative choices were available to this person when the decision was made? What criteria was the person likely to have applied in making the decision? What were the possible trade-offs in selecting one alternative over another? What were the risks, the rewards, and the consequences? How might these have been measured? Without benefit of hindsight, would you have made the same decision? Why or why not?

Such an outcomes-based task enables assessment of several indicators of good problem solving:

1. Clear definition of a decision question
   
   
   
   
2. Clear articulation of alternatives
   
   
3. Appropriateness of alternatives
   
   
4. Clear identification of criteria on which the alternatives were assessed
   
   
5. Value judgment of the importance of the identified criteria to the overall decision
   
   
6. Assessment of the extent to which each alternative included or matched up with each of the identified criteria
   
   
7. Commitment to a final selection among the alternatives
   
   
8. Adequacy of the final selection to the initial dilemma
   

Criterion-based assessments and authentic tasks focus on what adult learners can eventually learn to do well, rather than on how well they do the first time they encounter something. Authentic tasks address the outcomes we value and simulate challenges facing an individual on the job. We will explore these specific notions of assessment in the sections that follow.

What Norm-Referenced or Standardized Testing Can and Cannot Tell Us. What norm-referenced or standardized testing tells us is what learners know in general. If we look at a standardized reading test, for example, and focus on all learners with high scores and all learners with low scores, it is likely that more reading difficulties will exist among learners with low scores. Of what use is this kind of information? Standardized tests enable us to get a global picture. In general, are sixth graders learning reading at the sixth grade level? In general, are college algebra students learning college algebra?

However, what standardized testing does not tell us, nor is it meant to tell us, are nuances of performance that characterize the full range of a learner's skill, ability, and learning style. The fact that a reader scores well on a standardized test does not mean that we can say with confidence that the person is a wonderful reader and always will be in all circumstances.

Furthermore, traditional tests are arbitrarily timed, superficial in the content they test, and given only once or twice. They leave us with no way of gauging a learner's ability to make progress over time. So we must ask whether standardized tests provide sufficient information to allow intelligent instructional and program decision making. For example, what does it mean if a learner ranks in the eightieth percentile on a reading test? We may say that person has performed better than 80 percent of the others who took the same test. This sounds impressive. However, a difference in performance on one test item can significantly raise or lower an individual's percentile ranking.

In addition, reading passages in traditional tests are generally shorter and less complex than the texts that learners encounter in daily work. Standardized reading tests use a small number of item types to avoid confusing test takers with frequent changes in for-mat. This is problematic for there is considerable evidence that individuals "may appear to know a concept or skill when it is measured in one format but not know it if measured in another way." Given this, how much confidence do we want to place in standardized test scores for program and personnel decision making?

Do standardized tests have value? Yes, they do. The most immediate value for pre- and post-standardized tests is as an indicator of a learner's initial, general knowledge base. However, as noted in our discussion of the nature of standardized norm-referenced tests, standardized tests are not generally aligned with training and development curricula. Thus, it is important to examine the match between competencies measured on a standardized test and the curriculum of the training or instructional program. It is also important never to assume that test scores are infallible or to use a single test to make an important decision about an individual or a program.

If the primary purpose of assessment is to guide instructional decision making and to provide feedback to learners, then standardized norm-referenced testing falls short of this purpose. More can be accomplished with a combination of methods that include, but are not limited to, standardized measures. Pre and post standardized tests, when supplemented by criterion-referenced tests or alternative assessments, such as authentic tasks or curriculum-based measurements, can provide indicators of changes in a learner's knowledge base associated with a training or instructional program. In the following paragraphs we will discuss such alternatives to standardized testing.

Criterion-Referenced Testing. Criterion-referenced tests are tests of specific skills that are scored with reference to examples of poor, fair, good, and excellent performance of those skills. Unlike norm-referenced or standardized tests, which compare any given student's performance in general with that of other test takers, criterion referenced testing is interpreted in terms of performance criteria that can be more closely aligned with the content and conduct of specific training or instruction. Criterion-referenced tests tell us about a learner's level of proficiency in or mastery of some skill or set of skills. Learners are not compared with others, as in norm-referenced testing, but with a standard of mastery called a criterion.

This information is far more useful in making ongoing training or instructional decisions. Criterion referenced testing enables deficiencies to be precisely diagnosed and training or instruction to be targeted at addressing those deficiencies.

Criterion-referenced tests must be very specific if they are to yield information about individual skills. Such specificity is useful for enabling teachers and trainers to be relatively certain that learners have mastered or failed to master the skill in question. The disadvantage is that a number of criterion-referenced tests are needed to make decisions about multiple skills.