Home > Resources > Classroom Practice > Assessment

Keeping Student Performance Central: The New York Assessment Collection

Author(s): David Allen, Joe McDonald Source: Coalition of Essential Schools, October 1993. Ordering Information

Web Version of New York Assessment Collection

Educators now find themselves at the intersection of two rapidly advancing technologies. Both are sometimes touted as saviors for our failing schools. Both are looked upon hopefully, but with a measure of skepticism and anxiety by already overworked school people. Like all technologies, both have the danger of shifting attention from what really matters --in the case of schooling, student performance -- to the technologies themselves.

The first of these technologies is more easily identified as such: the computer driven revolution in communication, with its CD- ROMs, LANs, modems, electronic mail networks, and more and more powerful computers. While school budgets keep such tools outside the immediate circle of concerns for many, school people sense them looming and already infiltrating some schools and districts. "Preparing students for the twenty-first century" is a mission widely adopted by schools, with an implicit agenda involving information and communication tools. But little has been done to develop the capacity to deliver on such a promise, in terms of equipment especially, as well as the professional development necessary to utilize it and teach it or the reorganization of schools and systems to support it (Cuban, 1992).

The second technology of interest here is a technology of assessment. Calls for better accountability, along with dissatisfaction with current reliance on standardized tests and textbook derived exams, have given rise to a movement toward performance assessment (Mitchell, 1992). As with computers, this technology is already a presence in schools, though rarely central to the schooling that goes on there. Even so, more and more states such as New York (as well as Kentucky, California, and Vermont) have initiated reform of assessment practice placing a premium on performance assessment.

New York's New Compact for Learning, adopted by the state Board of Regents in 1991, calls for assessments of student performance that are as authentic as possible, representing realworld tasks and situations requiring higher-order thinking and complex, integrated performances (New York State Curriculum and Assessment Council, 1992, p. v). How will schools begin to develop the capacity for this kind of assessment practice?

In 1990, the IBM Corporation underwrote a project at the Coalition of Essential Schools to explore the intersection of these two technologies, looking particularly at how that intersection might affect the design of the public high school. The three-year study that followed has resulted in a number of publications, particularly concerning a strategy for school change termed "planning backwards" (McDonald, 1992), and in the development of several promising tools to support school change. The publications and the tools reflect a core idea of the Coalition of Essential Schools; namely, that student performance should be central, not marginal, to the work of schools.

One of the tools we have developed is "The Exhibitions Collection,"1 a computer sampler of more than thirty actual performance assessments, mainly from Essential schools. Since its release in 1992, the collection has been widely used in schools across the country. This kind of resource for schools appealed especially to New York State's Council for Curriculum and Assessment, which is charged with creating student outcome standards, curriculum frameworks, and assessment strategies as part of the New Compact for Learning.

The New York Assessment Strategy

The New Compact's general assessment strategy represents a significant shift from pencil and paper testing to performance assessments, including an emphasis on student portfolios and exhibitions. Since the New Compact also calls for performance assessment systems to be locally developed and to be compatible with state goals, it requires a dramatic change in the roles of the State Education Department, the districts, and individual schools: The State should encourage local practitioners to develop innovative and thoughtful assessment programs by inviting local initiatives and supporting local development with assess-ment options. Among the resources the State can provide is access to a portfolio or bank of assessment ideas, tasks, and instruments [emphasis added]. (New York State Curriculum and Assessment Council, 1992, p. 11)

Members of the Council and team leaders from the State Education Department, along with the staff of the Coalition's Exhibitions Project and our IBM colleagues, saw an opportunity to take advantage of communication technology to create and disseminate such a resource bank. In this paper, we describe how, building on our experience with the Exhibitions Collection and our research in schools, we will build a "New York Assessment Collection."2 Through this tool, we bring together the potential energy of both the communication and performance assessment technologies in a format most relevant to educators in that it keeps student performance central to its content and design."

The Six Dimensions of Assessment

Before we describe the New York Assessment Collection and the communication technology that underlies it, we'll briefly summarize the findings about performance assessment that have emerged from our research and shape our technological development.

Every assessment that calls for authentic performance is really a system of interacting elements. We have identified six components, or dimensions, that inhere in every good assessment, no matter how different in form each assessment may be from another: for example, one student's individual exhibition on the concept of whole numbers and an entire class's simultation of a U.N. peace conference.

Our six dimensions of assessment are:

1. Vision
2. Prompt
3. Coaching context
4. Performance
5. Standards
6. Reflection

There are two points to be made about these six dimensions. First, they are not always easy to differentiate; for example, coaching context or instruction may overlap in subtle ways with student performance. Second, whereas in the past, the design criteria for assessments were often limited to a linear schema (assignment performance assessment), our experience suggests that these six dimensions are nonlinear and therefore constantly interacting with one another.

When illustrated by assessments actually used in schools, these dimensions have become useful as trail markers for schools as they begin to design assessments for their students. It follows that any technology that supports performance assessment should pay heed to these dimensions and the complexity of the system of assessment they constitute.

Goals For the NY Assessment Collection

The primary goal for the New York Assessment Collection is that it portray a variety of real performance assessments from New York schools in a format that is useful for educators as they develop their own assessments. What do these performance assessments look like? How do they work? In creating the collection, we have drawn upon the six dimensions above as the key design element for the tool itself. As we discuss each dimension in the context of the collection, we suggest a few assessment design criteria that have emerged for each.

The second goal for the collection is to make use of technology readily available to teachers now and to anticipate a time when far more powerful technologies will be available to them. How will educators take advantage of these technologies? In the section that follows, we describe how an existing technology --hypermedia --allows us to portray assessments and disseminate them in a format accessible right now. Later, we'll look at some possible adaptations for the information technologies of the (not too distant) future.

What Is Hypermedia?

Before we turn to the six dimensions of the Assessment Collection, it will be useful to understand what hypermedia is and how it supports our goals for the collection.

Hypermedia3 refers to a computer document in which information is stored in pieces, called nodes. Rather than being organized in a linear fashion, as with a wordprocessing document, the nodes are linked to each other in the same way cities and towns are linked by roads. By clicking on buttons on the screen, the user --not the designer --chooses the path by which the information is accessed.

Hypermedia represents a promising technology for all kinds of uses in education because it encourages students to discover the links between bits of knowledge. Hypermedia documents can incorporate text, graphics, scanned photographs --even audio --in place of a traditional report (Scheidler, 1993).

We chose hypermedia as a vehicle for disseminating the New York Assessment Collection because we view assessments as complex systems of interrelated components, rather than linear progressions from assignment to test. The hypermedia document we are creating encourages users to explore the assessments as they choose, holding up one part against another to judge for coherence, fairness, and depth.

The "tour" that follows demonstrates how hypermedia allows users to "view" an assessment.

A Tour of the New York Assessment Collection

It is important to note that the tool we describe here4 is now only a prototype, a "shell" we will fill with real examples of assessments from New York elementary and secondary schools. We welcome feedback on the design, as well as assessments that might be included in the collection.

The best way to tour a user-friendly computer application, of course, is to sit down in front of your own computer and experiment with it. Nonetheless this paper format allows for some of our observations about the choices we've made in designing the collection.

The Assessments Menu

The collection opens with an introductory screen that includes purposes, information about the New Compact for Learning, and some criteria for "assessing assessments," but we'll begin our tour at the menu screen. The menu is really the nexus of the collection, and so always a choice on the screen. Here the user can choose to view any one of the assessments included. (In this prototype, only the first two are from New York schools.)

The commencement-level5 assessments menu represents one of three menu screens; assessments will be divided into elementary, intermediate and commencement. The viewer clicking on the first assessment listed --the Senior Institute at Central Park East Secondary School (CPESS)6 --is automatically placed into the first of the six dimensions by which each assessment is portrayed, the vision. Along the righthand side, the buttons for all six dimensions allow users to move around within the assessment as they choose.

Vision

Assessments are often led by the assignment, or prompt, and not by any vision of actual student performance. The emphasis therefore falls not on what students are actually doing, but on whether the assignment itself is clever or fits neatly into the curriculum. This is why we've chosen to begin looking at each assessment with the vision, and we encourage frequent revisiting of the vision as the user examines the other dimensions of the assessment, including actual student performances --an advantage of hypermedia.

The vision represents the image of performance that underlies the assessment. Often, as is the case at CPESS, this image will be part of the school's vision of its ideal graduate. In elementary schools especially, it may be associated with an idea of child development, and in every school with what it means to be a successful and fulfilled human being.

Criteria for designing an assessment might pose these questions:

1. Is the vision simple and coherent?
2. Is it "grounded" (for example, in the "habits of mind" valued at CPESS; elsewhere perhaps in the National Council of Teachers of Mathematics standards or the New Compact outcomes)?
3. Is it expressed, in David Perkins's phrase, as an "understanding performance"? (see Perkins, 1992, pp. 7779).

Prompt

From vision, the user may choose to follow any path in viewing the assessment. Most typically, one will turn to the prompt, or assignment. The prompt defines the work of the assessment for both the teacher and the student. It may be brief (with details left to coaching context) or elaborate and complex, as with CPESS's Senior Institute, which involves putting together a portfolio of fourteen separate items and performances.

Design criteria that may help to develop the prompt, in addition to careful consideration of the vision, might include questions such as the following:

1. Is it authentic --derived from something outside pure schoolwork?
2. Does it ask students to do something that matters?
3. Is it essential to the domain in which the student is working, rather than peripheral to it?

Standards

From here, the user might choose to look at the standards that accompany the performance. Standards are the explicit criteria against which the performance is judged. They provide the mark to shoot for and so must be available to students up front, not hidden from them. These criteria derive from an analysis of the vision, but also seek to answer two questions:

1. Where is this student's performance in relation to the vision?
2. How can we help the student reach higher?

With all the talk about standards, "world class" and otherwise, it is all too rare to see them applied to actual student performance. As we've written earlier, standards "come to life in the face of performance, then fade in clarity and power as images from the performance fade" (Sizer, McDonald and Rogers, 1992/93, p. 30). The collection encourages users to jump from the standards to actual images of performance and back to standards --bringing in vision and coaching context as well --to check for validity and fairness.

The guiding design criterion that might be applied to standards is, borrowing a phrase from Grant Wiggins, What are "the most salient and insightful discriminators in judging [this] performance"? (Wiggins, 1992, p. 26). Against these discriminators, the school may develop indicators of level (for example, beginner, apprentice, and expert) and begin to compile benchmarks for each level.

Performances

Performance is central to the whole enterprise of assessment. In our Exhibitions Collection we generally included just one sample of authentic student work. In the New York Assessment Collection, we will provide multiple examples at different levels of student performance (with an indication of how each has been assessed according to the standards). Initially all the samples will be text- or graphics-based (with an accompanying videotape), but in future multimedia versions, digitized video and audio of student performances will be included.

The performances are the student responses to the prompt. They should live on in the school, through careful documentation and archiving, to become benchmarks for subsequent performers and judges. They also function as benchmarks here for considering each assessment included in the collection --the user should put the actual performance up against the vision, the standards, and the coaching context to check for coherence and adherence to vision.

Coaching Context

The coaching context is meant to be more than just the logistics that support the performance. If we believe that good assessments provoke students to delve deeper or stretch farther, then the coaching that surrounds performance is crucial. It might include consultation, schedules, tools, technical advice, formative critique, peer collaboration, a timely push, a word of support, or even a support strategically withheld. At CPESS, the coaching context includes putting together and meeting with a graduation committee.

As with performances, future versions of the collection may be enhanced with video images of the coaching context, so that in the CPESS assessment, for example, the viewer might see an excerpt from a conference with the student's graduation committee.

Some criteria for coaching context might be derived from these questions:

1. Is it strategic --with an image of performance in mind?
2. Is it illustrative --full of images of good performance?
3. Is it equitable --flexible enough to provide resources to the students who really need them when they need them?

Reflection

The sixth dimension is reflection. In one sense, the entire Assessment Collection is intended to foster reflection --individual and collective. Because it brings real performances out into the open, it provides educators (and potentially students and parents) something on which to reflect. Very often the reflection component of the Exhibitions Collection is comprised of just one teacher's reflection, often taken from a brief interview after the performances: What worked? What would I change? The New York Assessment Collection will include reflections from the teachers involved, of course, but also from students, outside judges, and perhaps even state education team leaders. In a future version of the collection, the reflections screen will offer built-in access to an electronic bulletin board, allowing users to ask the developers of the assessment questions such as: Why did you choose a collaborative project? and How did you assess computational skills?

Ultimately, we believe that reflection must be "built into" every assessment. It is the element of the system that monitors the system itself, collecting and analyzing feedback from performances, performers, and judges. It may pose questions about the reliability and fairness of the system. It may also "tune" the assessment to external values, including, in New York, the aims of the New Compact.

Questions that may serve as criteria for schools as they think about building or enhancing reflective capacity are:

1. Does it provide feedback to the students?
2. Does it supply feedback to the faculty about the system itself?
3. Does it provide both "warm" and "cool" feedback? (see McDonald, 1993).

Taking Advantage of the Technologies of Tomorrow

The first versions of the New York Assessment Collection will be disseminated, like the existing Exhibitions Collection, on diskette. This makes the information it contains accessible to every school with an IBM-compatible or Macintosh computer. We are anticipating a time when schools will have access to far more powerful technology for sharing information. Two fast approaching developments with which educators have just begun to contend are multimedia and "data highways."

Multimedia is really just the logical next step from hypermedia, integrating video, sound, text, and graphics into single documents or databases. When this technology becomes more accessible to schools, how will they use it? The Assessment Collection offers one possible answer. The sharing of good assessment practice will be more powerful when schools and educators are able to view video clips of student performances, coaching sessions, even reflective feedback discussions, along with the written descriptions of these dimensions.

Other uses of multimedia by schools might be the development of a "digital school profile" to portray the school's systems to parents, community members, and other stakeholders, or a "digital student portfolio" (Niguidula, 1993) to portray a student's academic growth with new depth.

The second advance will be in communications technology. Already, national and regional networks (the Internet, NYCENet, NYSERNet) connect educators and researchers electronically. In the next decade, the capacity to access information and communicate instantly with other educators across a national 'data highway' will become available to nearly every school in the country. Not only will we be able to communicate words but --via fiber-optic networks -- video and sound as well.

The implications for elementary and secondary education are tremendous and openended. Already some teachers take advantage of the Internet to put their students in touch with others around the world (West, 1993). But how will teachers use these technologies to communicate with each other?

Again, we suggest that the Assessment Collection presents a starting point. As the national discussion --or debate --about standards continues, teachers need means to communicate about student performance. They need to be able to see actual student performances from other schools, discuss assessments with other educators, question each other about aspects of their work. Of course, these conversations can happen in many forums. Some of these should be electronic.

As mentioned above, we will build into the Assessment Collection a hook-up to electronic mail, so that viewers can enter into dialog with the assessment's creators. This is just a first step toward the point where educators are regularly viewing and discussing student performance, but one teachers must take so that the technology revolution supports their work, not the other way around.

Conclusion

Both the technologies referred to in this paper, communication and performance assessment, are at once enormously exciting and intimidating, especially in their capacity to put new strains on our already overburdened systems of schooling. Historically, the social organization of schools has dealt with new technologies by keeping them at the margins of instruction, rather than allowing them to substantially affect it (Cohen, 1988; Cuban, 1992). These technologies, with all the good they offer, will remain at the margins as long as student performance itself is marginalized.

The power of these two technologies, as we conceive of it, lies in their capacity to bring student performance to the center of schooling. Where performance assessment differs from assessment technologies of the past (for example, standardized testing) is in its basic tenet that images of authentic student performance --not an abstraction of it --guide all aspects of instruction and assessment.

This strategy is not without its critics. When we showed the prototype of the Assessment Collection at a conference recently, an educator homed in on a syntactical error made in one of the samples of student performance. He insisted that not only did the error invalidate the worth of the assessment described, but of the entire collection. It is no wonder that teachers have been hesitant to share student performance publicly. Until we develop tools and protocols for discussing work within and among schools, we will continue to concentrate on what goes into the system of schooling rather than who comes out of it.

In our development of communications technology, we insist that we cannot talk about assessment without the student performances; they are essential and central to the technological tool we will create, the New York Assessment Collection. As communication technology advances beyond this modest tool, and educators develop the capacity to use that technology, it will be vital to keep student performance --Tamika's exhibition on urban planning, Tony's community service portfolio --at the heart of technology use. Only by keeping our eyes on the real goal of schooling will we keep from being lost at the intersection of technologies, and rather be in a position to take advantage of the choices it offers.

References

Cohen, D. K. (1988). "Educational Technology and School Organization." In R. S. Nickerson & R. O. Zodhiates (eds.), Technology in Education: Looking Toward 2020 (pp. 231-264). Hillsdale, NJ: Lawrence Erlbaum.

Cuban, L. (1992). "Computers Meet Classroom; Classroom Wins." Education Week (November 11), pp. 36, 27.

McDonald, J. P. (1992). "Dilemmas of Planning Backwards: Rescuing a Good Idea." (CES Studies on Exhibitions, No. 3) Teachers College Record 94(1), pp. 152169.

McDonald, J. P. (1993). "Three Pictures of School Reform: Warm, Cool and Hard." (CES Studies on Exhibitions, No. 1) Phi Delta Kappan 74(6), pp. 480-485.

Mitchell, R. (1992). Testing for Learning. New York: Free Press. New York State Curriculum and Assessment Council. (1992). Building a Learning-Centered Curriculum for Learner-Centered Schools (Interim Report to the Commissioner and the Regents). New York: Council.

Niguidula, D. (1993). "The Digital Portfolio: A Richer Picture of Student Performance." Studies on Exhibitions (No. 13), Coalition of Essential Schools, Brown University.

Perkins, D. (1992). Smart Schools: From Training Memories to Educating Minds. New York: Free Press.

Scheidler, K. (1993). "Students Cross Discipline Boundaries with Hypermedia." The Computing Teacher 20(5), pp. 16-20.

Sizer, T. R., J. P. McDonald & B. Rogers (1992/93). "Standards and School Reform: Asking the Basic Questions." (CES Studies on Exhbitions, No. 8) Stanford Law & Policy Review 4, pp. 27-35.

West, P. (1993). "A Window on the World." Education Week (January 13), pp. 1, 2527.

Wiggins, G. (1992). "Creating Tests Worth Taking." Educational Leadership 49(8), pp. 26-33.

Footnotes

1. The Exhibitions Collection,version 2.0, is available in IBM and Macintosh formats from the Coalition of Essential Schools, Box 1969, Brown University, Providence, RI 02912.
2. The New York Assessment Collection will be developed in association with the New York State Education Department and with the support of IBM. The first version is planned for release on diskette (IBM and Macintosh formats) for use by New York schools in July 1994.
3. Some common programs used to create hypermedia documents are HyperCard for the Macintosh, and ToolBook and Authorware for IBM and IBM-compatible computers.
4. The principal designer for the prototype presented here is Michelle Riconscente, program assistant for technology at the Coalition of Essential Schools, whose ideas have also contributed to this paper. The authors would also like to acknowledge the contributions of Roseanne DeFabio and Susan Agruso of the New York State Education Department.
5. "Commencement-level" in the parlance of the New Compact refers typically to student work in the final two years of high school. 6. Central Park East Secondary School was founded in 1985. It is a New York City public school, serving grades 7-12, located at 1573 Madison Avenue, New York, NY 10029. It is a member of the Coalition of Essential Schools and the Center for Collaborative Education.

The Coalition of Essential Schools gratefully acknowledges the IBM Corporation and the UPS Foundation for their support of its research on Exhibitions.

Joseph McDonald has headed the IBM Exhibitions project for the past three years. David Allen, who joined the project in 1992, will coordinate development of the New York Assessment Collection. A slightly different version of this paper will appear in a monograph to be published by the New York State Council of Educational Associations.

Price: $5        Code: EX14
To order a hard copy of this resource you will need the title, price, and code to fill out your order form.

This resource last updated: June 07, 2002

Database Information:

About CES | CES Network | Fall Forum | Small Schools Project | Resources
My Homebase | Jobs | Search | Site Map | Contact Us | Home