The Digital Portfolio: A Richer Picture of Student Performance
What do we really know about a high school graduate's academic career? If we can't talk to the student personally -- and even if we can -- we rely on records. For the most part, a student record con- sists of two things: grades on a transcript and stan-dardized test scores. If a potential employer wants to know if the graduate has the ability to write a business report, she has to make do with a "B-" in English 2 or a 530 on the verbal portion of the SAT -- or, even more generally, something like a 2.85 grade-point average or a 57th percentile marking.
Grant Wiggins suggests this experiment.1 Go into any sports bar and ask, "Who's the best active major league baseball player?" In the ensuing argument, you'll hear about the quality of a catcher's defensive play or the leadership he brings to the team. You'll hear a bunch of numbers, too, from home runs to batting averages to errors, and arguments over which numbers are more significant. The truth is that any sports fan knows that no one statistic can capture a player's entire career -- even the tables of fine print on the back of every baseball card cannot entirely replicate images of play lodged in a fan's memory (or video collection). Yet, we have a tendency to reduce academic careers to just a few numbers.
We can gain a better understanding of a student's abilities and accomplishments by simply looking at the student's work, rather than the abstracted final grade. This, of course, is the primary idea behind portfolio assessment: students can compile their best work in writing, math, science, art, and even community service and present a more vivid record of what they are able to do. Portfolios are gaining currency as an assessment alternative to report cards and transcripts; numbers of schools, and even entire states -- notably Kentucky and Vermont -- are using portfolios to get a more accurate description of a student's capabilities.
To a school interested in reform, the idea sounds terrific; that portfolios can provide a more detailed picture than test scores and letter grade transcripts is beyond question. Then reality strikes. What are we going to do with ALL THIS STUFF? In Kentucky -- where, as part of the Education Reform Act, portfolios are now in place in writing and math and will soon be in place for other disciplines -- there are stories of schools budgeting for a hundred filing cabinets.
Even beyond the logistical nightmare of thousands of papers turning brittle and collecting dust, there are at least two other practical drawbacks. First, not everything may be on paper. A student's best demonstration of mathematical reasoning ability may be a three-dimensional model; a student's best articulation of historical under-standing may have come during a debate. Yes, we can use photographs and videotapes -- but these give rise to even more issues of storage. Second, the mass of paper and other materials isn't easily managed for presentation in a filing cabinet. What if a research paper in history is the best example of writing a student completed? Do we make two copies of the paper and file one under "history" and one under "writing"? Do we keep a paper index or develop an elaborate system for cross-referencing?
Further, how does this student's piece compare with those of students from other years? How can a younger student see a "benchmark" performance that meets the high standard that she is striving to achieve? Portfolios are typically filed by student -- last name first, etc. -- but what if we want to compare two or more students' writing samples? A paper chase ensues.
The Exhibitions Project at the Coalition of Essential Schools, with support from the IBM Corporation, has been developing a way to address these problems. Using computer technology, we can create a richer picture of what a student can know and do. Our task is to create a tool to communicate that picture. This paper describes our prototype for a "digital" portfolio.2
A Tool for Planning Backwards
The Digital Portfolio is a computer-based tool for what the Exhibitions Project of the Coalition has termed "planning backwards."3 A school can approach decisions about its system of education by addressing a series of three questions:
- What should a graduate of this school know and be able to do? The answer to this question is the "vision" of the qualities that a graduate -- any graduate -- should possess. The vision needs to be created by the school community, led by the faculty that will be looking for those qualities in their classrooms.
- How can a student demonstrate the skills and knowledge that a graduate should have? In other words, how would we know that a student has the qualities to fulfill our vision? Here, the answer might take the form of a set of "exhibitions" or other performance- based assessments completed either individually or collaboratively.
- How can the school arrange its systems so that all students can exhibit the desired skills and knowledge? This is where the real work of school change begins. The school needs to look at all its systems -- curriculum, pedagogy, assessment, schedul-ing, administration, and so on -- in an honest attempt to determine if the decisions it makes are helping students fulfill the vision. Often this takes a great deal of "rewiring" of communication within the school -- not just with physical wires, but by creating a structure of time and space that allows everyone involved with the school to communicate.
The Digital Portfolio represents a tool that may help a school develop some preliminary answers to these questions.
A Walk Through the Digital Portfolio Prototype
The Digital Portfolio is a "hypermedia" document. This simply means that the portfolio is a set of screens (or pages, in the terminology of the software) that are linked by buttons on the screen. When the user clicks the mouse to select a button, the program reacts, typically by navigating to another screen or by showing some additional information. In this section, we will walk through the prototype. Keep in mind that this is a work in progress -- a demonstration to begin the conversation, rather than a completed product.
The opening page of the Digital Portfolio is an identification of the student and her school. In the bar along the bottom of the screen are several buttons; clicking on any of the words in that bar elicits an action. For example, clicking on the Photo button allows the user to see a digitized photo of the student. Clicking on the Information button shows a screen with additional information about the student, such as her years of enrollment and other administrative data. Clicking on the Instructions button brings up a screen that tells the user how to navigate through the portfolio.
When the user clicks on the Main Menu button on the Introductory screen, the portfolio moves to the Main Menu screen. The design of the portfolio begins with an answer to the first question of planning backwards: What should a graduate know and be able to do? In our example, we have used the Kentucky Education Reform Act's six "learning goals," each of which comprises a number of outcomes. (The state has defined seventy-five outcomes under these six goals.) The six learning goals are listed on the left side of the screen. Again, these particular goals are simply an example; the set of goals used in any school's portfolio would be established by the local community.
Our prototype contains an additional column of goals to the right of the six primary learning goals. The right-hand column represents the subgoals of Goal 2, Core Concepts. Goal 2 is divided into smaller sections, primarily because the design of the portfolio parallels in many ways how teachers organize paper portfolios. If a student filed all of his or her work in folders representing the six learning goals, the folder for the Core Concepts would be bursting at the seams. Thus, our Main Menu provides folders for what Kentucky has defined as the six "core concept areas" of learning: science, mathematics, social sciences, arts and humanities, living skills, vocational studies.
The names of the goals -- Communication, Self-Sufficiency, and so on -- require some elaboration. Clicking on the round button next to a goal (or subgoal under Core Concepts) brings up a window showing the outcomes associated with that goal. In our example, the detailed description consists of the outcomes defined for each learning goal by the state. There is a list of outcomes associated with the goal of Communication. We can scroll down the list to see the rest of the outcomes.
We imagine that when a student enters a school, he or she will be given a blank portfolio, containing only screens. The student can browse through the goals and see what it is that he or she is expected to be able to know and do before graduating. One might also expect that incoming students will want to see benchmark performances in goal areas from earlier graduates' folders.
Next, we need to ask the second planning backwards question: How can a student demonstrate those skills and knowledge that a graduate should have? The Coalition refers to such demon-strations as "exhibitions," and they can take many forms: seminars, research projects, oral and written presentations, and yes, even timed tests.
An exhibition is an assessment that provides the opportunity for students to demonstrate a skill or knowledge that is highly valued by the school. It grows from a vision of students performing as their teachers hoped they would. As Joe McDonald has written, an exhibition "faces inward, points out-ward, and presses deeper."4 It allows a school to see what its students have done and to show the community what the school itself is doing, and it provides grist for the conversation mill within the school and community about standards and systems. So the Digital Portfolio, with exhibitions as its core content, presents not only the student but also the values and contours of the school community.
While any kind of information could go into the Digital Portfolio -- including the familiar transcript, report cards, and standardized test scores -- its strength is its ability to present authentic performance, such as exhibitions, in any media.
Looking Into a Folder
Let's consider some specific examples. How will we know if a student has demonstrated the ability to communicate or to solve problems? If we click on the folder icon next to a goal, we see a list of performances representing the student's exhibitions of that goal.
This list of performances is essentially a "table of contents" for a goal. It is meant to represent the student's performances over a period of time. The list might include performances required by the school or the state, such as the entries for the twelfth- grade writing portfolio, or it might include performances that the student feels are good representations of her work.
Any single performance might be a representation of more than one outcome or goal. In this example, the Waterfront exhibition is listed in both the Communications and Problem Solving folders. Because the portfolio is a hypermedia document, multiple paths can lead to the same information -- that is, a user can get to the same point in more than one way. One advantage of the technological version of a portfolio is that we only need to retain one copy of the actual student work on the disk, no matter how many places it is filed.
From this table of contents, a user can click on any of the titles to see the work. Let's suppose the user clicked on the Waterfront exhibition.
On the left side of this screen is the actual student work; by scrolling through that window, the user can read the entire work as it was submitted by the student (or, as in this case, a group of students). Here we see a description of how this student group proposes to bring more people to the Louisville waterfront.
The box on the right side of the screen displays additional information about the current portfolio item. The "demonstrated goals" -- meaning the outcomes that a reader should find in this piece -- are taken from the list of outcomes we saw in the Main Menu. This exhibition demonstrates a number of goals including "productive team membership" and "critical thinking skills."5 It is not, however, listed as a demonstration of "visual arts," though there is a visual component to the work; presumably, some other piece of work in the portfolio is a better demonstration of that outcome. This kind of selection and annotation can be made by the student, building in a reflective dimension to her school career.
One advantage of the digital portfolio is its ability to store multiple media. Graphics, for example, are incorporated into this student's work. Below the text are three buttons, labeled Video, Audio, and Graphic. Clicking on the Graphic button displays a list of graphics that the student has created for her presentation. Our example shows a sketch done by the student that was included with the project. In this case, the sketch was drawn by hand and scanned into the portfolio; drawings created by computer paint, drawing, and animation programs can be included in the portfolio as well.
Returning to the Goals screen, the menu along the bottom of the screen allows us to look at other aspects of the exhibition. Clicking on the Assignment button brings up another screen.
Again, on the left is the student work. On the right of this screen is the project description, the assignment that the student received from the teacher. Having a view of the assignment helps to put the performance in context; if one understands the constraints of the assignment, one is better able to judge the student work.
The menu bar now contains a Goals button. It also contains a button labeled Evaluation. If we click on the Evaluation button, we see a new screen.
The right side of the screen now contains a list of evaluators. If we click on any of the evaluators in this list, we will see an evaluation of the student work. Presumably, there will be a teacher evaluation for all items in a portfolio. The prototype allows for the student and other judges to evaluate the work as well. In fact, any reader may add an evaluation "on the fly" by clicking on the top line of the evaluators list and entering comments in a text box.
In our example, we can see the rubric used by the teacher in evaluating the project. If we scroll down the list, we can see what an "A" project represents, what a "B" project represents, and so on. The outside evaluators for this exhibition were representatives from the city's office of planning and development, whose evaluation of the exhibition offers other insights to the student work.
The Digital Portfolio prototype, as it exists now, contains information for just one student. As such, it contains useful information about a student's abilities and can provide a demonstration of growth over time. Imagine, for example, storing a recorded oral presentation in each of the four years of a student's high school career. This longitudinal dimension, so valued by teachers, is woefully underserved by the documentation we currently maintain on students.
It can also place student work in a context. Rather than simply identifying papers and projects as "best work," it identifies that a particular piece of work is a "best" demonstration of a particular outcome. Still, further elements need to go into the design. We assume that some users -- for instance, college admissions officers -- may want to see more than one student's work at a time. We need to design a multimedia database that allows easy searches through multiple portfolios, and that can answer queries such as, "Show me outstanding performances in writing portfolios" or "Show me all the science folders for my incoming chemistry class." Students can also see what valued outcomes they need to fulfill before they graduate and benchmark performances by recent graduates.
A database does not imply, though, that all student information should be available for all to view. Students should be seen as the owners of their own work. While a school should be able to retain a copy of some work as evidence of a student's performance, and may physically store the student work, the student should have at least some degree of control of what is included in the portfolio. For example, the school might legiti- mately require a writing sample; the particular sample that is submitted should be chosen by the student.
We need to answer the question, "What information is useful?" The answer, we presume, would be different for different audiences. We might presume that a student would want to keep everything that he or she produced in one version of the portfolio, and that the school would maintain a subset of that material as the "transcript." The school and the student might reduce that information even further for other audiences (e.g., college admissions officers, employers, accreditation agencies), combining selected exhibitions with more traditional information (test scores, grades, records of participation in clubs, sports, and so on).
There are also technical hurdles to be overcome. Multimedia documents -- particularly video clips -- occupy a great deal of disk space. (One earlier prototype, with about four minutes total of non-compressed video, occupies about twenty mega-bytes on a hard disk.) With hundreds or thousands of students developing portfolios within a school, issues of storing bits of information are just as real as issues of storing paper copies of everything. Still, compression technologies (such as Intel's Digital Video Interactive) and read-write optical storage media (for example, a "writable" CD-ROM) are making such storage more viable.
In addition, we would want to communicate this information across distances -- some distances as short as between classrooms, some as long as from a school to a state education department. Networked multi-media is still in its early stages; transmitting large amounts of data across wires is not as fast or as efficient as it needs to be. Again, though, much work is being done in this area, from advanced forms of local-area networking to the development of the "information superhighway."
Because multimedia storage and networking are issues that touch most every application of computing, the computer scientists and engineers of academia and industry are committing great resources to creating reasonable solutions to these problems. The solutions are some time away, but we can anticipate the initial technological solutions fairly quickly -- in less time, perhaps, than it will take for the educational establishment to embrace alternative assessments across the board. To that end, the digital portfolio represents a vision of what is possible.
Using the Portfolio in School
Let's assume then that we have all of the technical kinks worked out, and that the design of the portfolio and database have been established. Now, how would a school use this tool? In essence, we are now asking the third of our planning backwards questions: How do we arrange our systems so that students will be able to exhibit the desired skills and knowledge? We present here conjectures, some based on the work being done by schools that are pilot testing the Digital Portfolio (Eastern High School in Jefferson County, Kentucky, and Thayer High School in Winchester, New Hampshire), and some based on our theoretical musings.
A school's first step would be to determine what it wants its students to know and be able to do. This means that the labels on the folders in the Main Menu would vary from school to school and from state to state. Most schools have some kind of listing of goals and outcomes, but often those are collecting dust in a filing cabinet.6 By providing every student and every teacher with a copy of those goals, and a reason to know them, the goals have a better chance of driving instruction and learning in the school.
Then the exhibitions need to be established. That is, teachers and students need to determine what will be in the "table of contents" for each folder. Many Essential schools involved in planning backwards are establishing their own goals and exhibitions. Even within more structured frameworks, such as Kentucky's, there is much room for improvisation and experimentation.
A next step is to tie classroom activities to the portfolio- bound work. At Eastern High School, during the 1993-94 school year, members of the senior class are putting together digital portfolios. (The hope is that what this class does this year can be applied to the school as a whole in the next year.) The students will certainly be including those pieces that are part of the required state portfolio for twelfth-graders in writing. In addition, students will select pieces from other classes.
When a student enters an item into the portfolio, he or she needs to know what outcomes are being demonstrated. The faculty has discussed developing a system where students can seek the counsel of a "portfolio advisor" on the faculty to determine which outcomes have been met by that work. Ideally, teachers will be able to tell students which outcomes are being addressed before the students begin their work; the goals of the school need to permeate and shape its activities.
Logistically, students and faculty need access to equipment; however, we do not assume that every student and every teacher needs his or her own machine. The approach at Eastern takes advan- tage of an important technical resource: student power. Two groups of students are involved. First, the seniors who are creating their portfolios are responsible for getting their work into digital form. When possible, they will word-process or otherwise use computers to generate their work in the first place. However, they also have access to the school's scanners to input work that they have not previously digitized. The school also has a video camera and audio recording devices, which will be used by the students to record performances or to annotate their work. The students who are putting portfolios together are responsible for making sure a video camera is in the room when a performance is being given or for recording significant elements in the process of a project.
Second, a group of about twenty students is providing support for the first group. This second group is taking a class in multimedia, learning the intricacies of hypermedia systems such as Toolbook. The teacher, Scott Horan, makes an analogy between this production staff and the school newspaper staff. Just as some students write articles and other students lay out the paper, the seniors will put their work into digital form (and determine in which folder the work will reside), and the portfolio production team will provide the technical support to place the work in the digital portfolio. The construction of the digital portfolios in schools can itself become an "episode of learning."7
The school has arranged systems so that students can provide maximum support for each other. Mr. Horan's class takes place during the lunch period, allowing the portfolio production team to find and assist other students as the need arises.
The faculty, of course, plays a key role in helping to make the portfolios happen. Portfolio advisors for groups of students can help to map the student work to the outcomes and help the students determine what is "best" work. The implication, of course, is that the faculty needs to be consistent in determining what is "good enough" to go into the portfolio and which pieces show mastery of the corresponding goals. Eastern also has the benefit of a local-area network established for teachers, which allows for sharing of electronic mail and other information. More importantly, it has developed a culture where teachers will talk with each other about student work and about standards of achievement.
The development at Thayer has been somewhat different. There, the portfolio protocol has been articulated for the entire school; Thayer has a list of nineteen graduation skills in place of Kentucky's seventy-five outcomes.
Thayer is a much smaller school than Eastern, and the school is already designed to get to know students well. Almost all the teachers belong to teams that work with one group of students in all of their subject areas.8
Computer support is provided by a teacher, Rick Durkee, and the area computing coordinator, Tom Warner. While there is no organized class, student power is still in evidence at the school; the kids know who among them has an interest in com-puters, and they seek help when necessary.
At Thayer, the development of digital portfolios represents an addition of technology to an existing structure. Since teachers already have advisories where they talk with students about their work and a portfolio system that allows students to collect their work, the process of adding the Digital Portfolio is "merely" the process of employing the technology within an existing structure.
This strategy works at Thayer, we believe, because the school has already changed its structure from many of the traditional norms. It is highly probable that the Digital Portfolio would be an overwhelming burden for a school that had not changed its systems of assessment and its use of time. The Digital Portfolio makes assessment much more public and thus focuses conversations in the school on the issues of standards and student achievement. A teacher cannot simply close the classroom door when the assignments and evaluations she provides will be as much a part of the student record as the student work itself.
In fact, we would propose the following hypothesis: the most important thing in the adoption and adaptation of a new technological tool is the culture of the school. A school that is willing to change will find the possibilities in the new tool; a school that is reluctant to change will find the burdens. Further work with Digital Portfolios will provide many tests of this hypothesis.
The experiments at Eastern and Thayer will help define some of the issues of implementing the Digital Portfolio, including the time and space requirements needed to put them together, and the technological requirements for maintaining them. Once we have begun to collect some actual student work, we can begin to address the next set of questions. How would outside audiences -- parents, colleges, businesses, states, and so on -- use these portfolios? What additional features are neces-sary? How would those within the school use the portfolios? How would a school handle issues of confidentiality and access to student information? We plan to have conversations during the coming year with these audiences -- conversations built around actual student work as represented in the digital portfolios.
We can imagine many possibilities for the use of digital portfolios, but as often happens with technology, imagination runs far ahead of reality. Our primary goal, however, is not techno- logical; rather it is to devise a tool that can help schools develop a richer picture of what students are capable of doing.
- Grant Wiggins, panelist for "Performance Assessment: Exploring the Role of Performance Criteria and Standards," Education Commission of the States/Colorado Department of Education Con- ference on Alternative Assessment, Breckenridge, CO, June 1991.
- The concept for the Digital Portfolio was developed by David Niguidula. A first implementation was developed by Richard Bourgon at the Coalition of Essential Schools. Primary development of the current prototype was done by Michelle Riconscente of the Coalition staff using Toolbook software from Asymetrix. An earlier prototype, developed using Linkway Live! software from EduQuest, was created by Ms. Riconscente with further enhancements, including development of the interface and the use of multimedia, developed by Terry Waltman, Errol Rosser, and Jane Patla of EduQuest. Yolanda Jenkins has taken the lead for further development of the product at EduQuest. Revisions for the current version have come from suggestions at numerous conferences and from the students and staff at Eastern High School (Jefferson County, Kentucky) and Thayer High School (Winchester, New Hampshire).
- Joseph P. McDonald, "Steps in Planning Backwards," Studies on Exhibitions (No. 5), Coalition of Essential Schools, Brown University, 1992.
- Joseph P. McDonald, "Exhibitions: Facing Out-ward, Pointing Inward," Studies on Exhibitions (No. 4), Coalition of Essential Schools, Brown University, 1991.
- The particular demonstrated goals here, and elsewhere in the portfolio, are taken from Kentucky's "New Learning Goals and Valued Outcomes."
- Theodore R. Sizer, Horace's Compromise: The Dilemma of the American High School (Boston: Houghton Mifflin, 1985), pp. 76-78.
- Dennie Wolf, Janet Bixby, John Glenn III, and Howard Gardner, "To Use Their Minds Well: Investigating New Forms of Assessment," in Review of Research in Education 17, ed. Gerald Grant (Washington, DC: American Educational Research Association, 1991).
- Barbara A. Eibell, "Developing an Exhibition of Skills: A Cooperative Venture," Studies on Exhibitions (No. 11), Coalition of Essential Schools, Brown University, 1993.
For more details on the "dimensions" of an exhibition, see the Exhibitions Collection, a disk-based hypermedia collection that runs on IBM or Macintosh; see also David Allen and Joseph McDonald, "Keeping Student Performance Central: The New York Assessment Collection," Studies on Exhibitions (No. 14), Coalition of Essential Schools, Brown University, 1993.