Assessing Our Students, Assessing Ourselves is the third volume of the Rethinking Negotiation Teaching project. It is edited by Noam Ebner, James Coben and Christopher Honeyman. Published 2012 by DRI Press, an imprint of the Dispute Resolution Institute at Michell Hamline School of Law, the book is available from Amazon.com; with the generous financial support of the JAMS Foundation, all chapters in the book are also available free in PDF format.
Copyright: We encourage teachers to use any chapters in your teaching and training. There is no charge provided that copies are distributed at or below cost, you notify the publisher (spress01@hamline.edu) that you are using the material and describe the context, and you provide attribution as follows: Copyright 2012 DRI Press, Mitchell Hamline School of Law, Author and Chapter Title.
Description from DRI Press:
Contents
1. Introduction: Assessment as Mirror
Noam Ebner, James Coben & Christopher Honeyman
2. Evaluating Our Evaluation: Rethinking Student Assessment in Negotiation Courses
Noam Ebner, Yael Efron & Kimberlee K. Kovach
Surveying the history of the first thirty years of assessment of negotiation students, the authors find a great deal of creativity, but also a great deal of illogic. To set the stage for the detailed chapters which follow, the authors analyze the multiple purposes behind assessment, and outline a range of possibilities. They also identify seven criteria which any responsible evaluation system should try to satisfy, and discuss a key distinction: the different purposes of formative and summative assessment. It is essential, they contend, that modern systems of assessment address both.
II. Looking Harder at the Tried and True
3. Pop Quiz: Do You Use This Evaluation Method?
Noam Ebner & Yael Efron
Some things should never go out of style. In the midst of an impressive array of new tools, Ebner and Efron argue that there is still a role for a surprisingly basic tool which has been underused in our field. The quiz is a device to which most students have been subjected many times before they take a course in negotiation. Few other tools are so well adapted to the basic educational need to find out quickly if the student “knows“ something that has been presented as fact. Surprisingly, however, Ebner and Efron find few teachers of negotiation using this tool. They examine why this might be, and suggest strategies suited to our field.
4. Reflective Journal Assignments in Teaching Negotiation
Bobbi McAdoo
In contrast to quizzes, student journals are widely used in teaching negotiation, and are one of the most common methods of assessment. However, precisely how teachers use this versatile assessment tool varies widely. In support of her argument that journals have particular value in formative assessment, McAdoo outlines some of the different uses made of this tool by teachers from disciplines as diverse as law, psychology, and political science. McAdoo also analyzes and answers key challenges to using journals for assessment.
5 The Black Box of Student Evaluation: Course Participation
Noam Ebner & Yael Efron
Course participation is definitely at the “subjective“ end of the subjective-to-objective scale of possible assessment methods. It may be surprising, therefore, that so subjective a measure is almost universally a part of negotiation teachers’ assessment methods. Ebner and Efron conclude, however, that just because a method is widely used does not mean it is used effectively. Indeed they find, in surveying colleagues, a startling vagueness at the heart of current use. Picking apart their own prior practice, they discover that they themselves have used course participation in grading in ways that do not survive close examination. From these discoveries they proceed to analyze problems that seem inherent in the method, some of which appear daunting. They modestly conclude that to define “best practices” would be premature. As a step toward greater intellectual rigor and future development, however, Ebner and Efron propose five questions which any teacher interested in rethinking the use of course participation might profitably ask.
6. Using Recordings
Melissa Manwaring & Kimberlee Kovach
Technological progress may have made Sergei Eisenstein’s and Richard Nixon’s methods of recording obsolete, but it is still easy for a teacher to imagine that recording students at work will either become a Herculean endeavor, or somehow intrusive, if not both. That is not necessary. The cost and labor of using videorecordings has declined dramatically in recent years; in many countries, students not only bring the necessary equipment to class with them routinely, but have free access to the necessary software to process the raw recordings. Meanwhile, the nervousness that recording has classically inspired may be declining, as a result of the sheer volume of student-inspired YouTube and other videos made every day. Yet the utility of the recordings has not been diminished by this familiarity. Indeed, it has been enhanced, as well as made more transparent. The authors review a significant variety of the recent research, and place videorecording squarely at the heart of efforts to make assessment more meaningful, more reliable, and more responsive to students’ needs.
III. New Roles for Teachers, New Roles for Students
7. Negotiating the Assessment Criteria
Joel Lee
A series of authors in the Rethinking Negotiation Teaching project have argued for greater autonomy and responsibility to be given to students, in a variety of ways. Lee takes this trend a step further, reviewing the history of student disbelief in the rightness or appropriateness of various evaluation criteria, and experimenting with making an explicit negotiation over those criteria a key part of the educational experience. The negotiation is conducted among the students themselves, with the instructor agreeing in advance to accept the chosen criteria, provided that a specified (and high) percentage of the students have agreed on it. But there is a twist, which we will not divulge here…and students’ outraged responses to the twist have led to one of their best lessons of all.
8. Interviews as an Assessment Tool
Boyd Fuller
Some assessment methods constitute a kind of blunt instrument, in which the subtleties of students’ understanding or lack of it may be evaluated only approximately, with no granular detail. Fuller finds this problematic. If a focus of rethinking teaching in our field has been, and must be, on improving the responsiveness of teaching to students’ individual needs, some form of individuation in finding out just where a given student is weak is essential. Fuller argues for using interviews. In a series of experiments, he finds that three distinct kinds of interview support assessment of different skills, different methods of testing those skills, and different lessons drawn from the experience by the student. One of the three, Fuller concludes, largely failed his goals for it; but the other two are more promising.
9. Simulation Design for Learning and Assessment
Noam Ebner & Daniel Druckman
In previous work in this series, Ebner and Druckman have analyzed the widely assumed (but surprisingly unproven) benefits of role-plays, and concluded that students learn more from designing role-plays than from playing them out. Now, they take the logical next step — explicit assessment of students’ performance in simulation design. Ebner and Druckman have found it both valuable and practical to assess the concepts that students weave into the simulation instructions, the relationships constructed between them, the way the simulation design provides opportunities for integrative or distributive behavior, and the way it encourages particular communication behavior. While they find that “skill development“ may be better demonstrated through participating in the role-plays resulting from the design exercise, assessment of the design phase itself proves particularly useful for increasing students’ understanding of negotiation concepts.
10. Empowerment and Recognition: Students Grade Each Other’s Negotiation Outcomes
James Coben
Negotiating a settlement that satisfies a client’s nonmonetary interests is immensely empowering for students. But all toooften, student excitement about creative solutions to non-monetaryproblems blinds them to an abject failure to negotiate effectively for payments or benefits that their client needs, and that the other side was willing to offer. Or, conversely, they pay out far too much, often by not fully recognizing the time/value of money. How to make these points so they “take”? Coben here describes one solution – a form of peer assessment. After negotiating in pairs and recording settlement summaries which are standardized in format by the instructor (including discounting of all future monetary benefits to present value), students rank the quality of all settlements reached other than their own. The student’s grade for the exercise depends on the average rank their settlement earned from the class as a whole. An added benefit arises when (almost inevitably) a number of settlements result in high grades for both negotiators in a competing pair – providing a powerful and very concrete illustration of Pareto efficiency
11. Making Reputation Salient: Using the Reputation Index with Law Students
Nancy A. Welsh
Would negotiation students act the way they do in simulations, if they knew this might come back to haunt them? Analyzing the consequences of basing part of students’ final grades on objective results they achieve in negotiation simulations, Welsh found a need for something to counterbalance the expected incentives to engage in distributive tactics and “sharp practice.” She settled on an explicit focus on reputation. While the “reputation index” compiled for each student is only a small fraction of the student’s grade, it is based on recent thinking about the value of negotiators’ reputations even in the short term, and gives some reason for pause to any student who contemplates “defecting,” in classic game theory terms. What’s more, it supports the trend toward giving greater autonomy and responsibility to students themselves – because the reputation that counts here is not the student’s reputation with the teacher, but her reputation with other students
IV. Special Tools for Special Contexts
12. Assessing the Adventure
Sharon Press, Noam Ebner & Lynn P. Cohen
A significant series of prior writings in the Rethinking Negotiation Teaching project has focused on adventure learning. Here, the authors tackle a knotty series of assessment problems which adventure learning creates. Experiments at four different universities show that the practical problems are significant, but that they can be overcome. The authors offer a series of tools respectively suited to evaluation of the adventure learning experience itself, its outcomes, or a resulting but separate assignment.
13. Evaluating Email Negotiations
Melissa Nelken
Recent developments in the art of teaching negotiation via email, including contributions from this project, have created the need for robust methods of evaluating students’ success at using this medium. Negotiating via email provides an important learning experience which, Nelken points out, can include points poorly addressed in other negotiation assignment contexts, including students’ discovery for themselves of the limitations of multitasking and the importance of nuance. Nelken reviews the relative strengths, for particular purposes, of a variety of both direct and indirect methods of assessing the process and results of student negotiations conducted by e-mail.
14. Assessing Negotiation Competitions
Nuno Delicado, Horacio Falcão, Ellen Deason, Sharon Press, Shahla Ali, Eric Blanchot & Habib Chamoun-Nicolas
For many students, the first moment when “the rubber hits the road” in their learning is when they themselves hit the road, at a multi-institution competition. Across a remarkable array of institutions and cultures, the authors assess what has been learned about assessing students in competition environments. Noting that by definition, a competition requires winners (or does it? See next chapter) they suggest that students might be judged either by the “outcome” of the negotiation, or, either in combination with outcome or exclusively, by criteria more related to “style and process,” in the way that competitions in gymnastics, figure skating and diving are judged.
15. A Competition Without Winners or Losers? The Ontogeny of a New Negotiation Event Format
Horacio Falcão
Falcão examines the conduct of students in negotiation competitions, and finds much going on that contravenes key lessons we have been trying to teach. Typical incentives in typical competitions, it turns out, are antithetical to much of what we believe is most important for students to learn. Can anything be done about this? Falcão concludes that if the concept of “competition” is central, perhaps not, but if that concept is replaced by “challenge,” possibilities emerge for creative responses that might be equally exciting for students, but have stronger educational value all round. He outlines a series of distinct approaches with different likely advantages, and offers a preliminary assessment as to which might work best for what purpose.
16. A Benchmarking System for Assessment: An Experiment Creating More Transparency in Grading
Boyd Fuller & Sohni Kaur
After analyzing a tendency for existing systems of assessment to imply something that is obviously not true – that all topics taught in a negotiation course are equally important – and realizing that typical systems of grading tend to conflate data in such a way as to obscure which particular subtopics a given student may need more help with, the authors decided to try something new. Their “benchmark” approach provides for two different kinds of benchmarks, absolute and accumulative. Absolute benchmarks determine whether a student has demonstrated understanding of certain concepts or skills, and set the bar for achievement up to a certain grade. Accumulative benchmarks are used for the higher grades, and allow a degree of student choice as to what a given student seeks to focus on. Fuller and Kaur report on initial use of this system, and find that while it requires further development to be effective in encouraging students to prioritize and excel, it already works well for distinguishing which particular concepts students have learned well, and which ones demand either more coaching, or more subtle teaching.
17. Portfolio Evaluation: Kaleidoscopic Insights Into Learning
Michelle LeBaron
Like the benchmark approach, portfolio evaluation seeks to distinguish particular elements that have been learned well from those which will require more work from the student, or from the teacher. Unlike the benchmark approach, portfolio evaluation welcomes, indeed encourages, affective and sensory reflection, rather than an exclusive focus on the cognitive. LeBaron has a strong commitment to a broad view of the field, and her particular use of portfolios gives some room to cultural, physical and even aesthetic elements which, she argues, are often underappreciated in our field. Even in more conventional practice, however, LeBaron contends that portfolio evaluation addresses a series of formative and summative assessment needs – particularly fostering double-loop learning – better than standard approaches have.
V. Epilogue: Assessing Ourselves
Noam Ebner, James Coben & Christopher Honeyman