Specifications Grading (and IBL)
Written by: Dr. Christine von Renesse.
Why change my Grading...?
There were several things I didn’t like about grading...
- I didn’t like agonizing about the exact number of points to give. Is this worth 18 or 19 points? Is it “fair” compared to other students?
- Grading homework felt more summative than formative to me and I wasn’t sure if students learned from my written comments.
- I was also confused about the fact that a “B” could look many different ways. A student who did great in participation and homework but really bad on all exams could end up with the same grade as a student who got A’s on the exams but never talked and never handed in homework. How could I guarantee that a student passing my class knows particular facts, skills, methods, concepts, and improved in the meta skills (like persistence)?
- Recognize and challenge their own beliefs and feelings about mathematics.
- Work well with other students.
- Find work partners that have similar speed, curiosity and learning edge.
- Gain social competence in negotiating different ways of thinking.
- Persevere when it gets difficult and frustrating.
- Be ok making mistakes and learning from them.
- Be more confident in doing mathematics.
- Admit not knowing and ask questions to learn.
- Be prepared and ready to try/learn when coming to class.
- Realize that mathematics is more about creating and deep thinking than memorizing
procedures. - Be curious about mathematics.
- Recognize how they learn best and put a plan into action.
- Enjoy the challenge of reasoning.
- Only accept mathematics that makes sense to them.
- Communicate mathematical ideals in writing and orally.
Then last summer I read the book “Specifications Grading” by Linda B. Nilson. While this book doesn’t focus on mathematics it gave me lots of ideas and the goal to change what I am doing.
Ideas that were new to me...
What if a D-student doesn’t just know “everything” on a D level, but I get to decide which content or meta outcomes a D-student really needs to satisfy in my class? So here is my first revelation: In specifications grading you grade almost everything on a pass/fail basis (using growth mindset language: passed/ not yet passed). No more partial credit, either they got it or they don’t. Of course you need to write good rubrics to communicate what a “passed” looks like. I get to decide which exact items on my specifications list a D-student has to pass. Sounds too strict? Well, there are many ways for students to revise their work as you will see below.
Sample Syllabus
Here is a syllabus from my Spring 2018 calculus 2 class to give you a sense of what such a syllabus and one of the rubrics could look like.
Calculus 2 Syllabus
Homework Stories Rubric
The book suggested to map all the course outcomes to the different assessments that I use. I was used to just thinking about content. (Actually outcomes used to be an annoying item for me that one has to put on the syllabus). My next revelation was that the meta-goals from my class need to be explained and mapped to assessments as well! I have always included journals and used inquiry and group work in my classes but I had not systematically mapped these meta learning goals to the ways I assess. To do this I had to be more nuanced in formulating my meta-outcomes:
Meta Objectives:
Upon successful completion of this course students will:
Content Objectives and Exams
To improve my content outcomes I used the ideas of Mairead Greene and Paula Shorter, see their paper "Conceptual Understanding Weighting System: a targeted assessment tool". They suggest to differentiate between facts, skills (one step procedures), methods (several steps procedures) and conceptual understanding. Using these categories also really helped me improve my exam writing skills. I used to ask a few “big” questions that assessed across all categories. If students didn’t understand the larger question they often didn’t have a chance to show what they did know. I was often surprised that the exams didn’t seem to reflect what I thought the students were able to do. Using the categories has allowed me to get exam results that are relatively close to what I expect from observing students during class and from assessing their homework.
sample exam calculus 2 .
So which assessments do I choose?
Of course it depends on the class. Above you saw that in my calculus class I used: journals, homework stories, webwork assignments, 2 exams, a productive mistake and participation.
But for my class for future elementary school teachers I included reading assignments (which included looking at students working in videos and written student work) instead of procedural practice problems. Here is my Syllabus to Patterns, Algebra and Reasoning for future elementary school teachers. And in my mathematics for liberal arts class, I used a project and a portfolio instead of exams.
Big Lessons learned:
Growth Mindset Language:
You need to have the “not-yet” mindset language well established in the beginning of the semester so that students don’t see a “not passed” as a punishment but as an opportunity to grow. If you don’t do this you get push back from students. My first journal prompt helped me accomplish this: Journal 1 Assignment and Rubric.
Student learn more. Definitely.
A "D-student" now looks much better than a "D-student" a year ago in my class. I feel they actually deserve to pass the class and have a chance in the next class. But not just the weaker students learned more. All students had to work harder and more consistently to earn their grade.
Grading "homework stories" and revisions:
The book promised that I would spend less time on grading. Unfortunately this wasn’t true for me: while grading each assignment was much faster, I had now many revisions to grade. So grading wasn’t less time consuming but it was much more meaningful for me. Since student had to revise their work in order to pass the class or get the grade they wanted, they actually read my comments and tried to make sense of them. They learned!
Here is an example of an Excellent Homework Story by Morgan Shia. You can see how personalized the story is which makes it easy to read and grade. I can really see what Morgan is thinking and (if necessary) give very specific feedback.
Grading revisions lead to another insight on my end: Students probably never read my comments or made sense of them before I used specifications grading. Really. Watching students try to make sense of my comments “live” was eye-opening (and sad). The good news is that my comment writing and their comment reading was improving. A lot.
Office Hours:
Did you ever have the problem that the students who need to see you are not coming to your office? I did, but now I have a ton of students in office hours all the time. There are still some that don’t come or show up too late during the semester but I had 5-15 students in my office for 3-6 hours every week. And since I don’t just give them the answers during office hours either, they stayed with me, working hard on their problems and checking in with me now and then.
Exams:
Exams are difficult for me to integrate into the specifications grading – especially since the final exam doesn’t have a revision chance. I need to include some summative assessments though in which I only see the student’s work without the help of peers, internet, or tutor.
I am considering taking the average exam grade to take some pressure off. Doing two midterms and no final exam is also a possibility.
Participation Grades:
I learned that preparation needs to be an aspect of my grading since otherwise some of the less hard-working students just show up to class and try to catch up. Well, they actually still do this, but now they lose one grade level on the participation grading scale.
I also learned that some very fast students may deliberately work at a slower pace because they know that I have extension questions ready for them. To counter this problem, I included “learning at the learning edge” into the rubric. Some of my students who pulled this trick on me in calculus 1, were now working much harder in calculus 2.
Assessing Meta-Outcomes
To assess which meta-goals the students were working toward, I used journals and participation.
While I always asked students to write journals, I am now thinking more clearly about which aspects of their learning I want them to think about. Instead of a generic autobiography or reflection of their learning, I provide them with small video clips and then ask them to make connections to their learning. For example, I let students watch the movie Hidden Figures, short clips by Vi Hart, Jo Boaler, and Carol Dweck, Ted Talks (e.g. about mindset, perseverance or mistakes) and videos that show applications of mathematics or mathematical thinking.
Presenting productive mistakes.
Thanks to a great talk by Milos Savic, I included productive mistakes in my specifications. Every student had to present one productive mistake to the class during the semester. Students had to briefly check in with me before they presented so I could decide if it really was a productive mistake. The presentations were very informal and short. The audience was asked to reflect back what they had just heard or ask clarifying questions about the mistake. While I had only mathematical mistakes in mind, some students asked to present other mistakes like “not taking good enough notes”. These presentation turned out to be helpful to other students as well. I had to be careful though that these didn’t turn into an “easy way out”.
An activity to make sense of the grading system:
In my first semester of trying the new grading system I only told students about it without assessing if they actually made sense of it. It turned out that most of them didn’t understand it or didn’t take it seriously enough and I had to work hard during the semester to help them catch up. In the second semester I started with a syllabus activity and most of the students did not struggle to understand the system. I heard the inevitable “What?? You can fail the class just because you don’t hand in one journal??” right away and then it was understood.
Handing in work on time:
Once students realized that they can revise work, some started not to hand in work on time anymore. For my next specifications grading round I will make sure to include that you can only receive a “good grade” if most of your work is handed in on time.
Handing in and grading work “online”:
During my first semester, students could hand in work on paper or on blackboard. They were supposed to turn in revision with the old versions, but of course old versions were lost, or students claimed I had not returned work, etc. So the second time around, I made all work due on blackboard with a setting of “unlimited revisions”. Students that preferred to hand-write their work could use a scanning app to create pdfs, other students handed in word documents. I tried to comment directly on blackboard (since this is easiest) but at times I downloaded the documents and made comments into the documents before I uploaded them again. I find it much easier to type comments now: I write more since I can type faster than hand-write, there is no problem to fit comments into tight margins, and I can revise my comments if necessary.
Questions and Reflections:
1. Enforcing Meaningful Revisions
Sometimes students revised some of the (easy) problems but avoided fixing the actual gaps or mistakes. Can I somehow force students to make revisions more meaningful? Maybe I should add: “If you don’t address all my comments in your revision or ask me your questions your story grade is permanently zero.” Or is this too harsh?
2. Grading Webwork Problems
While students probably get help from the internet to pass these webwork questions, it gives them a chance to practice without me having to grade lots of procedural problems. But what does it mean to get an A for webwork activities? 90% as an average of all webwork assignments? 90% on each webwork assignment? How about 100% on each webwork assignment? I want to know that all students work on all webwork assignments not just some, so I can’t just use the accumulated percentage. Maybe a D means 65% in each one? This means that a D student shows less methods practice than an A-C student which makes sense in terms of learning attitude. But I can be sure that they have practiced at least some of each.
3. Essential Homework Stories
If I want to guarantee that students learn specific outcomes, I have to make clear which homework stories are essential for passing and which are necessary for better grades. Now that I have taught calculus 1 and 2 once using specifications grading, I can look through the stories to decide which ones should be essential. I think that this may be different in future teacher classes or mathematics for liberal arts classes. Not all classes will require “essential stories.” The specific content may not be as crucial as the process of proving “something”.
4. Extension Projects
I found that even in my second semester some A students could have used more extension projects. Some of the A students had to work less than others because they either had more prior knowledge or worked faster. How can I make them work/learn just as much? I usually bring extension questions every class, so that faster students don't work too far ahead in the material but work on other interesting, related and challenging problems instead. But how do I motivate A-students to want to work on these challenges? I have the “learning edge” in my participation rubric now and it helped some but I think there could be something even better...
5. Giving Up Early (or not)
Some students see that they cannot achieve a higher grade anymore because of their grade on the midterm exam. So they stop working hard. I allowed three students to retake the midterm (different version of course) to show me that they now mastered the content because they wanted to go for a higher grade. This seems like a good idea (but was also more work on my end).
Student Feedback:
I asked student on their final evaluation about the grading system. In hindsight this was not a good time to ask them since they were all very anxious about the final exam, final grades etc. The feedback I received was mixed. About 50% of the students liked the grading system, 50% did not.
Since this grading system forced students to work more, I am not surprised that it wasn't popular.
It is interesting that students didn't complain (much) during the semester, but chose to complain on the anonymous evaluation instead. But maybe they could tell that I was not going to change the system anyway. I constantly repeated why the system is so awesome ;-)
I also wonder if the students realized that they learned significantly more than they would have with traditional grading? (Why did I not ask about that??) I am planning to collect more (earlier) feedback next semester to find out... Here are some excerpts of diverse comments:
Future Elementary School Teachers:
“I liked how we knew what was expected to receive a particular grade. However, I don’t agree if you get a particular score on a test, you can only receive the grade you got on the test.”“I don’t think it’s fair to have participation knock your grade down if everything else is an A. I also think the final should be mixed in and not be as heavy.”
“I like being able to resubmit work and make corrections because it leaves room for a second chance to pass the class.”
“I think the grading method is stressful. Even if you try really hard with this course you can still end with a lower grade which I don’t believe is very fair.”
“The course load was extremely demanding especially compared to other courses.”
Calculus 2 Students:
“I think this is a fair way of giving students the opportunity to learn from their mistakes by revising.”“You made it hard enough you have to EARN a good grade, but not too hard where it’s impossible.”
“I do not like pass/fail on the homework stories. Redoing them multiple times seems a bit of a waste. Pass/fail on the test is awesome though. Really incentivizes you to learn!”
“The grading system allowed people who typically struggle with math to do well and succeed.”
“I did not like the pass/fail” at all. I didn’t mind for the anti-derivative test, but for the homework stories I hated it, the fact that I have to keep revising them until perfect just to get credit is horrible to me.”
“The grading helped keep me on track and know kind of where I stand in the class. It helped me to understand math more and want to understand it.”
“I think it’s fair. If you want to get a god grade in the class you have to put in the effort. People who don’t do so well in the class don’t put in the effort in their learning of the material.”
I also suggest Kristopher Williams's paper: "Specifications-Based Grading in an Introduction to Proofs Course" in PRIMUS (2018). His specifications are very different from mine!