Case study - Working with markers

 

Working with markers

Context

Shirley Earl coordinates a first year Business course with over 400 students. With that number of students the amount of time spent on the administration of assignments had become considerable. Last year Shirley had started to use the WebCT assignment tool for assignment submission. This had made her life much easier – all assignments arrived in one place and were automatically time-stamped. Still, there were a few aspects of using the WebCT assignment tool Shirley wasn’t fully satisfied with.

Because of the large number of students Shirley employs markers, usually about six per assignment. WebCT doesn’t support her in coordinating the work of these markers. Shirley uses marking sheets that contain the details of the marking rubric. For each assignment her team created an electronic copy of the marking sheet (entering the student id and name manually …) and provided individualised comments. When returning these marking sheets via WebCT one has to be very careful to send the right document to the right student - mistakes can be made quite easily.

Shirley heard about a new tool called WebCTConnect that supposedly addresses the issues she is facing. She set out to explore this tool.

Exploration

The first step was to check out the WebCTConnect website (http://www-ist.massey.ac.nz/marktool). Shirley stepped through the online tutorial and saw that there was a manual available as well. The download and installation of the tool worked without problems. One thing Shirley liked very much when thinking about using WebCTConnect was that nothing changes from the students’ perspective – students continue to submit to WebCT and still receive their marking results via WebCT. But, just to make sure, Shirley used WebCTConnect to download the assignments from last year’s course. She setup a trial marking scheme and marked a few of last year’s assignments. In every WebCT course there is a record for a dummy student. Shirley returned her pretend marking results to WebCT just for this student so she could check in WebCT how things looked (to be able to do this Shirley used the feature of WebCTConnect that allows you to add an assignment even if the student, like the dummy student, hadn’t submitted via WebCT).

Next Shirley explored the features around working with markers. She could again test these features by using the data from her old course. There was no danger in inadvertently changing any student records as all her explorations were contained in her own copy of WebCTConnect. Shirley decided to go ahead and use WebCTConnect with her large class. She liked the features of WebCTConnect and knew that she could simply go back to just using the WebCT assignment tool, without any impact on her students.

Getting the markers up to speed

Just before the assignment deadline Shirley met with her markers. Most of the markers had helped Shirley the previous year and were already familiar with the marking of electronic assignment copies. Shirley reminded her markers of the requirement of storing the assignments on a password protected drive. She then demonstrated WebCTConnect to the markers and discussed how the marking team was going to use the features of WebCTConnect to electronically exchange documents for coordination and quality assurance.

The markers went away to install WebCTConnect on their own computers and imported the marking scheme Shirley had emailed them.

A marker’s experience of using WebCTConnect

Jim was one of Shirley’s markers. In WebCTConnect, Jim could clearly see which assignments Shirley had allocated to him. For each assignment he marked, WebCTConnect immediately had a personalised copy of the marking sheet ready, eliminating the time spent previously on copy-and-pasting. Jim was really pleased about two other features of WebCTConnect, the in-built list of frequently-used-comments he could expand on as he went along and a field for comments to himself for each assignment, which he sometimes used to remind him to go back and check on some details. At the end of each marking session Jim simply emailed his marking to Shirley. As these emails contained only the marking information and not the actual assignments this didn’t take long at all.

Shirley’s experience of using WebCTConnect

Shirley was very pleased how WebCTConnect supported her in monitoring the work of her markers. She had asked all markers to start by assessing the same three assignments and email her the marking. In WebCTConnect Shirley could compare the marking of all three assignments and pick up on some different interpretations of the marking scheme right at the beginning of the process. Receiving the frequent marking updates allowed Shirley to track progress and do consistency checks across the markers. All of the communication and exchange with the markers was done electronically. This was a big advantage as some of the markers were off-campus and preferred to work after hours, making frequent face-to-face meeting difficult.

After all marking was finished and she was satisfied with the marking quality Shirley uploaded all results and feedback from WebCTConnect to WebCT. She could do that with just a few clicks – WebCTConnect automatically sent the right feedback sheet to each student.

The students’ experience

The students didn’t even know that Shirley and her marking team were using WebCTConnect. All the students saw was the interface to WebCT. They did appreciate submitting via WebCT and getting their results back the same way as this was fast and convenient. Plus, they liked the comprehensive feedback they received, starting with a clear outline of the marking criteria and sections of individual comments linked to those criteria.

Summary

Adding WebCTConnect to the WebCT assignment tool helps with additional efficiency gains and makes it practical to implement schemes for ensuring marker consistency. The students only interact with WebCT.
This approach has helped the lecturer by:

  1. Being able to allocate markers to assignments.
  2. Electronically exchange marking information within the marking team.
  3. Monitor the marking process.
  4. Return individualised feedback sheets for the whole class within a few clicks.

(WebCTConnect provides further advantages, like dealing with group assignments, automatically unzipping files and exporting all marks and comments, that have not been discussed in this scenario.)

< Back