Pilot of electronic plagiarism, interim report August 2004
1. Background
Over the last year the institutional framework for dealing with student plagiarism has been rolled out, leading to a growing need for the use of computer based detection methods. These detection methods have been trialled with8 members of staff in 7 subjects over the past year to provide an overview of the work needed before computer based plagiarism detection can be expected to feasibly run campus wide.
The targets of this trial have been to determine the resources required, both in terms of staff time and in staff training, to determine likely procedures required for use of the system, and to evaluate the available software.
This initial pilot was conducted over a period from November 2003 to July 2004 and took up one day a week of my time. Other staff involved and the numbers of students who participated via these members of staff are listed below:
| Fiona Measham, Applied Social Science | 16 |
| Tamzin Doggart, Independent Studies | 2 |
| Frank Martin, Biology | 71 |
| Carolyn Downs, Independent Studies/History | 9 |
| David Simm, Management | 1 |
| Steve Quayle, Engineering | 47 |
| Chris Park, Geography | 15 |
| Eric Evans, History | 7 |
For the purposes of the trial, staff were taken through the process of submitting their collected work to JISC and interpreting the results returned.
2.
Procedure
The vast majority of work submitted for marking by students is done on paper, and at the start of this year there was no system for collecting electronically submitted work in place. For the purposes of these trials a simple web based system was set up, which a few of the staff listed used successfully and which could potentially be expanded to incorporate larger numbers of staff and students, although this would take time. Other staff chose to collect work via email, which may be more time consuming, but is more familiar to most users – both staff and student. It is also possible for students to submit directly toJISC (the plagiarism detection web site) although this has significant set up overheads.
3.
A
spects
of the trial
An important aspect of the trial that emerged was the need for students to give written permission for their work to be submitted and anyone becoming involved in the pilot in 2004/05 will need to ensure this occurs at the beginning of the year. In due course a system will need to be developed to seek all students’ permission at the beginning of each academic year.
4.
Processing Results
One aspect of the cost involved in the use of the computer based detection system, is the time taken for work to be processed. This should include set up time, processing time and interpretation time. Based on the trials conducted for this pilot, it can be estimated that the average user after training would take approximately 30 minutes to submit work (this doesn’t include collection time, which may vary dramatically depending on the method used) and then interpret the results. However, although as a more experienced user I may usually submit work more quickly, I have also through the course of my plagiarism work in computing occasionally experienced courses which took up to 1 day to submit correctly because of certain limitations ofJISC .
5.
Staff Training
Needs
The staff who volunteered to participate in this trial were from a range of backgrounds, although it should be noted that they were often the more computer literate amongst their department al colleague and this may skew results slightly. The majority of the computing knowledge required in order to use the JISC service is basic computer knowledge, but is not necessarily knowledge that staff will already have. Staff who regularly use a computer and are reasonably comfortable with the technology would probably require approximately 30 minutes technical training, with an additional 30 minutes training in the interpretation of results from JISC .
Further staff time may also be required for installation of software necessary for use of this electronic submission system. For many departments staff themselves will not necessarily have the access to install software themselves and so this may require support staff time.
It should also be noted that although after 30 minutes training most staff will be able to use theJISC system competently, there may need to be technical help readily available so that problems such as those I encountered with JISC ’s limitations do not either take up disproportionate amounts of staff time or deter staff from using the JISC system at all.
6.
Summary and Conclusions
Although it has been difficult to engage staff in the trial , once people have tried out the system they have as a whole been highly interested in the results, even when no plagiarism was detected. With greater awareness and participation from staff, and with time put into training, JISC plagiarism detection could be put into use quite successfully to support the plagiarism framework.
At the start of next term, two plagiarism workshops will be held to provide a greater awareness to staff, with the aim to encourage a single participant from every department. From these workshops we hope to offer a plagiarism detection service, with each department in attendance subscribing to use the facility with a single class, whether they chose to simply interpret results or to step through the entire submission process. In this way we hope that each department will have some expertise in and awareness of the JISC system. At the end of this second year of the trial we will be in a position to make concrete proposals about the implementation of the JISC service across the institution.
Angie Chandler, Computing Department
Terry Wareham, Centre for the Enhancement of Learning and Teaching
August 2004
Appendix
– Remaining Problems Encountered with the
JISC
plagiarism detection
software
From an interpretation perspective, whilst in earlier versions it was difficult to even submit groups of students, the largest remaining problem withJISC is the method by which it displays the results. This is because, although easy to understand, the results returned by JISC (a report on each document submitted) are only distinguished by a simple colour code. Each report is given a colour code (one of five) based on the percentage of plagiarism within, but the green colour code is such that it includes all reports with between 1% and 25% plagiarism detected. This is likely to include nearly all students in a class and requires the member of staff to individually check each document. The next version of the system is expected to rectify the problem by listing in percentage order, thus allowing staff to concentrate their interpretation of results on those which look to be the most suspicious.
Other issues, such as removal of quotes, and possibly references, are less likely to be dealt with, and may only be relevant to certain disciplines. An additional Java language program is under construction which is expected to address these problems and provide department level control of the results given, rather than being forced to rely on changes toJISC .
From a submission perspective, although much improved, there remain a few limitations placed onJISC which are unlikely to be removed. These are all in relation to the submission of large files, whether it is a large document submitted by a student or a particularly large zip file with a number of students’ work in (zip files are the easiest means by which a batch of work can be submitted).
Zip files, although unfamiliar to a significant proportion of staff, are an easily created and convenient means of collecting together a batch of work, and in the case ofJISC are the only means by which a group of students’ work can be easily submitted. However, once the zip file is larger than JISC will permit , staff will suddenly be required to manipulate a single large zip file into a number of smaller ones, which can prove quite fiddly as sizes can be unpredictable. The difference between straightforward creation of zip files and more complex manipulation of zip files is significant and is likely to prove to be a problem for less experienced staff, although the problem is only likely to occur in exceptionally large classes or with unusually large and pictorial pieces of work.
Similarly, with longer pieces of work such as projects and dissertations, it is likely that theJISC file size limit would be reached. This can be counteracted to some extent by converting Word documents to plain text (which can be done as a batch) but there is also a character limit. If the character limit is reached, in order to submit the document it will be necessary to either split it into parts or take a sample, again increasing the complexity of the submission procedure.