In at the deep end of essay marking

Martin Gough

Background
Rationale for the Topic
How to Use a Criteria-based System for the First Time

Article

This is a paper designed to provoke thought and discussion about good practice in marking essays, applicable primarily in Philosophy but intended to serve as a model for argument-based writing in other subject areas. The system I promote is described in section (3) if the reader wishes to skip to that part. But my claims in the earlier sections are by no means uncontroversial.

1. Background

I reflect at length on my experiences as a new tutor in a project I completed in February 1996 entitled 'Grading and commenting on written essays in Philosophy (with particular reference to marking for novices)'. The project aims in part to give practical advice for new tutors, particularly those in the Philosophy subject area, and outline good practice more widely. The project was the additional distance learning component of a course I attended in July 1995, 'New Approaches To University Teaching: A training course for postgraduates and others planning an academic career', a residential M-level course run by the Unit for Innovation in Higher Education at Lancaster University, convened by Dr John Wakeford of the School of Independent Studies. Costs of attendance were met partly from the Open University Flexible Fund, partly from Leeds University Union Postgraduate Students Representative Council and the rest self-funded. There was a variety of workshop and practical sessions on various aspects of teaching in Higher Education. The description of staff development provision for postgraduates at Lancaster University, in Stephen Pritchard's report in The Independent (Thursday 25th January 1996, section 2, p14), reflects the content of the course.

As further output arising from the course, I wrote a chapter entitled 'The Postgraduate Philosopher' for the volume In at the Deep End: first experiences of university teaching, David Allan (ed.), Lancaster University Unit for Innovation in Higher Education, in collaboration with the Times Higher Education Supplement, (1996). This further piece acknowledges pedagogical issues facing particularly the new teacher but concentrates more on ideal institutional provision in Higher Education for supporting the development of new teachers into good teachers, albeit from the perspective of Philosophy. This piece draws also from the National Postgraduate Committee's (NPC) pioneering Guidelines series (1993) pamphlet Guidelines on the Employment of Postgraduates as Teachers. This pamphlet in turn was a follow-up to the piece written by NPC officers for the AUT Bulletin (1990), 'Postgraduates teaching undergraduates: the blind leading the blind?'. I fed the work done to produce these publications into for instance the UK Council for Graduate Education working group on which I sat, which produced the report in booklet form: Preparing Postgraduates to Teach in Higher Education (1999). Certain other ideas mentioned in my piece for the In at the Deep End book are elaborated in my article, "An Alternative Future for Postgraduate Research in the Humanities", Journal of Graduate Education, vol.1 (no.4, Spring 1995), pp127-133. The title of my paper continuing on below acknowledges the title of the book.

2. Rationale for the Topic

I completed the Lancaster course project in what was my sixth year of teaching Philosophy in Higher Education, encompassing all three levels of the undergraduate programme and Masters level, including authoring my own 3rd-level lecture course. I was being employed in the Departments of Philosophy at the Universities of Leeds and Nottingham and as Yorkshire Regional Course Tutor for the Open University. I had formulated my views about how to mark essays at the start of this period, when first presented with the challenge of the first essay round from my first cohort of students. Five years on I was in a good position to reflect on strengths and weaknesses.

My first introduction to paid teaching work was on the 1991-92 First-Year course in the Department of Philosophy at the University of Leeds. Most of the students are not trained at all in Philosophy from their school studies, so a general challenge is to make the subject accessible, as well as interesting, whilst instilling the culture of the discipline particularly into those students who may have no, or the wrong, idea about the subject when they start. That is a challenge at the level of tutorial seminar classes. The Leeds Department had put substance behind their rhetoric that classroom contact and group discussion is an essential part of learning Philosophy, by attaching assessment weightings to classroom performance by students, for example thoroughness and clarity of oral presentation of topics. The tutorial structure was bolstered also by experiments such as proctorial seminars. The Department's home grown pamphlet reviews these experiments, alongside surveys of literature on the area and actual practice in other UK Philosophy Departments at the time: George MacDonald Ross, Jim Parry & Martin Cohen, Philosophy & Enterprise: The Implications for Philosophy of the Enterprise in Higher Education Initiative, (Department of Philosophy, University of Leeds, 1993).

This pamphlet offers advice on developing learner-centred approaches to teaching at the expense of the more traditional didactic, "academic" approach. Thus it is a useful contribution to the best means of developing the experience of Higher Education for undergraduates. However, if only due to the boundaries of its brief, it is reticent on the issue from the point of view of the interests of for example postgraduates, as typical new tutors, except to acknowledge that part-time staff such as postgraduates are a valuable resource who need and deserve provision for their development. This is not to say that effort was lacking at Leeds in preparing new tutors for their new responsibilities, quite the contrary. The University's central staff development unit was also at this time offering generic workshops on small group teaching, useful for those (such as new tutors) who need tuition and practice in developing their competence, although requiring subject-specific guidance that only Departments can give by way of follow-up.

Whilst more emphasis was now being placed on teaching competence in and assessment of class-based work in Philosophy at Leeds, as with most Higher Education courses past and present the greater part of the assessment weighting for students was on written work. The students knew this and were thus most concerned about doing the right thing in this area. And yet direction given to tutors in terms of training in this area was lacking in comparison with other areas. I suggest that one reason for this has been a general secretiveness about individual practice, as if markers (albeit not without exception) at all levels of seniority are in fear of being exposed as unwittingly doing something wrong.

More positively, markers are presumed to have integrity and competence in judging standards of written work by students. I suggest that this presumption is not warranted for absolute novices. With a system of second-marking or monitoring by a more senior teacher than a postgraduate in the department, and ultimately the External Examiner for relevant modules, at the most general level whether we are considering postgraduates or senior lecturers as first-markers, there is no intrinsic problem of ensuring ex post facto quality control in the form of uniformity of numerical grades for a given standard of work. Objectivity in assessment is best determined, if not entirely constituted by, concurrence in grades awarded by difference markers for given pieces of work. Nonetheless, there is a need for substantial guidance for novices ex ante facto.

There is additional assistance in form of the valuable "Grade Score System" guidelines, whereby a standard numerical grade is advertised as representing a definite point on the Degree classification scale such as experienced markers can determine with a judgemental "overall impression", and at the same time not encroaching upon different individual practices of determining that overall impression. An overall impression grade is not lacking in objectivity in the important sense by virtue of it being just an impression on reading through an essay in a reasonable time. Apart from the fact that the Grade Score System was developed at Leeds too late to assist those alongside me in my first years of teaching, it was also insufficient guidance anyway on its own for most new tutors who were stuck on just that requirement to quantify an "overall impression".

To illustrate, one of my postgraduate tutor colleagues (more experienced by one year) in the Leeds Department claimed that they could judge the overall mark in one decision on reading through the essay, with implicit regard for the different criteria of assessment (as I provide below). They had not up to that point articulated the criteria explicitly, although they did not find any criteria inappropriate. I do not and did not then doubt their ability to judge the overall grade as accurately as could be done, even though failing to consider all the criteria might lead even the senior experienced marker, perhaps through good habits gradually slipping, to develop an implicit bias in favour of some criteria at the expense of others. And I might add that their wisdom has allowed them veritably to sail into a strong position on the career ladder of the UK Philosophy establishment, so they shall remain nameless. The problem I had with my colleague's approach was that it appeared that they would not be able to explain adequately what they were doing, whether they might be trying (in what would be an appropriate spirit of co-operation) to do so for novice tutors or, perhaps more seriously, for the students whom they were teaching. The way they talked at one point sounded almost like they were able by some sort of meditation to savour the essence and value of an essay just read and discover a number ex.100 as if it were crystallising in their mind's eye which they would award to the essay. In sum, they knew how to grade an essay but not how to explain it in terms of good practice.

So there is still room for preparing the new tutor, at least, if not re-training more experienced tutors, for a task which carries so much responsibility in terms of determining the fates of students, who are seeking to pass courses as conditions of attaining a Degree as one of the main aims of attending university. Monitoring marking after the event is one thing but knowing how to set about marking beforehand is more important still. This is so whether or not the tutor is apprised of the guidelines of the "Grade Score System": we may be satisfied that 68 is to be awarded for a high 2(i) but not first-class grade piece of work but not know what a high 2(i) grade is exactly. Better initiation into the work of marking would be for the most efficient use of the resources of the tutors themselves, with consequent peace of mind at the start when the full realisation of the task of marking in "reasonable" time approaching a hundred or so essays with explanatory comments has sunk in; and it would be for the sake also of the resources of the monitors who must tidy up the marking process afterwards, to ensure quality control and the reasonable satisfaction of undergraduate students.

What I offer in the rest of this paper is not a fixed set of procedures and failsafe formulae for determining exactly the grades of written work by students. Just as the written word, generally speaking in the form of sentences and larger units of meaning such as whole essays, is continually open to reinterpretation, so are formulae and other rules themselves, particularly in the sense of latitude in the way that such rules can be applied, without hope of absolute control. Rather I offer a framework for the new tutor to obtain a foothold in the very idea of approaching written work from the point of view of assessment and to develop their practice from this starting point. I shall limit my study to written essays as paradigmatical of written work, if only because objectivity in criteria of assessment are prima facie the most elusive here. I intend that my approach to Philosophy essay writing would continue to be a useful rule of thumb for efficient and accurate marking for tutors as they become more experienced, and indeed useful for those who are already experienced. And I anticipate that it is applicable in principle, with adaptation, to other disciplines, albeit that my own first-hand experience of essay marking has been for Philosophy courses only.

What follows are annotated versions of grading and commenting schedules I have used for various courses, with historical background as to how they worked.

3How to Use a Criteria-based System for the First Time

With my task of marking upwards of eighty essays in "reasonable" time fast approaching I spent not a short time constructing a schedule for grading and offering general comments which could provide a usable basis for getting through such a task with as mechanical an efficiency as is possible. Setting it out in the form of a word-processed document, which could be edited before printing out and returned to the student with their essay, created an indispensable tool. I created a lengthy key of abbreviations I would write in the margin of scripts, each abbreviation linked to five main criteria under which the essay would be judged separately. The criteria were separate in the most absolute sense in this context: each received a different mark out of a sub-total, where these sub-totals sum to 100. It might seem at first sight as though the essay was being taken in five bits physically sectioned off. It was not like this: rather the whole essay was considered five times over from five perspectives defined by the criteria. The criteria and their weightings were:

ARGUMENT:	/45
(answering the question properly or fully	[ /10]
-- introductory outline	[ /05]
-- clear structure to essay	[ /15]
-- use of examples & expansion of points	[ /15])
KNOWLEDGE:	/25
(any mistakes in referring to the course subject matter?
-- enough detail with respect to subject matter?)
ORIGINALITY:	/05
RELEVANCE:	/05
(the scheme of marking is different here, more simply negative whereas the above is based on more positive reward, since I knock a mark off for each irrelevant point up to a specified maximum)
STYLE:	/20
(English, clarity of writing, use of references: I knock a mark off for each offence up to a specified maximum)

Calculation of the Grade

The total grade for the essay has to be an exact mark out of 100, for the purposes of summing with other assessed work for an overall module grade which can be used as the system demands to be put towards determining Degree classifications. In that spirit, aspects of the essay can have their own exact mark and the essay is simply the sum of those marks. There need be no overall impression of a whole essay which is used to determine its mark. This analytical approach to essay grading is preferable for those who need to know how to set about grading, if we are to give them any more instruction than simply "Well, read it!". Having to deal with separated aspects of an essay in turn is easier, since the marker can focus on a more clearly defined concept which is the respective criterion. We should not assume that everyone new to marking essays can hit upon the right grade as an exact percentage just by reading the essay and gaining an overall impression.

Having said that, I judge that some people as novices are and some others always have been able to judge the grade of a whole essay with an overall impression. And all of us should be able to develop the skill. Being able to do so is advantageous in two respects. One respect is that it provides a check on the more analytically derived grade, in case your calculation comes out as a whole class or more above or below the rough class which is suggested to you prior to calculation. There is a strong case for a compromise between the two grades in such cases. I can reassure the reader that my experience is that such cases are rare and that the calculated grade tends to confirm the overall first impression one such as it is made. Rather than showing the redundancy of the analytical approach, even on this consideration alone to the exclusion of other ones, however, for the novice marker at least, it is a useful way to build up confidence in your ability to assess such essays with an overall first impression. The other respect in which the overall impression is advantageous is in the context of a marking task which needs to be done as quickly as possible, such as grading without the need for commenting, for example "exam" marking: in the Leeds Department First-Year Course, as in some others, there were assessed essays instead of a sat exam but where the convention remains that the scripts should not be annotated, so that an unclouded first impression on the part of the second-marker can be better ensured.

I wish to concentrate upon essays where a significant amount of commenting has to be offered for the student. Here, particularly for novice markers, and especially with poor essays, the task can take so long that any overall impression developing is simply lost in the duration. In the schedule above, I include the sub-total mark I assigned for weightings of the different criteria and sub-criteria of assessment. For the 1991-92 year I showed the exact mark they received out of the sub-totals for each criterion as a whole, ex.45 here for ARGUMENT. Rather than show the students the exact mark for sub-criteria under each criterion, in each of the [ ] under e.g. ARGUMENT I listed one of the three vaguer grades GOOD/O.K./POOR, but keeping a private record for myself of the exact mark.

The marks for ARGUMENT, KNOWLEDGE and ORIGINALITY are calculated on the basis of how well the student seems to have done (under each sub-criterion where applicable), a "positive" assessment, as it were. In contrast, the marks for RELEVANCE and STYLE are calculated more on a "negative" penalty basis. So, for STYLE, it is quite possible to get 0/20 without it being an entire write-off as regards how well it is written, just due to 20 or more "crimes", as it were, against written English in the essay. A rule of thumb for working out the penalty is by adding up how many margin note abbreviations you as the marker found that you had made.

This works too for the sub-criteria under ARGUMENT and KNOWLEDGE, e.g. a rough penalty taken into account for the number of instances of respective margin abbreviations and other respective margin comments which are critical of the passage. This is still a useful rule of thumb but one which should not govern the assessment under these criteria as respective whole units, since it is important with these ones to consider how well the student has done qualitatively in the specific respect rather than merely quantitatively how many times they have slipped up. For example, for KNOWLEDGE, extra credit should be awarded for good use of sources on top of what can be picked up in classes. I tended to give full marks for ORIGINALITY if the student showed a hint of having tried to approach the issue in their own way, as opposed to relying on the facts and arguments in the form that they are presented by the course.

The size of the weighting for STYLE might be queried as too excessive, 20% for non-philosophical points, that is: many of my students were unhappy that they lost marks for what they felt was not the business of a tutor whose job is to teach them Philosophy. I would defend myself by warning new markers that they can expect some very unfortunate writing habits in a large number of students. This is best sorted out sooner rather than later in a Degree scheme, given that their pre-degree schooling has failed in its duty to achieve this, and it may be that no-one else in the H.E.I. may serve the students on this matter. Graduating students should be proficient in written English, as a general skill as well as a philosophical one (we need to teach them how to walk before they can do philosophical gymnastics, as it were), a skill for which they will undoubtedly be grateful whatever their chosen employment. The EHE guidelines required that under its scheme it is important to develop such general skills; and we as tutors could be proud that it will have been the students' Philosophy course which will have taught them these skills. The 20% is an unambiguous incentive for them to tidy up their approach, and a good start on the marks front if the students take some effort to do this (given that they want decent marks!).

As well as the key for margin note abbreviations, I had a list of general comments which could be deleted or retained as appropriate as linked to one of the above five criteria (see section below on STOCK GENERAL COMMENTS). This first effort at grading and commenting would fairly be described as my harshest. One of the comments which came back on the students' feedback questionnaires at the end of the course was that the commenting was "pointless because everybody received the same comments" (or words to that effect), augmenting the appearance of mechanism in the marking process. But, apart from this missing the point that the pre-prepared sections of the commenting were meant to be general and not responding directly so much to the content of particular contributions within any one essay, the fact remains that a significant proportion of students at least at the beginners' level tend to make the same sort of mistakes in approach and it is useful to have available a stock comment which can be called up when an essay, considered individually, demands it. And there is, anyway, room too in this editing stage to add some more specific comments directed to particular content and still under the umbrella of one of the criteria.

I do not regret adopting the seemingly more mechanical approach as a tutor marking for the first time. It certainly provoked discussion in the Leeds Department. It also raised a stir a few weeks later when I came to mark my first round of TMAs (Tutor Marked Assignments) for my first cohort of Open University students on the 3^rd-level course Philosophy of the Arts. Commenting on essays being the only form of tutor contact for some students, classes and use of the telephone being optional, the OU expects much more particularised commenting. I honoured that, to an extent at least, but also tried to apply my pre-prepared commenting and criteria with strict weightings (albeit different ones - see section below on APPLICATION TO UPPER LEVELS). The OU's highly organised monitoring system picked this up straightaway as appearing too impersonal, especially the abbreviations used in margins. There was objection to the strict sub-total quantitative grading for the criteria but, ultimately, no successful argument against the principle of it. What I had to change was the appearance to the student of it being impersonal. I think that this was fair comment for OU students many of whom would be adult returners with a residual sensitivity to unsubtle criticism hanging over them from a dispiriting school experience. New school leavers on traditional university courses have the luxury of frequent classes and pastoral time with tutors for sorting out their particular worries, and so the more mechanical appearance of a marking schedule should be of less concern.

Having said this, for my teaching of the Leeds Department First-Year course in the 1992-93 session, I did revise my marking schedule, at least cosmetically, re-configuring the order of my five criteria and the respective weightings to some extent. This followed well organised meetings amongst tutors and the lecturers responsible well in advance of the first essay deadline of the course. Although there was general support for the separate criteria for marking, we acknowledged that there are different ways of defining those criteria, and variety in the weightings for each acceptable. My own reduction in the STYLE sub-total resulted from an absolute directive from my senior. My revised schedule is below. As regards filling in the schedule I decided that there was no need to inform the students of their exact sub-total marks, instead awarding one of the A-F range of grades as appropriate under each criterion.

ARGUMENT:	/40
Have you answered the question properly and fully?	15
-- Is your introductory outline clear & explanatory, yet succinct?	05
Have you used examples & expanded points sufficiently?	20
STRUCTURE & RELEVANCE:	/20
Have you provided a clear structure to essay?
-- Are there any points, from particular clauses all the way up to whole pages, which do not help your argument towards your answer?
KNOWLEDGE:	/20
Any mistakes in referring to the course subject matter?
-- enough detail with respect to subject matter?
INDEPENDENCE:	/05
How far have you constructed your argument in your own way differently from other philosophers and commentators you have read?
STYLE:	/15
Are these up to scratch? use of references and presentation?	05
And spelling, grammar and clarity of expression?	10

The reader will notice a change of nomenclature to INDEPENDENCE (from "originality"): this is designed to suggest that ground-breaking new work is not required from the undergraduate, but rather just some individual "flair", in order to achieve a good mark. More fundamentally, I separated out STRUCTURE from ARGUMENT and put it with RELEVANCE. I have stuck with this new arrangement of criteria since but, as it stood then, this would have left ARGUMENT looking like it had rather a low weighting. ARGUMENT being the chief purpose of typical writing in Philosophy, and so a basic philosophical skill, I decided to bolster it up with the deficit from STYLE and a bit from KNOWLEDGE, compared to the previous year's weightings.

Application to Upper Levels

We are trying to induce students into the discipline of Philosophy, and progressively so with each undergraduate level. This leaves open the likelihood that we need shifts in emphasis in what is required in essay writing. I have alluded above to the common discovery that new undergraduates often lack general writing skills, so it is important to concentrate on that where necessary, in order to pave the way for the individual to realize their philosophical potential via the written word more easily. A further issue for best practice in Philosophy essay marking, then, is determining the appropriate weightings. These are my weightings for the 3^rd-level courses I have taught:

ARGUMENT:	/30
Have you answered the question properly and fully?	10
-- Is your introductory outline clear & explanatory, yet succinct?	05
-- Have you used examples & expanded points sufficiently?	15
STRUCTURE & RELEVANCE:
Have you provided a clear structure to essay?	/20
-- Are there any points, from particular clauses all the way up to whole pages, which do not help your argument towards your answer?
KNOWLEDGE:	/30
Any mistakes in referring to the course subject matter?
-- enough detail with respect to subject matter?
INDEPENDENCE:	/10
How far have you constructed your argument in your own way differently from other philosophers and commentators you have read?
STYLE:	/10
Are these up to scratch? use of references and presentation?
And spelling, grammar and clarity of expression?

The STYLE criterion should be carry less weight still in upper levels of study, since all students should have learnt to write by the end of the first level study: rigour in citing references comes under this heading, so this remains important to highlight. The drop in weighting for STYLE has in effect been transferred to INDEPENDENCE, since this latter area can then reward better a flair for developing ideas which you need for postgraduate level work. I described ARGUMENT as a basic skill and, as such, can lose out in upper levels in terms of weighting to KNOWLEDGE: this would be to reward both initiative in individual research through a broader range of sources and thoroughness of reading of those sources.

At the time of writing this, Martin Gough was a Research Officer at The University of London's Centre for Research on the Wider Benefits of Learning http://www.learningbenefits.net, having taught Philosophy for Leeds, Nottingham and the Open University. He is currently Lecturer in Higher Education and Academic Practice at the University of Kent and co-convenor of the Society for Research in Higher Education Postgraduate Issues Network, which produces a series of Guides: http://www.srhe.ac.uk/publications.gpi.asp

Created on: April 17th 2007

Updated on: April 29th 2008

Supporting Philosophy, Theology,
Religious Studies, History of Science and Philosophy of Science

Please note that as of the 1st of August 2011 the Subject Centre for PRS entered a 5 month transition phase prior to its closure on the 31st of December 2011. Most pages on this website are no longer being updated, apart from resources from the last academic year's activities.

PDF Download