Designing test questions for equivalence, accessibility and open-book exams.
Online alternatives to traditional assessments and coursework.
Assessment design checklist
A checklist based on the University’s nine principles of assessments.
About assessment design
At the heart of good learning and assessment design is the concept of constructive alignment (Biggs, 1999). Mayes and de Freitas explain good pedagogical design, following Biggs, as:
“… ensuring that there are absolutely no inconsistencies between the curriculum we teach, the teaching methods we use, the learning environment we choose, and the assessment procedures we adopt …
Thus, we need to start with carefully defined intended learning outcomes, we then need to choose learning and teaching activities that stand a good chance of allowing the students to achieve that learning, then we need to design assessment tasks which will genuinely test whether the outcomes have been reached.” (Mayes & de Freitas, 2004).
This is not as simple as it may seem but is helped by keeping in mind what you are asking the learner to do. Goodyear (2000) suggests the idea of a ‘cognitive walkthrough’ of your course from the view of the learner. Is the current task relevant? Does what they are asked to do at each stage contribute to their achievement of the intended outcomes? Assessment tasks can be thought of as rungs on a ladder towards the learning goals.
Types of assessment
Assessment within a course measures knowledge acquisition, performance and skills development in order to determine students’ progress towards attainment of learning outcomes. Assessed tasks may include a variety of activities such as quizzes, tutorials, labs, journals, tutorials, case studies, group work, presentations, projects, tests and more. Ideally, academics use a mix approach to suit the subject, course, students and their style of teaching.
A distinction is sometimes made between assessment FOR learning (formative) and assessment OF learning (summative).
- Formative assessments are conducted at intermediate points throughout a course to give students feedback on their performance and to give instructors information on how students are learning.
“Specific, constructive feedback about learning, as it is unfolding, is one of the most powerful influences on student achievement. Positive feedback celebrates success and helps keep students motivated, whilst constructive feedback highlights important aspects to focus on. Feed-forward provides an outline of the next steps to be taken. Feedback/feed-forward includes all dialogue to support learning in both formal and informal situations.” – Te Kete Ipurangi.
- Summative assessments are conducted at the end of a unit or course for the purpose of assigning a grade.
While in remote teaching mode, aiming for questions that promote and test higher-order thinking can help to minimise academic dishonesty. As a general rule, asking students to think critically is a better measure of learning than providing MCQs as there is little to no element of guesswork.
Intended learning outcomes
It is important to align your redesigned assessment task with the intended learning outcomes (ILOs), and consider how it fits with other learning activities within your course.
For example, if an ILO requires the learner to produce comprehensible, appropriate and spontaneous spoken responses to a situation in the target language, then an aligned task might be to present a scenario and ask the learner to respond orally to questions about the situation. A non-aligned task might ask the learner to write a paragraph on how they would respond, since, in that case, the response is not spontaneous, is not spoken, and does not test pronunciation.
Whilst live Zoom assessments are not normally recommended if your learning outcomes require a test of live performance then a live test may be necessary. In the language learning example above, the usually face-to-face task might be redesigned as a live assessment using Zoom, presenting scenario-based questions that students are able to research and practice for, but not pre-prepare exact responses. Using breakout rooms would allow for the whole class to conduct a live assessment, which can be recorded.
A common and very clear mismatch between approaches to learning, types of activities, and ILOs often occurs in online courses claiming to be based on social constructivism and communities of learning. In these cases, much of the process of the course is conducted through online discussions with students focussed on collaboratively negotiating meanings, but is then assessed individually through a final assessment.
Short term ‘Minimal Viable Product’ changes to accommodate remote learners must be aligned to ILOs and be relevant, but also feasible, for students to do remotely, under a range of circumstances and with a reasonable expectation of meeting the outcomes. The task also needs to be secure and minimise opportunities for cheating.
To ensure equivalence for students studying remotely all coursework assessments must be the same for all enrolled students and must be able to be completed remotely and submitted electronically. It is likely that some assessments may need to be completed in an open book format, with the specific questions within the assessment being designed to test deep learning, and avoid/minimise content recall.
While the online environment may invite an open book approach, you may still wish to categorise your assessment depending on the resources you would like students to utilise and access during the assessment period.