31 Portfolio assessment
What is a portfolio?
Advantages
Portfolios offer a number of advantages as an assessment tool:
• Outcomes and competencies can be assessed that other tools have difficulty in reaching. These may be skills necessary for life-long learning such as self-assessment, reflection and the adoption of appropriate learning strategies. They provide a tool to assess attitudes and professionalism.
• Portfolios include evidence collected over a period of time and provide an overall and holistic view of a student’s competence.
• With the range of quantitative and qualitative evidence included and the triangulation from the different sources of evidence, portfolios provide a comprehensive and reliable interpretation of a student’s achievement.
• Portfolios represent a personalised approach to assessment, focusing on what individual students achieve in contrast to the more standardised approach with instruments such as the OSCE.
• As the students go through their course of studies the portfolio integrates learning and assessment and focuses the student’s attention on the learning outcomes expected. The student’s reflection on the basic sciences relevant to a documented patient encounter reinforces the application of theory to practice.
• The portfolio reinforces a student-centred approach to the curriculum with students being given greater responsibility for their own learning and assessment.
Implementing portfolio assessment in practice
1. Define the purpose. It should be made clear whether the portfolio is to be used for summative or formative decisions and how it relates to other elements in the assessment process.
2. Determine the competencies to be assessed. Identifying the competencies to be assessed by the portfolio is part of a systematic approach to assessment that ensures that all of the expected learning outcomes are assessed. In particular, the portfolio is valuable in assessing learning outcomes such as attitudes and professionalism and the higher levels of Miller’s pyramid.
3. Define the portfolio content. Evidence should be included in the portfolio that demonstrates a student’s achievement of the learning outcomes to be assessed. Students should be given guidelines as to the type of evidence that is acceptable, but there should be a certain amount of freedom of choice. The type of evidence that is included, the student’s comments on it, and their ability to assess their own competence is a measurement of their understanding of the learning outcomes.
Examples of the type of evidence that might be included are:
4. Develop a marking system. Specific criteria may be set out for each of the learning outcomes to be assessed with the portfolio, and a global rating used for the achievement of each outcome:
Several assessors should review each portfolio with an assessment committee taking a final decision.
5. Select and train the examiners. The choice of examiners will depend on the purpose of the assessment and the learning outcomes to be assessed. The examiners should include a range of staff from the basic sciences and clinical medicine. Examiners with less experience can be paired with more senior examiners. The training of the examiners is essential for the success of the programme. Faculty members often appreciate their participation in portfolio examinations as it allows them to get to know more about the individual student and his or her capabilities.
6. Plan the examination process and timetable. It is necessary to set a deadline for portfolio submissions. Students’ failure to meet the deadline is itself evidence of a lack of professionalism. Time should be scheduled for examiners to read each portfolio and to meet to discuss them. An opportunity should be provided for them to meet, possibly in pairs, with each student to allow the student to defend the portfolio. Finally, time needs to be set aside for the examiners to discuss each student’s performance in order to come to a final decision as to the student’s achievement of the learning outcomes assessed.
7. Student orientation. Students should be informed in writing about the portfolio assessment process and what is expected of them. In general, the more information given to students the more positive they are about the portfolio.
8. Develop guidelines for decisions. If portfolios are used for summative pass/fail decisions, standards need to be specified so that there is no doubt about what constitutes a pass or fail. A decision needs to be made whether poor performance in relation to one outcome assessed can be compensated by good or excellent performance in another area, or whether areas are not compensatory. The medical school in Dundee adopted the approach that a student cannot compensate for deficits in one domain, such as attitudes or professionalism, by a good performance in another domain. Students must achieve the minimum expected standard in every domain.
9. Establish reliability and validity evidence. What constitutes good reliable evidence should be agreed prior to the implementation of the portfolio. The degree of reliability may be determined in a pilot study. For example, should there be two pairs of examiners for each portfolio or one pair? Triangulation of the evidence in the portfolio from different sources will increase the validity of the decision reached and guide the faculty as to the use of the portfolio results.
10. Evaluate the portfolio assessment. Students’ and examiners’ opinions on the strengths and weaknesses of portfolios as an assessment tool should be sought. Students’ performance with the portfolio assessment should be compared with their performance in an OSCE or written examination and ultimately with their subsequent performance as a doctor. When problems are identified relating to the professionalism or performance of practising doctors, were the same issues identified in their portfolios as students?
Reflect and react
1. Review the value of portfolios as a tool for student assessment. Which of the advantages listed are applicable to the programme for which you have a responsibility?
2. What could be included in your student’s or trainee’s portfolio that would provide evidence of their achievement of the expected learning outcomes?
3. If you are an examiner for a portfolio assessment it is important that you fully understand the assessment process and the marking scheme used.
Davis M.H., Friedman Ben-David M., Harden R.M., et al. Portfolio assessment in medical students’ final examinations. Med. Teach.. 2001;23:357-366.
Driessen E., van Tartwijk J., van der Vleuten C., Wass V. Portfolios in medical education: why do they meet with mixed success? A systematic review. Med. Educ.. 2007;41:1224-1233.
A description of how portfolios can be used for summative and formative purposes.
Friedman Ben-David M., Davis M.H., Harden R.M., et al. Portfolios as a Method of Student Aassessment. AMEE Medical Education Guide No. 24. Dundee: AMEE, 2001.
Practical advice on the use of portfolios for the purpose of assessment.
van Tartwijk J., Driessen E.W. Portfolios for Assessment and Learning. AMEE Guide No. 45. Dundee: AMEE, 2010.
Buckley S., Coleman J., Davidson I., et al. The Educational Effects of Portfolios on Undergraduate Student Learning: a Best Evidence Medical Education (BEME) systematic review. Dundee: AMEE; 2009.
Gronlund N.E., Waugh C.K. Assessment of Student Achievement, ninth ed. Upper Saddle River, New Jersey: Pearson; 2009.
Chapter 11 describes the value of portfolios as a means of assessment and how they can be scored.
Tochel C., Haig A., Hesketh A., Cadzow A., et al. The effectiveness of portfolios for post-graduate assessment and education. BEME Guide No. 12. Med. Teach.. 2009;31:299-318.
