The Scorion e-portfolio system within HBO-ICT at the HvA

This interview took place in early June 2023

"Since 2022, we have been working with Scorion within the propaedeutic phaseof the programme," says Jan Berends Lecturer Business, IT & Management. Institution-wide, we work with the e-portfolio system Portflow. "Should this meet our functional requirements, there is a chance that we will eventually switch to this e-portfolio system," says Jedeloo.

Functions of e-portfolio system

Arthur Jedeloo, Lecturer Business, IT & Management says: "We have a clear idea about the function of e-portfolio. For us, e-portfolio is a learning tool for students to give feedback, feedup and feedforward. Students are able to see their own progress and link it to standards/criteria against which they are assessed. It is also important that the lecturer and/or coach is able to monitor the student."

Berends adds, "We also have data points that allow us to track the student's development". These data points are also assessed: it is always assessed whether the student scores below, at or above level. Here, it is important that the decision committee is empowered to ultimately do a summative assessment. The decision committee is responsible for making decisions regarding the assessment and evaluation of students. It decides whether a student meets the set assessment criteria and whether they are fit to move on to the next stage of their programme or obtain a degree. They have a dashboard where all information (the learning outcomes) is presented in a very accessible way. "With our e-portfolio system, you are able to properly collect all the data points and relate them to the learning outcomes."

The overall learning and testing process

Within the propaedeutic phase, there is a 10-week block with a total of 3 sprints. At the beginning of each sprint, the student formulates a learning objective for that sprint and at the end he then submits his products. These form the 'data points'. "Important to mention is that the student himself can also create a data point to provide evidence," Berends points out.

There are several measuring moments (for an example, see Figure 1). When assessing at the end of a sprint, there is a sprint review where the professional product is looked at. There are also retrospectives during a sprint where collaboration and other professional skills are looked at. In between, there is a standard coaching session at each block and a look 'under the bonnet' at an expert review. "This coaching further varies per semester/module," Jedeloo says. It is done in different places in different ways. Coaching can be from the coach, study advisor, (subject) lecturers, peers and students. "We therefore find it very important that the system is accessible to external people," says Berends.

At the end of the three sprints, the decision committee looks carefully at the student's portfolio. The assessment process has recently become more formalised. Thus, the decision committee consists of three people, with the coach allowed to give advice. A decision procedure was created for this, to take out the unwanted assessor effects. "In such a committee, of course, there should also be someone who has a good view of the teaching that comes next in that period. There should also be calibration above all decision committees".

There is always a distinction between rating and assessment, according to Jedeloo. "This is in the didactic process," he says. You can discuss what is meant by valuing. You will also get different opinions about that side by side. An important element is being able to follow a student during the process. For a decision committee, it is important that they can see afterwards what development there was. "You all have cross-sections that you want to have the overview of". What is important here is that such a decision committee gets an overview, from which they can easily pull the right information. During each sprint, students have the opportunity to reflect on their own performance over the past period. The online dashboard makes this clear for the student.

Student ownership

"Student ownership is very important to us," Jedeloo says. For example, the student has the possibility to design the e-portfolio himself without using a template. Students can also add their own data points and feedback to their portfolio. Berends: "After all, with programmatic testing, you always have fixed and free data points".

In addition, students are able to formulate their own learning objectives and link them to metadata. "Of course, at the end of a sprint, your product has to meet certain quality criteria. These skills are elaborated in behavioural criteria that you use to assess those skills," says Berends. Jedeloo points out that there is sometimes room to flesh this out yourself. Finally, students have the opportunity to link products/evidence to learning outcomes/learning objectives/competencies.

An online dashboard

Jedeloo: "The students gain insight into their own development using an online dashboard. Both the coach and the student can access this dashboard at any time." Currently, this online dashboard is being further developed. Figure 2 shows an example of what the dashboard will (presumably) look like in the next academic year. In addition, within the online dashboard, both the student and the coach can always see what feedback has been given previously by the coach or another person. "This can be compared to a doctor looking in the file and finding all the patient data in it," Jedeloo points out.

Feedback on learning outcomes and reflections

Students receive peer feedback, lecturer feedback as well as feedback from the field. Students exhibit a lot of 'feedback seeking behaviour', appoints Jedeloo. "For example, we often see first-year students seeking feedback on their own from senior students (who function as student assistants)."

Key success factors

A key success factor for the process is that companies have access to the system. "We consider this an important functionality that supports our current e-portfolio system," Jedeloo appoints.

Another key success factor is that remediation needs to be well supported. By already being able to give feedback to the student in the interim, you gain insight into the whole process and can send a signal to the student in time. Student and coach should both be able to see the history, in a clear and convenient way. This way, other lecturers who were not involved in the student's development process in the interim are also able to see where the student stands and what he or she specifically needs to work on. So both parties, i.e. the student and the coach/lecturers, should be able to see the history in a quick and clear way. "Our current e-portfolio system supports this reasonably well, but we want to move towards having the feedback and the dashboard in one," Jedeloo states.

Functional bottlenecks

"What I find a big disadvantage is that you have to click through all the time," Jedeloo says. Information is only available when you have clicked several times. "At times like this, the system starts to become leading and also begins to drive the process in the perception of coaching and feedback. As a result, you cannot access the feedback that is available in an orderly way. You cannot see at a glance whether there is any pattern in the various feedback given. You then lose the overview right away. "We are therefore redesigning our e-portfolio system to make the process more efficient and thus make it easier to give feedback," Jedeloo says. This also allows you to get more data points. Jedeloo: "Fortunately, our e-portfolio system has the ability to set up the process completely the way you want. We add our own sauce to it".