About the project
Objective
The Responsible Digital Assessment Futures in Higher Education (REFINE) project aims to envision digital assessment futures by unpacking the opportunities and challenges that digital assessment brings to higher education for students’ improved learning. It specifically focuses on stakeholders directly and indirectly involved in the design and implementation of effective and responsible digital assessment practices. The project addresses the following research questions:
- What are teachers’ experiences of digital assessment in higher education?
- Which are the key opportunities and challenges (e.g., legal, ethical, organizational, technical, and pedagogical) of trusted digital assessment?
In this project, a mixed-method human-centred approach will be undertaken. An online national survey will be distributed to educators across several higher education institutions in Sweden. Two workshops will follow this with stakeholders – educators, lawyers, managers, and system developers – involved in designing and implementing digital assessment in higher education. By adopting a multi-stakeholder perspective, we aim to address several key challenges of digital assessment and suggest ways forward for students’ improved learning, improved learner support and teaching in the post-pandemic higher education setting.
The project will be a collaborative effort by a group of researchers from KTH and Stockholm University, representing the departments of Digital Learning, Media Technology and Interaction Design, Network and Systems Engineering at KTH and the Department of Computer and Systems Science at Stockholm University. All project members have much-needed complementary expertise and knowledge in technology-enhanced learning, organizational aspects of higher education, assessment, learning analytics and responsible approaches related to the use of digital tools and student data in higher education.
The outcomes of this seed project will contribute to digital assessment research and practice.
- First, it will enhance our understanding of current digital assessment practice and their associated challenges and opportunities in the setting of Swedish higher education.
- Second, based on this understanding, a set of recommendations for sustainable future digital assessment practices will be offered.
- Third, it will provide a foundation for creating and advancing competitive research proposals on digital assessment futures.
Background
Effective assessment practices are central to student learning and academic achievement. Whereas online assessment forms have to some degree, been present prior to the COVID-19 pandemic, the traditional examination forms had a stronghold within higher education and have often been the preferred way of assessing students. Restrictions during the COVID-19 pandemic led to a dramatic increase in the use of digital assessment, including formative and summative digital assessment practices in higher education.
In this regard, scholars and practitioners have shown some of the advantages of adopting digital assessment tools, such as personalized and accurate assessment, but also concerns, including issues of ethics, privacy, fairness, trust and security associated with the use of digital assessment tools (e.g., proctoring software).
Crossdisciplinary collaboration
The researchers in the team represent the KTH Royal Institute of Technology and Stockholm University.
About the project
Objective
This project explores the transformation of education by working with and learning from students and teachers with cognitive disabilities. It is part of a larger endeavour to understand how education empowers and takes in diverse people’s experiences. We focus on the digitalisation of education and how digital tools can be better used to respond to the experiences and needs of students and teachers with cognitive disabilities. Implemented as an action research project, the project’s aim is two-fold: 1) Research-informed changes to higher education for greater inclusivity and 2) knowledge of this process and the experiences of those involved. We will conduct focus group meetings and “shadow” students and teachers with cognitive disabilities to learn about their experiences and needs and collect ideas for change.
Background
About 10% of the students at KTH, and about 33% of the students getting support from the student support administration, are registered as having cognitive impairments such as dyslexia, autism or ADHD. More students are assumed to have “lived experience” of cognitive impairments. While there is quite some research on students, we know little about teachers with cognitive impairments. In light of our great sustainability challenges, education is transforming to become more relevant for sustainability—equality and high-quality education for all our sustainability goals. More so, education has a great potential to promote just societal transformation if it becomes a space where diverse people can participate and are valued.
Crossdisciplinary collaboration
The project spans research in education and human-computer interaction at KTH. Further, several organisations at KTH participate in the collaboration (administration, student union, equal opportunity and sustainability office) and the organisation “Begripsam” in Stockholm. Through different channels, we will invite all students and teachers to participate in the project, especially those with lived experiences of cognitive impairment, to develop future education for all. PI Jan Gulliksen focuses on education and research for usability and accessibility, User-centred systems design, digitalization and digital work environments. Co-PI Anne-Kathrin Peters focuses on education for sustainability, especially equality, diversity, and justice.
About the project
Objective
The objective of the Swedish Question Generation for Assessing Reading Comprehension (SWE-QUEST) project is to develop a demo system that, given a text, automatically generates multiple-choice reading comprehension questions on the text, as shown in the picture below.
Using a neural-network-based generative model, the demo system will generate the whole multiple-choice question, both the question itself and the answer alternatives. Although the example above was in English, our demo system will work for Swedish text. The project will push the state-of-the-art in natural language generation. The system is intended to be used by teachers of SFI (Svenska för invandrare) to facilitate test construction and the development of teaching materials. Still, it can also be used for self-studies of Swedish.
Background
The SFI students form a heterogeneous group, and the SFI classes are often too big to allow the teacher time to adapt the level of teaching to cater to each student individually. Therefore, developing such a tool could be highly useful. It would rapidly and easily generate several suggested multiple-choice questions (MCQ) on text material for teaching and assessment.
The task of automatically generating reading comprehension questions (without distractors) using neural methods has been studied before, primarily for English. There have also been some attempts at generating distractors, given the question and the correct answer, using neural methods. However, no attempt has been made to construct a trainable model that generates the whole MCQ in one go.
Crossdisciplinary collaboration
The researchers in the team represent the KTH Schools of Electrical Engineering and Computer Science, the Division of Speech, Music and Hearing and the Department of Swedish Language and Multilingualism at Stockholm University.
About the project
Objective
This project aims to develop and study a demonstrator of semi-automated online math tutoring by combining a human approach and automated tutoring. The automation will be based on Natural Language Processing analysis of previous tutor-student interactions. To evaluate the semi-automated tutoring concept, we will conduct a randomized controlled trial by randomly assigning students into a treatment group where tutoring is semi-automated and to a control group with only human online tutoring. We will also conduct a thematic comparative analysis of student-automated tutor interaction compared to student-human tutor. The findings will contribute to research on semi-automated tutoring, a scientific area that has received limited attention. The semi-automated approach could greatly impact society because tutors can help more students. The project will serve as an example of beneficially using semi-automation without losing the human touch of tutoring.
Background
It has been known for decades that one-to-one tutoring is a very effective teaching method, although the key challenge is to scale it up. Maths Coach Online (mattecoach.se) has been offering one-to-one tutoring by teacher students to K-12 students using chat and interactive whiteboard since 2009 and has conducted more than 70,000 tutoring conversations. We are transforming Maths Coach Online into a national service, making scaling to a larger volume of students important. Therefore, we want to explore how to support high-quality math learning for as many students as possible by incorporating semi-automated intelligent tutoring.
Crossdisciplinary collaboration
The researchers in the team represent the KTH School of Industrial Engineering and Management, Department of Learning in Engineering Sciences and the KTH School of Electrical Engineering and Computer Science, Department of Intelligent Systems.
About the project
Objective
In recent years, Sweden has emerged as one of the most welcoming European countries for migrants from diverse backgrounds and cultures. However, the challenge lies in effectively integrating these individuals into Swedish society, given the nation’s relatively small population and the rapid influx of migrants. Language learning is critical to this integration process, posing significant challenges for learners and educators. We aim to develop a language learning system focusing specifically on conversational skills utilizing social robots and conversational AI. This will enable learners to practice language through conversation with diverse personas in real-life situations.
Background
Recent advancements in Large Language Models (LLMs) have transformed our approach to complex challenges such as language learning, opening up for new opportunities that were not available a few years ago. Alongside LLMs, advancements in speech technology — specifically in the fields of speech recognition and synthesis — enable the creation of much more natural, lifelike conversational experiences with AI. This progress empowers us to develop a system that not only comprehensively understands Swedish but can also understand the user’s native language. This multilingual competence facilitates a more effective, personalized, and user-centric language learning platform.
Crossdisciplinary collaboration
Our project benefits from the expertise of a diverse team drawn from the Division of Speech, Music, and Hearing at KTH and the Department of Education at Stockholm University. This interdisciplinary collaboration ensures that we have the necessary theoretical and practical background to develop an innovative solution that addresses the challenges of language development and cultural integration in the context of digital transformation.