About the project
Objective
This project argues that urban autonomous systems should be understood as publicly sustained rather than self-sufficient. It aims to conceptualise the invisible public work through which this dependency is maintained and to explore how design can make it more legible and socially sustainable.
Background
Urban digitalisation is bringing autonomous technologies into urban public space, from service robots on sidewalks to autonomous mobility services and intelligent urban infrastructure. These systems are often framed and deployed as self-sufficient forms of machine autonomy. Yet in practice, their everyday operation frequently depends on co-present people who interpret their behaviour, adapt their own actions, and at times intervene to resolve issues. This is exemplified by pedestrians altering their paths to grant robots passage and removing obstacles or providing instructions when robots become immobilised, and by drivers adjusting their behaviour to ensure safe interaction with autonomous vehicles. This suggests that what appears as machine autonomy is often sustained through invisible public work.
However, this dependency remains under-recognised in both design and deployment. As a result, the work required from the public is often left to emerge informally, without being made legible or supported through design. This can place unfair burdens on people and create friction in shared spaces, undermining public trust and acceptance of these technologies. To address this gap, the project aims to propose and develop the concept of publicly sustained autonomy through empirical and design research. This concept reframes urban autonomous systems as sociotechnical arrangements whose operation depends not only on technical autonomy, but also on distributed support from the public. Building on this framing, the project investigates how urban autonomous systems can be designed to make the public work that sustains them more legible and socially sustainable, and under what institutional conditions such approaches can be deployed responsibly in public space.
About the Digital Futures Postdoc Fellow
Xinyan Yu is a Design and Human–Computer Interaction researcher. She received her PhD from the Design Lab at the University of Sydney. Her research examines how emerging autonomous technologies become part of everyd
ay public life, with a particular focus on human interactions with service robots and autonomous vehicles in urban environments. Situated at the intersection of design research, human-computer interaction, and human robot interaction, her work investigates how these systems are encountered and experienced in everyday settings, and how design can support their integration in more socially meaningful ways.
Main supervisor
Andrii Matviienko, KTH Royal Institute of Technology.
Co-supervisor
Airi Lampinen, Stockholm University
About the project
Objective
- Develop an accurate and computationally efficient model of power-electronics-dominated microgrids with distributed renewable energy sources.
- Establish electrical-parameter-dependent Lyapunov methods for analysing microgrid stability and robustness under time-varying loads and network conditions.
- Design distributed event-triggered control strategies that enable reliable, communication-efficient, and scalable operation of renewable-energy microgrids.
Background
The transition towards renewable energy is increasing the complexity of modern power systems. Microgrids with high shares of distributed renewable energy sources rely on power-electronic converters, communication networks, and coordinated control, making their modelling, stability analysis, and operation challenging.
This project will develop event-triggered networked control methods for microgrids, combining control theory, power systems, network science, and data-driven methods. The goal is to improve the reliability, resilience, and efficiency of future renewable-energy systems while reducing unnecessary communication and supporting the sustainable digital transformation of energy infrastructure.
About the Digital Futures Postdoc Fellow
Dr. Libei Sun is a postdoctoral researcher at the Department of Electrical Engineering at KTH Royal Institute of Technology. Libei received a PhD in Control Theory and Control Engineering from Chongqing University, China, and was previously a postdoctoral researcher at the University of Hong Kong and a researcher at Nanyang Technological University. Libei’s research focuses on event-triggered control, smart grids, safe control, and robotics.
At KTH, Libei works with Prof. Qianwen Xu and Prof. Dimos Dimarogonas on reliable and communication-efficient control solutions for sustainable power systems.
Main supervisor
Prof. Qianwen Xu, KTH Royal Institute of Technology.
Co-supervisor
Dimos Dimarogonas, KTH Royal Institute of Technology.
About the project
Position in the Digital Futures research matrix
This project sits within the Trust research theme, which addresses the foundations of safe, reliable, and accountable digital systems, and connects to the Smart Society societal context, which looks at how digital technologies shape public life and institutions. By working at this intersection, the project bridges technical AI safety research with the societal and human dimensions of AI misuse.
Objective
The project sets out to understand how generative AI systems can be exploited for socially harmful purposes, through jailbreaks, adversarial prompting, and other unsafe forms of human-AI interaction. Rather than treating this as a narrow cybersecurity issue, the project looks at broader harms such as manipulation, coercive persuasion, misinformation, and intimate surveillance. Its goals are to map emerging misuse scenarios, identify research gaps across AI safety, HCI, and sociotechnical security, build interdisciplinary collaborations, explore ethical approaches to red-teaming, and produce an initial roadmap for future research in this space.
Background
The project responds to a growing body of evidence that safety-aligned AI systems can still be manipulated into producing harmful or restricted outputs, despite existing safeguards. Techniques for bypassing these safeguards continue to evolve and circulate within online communities, often outpacing efforts to detect and prevent them. While most existing AI safety research concentrates on technical robustness and content filtering, far less attention has gone to socially situated harms like coercive control, deceptive influence, and interpersonal abuse enabled by AI. This project aims to help close that gap between technical jailbreak research and human-centred research on digital harm.
Cross-disciplinary collaboration
The project brings together expertise from human-centred cybersecurity, HCI, control systems security, and software supply chain security, combined through a shared program of exploratory research and four interdisciplinary workshops covering AI-enabled interpersonal harm, cybersecurity misuse, misinformation and manipulative communication, and ethical red-teaming methodologies.
PIs
- Asreen Rostami (PI) – Senior Researcher, RISE Research Institutes of Sweden
- Nicolas Harrand (Co-PI) – Senior Lecturer, Stockholm University
- Henrik Sandberg (Co-PI) – Professor, KTH Royal Institute of Technology
About the project
Objective
- Careful assessment of what new information due to personal data of an individual has been memorised;
- Guidelines outlining what residuals of personal information in a ML model should be legally acceptable;
- Explore how the RTBF can be effectively implemented, ultimately in the context of ML systems.
Background
In the context of machine learning (ML), there is an urgent need to clarify what “deletion” under the Right to be Forgotten (RTBF) truly entails. As ML models generalize and internalize patterns from data, achieving complete removal of an individual data point remains a major technical challenge. We argue, however, that such full deletion often exceeds what the law actually requires. REFLECT-ML aims to explore how the RTBF can be effectively implemented, ultimately in the context of ML systems. We will address the disconnect between the abstract legal language of regulators and operationally founded technical theoretical measures that can be used to quantify the new information associated with individual data as well as the practical complexities in estimating those. Further, we will develop different technical approaches to control its memorization as well as exploring machine unlearning attempts.
Cross-disciplinary collaboration
The team consists of PI Oechtering (researcher in information theory, relevant for information quantification), PI Colonna (researcher in law, relevant knowledge related to the GDPR and AI Act), and PI Johansson (researcher in optimization, relevant for the design of efficient learning algorithms). The outlined work is a cross-disciplinary effort since information measures need to be legal compliant, legal requests need to be algorithmically feasible, and algorithms need to aim for the right objective.
About the project
Objective
Code generation with AI is making very fast progress. Nonetheless, it suffers from a fundamental drawback: it is unreliable, and users developers cannot trust the generated code, which is a major blocker. Our project TRUST-SW aims to solve this problem: AI will generate code alongside verification materials using state-of-the-art formal verification. By doing so, project TRUST-SW will enable users to produce dependable, correct code with generative AI. The specific objectives are:
- Generate specification from human language using LLMs: Develop capabilities for LLMs to translate human language into specification with no or minimal human supervision.
- Empower LLMs with deductive, explainable reasoning: Equip LLMs with deductive reasoning skills to debug and validate code based on natural language instructions and novel techniques.
- Embed and evaluate real-world constraints into code generation: Ensure that generated code aligns with practical requirements, handling complex constraints of real-world applications.
Background
Recent advances in generative AI, particularly large language models, have made it possible to automatically generate software from natural language prompts. While these tools significantly accelerate software development, the generated code may contain subtle bugs, security vulnerabilities, or logical errors that are difficult to detect. At the same time, formal verification techniques can mathematically guarantee software correctness but often require significant expertise and manual effort. TRUST-SW addresses this gap by exploring how AI systems can work together with formal reasoning and verification tools to automatically generate software that is not only functional but also provably correct and trustworthy.
Cross-disciplinary collaboration
TRUST-SW brings together expertise from several research areas, including artificial intelligence, formal verification, programming languages, and software engineering. The project combines advances in large language models with rigorous methods from formal methods and automated reasoning. This interdisciplinary collaboration enables the development of novel approaches where AI-generated code can be systematically verified and validated, bridging the gap between machine learning–based code generation and mathematically grounded software assurance.
PIs: Marco Chiesa, Martin Monperrus, Mariano Scazzariello.
About the project
Objective
Study the issue of outliers’ privacy from an information-theoretic point of view, propose an adapted privacy notion, such as pointwise maximal leakage, to solve it and design sanitising mechanisms in the light of these principled insights.
Background
The most popular privacy measure, differential privacy, can only protect outliers at the cost of destroying accuracy. Its relaxation, metric differential privacy, fails to guarantee the privacy of such isolated points.
About the Digital Futures Postdoc Fellow
Arnaud Grivet Sébert completed his PhD in CEA List, Gif-sur-Yvette, France, under the direction of Renaud Sirdey and the co-supervision of Cédric Gouy-Pailler. He proposed approaches that combine differential privacy and homomorphic encryption to protect the training data privacy in distributed machine learning.
He then worked on the privacy of textual data, and especially outliers, as a post-doctoral researcher in LIX (Laboratoire d’Informatique de l’Ecole Polytechnique), Palaiseau, France, with Catuscia Palamidessi and Sonia Vanier, and in Macquarie University, Sydney, Australia, with Annabelle McIver and Mark Dras.
He is now starting a post-doctoral contract in KTH, funded by Digital Futures and supervised by Tobias Oechtering and Martina Scolamiero. He is especially interested in the theoretical aspects of privacy, but also in its relations with other ethical properties like frugality, robustness, fairness.
Main supervisor
Tobias Oechtering, KTH
Co-supervisor
Martina Scolamiero, KTH
About the project
Objective
The project primarily addresses trustworthy AI deployment for mission-critical robotic systems operating in industrial and adversarial environments.
EdgeWise aims to redefine how Vision-Language-Action (VLA) models are deployed, executed, and secured for next-generation humanoid robots operating in connectivity-constrained and adversarial environments.
The main objectives are to:
- Design a novel three-tier VLA deployment architecture (on-device, edge, cloud) to reduce end-to-end control loop latency and increase resilience in unstable network conditions.
- Enable efficient multi-tenant edge model serving by compressing fine-tuned models into lightweight adapters that share a common base model, significantly improving resource utilization.
- Introduce verifiable and interpretable execution planning, where robots generate structured code from natural language instructions and apply formal verification methods to ensure correctness and safety before execution.
Background
Humanoid robots powered by Vision-Language-Action models have the potential to transform healthcare, construction, search-and-rescue, and other labor-intensive or hazardous domains. However, current VLA architectures rely heavily on cloud-based reasoning models, which introduce latency and depend on stable connectivity. This makes them unsuitable for many real-world scenarios such as disaster zones, underground mining, or remote medical environments.
Moreover, VLA systems often generate opaque action tokens that are difficult to interpret or verify, raising serious safety and security concerns. Recent studies show that language-model-controlled robots can be manipulated into performing unsafe actions, highlighting the urgent need for transparency and formal guarantees.
EdgeWise addresses these limitations by combining edge computing, efficient model sharing mechanisms, and formal verification techniques to enable secure, resilient, and low-latency robotic AI systems.
Crossdisciplinary collaboration
EdgeWise is a collaboration between RISE and KTH, combining expertise in:
- Distributed systems and edge/cloud orchestration
- Machine learning systems and large-model deployment
- Formal verification and secure software systems
- Robotics and real-world experimental testbeds
The project integrates systems research, AI model optimization, networking (5G/6G), and formal methods. The experimental platform includes robotic systems connected to a private 5G infrastructure and research data center resources, enabling controlled evaluation of latency, resilience, and safety mechanisms.
PI: Joakim Eriksson, RISE
Co-PI: Marco Chiesa, KTH