About the project

Objective
In the Data-Limited Learning of Complex Dynamical Systems (DLL) project, we develop methods and tools to learn to control complex dynamic systems using limited data samples. In contrast to traditional machine learning techniques that require large amounts of data for training, this project aims to utilize a priori knowledge of the system and combine such structural knowledge reliably with a limited number of data samples.

The project focuses on two application domains: (i) continuous bioprocessing for safer and more efficient production of biopharmaceuticals and (ii) reinforcement learning of cyber-physical systems in general and robotics in particular. In these domains, probing large amounts of data from the system can be expensive or even impossible without wearing out the system.

Background
Recent years have witnessed spectacular successes in applying machine learning tools to the decision-making and control of complex dynamical systems. These techniques typically combine reinforcement learning with large neural networks, thus requiring a tremendous amount of training data, even to learn simple tasks. Their applications have been mainly limited to specific scenarios, such as board and video games, where generating and gathering data is inexpensive. However, in many biological or physical application domains, data is limited, and probing the system for more data can be expensive or even impossible without destroying the system.

Cross-disciplinary collaboration
This project involves co-PIs and researchers from different disciplines, including computer science, automatic control, machine learning, and biotechnology. To make the interaction as fruitful as possible, the project is divided into three sub-projects: continuous bioprocessing, (ii) reinforcement learning in cyber-physical systems, and (iii) theory. The research involves fundamental theory and practical applications, including involvement from industrial partners. The collaboration between the research team at Digital Futures and the Competence Centre for Advanced BioProduction (AdBIOPRO) will establish a strong, visible, and sustainable research environment that will overarch digitalization and life science research at KTH.

Activities & Results

Find out what’s going on!

Activities, awards, and other outputs

Results

This project has so far resulted in several fundamental research results. The results include but are not limited to: a new biological modelling approach that includes transcriptional information, new system identification techniques for differential-algebraic equations subject to disturbances, fundamental limits for sample complexity and lower bounds for the regret of deterministic discrete dynamical systems.

The results are published in top-tier journals and conferences such as ICML, NeurIPS, and CDC.

Publications

  1. Filippo Vannella, Jaeseong Jeong, and Alexandre Proutiere. Off-Policy Learning in Contextual Bandits for Remote Electrical Tilt Optimization. IEEE transactions on Vehicular Technology, 2022.
  2. Filippo Vannella, Alexandre Proutiere, Yassir Jedra, and Jaeseong Jeong. Remote Electrical Tilt Optimization: a contextual bandit approach. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM) 2022.
  3. Daniel Lundén, Joey Öhman, Jan Kudlicka, Viktor Senderov, Fredrik Ronquist, and David Broman. Compiling Universal Probabilistic Programming Languages with Efficient Parallel Sequential Monte Carlo Inference. In the Proceedings of 31st European Symposium on Programming (ESOP), 2022.
  4. Kévin Colin, Mina Férizbegovic, and Håkan Hjalmarsson, Regret Minimization for Linear Quadratic Adaptive Controllers using Fisher Feedback Exploration, IEEE Control Systems Letters and In Proceedings of the 61th IEEE Conference on Decision and Control (CDC), 2022.
  5. Yassir Jedra and Alexandre Proutiere. Minimal expected regret in online LQR. In Proceedings of the International Conference on Artificial Intelligence and Statistics(AISTATS), 2022.
  6. Robert Bereza, Oscar Eriksson, Mohamed R.-H. Abdalmoaty, David Broman andHåkan Hjalmarsson. Stochastic Approximation for Identification of Non-Linear Differential-Algebraic Equations with Process Disturbances, In Proceedings of the IEEE Conference on Decision and Control (CDC), 2022.
  7. A Ghosh, A.E. Fontcurbeta, Mohamed R. Abdalmoaty, Saikat Chatterjee, Time-varying Normalizing Flow for Generative Modeling of Dynamical Signals, European Signal Processing Conference (EUSIPCO), 2022.
  8. S.Das, A.M. Javid, P.B. Gohain, Y.C. Eldar, S. Chatterjee. Neural Greedy Pursuit for Feature Selection, In the Proceedings of the International Joint Conference on Neural Networks (IJCNN), IEEE WCCI 2022.
  9. X Liang, AM Javid, M Skoglund, S Chatterjee, Decentralized learning of randomization-based neural networks with centralized equivalence, AppliedSoft Computing, 2022.
  10. David Broman. Interactive Programmatic Modeling. In ACM Transactions on Embedded Computing Systems (TECS), Volume 20, Issue 4, Article No 33, Pages 1-26, ACM, 2021.
  11. Mohamed R.-H. Abdalmoaty, Oscar Eriksson, Robert Bereza, David Broman and Håkan Hjalmarsson. Identification of Non-Linear Differential-Algebraic Equation Models with Process Disturbances, In Proceedings of the IEEE Conference on Decision and Control (CDC), 2021.
  12. Mina Ferizbegovic, Per Mattsson, Thomas Schön, and H. Hjalmarsson. Bayes Control of Hammerstein Systems, 19th IFAC Symposium on System Identification, 2021.
  13. Daniel Lundén, Johannes Borgström, and David Broman. Correctness of Sequential Monte Carlo Inference for Probabilistic Programming Languages. In Proceedings of 30th European Symposium on Programming (ESOP), LNCS vol. 12648, Springer, 2021.
  14. Viktor Palmkvist, Elias Castegren, Philipp Haller, and David Broman. Resolvable Ambiguity: Principled Resolution of Syntactically Ambiguous Programs. In Proceedings of the 30th ACM SIGPLAN International Conference on Compiler Construction (CC), ACM, 2021.
  15. A.M. Javid, S. Das, M. Skoglund and S. Chatterjee, A ReLU Dense Layer to Improve the Performance of Neural Networks, in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021.
  16. X. Liang, M. Skoglund, and S. Chatterjee, Feature reuse for a randomization based neural network, in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021.
  17. P.G. Jurado X. Liang, A.M. Javid, and S. Chatterjee, Use of deterministic transforms to design weight matrices of a neural network, in European Signal Processing Conference (EUSIPCO), 2021.
  18. Díogo Rodrigues, Mohamed R. Abdalmoaty, Elling W. Jacobsen, Véronqiue Chotteau, and Håkan Hjalmarsson. An Integrated Approach for Modeling and Identification of Perfusion Bioreactors via Basis Flux Modes, Computers & Chemical Engineering, 2021.
  19. Aymen Al Marjani and Alexandre Proutiere. Adaptive Sampling for Best Policy Identification in Markov Decision Processes. In Proceedings of International Conference on Machine Learning (ICML), 2021.
  20. Aymen Al Marjani, Aurélien Garivier, and Alexandre Proutiere: Navigating to the Best Policy in Markov Decision Processes. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2021.
  21. Damianos Tranos and Alexandre Proutiere: Regret Analysis in Deterministic Reinforcement Learning. In Proceedings of the IEEE Conference on Decision and Control (CDC), 2021.
  22. Zhang, L., Wang, M., Castan, A., Hjalmarsson, H. and Chotteau, V., Probabilistic model by Bayesian network for the prediction of antibody glycosylation in perfusion and fed‐batch cell cultures. Biotechnology and Bioengineering, 118(9), pp.3447-3459, 2021.
  23. Chotteau, V., Hagrot, E., Zhang, L., Mäkinen, M. Mathematical modelling of cell culture processes. In Cell Culture Engineering and Technology (pp. 467-484). Springer, Cham, 2021.
  24. Mingliang Wang, Riccardo S. Risuleo, Elling W. Jacobsen, Véronique Chotteau and Håkan Hjalmarsson. Identification of Nonlinear Kinetics of Macroscopic Bio-reactions Using Multilinear Gaussian Processes, Computers and Chemical Engineering, 123(2), 2020.
  25. Stefanie Fonken, Mina Ferizbegovic, and H. Hjalmarsson. Consistent Identification of Dynamic Networks Subject to White Noise Using Weighted Null-Space Fitting, In Proceedings of the IFAC 2020 World Congress, 2020.
  26. Riccardo S. Risuleo and Håkan Hjalmarsson. Nonparametric Models for Hammerstein-Wiener and Wiener-Hammerstein System Identification, In Proceedings of the IFAC 2020 World Congress, 2020.
  27. Saranya Natarajan and David Broman. Temporal Property-Based Testing of a Timed C Compiler using Time-Flow Graph Semantics. In the Proceedings of the Forum on specification & Design Languages (FDL 2020), IEEE, 2020.
  28. Mohamed R. Abdalmoaty and Håkan Hjalmarsson. Identification of Stochastic Nonlinear Models Using Optimal Estimating Functions, Automatica, 119, 2020. Mina Ferizbegovic, Jack Umenberger, Håkan Hjalmarsson and Thomas Schön. Learning robust LQ-controllers using application oriented exploration, IEEE Control Systems Letters, 4(4):19-24 and jointly published in the Proceedings of the IEEE Conference on Decision and Control (CDC), 2020.
  29. A.M. Javid, A. Venkitaraman, M. Skoglund, S. Chatterjee, High-dimensional neural feature design for layer-wise reduction of training cost, EURASIP Journal on Advances in Signal Processing, 2020.
  30. Yassir Jedra and Alexandre Proutiere. Finite-time Identification of Stable Linear Systems Optimality of the Least-Squares Estimator. In Proceedings of the IEEE Conference on Decision and Control (CDC), 2020.
  31. Filippo Vannella, Jaeseong Jeong, and Alexandre Proutiere. Off-policy Learning for Remote Electrical Tilt Optimization. VTC-Fall, 2020.
  32. Jack Umenberger, Mina Ferizbegovic, Thomas Schön and Håkan Hjalmarsson. Robust exploration in linear quadratic reinforcement learning, In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2019 (Spotlight paper).
  33. Yassir Jedra, Alexandre Proutiere. Sample Complexity Lower Bounds for Linear System Identification. In Proceedings of the IEEE Conference on Decision and Control (CDC), 2019.

About the project

Objective
The goal of the collaborative impact project is to leverage the novel technical and theoretical research results from the data-limited learning project to create added value and technical transfer to the Swedish industry (Saab and AstraZeneca) and to create interactive physical demonstrators for the Digital Futures Hub and the general public.

Background
This impact project is an extension to the 4-year project Data-Limited Learning of Complex Dynamical Systems (DLL), where the new project focuses on impact activities, software prototypes, and demonstrators. See the following for the previous DLL project.

The previous DLL project consisted of three connected sub-projects (i) bioprocessing, (ii) reinforcement learning for cyber-physical systems, and (iii) theoretical foundation. Within these sub-projects, our project has resulted in significant research results. In this new collaborative impact project, we focus on a subset of these results within the three tasks. The key aspect of the usefulness of this new project is to take the results from a theoretical or pure academic setting, to create demonstrators for the public, and to enable technical transfer to the Swedish industry.

Cross-disciplinary collaboration
This project involves co-PIs and researchers from different disciplines, including computer science, automatic control, machine learning, and biotechnology. The project is divided into three main project tasks, each aiming for separate impact activities:

Objective
With the integration of information and communications technology and intelligent electric devices, substation automation systems (SAS) greatly boost the efficiency of power system monitoring and control. However, substations also bring new vulnerabilities at the frontier of a bulk power system’s wide-area monitoring and control infrastructure. They are known to be attractive targets for attackers. In this project, we will research, develop, and validate algorithms that defend against cyberattacks that aim to disrupt substation operations by maliciously changing measurements and/or spoofing spurious control commands.

We propose multiple use-inspired AI innovations that crucially leverage concurrent capabilities of SAS to transform the cyber security of power systems, including (i) a framework that synergizes optimization-based attack modelling with inverse reinforcement learning for multi-stage attack detection, (ii) a decision-focused distributed CPS modelling approach, and (iii) a mathematical program with equilibrium constraints framework of adversarial unlearning for spoofing detection.

Background
In the IEC 61850-based Substation Automation System (SAS), integrating computing and communication technologies with Intelligent Electric Devices (IEDs) greatly enhances the efficiency of power system monitoring and control. The fast-growing connectivity via wide area networks (WAN) enables powerful automation functions but also brings cyber vulnerabilities concerning new attack vectors. The substations are known to be attractive targets for attackers since they form the frontier of the wide-area monitoring and control infrastructure of a bulk power system, which consists of a Supervisory Control And Data Acquisition (SCADA) system, an Energy Management System (EMS), and a control centre.

Cyberattacks at SASs may be performed by maliciously changing measurements from IEDs and merging units (MUs) and/or spoofing spurious control commands for one or more switching devices from IEDs. An attack can alter a device’s configuration even if commands and data comply with syntax, protocol, and the targeted device. The vulnerabilities of the modern grid are many, as described in a National Academies Report.

Crossdisciplinary collaboration
Anomaly detection can reduce cyber threats to substations and improve root cause analysis. Traditional anomaly data detection heavily relies on human experts to design rule-based detection mechanisms, which can be time-consuming, inefficient, less adaptive, and labour-intensive. More recently, sophisticated anomaly detection methods have been reported in the literature. Still, they largely ignore the special characteristics of attacks on SAS and practical system-level constraints on communication and computation.

Transformative and disruptive applications of use-inspired AI for SAS anomaly detection are in their infancy. The proposed project is among the first known efforts to develop and demonstrate AI-enabled SAS anomaly data detection that crucially leverages the cross-disciplinary collaboration between substation Information engineering and Communications Technology (especially distributed machine learning) for cyber defence.

The project is a collaboration between the University of California Berkeley, Virginia Tech and KTH Royal Institute of Technology.

About the project

Objective
Soil carbon sequestration in croplands has tremendous potential to help mitigate climate change; however, it is challenging to develop optimal management practices to maximise the sequestered carbon and crop yield. This project aims to develop an intelligent agricultural management system using deep reinforcement learning (RL) and large-scale soil and crop simulations. To achieve this, we propose to build a simulator to model and simulate the complex soil-water-plant-atmosphere interactions, which will run on high-performance computing platforms.

Background
Massive simulations using such platforms allow the evaluation of the effects of various management practices under different weather and soil conditions in a timely and cost-effective manner. By formulating the management decision as an RL problem, we can leverage the state-of-the-art algorithms to train management policies, which are expected to maximise the stored organic carbon while maximising the crop yield. The whole system will be tested using data on soil and crops in both the mid-west of the United States and the Mediterranean region. The proposed research has great potential to impact climate change and food security, two of humanity’s most significant challenges.

Crossdisciplinary collaboration
This project is a collaboration between the University of Illinois at Urbana-Champaign, KTH Department of Sustainable Development and Stockholm University, Department of Physical Geography.

About the project

Objective
In this project, Princeton University and KTH Royal Institute of Technology plan to develop a family of C3.ai-enabled methods for learning, optimization, and system stability analysis of grid-tied inverters and power electronics-based power systems. The goal is to develop a unified machine-learning platform for power electronics, power systems, and data science research. A bottom-up approach, from modelling a single inverter to modelling a cluster of inverters connected as a microgrid, will be used as a motivating case study to show a holistic hierarchical modelling approach supported by C3.ai.

Background
Distributed and renewable energy resources are massively deployed in electric power grids, driven by the sharp cost reduction and the demand for carbon neutrality. The grid of the future will be supported by clouds of distributed and renewable energy resources. Power electronics Inverters are pervasively needed to connect renewable energy resources to the grid, thanks to their full controllability over electric power. On the other hand, to meet a wide range of grid requirements, these grid-tied inverters are commonly equipped with sophisticated control systems, which pose challenges to the stability and control of power grids, threatening the energy security of modern society.

Crossdisciplinary collaboration
This project is a collaboration between KTH and Princeton University.

Objective
We propose a solution using machine learning and test generation, leveraging machine learning expertise from UIUC and testing and verification from KTH. Unlike previous approaches, we focus on explainable AI in our safety cage so that the cage itself and its effects on network traffic can be inspected and validated. Lightweight approaches guarantee that our safety cage can be embedded in programmable networks or operating system kernels. Machine learning will learn behavioural models that have their roots in formal modelling (access policies, protocol states, Petri Nets) and thus are inherently readable by humans. The test-case generation will validate diverse traces against the model and showcase potential malicious behaviour, validating both positive and negative outcomes.

Background
Industrial robots usually operate within a “safety cage” to ensure that a robot does not harm workers. We need the same type of security, simple and explainable, for IT systems. Novel mechanisms that can be embedded in the network, such as through hardware-accelerated programmable networks or kernel extensions, enable this type of security at the network level.

Crossdisciplinary collaboration
The project is a collaboration between the University of Illinois at Urbana-Champaign and the KTH Royal Institute of Technology. KTH will combine its experience in testing and verification with UIUC’s expertise in machine learning.

About the project

Objective
The team will address five objectives regarding cyberattacks on power systems based on state-of-the-art AI methods: (1) designing graph neural networks that can process power data to learn the state of the system and detect cyberattacks; (2) developing AI algorithms that utilize image recognition techniques using convolutional neural networks to detect denial of view and image replays resulting from cyberattacks; and (3) developing optimization techniques to robustify previously designed neural networks against adversarial data. Selecting power system operating points and policies through attack-aware methods creates a resilient system. If an attack is not immediately sensed, operating from such a position of strength buys time for detection algorithms. Objectives 4 and 5 aim to develop attack-aware AI methods via distributionally robust optimization and cascading failure analysis.

Background
The operation of power systems is becoming data-centric to improve the efficiency, resiliency, and sustainability of power systems and address climate change. Major operational problems, such as security-constrained optimal power flow, contingency analysis, and transient stability analysis, rely on the knowledge extracted from sensory data. Data manipulation by a malicious actor tampers with grid operation, with catastrophic consequences, including physical equipment damage and cascading failures. Developing frameworks and methodologies that help power operators protect the power grid against such malicious attacks is paramount to national security.

Crossdisciplinary collaboration
The project is a collaboration between the University of California Berkeley, California Institute of Technology, KTH Royal Institute of Technology and Electric Power Research Institute. Assistant Professor Jan Kronqvist leads the research in the Department of Mathematics at KTH. At KTH, the research is focused on developing optimization techniques to robustify previously designed neural networks against adversarial data and the fundamental mathematical theory needed to develop such optimization techniques.

Contacts at other participating institutes:

Javad Lavaei, Associate Professor, Industrial Engineering and Operations Research, University of California, Berkeley
Somayeh Sojoudi, Assistant Professor of Electrical Engineering & Computer Science, University of California, Berkeley
Steven Low, Professor of Computing and Mathematical Sciences and Electrical Engineering, California Institute of Technology
Jeremy Lawrence, Principal Technical Leader at Electric Power Research Institute, Electric Power Research Institute