Hi João, you have an extensive background in autonomous vehicles and cyber-physical systems. Could you share some insights into your research interests and how they have evolved over the years?

– My main research is about systems of autonomous maritime vehicle systems, including underwater, surface, and air vehicles, with applications to the ocean sciences, oil and gas and wind farms, security, and defense. The vision is to design and deploy systems of systems (SoS) that will deliver novel capabilities that are well beyond what can be delivered by isolated vehicles, or even by robotic swarms (as recognized by some of the proponents of robotic swarming approaches). Important concepts are teams, intra and inter-team coordination, team formation and maintenance, dynamic allocation of vehicles to teams, as well as high-level planning and execution control at the system of systems level. Most of the initial insights about this research area were developed when I was studying and working at the University of California at Berkeley (UCB) in the early 2000’s. I was a member of the SHIFT language development group, focused on the design and implementation of a modeling language for dynamic networks of hybrid automata – dynamic networks are an important component of SoS. The development of the language was partly inspired by UCB pioneering work in layered control architectures for automated highway systems (AHS) – the layering principles still inform our current designs. AHS work also highlighted the merits and limitations of formal verification, strongly influenced by Josepf Sifakis, Turing Award in 2007 along with Edmund M. Clarke and E. Allen Emerson, for their work on model checking. The compositional inspiration for SoS came from the ONR-funded Mobile Offshore Base, in which we developed a control architecture and implementation for scaled models to enable independent semi-submersible scaled platforms to form a runway, as it should happen in the middle of the ocean. Yet, in another project funded by DARPA, I started developing teaming concepts to solve a complex coordination problem under adversarial behavior. These concepts provided the inspiration for the design of a layered architecture of controllers in which vehicles, teams, sub-teams, and tasks controllers were dynamically created and linked for teams to perform complex missions.

Some of these insights and developments were later transitioned to developments in maritime robotics at the Laboratório de Sistemas e Tecnologias Subaquáticas (LSTS). This was at a time when the focus in maritime robotics was mainly about single vehicle operations in the presence of the challenges posed by the underwater environment (e.g., no GPS or refueling stations underwater, fast attenuation of radio waves, slow and unreliable acoustic communications, etc.). While the underwater environment presented major challenges for the deployment of isolated vehicles, it also presented opportunities to innovate in cooperative robotics to overcome the localization, power consumption, and communication challenges.

It was then that we understood the need to develop low-cost vehicles to test our novel developments in multi-vehicle systems – commercial-of-the-shelf vehicles were just too expensive and were mostly based on closed software systems, thus precluding their use for research and development. Until then, our developments were mainly about a technological push. The first application’s pull came from the Portuguese Navy. This started a unique partnership, initially focused on the development of an autonomous underwater vehicles program. In the process, we learned a lot from the operational know-how of the Portuguese Navy and strengthened our partnership.

Our operational know-how has been key to successful LSTS campaigns in the Atlantic, Arctic and Pacific Oceans, as well as in the Mediterranean and Adriatic seas. These campaigns added another dimension to our research: interdisciplinary ocean studies with robotic vehicles and ship-robotic surveys. This new dimension prompted the development of novel interdisciplinary approaches to find, track, and sample dynamic features of the ocean with unprecedented spatial and temporal resolution. Currently, we are especially interested in closing the observation, modeling, assimilation, and sampling cycle with networked unmanned vehicles and ship-robotic surveys. This is not a simple endeavor. In fact, the problem of modeling and assimilation is especially challenging because the spatial and temporal resolutions of ocean models are typically not consistent with the high-resolution data provided by unmanned vehicles. Moreover, lower resolution models fail, in some cases, to capture, in an aggregate fashion, behaviors exhibited in higher resolution models. Recent work in the field highlights the role of high-resolution modelling, sampling, and machine learning as one promising approach to address this problem. A reference should be made here to the fact that Syukuro Manabe and Klaus Hasselmann, two of the three 2021 Nobel laureates in Physics, were awarded “for the physical modelling of Earth’s climate, quantifying variability and reliably predicting global warming”.

In 2010 we started, in cooperation with the Portuguese Navy, what is known now as the Robotic Experimentation and Prototyping with Maritime Unmanned Systems (REPMUS) exercise. Today, this is the largest exercise of its kind in the world, co-organized by the Portuguese Navy, Porto University, the European Defense Agency, and the Maritime Unmanned Systems initiative and the Center for Maritime Research and Experimentation from NATO. The success of the exercise, developed in the framework of the triple helix (academia, industry, and armed forces), is predicated on the evaluation and testing of new developments in operational environments. REPMUS is also pushing the envelope in interoperability into interchangeability (I2I), with tens of networked heterogenous underwater, surface, and aerial vehicles coordinated with the help of a hierarchy of human operators, tactical units, and commanders. In doing so, the exercise provides a glimpse of future SoS operations, as well as additional motivation and challenges to our developments.

Finally, we observe that SoS are cyber-physical systems in which physical and computational dynamics are tightly coupled, while physical entities (e.g., vehicles) and computational entities (e.g., processes) dynamically interact over time – e.g., computational processes can migrate between vehicles and vehicles may interact with other vehicles, that may come and go. This is why research in SoS is highly interdisciplinary, at the intersection of control and computation. For example, models of SoS will have to incorporate, in addition to vehicle dynamic models, other types of dynamics, including mobile linkage and mobile locality, and related concepts of mobile computing and mobile computation. In this respect, the pioneering work of Robin Milner, Turing Award winner in 1991, was truly inspirational, mainly for his development of the Pi-calculus, which models systems with a dynamically changing communication topology, and of Bigraphs, which models dynamic linkage and mobility. This is done with the help of a place graph and of a link graph (actually, a hypergraph) together with reaction rules governing the evolution of the bigraph.

Now, it is the time for analysis, synthesis, evaluation and testing of SoS.

Your leadership at the Laboratório de Sistemas e Tecnologias Subaquáticas (LSTS) has led to significant advancements in underwater vehicle technology. Can you discuss some of the major accomplishments of the LSTS, including the design of the Light Autonomous Underwater Vehicle and the LSTS open-source software toolchain?

– We have been working for over 20 years in the field. I will briefly allude to some key accomplishments. One of the most significant accomplishments has to do with our trajectory as a laboratory. Starting from scratch, we designed, built, and deployed multiple vehicles and developed a reputation for excellence in networked vehicle systems.

Another significant accomplishment was the development of the award-winning Light Autonomous Underwater Vehicle (LAUV), which started almost two decades ago at a time when mainstream developments were focused on bigger and very expensive vehicles. In fact, we demonstrated that small vehicles could deliver comparable performance for some applications. The LAUV is commercialized through our spinoff, Oceanscan, Marine Systems and Technologies – there are over 100 units in operation in Europe, North America, and Asia. In addition, we built a frugal fleet of vehicles to evaluate and test novel multi-vehicle systems capabilities. This was at a time when the operation of fleets was prohibitive for most research institutions.

The LSTS open-source software tool chain (https://lsts.fe.up.pt/toolchain) was another major accomplishment. It supports underwater, surface, and aerial vehicles, as well as inter-operated communications networks and interoperability. One decade later, our software still provides some unmatched functionalities, and it is evolving to become a powerful cloud-based system.

Our unique cooperation with the Portuguese Navy is still a case study and an example to follow. In addition to having co-developed an underwater vehicle program for the Portuguese Navy, we jointly founded the REP, now REPMUS exercise, the largest exercise of its kind in the world.

Yet, another relevant accomplishment was the coordination of the H2020 EU Marine robotics research infrastructure network project. The project mobilized a comprehensive consortium of most of the key marine robotics players in Europe to open-up marine robotics research infrastructures to European and worldwide researchers and to establish a world-class marine robotics integrated infrastructure. Access to infrastructures was granted through open calls. Close to 60 projects from all over the world were granted access to our infrastructures, thus providing a glimpse of future best practices in cooperation and utilization of robotic infrastructures. Projects ranged from the exploration of the Blanes canyon with a HROV deployed from a ship, both from Ifremer, to cooperative control of multiple LAUVs from LSTS.

In 2018, we led an international team on board the R/V Falkor during the highly successful Schmidt Ocean Institute cruise entitled Exploring Fronts with Multiple Robots. The goal of the cruise was to demonstrate a novel approach to observe the ocean with multiple autonomous underwater, surface, and aerial vehicles operated from the R/V Falkor. These vehicles were powered by our software toolchain to study the Northern Pacific Subtropical front, located approximately 1000 nautical miles west of San Diego, United States of America. The duration of the cruise was 3 weeks, with two weeks on site working non-stop in 4 daily shifts.

Currently, we are developing a Command-and-Control Center for Atlantic Operations with the goal of developing, testing, and demonstrating long endurance operations in the Atlantic, with special emphasis on the triangle joining Continental Portugal and the Madeira and Azores islands. Operations will involve not only our expanding fleet, including long endurance vehicles, but also vehicles from partners and associates. The goal is also to connect our center to similar centers in Europe and the United States for cooperative missions to take place in the Atlantic.

You’ve been instrumental in fostering a global research community through conferences and workshops on hybrid systems and autonomous vehicles. How do you see the future of this field evolving, and what opportunities or challenges do you anticipate?

– Maritime unmanned vehicle systems are already impacting the ocean sciences, security and defense, offshore oil, gas, and wind projects, etc. But this is just the beginning. In fact, most of the current deployments are still based on single vehicle operations, closed software systems, limited interactions with other systems, and significant support from human operators. However, as ocean robotic operations become more complex, with increasing numbers of platforms, operators, manned vessels, and interactions, novel forms of organization, well beyond what is done today, will have to be developed. This will require a paradigm shift from operations with a few vehicle systems to systems of systems and underlying organizing principles. However, this paradigm shift presents yet another challenge summarized as Martec’s law: “Technology changes exponentially fast, yet organizations change logarithmically slow”. As the exponential / logarithmic gap widens over time, development and organizational resets may be needed. This will have profound impacts on models of engineering development, as well as on systems’ life cycles.

Complexity cannot be addressed in isolation. SoS are evolving into complex network structures, powered by Artificial Intelligence, targeting varied applications, and requiring unprecedent forms of interdisciplinary work. Similarly, research and development should become more networked and interdisciplinary, with agile development models closed around evaluation and testing, first in Digital Twins, before transitioning to operational environments. But the promise of Digital Twins should not be overstated (e.g., ocean models with varying degrees of spatial and temporal resolution may not be consistent), and more research into solving these challenges is needed. We can only speculate at this stage, but it seems that networked research and development could be one the ways for organizations to match exponential technological growth and complexity.

Finally, we will be facing significant challenges in cyber-security, ethics, interoperability, interchangeability, standardization, legal frameworks, and frugality, to name just a few.

As a member of various committees and advisory boards, including the NATO MUS Innovation Advisory Board and the Advisory Board of the Swedish Marine Robotics Center, how do you envision collaboration and innovation within the international research community in the coming years?

In line with my previous answer, international collaboration and innovation networks will be key to addressing the challenges posed by exponential technological growth and market demand. Moreover, some collaboration efforts should be tentatively aligned with developments in other areas (e.g., space). This is why some of the space agencies are considering underwater environments as space analogues, closer to home, to test space systems and technologies.

The effectiveness of collaboration and innovation networks poses significant challenges in terms of coordination mechanisms and integrity of intellectual property, to name just a few. Examples coming from projects like the H2020 EU Marine robotics research infrastructure network or from large scale exercises like REPMUS may provide templates and insights into effective work practices.

Last, but not least, you’re a Digital Futures Scholar in residence from February 2024 to September 2024 – can you elaborate on your experience so far? What are your main objectives during this period? What do you expect to be the key outcomes or achievements?

– I have just finished my first weeks with Digital Futures. I really like the informal and friendly atmosphere, the open space, the design of the building, and, of course, the interdisciplinary work being developed here. This is simply fantastic and intellectually stimulating, as I would expect from Digital Futures and KTH.

I had excellent discussions about the future of marine robotics with my host, Professor Ivan Stenius, and with other colleagues from SMARC. I had fantastic discussions about models of systems with dynamic structure, machine learning, and control of networked vehicles with Professor Karl Johansson and other colleagues from the division of decision and control systems. I am very grateful to all of them. But this is just the beginning of my Scholar in Residence program. Now, I will be away for several weeks, but I will stay in touch and will work on concrete plans for future collaborative efforts, aiming at addressing fundamental problems in models of computation and layered control architectures for multi-vehicle systems and nested ocean models. Equally important will be the co-development, with colleagues in the ocean sciences, of novel robotic systems for sustained, and economically sustainable, ocean exploration.

I am ambitious and tend to be very optimistic, but I will do my best to develop and strengthen an international network of collaborations at the intersection of science and technology with goal of starting innovative coursework development and collaborative projects that will help us to understand and monitor how key issues such as climate change, ocean acidification, unsustainable fishing, pollution, waste, loss of habitats and biodiversity, shipping, security, and mining are affecting global ocean sustainability and stewardship.

Of course, I will have to finish in Portuguese by saying “Muito obrigado”.