Unmanned maritime vehicles are more and more becoming important assets for military operations, not only on the battlefield, but also in logistics and in supportive roles. This also means that more and more of these vehicles are being deployed in the field, each with their own characteristics and specifications. This variety poses interoperability problems when deploying these assets together for operations, as there is to date no unified command structure for these unmanned assets and there are also no standardized data interchange platforms that allow for an easy transfer of sensory information from one platform to another. The result is that field commanders have to work in most circumstances with custom-built solutions that may be good in performing one task well, but that encompass little flexibility and modularity towards upgrading the task description for future needs or towards interoperability with other deployed assets. This study proposal aims to remedy that situation by developing an interoperability concept for heterogeneous unmanned assets that allows for the seamless integration of these tools into existing standard operating procedures.
Interoperability is the key that acts as the glue among the different units within the team, enabling efficient multi-robot cooperation. Seamless and non-ambiguous interaction between different robots of any provider and domain demands a common, well-defined interface. The ultimate goal of the work on the heterogeneous team is to consolidate a common command, control and payload interface to be agreed and adopted by all robotics platforms and control stations involved in an operation. This approach provides a common framework for the development of collaborative unmanned assets, minimizing the integration time and costs by avoiding ad-hoc implementations.
As a key concept, we develop a heterogeneous interoperability and collaboration framework which is seamlessly interoperable with the existing and future C4I and GIS infrastructure. The interoperability concept consists of a highly modular system of carrier platforms and sensor payloads, enabling straightforward switching of payloads from one system to another. This modular architecture will also ensure the future exploitability of the results of this project on other assets after the end of the project.
In terms of validation, three use cases are under preparation, each geared at a different application domain to maximize the exploitation potential of the IP developed within this study and to validate the proposed interoperability concept in a wide range of applications:
- Shared situational awareness and data traffic management for unmanned dredging vessels
This is fits in a VLAIO – Blue Cluster research and innovation project (SSAVE) studying the data management, cybersecurity and interoperability questions that are posed when (unmanned) assets start to exchange data on all levels (not only towards the cloud, but also between the edges). Preparatory work has been performed this year on unmanned surface vessels navigating autonomously within the port of Antwerp. - Maritime surveillance of offshore windmill parks
Within the scope of this validation use-case an off-shore windmill park has been outfitted with technology (hardware and software) to automatically detect trespassing vessels. - Unmanned mine counter measures
This validation use case fits into the much larger PESCO-MAS-MCM framework and is still under preparation.
Royal Higher Institute for Defence
with support from dotOcean and the Belgian Navy
2019 – 2023
0.8 M€
Project Publications
2021
- G. De Cubber, R. Lahouli, D. Doroftei, and R. Haelterman, “Distributed coverage optimisation for a fleet of unmanned maritime systems," ACTA IMEKO, vol. 10, iss. 3, pp. 36-43, 2021.
[BibTeX] [Abstract] [Download PDF] [DOI]
Unmanned maritime systems (UMS) can provide important benefits for maritime law enforcement agencies for tasks such as area surveillance and patrolling, especially when they are able to work together as one coordinated system. In this context, this paper proposes a methodology that optimises the coverage of a fleet of UMS, thereby maximising the opportunities for identifying threats. Unlike traditional approaches to maritime coverage optimisation, which are also used, for example, in search and rescue operations when searching for victims at sea, this approach takes into consideration the limited seaworthiness of small UMS, compared with traditional large ships, by incorporating the danger level into the design of the optimiser.
@ARTICLE{cubberimeko2021, author={De Cubber, Geert and Lahouli, Rihab and Doroftei, Daniela and Haelterman, Rob}, journal={ACTA IMEKO}, title={Distributed coverage optimisation for a fleet of unmanned maritime systems}, year={2021}, volume={10}, number={3}, pages={36-43}, issn={2221-870X}, url={https://acta.imeko.org/index.php/acta-imeko/article/view/IMEKO-ACTA-10%20%282021%29-03-07/pdf}, project={MarSur, SSAVE}, publisher={IMEKO}, month=oct, abstract = {Unmanned maritime systems (UMS) can provide important benefits for maritime law enforcement agencies for tasks such as area surveillance and patrolling, especially when they are able to work together as one coordinated system. In this context, this paper proposes a methodology that optimises the coverage of a fleet of UMS, thereby maximising the opportunities for identifying threats. Unlike traditional approaches to maritime coverage optimisation, which are also used, for example, in search and rescue operations when searching for victims at sea, this approach takes into consideration the limited seaworthiness of small UMS, compared with traditional large ships, by incorporating the danger level into the design of the optimiser. }, doi={http://dx.doi.org/10.21014/acta_imeko.v10i3.1031}, unit= {meca-ras}} - Y. Baudoin, G. De Cubber, and E. Cepolina, “Mobile Robots Supporting Risky Interventions, Humanitarian actions and Demining, in particular the promising DISARMADILLO Tool," in Proceedings of TC17-VRISE2021 – A VIRTUAL Topical Event of Technical Committee on Measurement and Control of Robotics (TC17), International Measurement Confederation (IMEKO), Theme: “Robotics for Risky Interventions and Environmental Surveillance", Houston, TX, USA, 2021, pp. 5-6.
[BibTeX] [Download PDF]@INPROCEEDINGS{knvrise, author={Baudoin, Yvan and De Cubber, Geert and Cepolina, Emanuela}, booktitle={Proceedings of TC17-VRISE2021 - A VIRTUAL Topical Event of Technical Committee on Measurement and Control of Robotics (TC17), International Measurement Confederation (IMEKO), Theme: "Robotics for Risky Interventions and Environmental Surveillance"}, title={Mobile Robots Supporting Risky Interventions, Humanitarian actions and Demining, in particular the promising DISARMADILLO Tool}, year={2021}, volume={}, number={}, pages={5-6}, url={https://mecatron.rma.ac.be/pub/2021/TC17-VRISE2021-Abstract%20Proceedings.pdf}, project={AIDED, Alphonse, MarSur, SSAVE, MarLand, iMUGs, ICARUS, TIRAMISU}, publisher={IMEKO}, address={Houston, TX, USA}, month=oct, unit= {meca-ras} }
2020
- H. Balta, J. Velagic, H. Beglerovic, G. De Cubber, and B. Siciliano, “3D Registration and Integrated Segmentation Framework for Heterogeneous Unmanned Robotic Systems," Remote Sensing, vol. 12, iss. 10, p. 1608, 2020.
[BibTeX] [Abstract] [Download PDF] [DOI]
The paper proposes a novel framework for registering and segmenting 3D point clouds of large-scale natural terrain and complex environments coming from a multisensor heterogeneous robotics system, consisting of unmanned aerial and ground vehicles. This framework involves data acquisition and pre-processing, 3D heterogeneous registration and integrated multi-sensor based segmentation modules. The first module provides robust and accurate homogeneous registrations of 3D environmental models based on sensors’ measurements acquired from the ground (UGV) and aerial (UAV) robots. For 3D UGV registration, we proposed a novel local minima escape ICP (LME-ICP) method, which is based on the well known iterative closest point (ICP) algorithm extending it by the introduction of our local minima estimation and local minima escape mechanisms. It did not require any prior known pose estimation information acquired from sensing systems like odometry, global positioning system (GPS), or inertial measurement units (IMU). The 3D UAV registration has been performed using the Structure from Motion (SfM) approach. In order to improve and speed up the process of outliers removal for large-scale outdoor environments, we introduced the Fast Cluster Statistical Outlier Removal (FCSOR) method. This method was used to filter out the noise and to downsample the input data, which will spare computational and memory resources for further processing steps. Then, we co-registered a point cloud acquired from a laser ranger (UGV) and a point cloud generated from images (UAV) generated by the SfM method. The 3D heterogeneous module consists of a semi-automated 3D scan registration system, developed with the aim to overcome the shortcomings of the existing fully automated 3D registration approaches. This semi-automated registration system is based on the novel Scale Invariant Registration Method (SIRM). The SIRM provides the initial scaling between two heterogenous point clouds and provides an adaptive mechanism for tuning the mean scale, based on the difference between two consecutive estimated point clouds’ alignment error values. Once aligned, the resulting homogeneous ground-aerial point cloud is further processed by a segmentation module. For this purpose, we have proposed a system for integrated multi-sensor based segmentation of 3D point clouds. This system followed a two steps sequence: ground-object segmentation and color-based region-growing segmentation. The experimental validation of the proposed 3D heterogeneous registration and integrated segmentation framework was performed on large-scale datasets representing unstructured outdoor environments, demonstrating the potential and benefits of the proposed semi-automated 3D registration system in real-world environments.
@Article{balta20203Dregistration, author = {Balta, Haris and Velagic, Jasmin and Beglerovic, Halil and De Cubber, Geert and Siciliano, Bruno}, journal = {Remote Sensing}, title = {3D Registration and Integrated Segmentation Framework for Heterogeneous Unmanned Robotic Systems}, year = {2020}, month = may, number = {10}, pages = {1608}, volume = {12}, abstract = {The paper proposes a novel framework for registering and segmenting 3D point clouds of large-scale natural terrain and complex environments coming from a multisensor heterogeneous robotics system, consisting of unmanned aerial and ground vehicles. This framework involves data acquisition and pre-processing, 3D heterogeneous registration and integrated multi-sensor based segmentation modules. The first module provides robust and accurate homogeneous registrations of 3D environmental models based on sensors’ measurements acquired from the ground (UGV) and aerial (UAV) robots. For 3D UGV registration, we proposed a novel local minima escape ICP (LME-ICP) method, which is based on the well known iterative closest point (ICP) algorithm extending it by the introduction of our local minima estimation and local minima escape mechanisms. It did not require any prior known pose estimation information acquired from sensing systems like odometry, global positioning system (GPS), or inertial measurement units (IMU). The 3D UAV registration has been performed using the Structure from Motion (SfM) approach. In order to improve and speed up the process of outliers removal for large-scale outdoor environments, we introduced the Fast Cluster Statistical Outlier Removal (FCSOR) method. This method was used to filter out the noise and to downsample the input data, which will spare computational and memory resources for further processing steps. Then, we co-registered a point cloud acquired from a laser ranger (UGV) and a point cloud generated from images (UAV) generated by the SfM method. The 3D heterogeneous module consists of a semi-automated 3D scan registration system, developed with the aim to overcome the shortcomings of the existing fully automated 3D registration approaches. This semi-automated registration system is based on the novel Scale Invariant Registration Method (SIRM). The SIRM provides the initial scaling between two heterogenous point clouds and provides an adaptive mechanism for tuning the mean scale, based on the difference between two consecutive estimated point clouds’ alignment error values. Once aligned, the resulting homogeneous ground-aerial point cloud is further processed by a segmentation module. For this purpose, we have proposed a system for integrated multi-sensor based segmentation of 3D point clouds. This system followed a two steps sequence: ground-object segmentation and color-based region-growing segmentation. The experimental validation of the proposed 3D heterogeneous registration and integrated segmentation framework was performed on large-scale datasets representing unstructured outdoor environments, demonstrating the potential and benefits of the proposed semi-automated 3D registration system in real-world environments.}, doi = {10.3390/rs12101608}, project = {NRTP,ICARUS,TIRAMISU,MarSur}, publisher = {MDPI}, url = {https://www.mdpi.com/2072-4292/12/10/1608/pdf}, unit= {meca-ras} } - A. Kakogawa, S. Ma, B. Ristic, C. Gilliam, A. K. Kamath, V. K. Tripathi, L. Behera, A. Ferrein, I. Scholl, T. Neumann, K. Krückel, S. Schiffer, A. Joukhadar, M. Alchehabi, and A. Jejeh, Unmanned Robotic Systems and Applications, M. Reyhanoglu and G. De Cubber, Eds., InTech, 2020.
[BibTeX] [Abstract] [Download PDF] [DOI]
This book presents recent studies of unmanned robotic systems and their applications. With its five chapters, the book brings together important contributions from renowned international researchers. Unmanned autonomous robots are ideal candidates for applications such as rescue missions, especially in areas that are difficult to access. Swarm robotics (multiple robots working together) is another exciting application of the unmanned robotics systems, for example, coordinated search by an interconnected group of moving robots for the purpose of finding a source of hazardous emissions. These robots can behave like individuals working in a group without a centralized control.
@Book{de2020unmanned, author = {Atsushi Kakogawa and Shugen Ma and Branko Ristic and Christopher Gilliam and Archit Krishna Kamath and Vibhu Kumar Tripathi and Laxmidhar Behera and Alexander Ferrein and Ingrid Scholl and Tobias Neumann and Kai Krückel and Stefan Schiffer and Abdulkader Joukhadar and Mohammad Alchehabi and Adnan Jejeh}, editor = {Reyhanoglu, Mahmut and De Cubber, Geert}, publisher = {{InTech}}, title = {Unmanned Robotic Systems and Applications}, year = {2020}, month = apr, abstract = {This book presents recent studies of unmanned robotic systems and their applications. With its five chapters, the book brings together important contributions from renowned international researchers. Unmanned autonomous robots are ideal candidates for applications such as rescue missions, especially in areas that are difficult to access. Swarm robotics (multiple robots working together) is another exciting application of the unmanned robotics systems, for example, coordinated search by an interconnected group of moving robots for the purpose of finding a source of hazardous emissions. These robots can behave like individuals working in a group without a centralized control.}, doi = {10.5772/intechopen.88414}, project = {NRTP,ICARUS,MarSur}, url = {https://www.intechopen.com/books/unmanned-robotic-systems-and-applications}, unit= {meca-ras} } - G. De Cubber, R. Lahouli, D. Doroftei, and R. Haelterman, “Distributed coverage optimization for a fleet of unmanned maritime systems for a maritime patrol and surveillance application," in ISMCR 2020: 23rd International Symposium on Measurement and Control in Robotics, Budapest, Hungary, 2020.
[BibTeX] [Abstract] [Download PDF] [DOI]
In order for unmanned maritime systems to provide added value for maritime law enforcement agencies, they have to be able to work together as a coordinated team for tasks such as area surveillance and patrolling. Therefore, this paper proposes a methodology that optimizes the coverage of a fleet of unmanned maritime systems, and thereby maximizes the chances of noticing threats. Unlike traditional approaches for maritime coverage optimization, which are also used for example in search and rescue operations when searching for victims at sea, this approaches takes into consideration the limited seaworthiness of small unmanned systems, as compared to traditional large ships, by incorporating the danger level in the design of the optimizer.
@InProceedings{decubber2020dco, author = {De Cubber, Geert and Lahouli, Rihab and Doroftei, Daniela and Haelterman, Rob}, booktitle = {ISMCR 2020: 23rd International Symposium on Measurement and Control in Robotics}, title = {Distributed coverage optimization for a fleet of unmanned maritime systems for a maritime patrol and surveillance application}, year = {2020}, month = oct, organization = {ISMCR}, publisher = {{IEEE}}, abstract = {In order for unmanned maritime systems to provide added value for maritime law enforcement agencies, they have to be able to work together as a coordinated team for tasks such as area surveillance and patrolling. Therefore, this paper proposes a methodology that optimizes the coverage of a fleet of unmanned maritime systems, and thereby maximizes the chances of noticing threats. Unlike traditional approaches for maritime coverage optimization, which are also used for example in search and rescue operations when searching for victims at sea, this approaches takes into consideration the limited seaworthiness of small unmanned systems, as compared to traditional large ships, by incorporating the danger level in the design of the optimizer.}, project = {SSAVE,MarSur}, address = {Budapest, Hungary}, doi = {10.1109/ISMCR51255.2020.9263740}, url = {http://mecatron.rma.ac.be/pub/2020/conference_101719.pdf}, unit= {meca-ras} }
2019
- G. De Cubber, “Opportunities and threats posed by new technologies," in SciFi-IT, Ghent, Belgium, 2019.
[BibTeX] [Abstract] [Download PDF] [DOI]
The technological evolution is introducing in a fast pace new technologies in our everyday lives. As always, these new technologies can be applied for good causes and thereby give us the opportunity to do many interesting new things. Think for example about drones transporting blood samples between hospitals. However, like always, new technologies can also be applied for bad causes. Think for example about the same drones, but this time transporting bomb parcels instead of blood. In this paper, we focus on a number of novel technologies and discuss how security actors are currently doing their best to maximize the good use of these tools while minimizing the bad use. We will focus on research actions taken by Belgian Royal Military Academy in the domains of: – Augmented reality, and showcase how this technology can be used to improve surveillance operations. – Unmanned Aerial Systems (Drones), and showcase how the potential security threats posed by these systems can be mitigated by novel drone detection systems. – Unmanned Maritime Systems, and showcase how this technology can be used to increase the safety at sea. – Unmanned Ground Systems, and more specifically the autonomous cars, showcasing how to prevent potential cyber-attacks on these future transportation tools.
@InProceedings{de2019opportunities, author = {De Cubber, Geert}, booktitle = {SciFi-IT}, title = {Opportunities and threats posed by new technologies}, year = {2019}, abstract = {The technological evolution is introducing in a fast pace new technologies in our everyday lives. As always, these new technologies can be applied for good causes and thereby give us the opportunity to do many interesting new things. Think for example about drones transporting blood samples between hospitals. However, like always, new technologies can also be applied for bad causes. Think for example about the same drones, but this time transporting bomb parcels instead of blood. In this paper, we focus on a number of novel technologies and discuss how security actors are currently doing their best to maximize the good use of these tools while minimizing the bad use. We will focus on research actions taken by Belgian Royal Military Academy in the domains of: - Augmented reality, and showcase how this technology can be used to improve surveillance operations. - Unmanned Aerial Systems (Drones), and showcase how the potential security threats posed by these systems can be mitigated by novel drone detection systems. - Unmanned Maritime Systems, and showcase how this technology can be used to increase the safety at sea. - Unmanned Ground Systems, and more specifically the autonomous cars, showcasing how to prevent potential cyber-attacks on these future transportation tools.}, doi = {https://doi.org/10.5281/zenodo.2628758}, address = {Ghent, Belgium}, project = {MarSur,SafeShore}, url = {http://mecatron.rma.ac.be/pub/2019/Sci-Fi-It-2019-DeCubber (2).pdf}, unit= {meca-ras} } - G. De Cubber and R. Haelterman, “Optimized distributed scheduling for a fleet of heterogeneous unmanned maritime systems," in 2019 IEEE International Symposium on Measurement and Control in Robotics (ISMCR), Houston, USA, 2019.
[BibTeX] [Abstract] [Download PDF] [DOI]
Due to the increase in embedded computing power, modern robotic systems are capable of running a wide range of perception and control algorithms simultaneously. This raises the question where to optimally allocate each robotic cognition process. In this paper, we present a concept for a novel load distribution approach. The proposed methodology adopts a decentralised approach towards the allocation of perception and control processes to different agents (unmanned vessels, fog or cloud services) based on an estimation of the communication parameters (bandwidth, latency, cost), the agent capabilities in terms of processing hardware (not only focusing on the CPU, but also taking into consideration the GPU, disk & memory speed and size) and the requirements in terms of timely delivery of quality output data. The presented approach is extensively validated in a simulation environment and shows promising properties.
@InProceedings{de2019optimized, author = {De Cubber, Geert and Haelterman, Rob}, booktitle = {2019 {IEEE} International Symposium on Measurement and Control in Robotics ({ISMCR})}, title = {Optimized distributed scheduling for a fleet of heterogeneous unmanned maritime systems}, year = {2019}, month = sep, number = {23}, publisher = {{IEEE}}, address = {Houston, USA}, abstract = {Due to the increase in embedded computing power, modern robotic systems are capable of running a wide range of perception and control algorithms simultaneously. This raises the question where to optimally allocate each robotic cognition process. In this paper, we present a concept for a novel load distribution approach. The proposed methodology adopts a decentralised approach towards the allocation of perception and control processes to different agents (unmanned vessels, fog or cloud services) based on an estimation of the communication parameters (bandwidth, latency, cost), the agent capabilities in terms of processing hardware (not only focusing on the CPU, but also taking into consideration the GPU, disk & memory speed and size) and the requirements in terms of timely delivery of quality output data. The presented approach is extensively validated in a simulation environment and shows promising properties.}, doi = {10.1109/ismcr47492.2019.8955727}, project = {MarSur}, url = {http://mecatron.rma.ac.be/pub/2019/ICMCR-DeCubber.pdf}, unit= {meca-ras} } - A. Coluccia, A. Fascista, A. Schumann, L. Sommer, M. Ghenescu, T. Piatrik, G. De Cubber, M. Nalamati, A. Kapoor, M. Saqib, N. Sharma, M. Blumenstein, V. Magoulianitis, D. Ataloglou, A. Dimou, D. Zarpalas, P. Daras, C. Craye, S. Ardjoune, D. De la Iglesia, M. Mández, R. Dosil, and I. González, “Drone-vs-Bird Detection Challenge at IEEE AVSS2019," in 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2019, pp. 1-7.
[BibTeX] [Abstract] [Download PDF] [DOI]
This paper presents the second edition of the “drone-vs-bird” detection challenge, launched within the activities of the 16-th IEEE International Conference on Advanced Video and Signal-based Surveillance (AVSS). The challenge’s goal is to detect one or more drones appearing at some point in video sequences where birds may be also present, together with motion in background or foreground. Submitted algorithms should raise an alarm and provide a position estimate only when a drone is present, while not issuing alarms on birds, nor being confused by the rest of the scene. This paper reports on the challenge results on the 2019 dataset, which extends the first edition dataset provided by the SafeShore project with additional footage under different conditions.
@INPROCEEDINGS{8909876, author={A. {Coluccia} and A. {Fascista} and A. {Schumann} and L. {Sommer} and M. {Ghenescu} and T. {Piatrik} and G. {De Cubber} and M. {Nalamati} and A. {Kapoor} and M. {Saqib} and N. {Sharma} and M. {Blumenstein} and V. {Magoulianitis} and D. {Ataloglou} and A. {Dimou} and D. {Zarpalas} and P. {Daras} and C. {Craye} and S. {Ardjoune} and D. {De la Iglesia} and M. {Mández} and R. {Dosil} and I. {González}}, booktitle={2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)}, title={Drone-vs-Bird Detection Challenge at IEEE AVSS2019}, year={2019}, volume={}, number={}, pages={1-7}, project = {SafeShore,MarSur}, doi = {10.1109/AVSS.2019.8909876}, abstract = {This paper presents the second edition of the “drone-vs-bird” detection challenge, launched within the activities of the 16-th IEEE International Conference on Advanced Video and Signal-based Surveillance (AVSS). The challenge's goal is to detect one or more drones appearing at some point in video sequences where birds may be also present, together with motion in background or foreground. Submitted algorithms should raise an alarm and provide a position estimate only when a drone is present, while not issuing alarms on birds, nor being confused by the rest of the scene. This paper reports on the challenge results on the 2019 dataset, which extends the first edition dataset provided by the SafeShore project with additional footage under different conditions.}, url = {https://ieeexplore.ieee.org/abstract/document/8909876}, unit= {meca-ras} }