The objective of the Near-Real-Time Processing (NRTP) project is to develop a toolkit allowing military drone operators to obtain high-quality situational awareness data from drone imagery. Examples are near-real time 2D and 3D mapping and automatic detection of enemy troops.
We tackle the problem of the acquisition and processing of spatial data in hazardous, inaccessible, and/or spatially-extensive regions in (near) real time without any physical interaction within dangerous zones. This drone-acquired information is highly beneficial for assessing potential site locations, designing site layouts, generating 3D visualizations, assessing site visibility, calculating volumes, monitoring operational progress and producing records. In addition, geo-referenced spatial models derived from RPAS data can also be integrated and improve the quality of existing operational maps.
Another example of the data analytics based on the 2D/3D representation is change detection. On a regular basis, the aerial vehicle autonomously takes off and executes a flight plan and detects any changes and intruders in the environment, alerting the base commander if necessary. Therefore, we proposed the development of an algorithm for change detection, intruder detection and semantic classification and segmentation to assist the military operators in their decision processes.
Project Publications
2020
- H. Balta, J. Velagic, H. Beglerovic, G. De Cubber, and B. Siciliano, “3D Registration and Integrated Segmentation Framework for Heterogeneous Unmanned Robotic Systems," Remote Sensing, vol. 12, iss. 10, p. 1608, 2020.
[BibTeX] [Abstract] [Download PDF] [DOI]
The paper proposes a novel framework for registering and segmenting 3D point clouds of large-scale natural terrain and complex environments coming from a multisensor heterogeneous robotics system, consisting of unmanned aerial and ground vehicles. This framework involves data acquisition and pre-processing, 3D heterogeneous registration and integrated multi-sensor based segmentation modules. The first module provides robust and accurate homogeneous registrations of 3D environmental models based on sensors’ measurements acquired from the ground (UGV) and aerial (UAV) robots. For 3D UGV registration, we proposed a novel local minima escape ICP (LME-ICP) method, which is based on the well known iterative closest point (ICP) algorithm extending it by the introduction of our local minima estimation and local minima escape mechanisms. It did not require any prior known pose estimation information acquired from sensing systems like odometry, global positioning system (GPS), or inertial measurement units (IMU). The 3D UAV registration has been performed using the Structure from Motion (SfM) approach. In order to improve and speed up the process of outliers removal for large-scale outdoor environments, we introduced the Fast Cluster Statistical Outlier Removal (FCSOR) method. This method was used to filter out the noise and to downsample the input data, which will spare computational and memory resources for further processing steps. Then, we co-registered a point cloud acquired from a laser ranger (UGV) and a point cloud generated from images (UAV) generated by the SfM method. The 3D heterogeneous module consists of a semi-automated 3D scan registration system, developed with the aim to overcome the shortcomings of the existing fully automated 3D registration approaches. This semi-automated registration system is based on the novel Scale Invariant Registration Method (SIRM). The SIRM provides the initial scaling between two heterogenous point clouds and provides an adaptive mechanism for tuning the mean scale, based on the difference between two consecutive estimated point clouds’ alignment error values. Once aligned, the resulting homogeneous ground-aerial point cloud is further processed by a segmentation module. For this purpose, we have proposed a system for integrated multi-sensor based segmentation of 3D point clouds. This system followed a two steps sequence: ground-object segmentation and color-based region-growing segmentation. The experimental validation of the proposed 3D heterogeneous registration and integrated segmentation framework was performed on large-scale datasets representing unstructured outdoor environments, demonstrating the potential and benefits of the proposed semi-automated 3D registration system in real-world environments.
@Article{balta20203Dregistration, author = {Balta, Haris and Velagic, Jasmin and Beglerovic, Halil and De Cubber, Geert and Siciliano, Bruno}, journal = {Remote Sensing}, title = {3D Registration and Integrated Segmentation Framework for Heterogeneous Unmanned Robotic Systems}, year = {2020}, month = may, number = {10}, pages = {1608}, volume = {12}, abstract = {The paper proposes a novel framework for registering and segmenting 3D point clouds of large-scale natural terrain and complex environments coming from a multisensor heterogeneous robotics system, consisting of unmanned aerial and ground vehicles. This framework involves data acquisition and pre-processing, 3D heterogeneous registration and integrated multi-sensor based segmentation modules. The first module provides robust and accurate homogeneous registrations of 3D environmental models based on sensors’ measurements acquired from the ground (UGV) and aerial (UAV) robots. For 3D UGV registration, we proposed a novel local minima escape ICP (LME-ICP) method, which is based on the well known iterative closest point (ICP) algorithm extending it by the introduction of our local minima estimation and local minima escape mechanisms. It did not require any prior known pose estimation information acquired from sensing systems like odometry, global positioning system (GPS), or inertial measurement units (IMU). The 3D UAV registration has been performed using the Structure from Motion (SfM) approach. In order to improve and speed up the process of outliers removal for large-scale outdoor environments, we introduced the Fast Cluster Statistical Outlier Removal (FCSOR) method. This method was used to filter out the noise and to downsample the input data, which will spare computational and memory resources for further processing steps. Then, we co-registered a point cloud acquired from a laser ranger (UGV) and a point cloud generated from images (UAV) generated by the SfM method. The 3D heterogeneous module consists of a semi-automated 3D scan registration system, developed with the aim to overcome the shortcomings of the existing fully automated 3D registration approaches. This semi-automated registration system is based on the novel Scale Invariant Registration Method (SIRM). The SIRM provides the initial scaling between two heterogenous point clouds and provides an adaptive mechanism for tuning the mean scale, based on the difference between two consecutive estimated point clouds’ alignment error values. Once aligned, the resulting homogeneous ground-aerial point cloud is further processed by a segmentation module. For this purpose, we have proposed a system for integrated multi-sensor based segmentation of 3D point clouds. This system followed a two steps sequence: ground-object segmentation and color-based region-growing segmentation. The experimental validation of the proposed 3D heterogeneous registration and integrated segmentation framework was performed on large-scale datasets representing unstructured outdoor environments, demonstrating the potential and benefits of the proposed semi-automated 3D registration system in real-world environments.}, doi = {10.3390/rs12101608}, project = {NRTP,ICARUS,TIRAMISU,MarSur}, publisher = {MDPI}, url = {https://www.mdpi.com/2072-4292/12/10/1608/pdf}, unit= {meca-ras} } - A. Kakogawa, S. Ma, B. Ristic, C. Gilliam, A. K. Kamath, V. K. Tripathi, L. Behera, A. Ferrein, I. Scholl, T. Neumann, K. Krückel, S. Schiffer, A. Joukhadar, M. Alchehabi, and A. Jejeh, Unmanned Robotic Systems and Applications, M. Reyhanoglu and G. De Cubber, Eds., InTech, 2020.
[BibTeX] [Abstract] [Download PDF] [DOI]
This book presents recent studies of unmanned robotic systems and their applications. With its five chapters, the book brings together important contributions from renowned international researchers. Unmanned autonomous robots are ideal candidates for applications such as rescue missions, especially in areas that are difficult to access. Swarm robotics (multiple robots working together) is another exciting application of the unmanned robotics systems, for example, coordinated search by an interconnected group of moving robots for the purpose of finding a source of hazardous emissions. These robots can behave like individuals working in a group without a centralized control.
@Book{de2020unmanned, author = {Atsushi Kakogawa and Shugen Ma and Branko Ristic and Christopher Gilliam and Archit Krishna Kamath and Vibhu Kumar Tripathi and Laxmidhar Behera and Alexander Ferrein and Ingrid Scholl and Tobias Neumann and Kai Krückel and Stefan Schiffer and Abdulkader Joukhadar and Mohammad Alchehabi and Adnan Jejeh}, editor = {Reyhanoglu, Mahmut and De Cubber, Geert}, publisher = {{InTech}}, title = {Unmanned Robotic Systems and Applications}, year = {2020}, month = apr, abstract = {This book presents recent studies of unmanned robotic systems and their applications. With its five chapters, the book brings together important contributions from renowned international researchers. Unmanned autonomous robots are ideal candidates for applications such as rescue missions, especially in areas that are difficult to access. Swarm robotics (multiple robots working together) is another exciting application of the unmanned robotics systems, for example, coordinated search by an interconnected group of moving robots for the purpose of finding a source of hazardous emissions. These robots can behave like individuals working in a group without a centralized control.}, doi = {10.5772/intechopen.88414}, project = {NRTP,ICARUS,MarSur}, url = {https://www.intechopen.com/books/unmanned-robotic-systems-and-applications}, unit= {meca-ras} }
2019
- H. Balta, J. Velagic, G. De Cubber, and B. Siciliano, “Semi-Automated 3D Registration for Heterogeneous Unmanned Robots Based on Scale Invariant Method," in 2019 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Wurzburg, Germany, 2019.
[BibTeX] [Abstract] [Download PDF] [DOI]
This paper addresses the problem of 3D registration of outdoor environments combining heterogeneous datasets acquired from unmanned aerial (UAV) and ground (UGV) vehicles. In order to solve this problem, we introduced a novel Scale Invariant Registration Method (SIRM) for semi-automated registration of 3D point clouds. The method is capable of coping with an arbitrary scale difference between the point clouds, without any information about their initial position and orientation. Furthermore, the SIRM does not require having a good initial overlap between two heterogeneous datasets. Our method strikes an elegant balance between the existing fully automated 3D registration systems (which often fail in the case of heterogeneous datasets and harsh outdoor environments) and fully manual registration approaches (which are labour-intensive). The experimental validation of the proposed 3D heterogeneous registration system was performed on large-scale datasets representing unstructured and harsh outdoor environments, demonstrating the potential and benefits of the proposed 3D registration system in real-world environments.
@InProceedings{balta2019semi, author = {Balta, Haris and Velagic, Jasmin and De Cubber, Geert and Siciliano, Bruno}, booktitle = {2019 {IEEE} International Symposium on Safety, Security, and Rescue Robotics ({SSRR})}, title = {Semi-Automated {3D} Registration for Heterogeneous Unmanned Robots Based on Scale Invariant Method}, year = {2019}, month = sep, publisher = {{IEEE}}, volume = {1}, address = {Wurzburg, Germany}, abstract = {This paper addresses the problem of 3D registration of outdoor environments combining heterogeneous datasets acquired from unmanned aerial (UAV) and ground (UGV) vehicles. In order to solve this problem, we introduced a novel Scale Invariant Registration Method (SIRM) for semi-automated registration of 3D point clouds. The method is capable of coping with an arbitrary scale difference between the point clouds, without any information about their initial position and orientation. Furthermore, the SIRM does not require having a good initial overlap between two heterogeneous datasets. Our method strikes an elegant balance between the existing fully automated 3D registration systems (which often fail in the case of heterogeneous datasets and harsh outdoor environments) and fully manual registration approaches (which are labour-intensive). The experimental validation of the proposed 3D heterogeneous registration system was performed on large-scale datasets representing unstructured and harsh outdoor environments, demonstrating the potential and benefits of the proposed 3D registration system in real-world environments.}, doi = {10.1109/ssrr.2019.8848951}, project = {NRTP}, url = {https://ieeexplore.ieee.org/document/8848951}, unit= {meca-ras} } - N. Nauwynck, H. Balta, G. De Cubber, and H. Sahli, “A proof of concept of the in-flight launch of unmanned aerial vehicles in a search and rescue scenario," ACTA IMEKO, vol. 8, iss. 4, p. 13–19, 2019.
[BibTeX] [Abstract] [Download PDF] [DOI]
This article considers the development of a system to enable the in-flight-launch of one aerial system by another. The article discusses how an optimal release mechanism was developed taking into account the aerodynamics of one specific mothership and child Unmanned Aerial Vehicle (UAV). Furthermore, it discusses the PID-based control concept that was introduced in order to autonomously stabilise the child UAV after being released from the mothership UAV. Finally, the article demonstrates how the concept of a mothership and child UAV combination could be taken advantage of in the context of a search and rescue operation.
@Article{nauwynck2019proof, author = {Nauwynck, Niels and Balta, Haris and De Cubber, Geert and Sahli, Hichem}, journal = {{ACTA} {IMEKO}}, title = {A proof of concept of the in-flight launch of unmanned aerial vehicles in a search and rescue scenario}, year = {2019}, month = dec, number = {4}, pages = {13--19}, volume = {8}, abstract = {This article considers the development of a system to enable the in-flight-launch of one aerial system by another. The article discusses how an optimal release mechanism was developed taking into account the aerodynamics of one specific mothership and child Unmanned Aerial Vehicle (UAV). Furthermore, it discusses the PID-based control concept that was introduced in order to autonomously stabilise the child UAV after being released from the mothership UAV. Finally, the article demonstrates how the concept of a mothership and child UAV combination could be taken advantage of in the context of a search and rescue operation.}, doi = {10.21014/acta_imeko.v8i4.681}, publisher = {{IMEKO} International Measurement Confederation}, project = {ICARUS, NRTP}, url = {https://acta.imeko.org/index.php/acta-imeko/article/view/IMEKO-ACTA-08 (2019)-04-04}, unit= {meca-ras} }
2018
- N. Nauwynck, H. Balta, G. De Cubber, and H. Sahli, “In-flight launch of unmanned aerial vehicles," in International Symposium on Measurement and Control in Robotics ISMCR2018, Mons, Belgium, 2018.
[BibTeX] [Abstract] [Download PDF] [DOI]
This paper considers the development of a system to enable the in-flight-launch of one aerial system by another. The paper will discuss how an optimal release mechanism was developed, taking into account the aerodynamics of one specific mother and child UAV. Furthermore, it will discuss the PID-based control concept that was introduced in order to autonomously stabilize the child UAV after being released from the mothership UAV. Finally, the paper will show how the concept of a mothership UAV + child UAV combination could be usefully taken into advantage in the context of a search and rescue operation.
@InProceedings{nauwynck2018flight, author = {Nauwynck, Niels and Balta, Haris and De Cubber, Geert and Sahli, Hichem}, booktitle = {International Symposium on Measurement and Control in Robotics ISMCR2018}, title = {In-flight launch of unmanned aerial vehicles}, year = {2018}, volume = {1}, abstract = {This paper considers the development of a system to enable the in-flight-launch of one aerial system by another. The paper will discuss how an optimal release mechanism was developed, taking into account the aerodynamics of one specific mother and child UAV. Furthermore, it will discuss the PID-based control concept that was introduced in order to autonomously stabilize the child UAV after being released from the mothership UAV. Finally, the paper will show how the concept of a mothership UAV + child UAV combination could be usefully taken into advantage in the context of a search and rescue operation.}, doi = {10.5281/zenodo.1462605}, file = {:nauwynck2018flight - In Flight Launch of Unmanned Aerial Vehicles.PDF:PDF}, keywords = {Unmanned Aerial Vehicles, Control, Autonomous stabilization, Search and Rescue drones, Heterogeneous systems}, project = {NRTP}, address = {Mons, Belgium}, url = {http://mecatron.rma.ac.be/pub/2018/Paper_Niels.pdf}, unit= {meca-ras} } - H. Balta, J. Velagic, G. De Cubber, W. Bosschaerts, and B. Siciliano, “Fast Statistical Outlier Removal Based Method for Large 3D Point Clouds of Outdoor Environments," in 12th IFAC SYMPOSIUM ON ROBOT CONTROL – SYROCO 2018, Budapest, Hungary, 2018, p. 348–353.
[BibTeX] [Abstract] [Download PDF] [DOI]
This paper proposes a very effective method for data handling and preparation of the input 3D scans acquired from laser scanner mounted on the Unmanned Ground Vehicle (UGV). The main objectives are to improve and speed up the process of outliers removal for large-scale outdoor environments. This process is necessary in order to filter out the noise and to downsample the input data which will spare computational and memory resources for further processing steps, such as 3D mapping of rough terrain and unstructured environments. It includes the Voxel-subsampling and Fast Cluster Statistical Outlier Removal (FCSOR) subprocesses. The introduced FCSOR represents an extension on the Statistical Outliers Removal (SOR) method which is effective for both homogeneous and heterogeneous point clouds. This method is evaluated on real data obtained in outdoor environment.
@InProceedings{balta2018fast01, author = {Balta, Haris and Velagic, Jasmin and De Cubber, Geert and Bosschaerts, Walter and Siciliano, Bruno}, booktitle = {12th IFAC SYMPOSIUM ON ROBOT CONTROL - SYROCO 2018}, title = {Fast Statistical Outlier Removal Based Method for Large {3D} Point Clouds of Outdoor Environments}, year = {2018}, number = {22}, pages = {348--353}, publisher = {Elsevier {BV}}, volume = {51}, abstract = {This paper proposes a very effective method for data handling and preparation of the input 3D scans acquired from laser scanner mounted on the Unmanned Ground Vehicle (UGV). The main objectives are to improve and speed up the process of outliers removal for large-scale outdoor environments. This process is necessary in order to filter out the noise and to downsample the input data which will spare computational and memory resources for further processing steps, such as 3D mapping of rough terrain and unstructured environments. It includes the Voxel-subsampling and Fast Cluster Statistical Outlier Removal (FCSOR) subprocesses. The introduced FCSOR represents an extension on the Statistical Outliers Removal (SOR) method which is effective for both homogeneous and heterogeneous point clouds. This method is evaluated on real data obtained in outdoor environment.}, doi = {10.1016/j.ifacol.2018.11.566}, file = {:balta2018fast - Fast Statistical Outlier Removal Based Method for Large 3D Point Clouds of Outdoor Environments.PDF:PDF}, journal = {{IFAC}-{PapersOnLine}}, project = {NRTP}, address = {Budapest, Hungary}, url = {https://www.sciencedirect.com/science/article/pii/S2405896318332725}, unit= {meca-ras} } - H. Balta, J. Velagic, G. De Cubber, W. Bosschaerts, and B. Siciliano, “Fast Iterative 3D Mapping for Large-Scale Outdoor Environments with Local Minima Escape Mechanism," in 12th IFAC SYMPOSIUM ON ROBOT CONTROL – SYROCO 2018, Budapest, Hungary, 2018, p. 298–305.
[BibTeX] [Abstract] [Download PDF] [DOI]
This paper introduces a novel iterative 3D mapping framework for large scale natural terrain and complex environments. The framework is based on an Iterative-Closest-Point (ICP) algorithm and an iterative error minimization mechanism, allowing robust 3D map registration. This was accomplished by performing pairwise scan registrations without any prior known pose estimation information and taking into account the measurement uncertainties due to the 6D coordinates (translation and rotation) deviations in the acquired scans. Since the ICP algorithm does not guarantee to escape from local minima during the mapping, new algorithms for the local minima estimation and local minima escape process were proposed. The proposed framework is validated using large scale field test data sets. The experimental results were compared with those of standard, generalized and non-linear ICP registration methods and the performance evaluation is presented, showing improved performance of the proposed 3D mapping framework.
@InProceedings{balta2018fast02, author = {Balta, Haris and Velagic, Jasmin and De Cubber, Geert and Bosschaerts, Walter and Siciliano, Bruno}, booktitle = {12th IFAC SYMPOSIUM ON ROBOT CONTROL - SYROCO 2018}, title = {Fast Iterative {3D} Mapping for Large-Scale Outdoor Environments with Local Minima Escape Mechanism}, year = {2018}, number = {22}, pages = {298--305}, publisher = {Elsevier {BV}}, volume = {51}, abstract = {This paper introduces a novel iterative 3D mapping framework for large scale natural terrain and complex environments. The framework is based on an Iterative-Closest-Point (ICP) algorithm and an iterative error minimization mechanism, allowing robust 3D map registration. This was accomplished by performing pairwise scan registrations without any prior known pose estimation information and taking into account the measurement uncertainties due to the 6D coordinates (translation and rotation) deviations in the acquired scans. Since the ICP algorithm does not guarantee to escape from local minima during the mapping, new algorithms for the local minima estimation and local minima escape process were proposed. The proposed framework is validated using large scale field test data sets. The experimental results were compared with those of standard, generalized and non-linear ICP registration methods and the performance evaluation is presented, showing improved performance of the proposed 3D mapping framework.}, doi = {10.1016/j.ifacol.2018.11.558}, journal = {{IFAC}-{PapersOnLine}}, address = {Budapest, Hungary}, project = {NRTP}, url = {https://www.sciencedirect.com/science/article/pii/S2405896318332646}, unit= {meca-ras} }
2017
- D. Lapandic, J. Velagic, and H. Balta, “Framework for automated reconstruction of 3D model from multiple 2D aerial images," in 2017 International Symposium ELMAR, Zadar, Croatia, 2017, pp. 173-176.
[BibTeX] [Abstract] [Download PDF] [DOI]
The paper considers a problem of 3D environment model reconstruction from a set of 2D images acquired by the Unmanned Aerial Vehicle (UAV) in near real-time. The designed framework combines the FAST (Features from Accelerated Segment Test) algorithm and optical flow approach for detection of interest image points and adjacent images reconstruction. The robust estimation of camera locations is performed using the image points tracking. The coordinates of 3D points and the projection matrix are computed simultaneously using Structure-from-Motion (SfM) algorithm, from which the 3D model of environment is generated. The designed framework is tested using real image data and video sequences captured with camera mounted on the UAV. The effectiveness and quality of the proposed framework are verified through analyses of accuracy of the 3D model reconstruction and its time execution.
@INPROCEEDINGS{8124461, author={D. {Lapandic} and J. {Velagic} and H. {Balta}}, booktitle={2017 International Symposium ELMAR}, title={Framework for automated reconstruction of 3D model from multiple 2D aerial images}, year={2017}, volume={}, number={}, pages={173-176}, abstract={The paper considers a problem of 3D environment model reconstruction from a set of 2D images acquired by the Unmanned Aerial Vehicle (UAV) in near real-time. The designed framework combines the FAST (Features from Accelerated Segment Test) algorithm and optical flow approach for detection of interest image points and adjacent images reconstruction. The robust estimation of camera locations is performed using the image points tracking. The coordinates of 3D points and the projection matrix are computed simultaneously using Structure-from-Motion (SfM) algorithm, from which the 3D model of environment is generated. The designed framework is tested using real image data and video sequences captured with camera mounted on the UAV. The effectiveness and quality of the proposed framework are verified through analyses of accuracy of the 3D model reconstruction and its time execution.}, keywords={autonomous aerial vehicles;cameras;feature extraction;image reconstruction;image segmentation;image sensors;image sequences;remotely operated vehicles;video signal processing;automated reconstruction;multiple 2D aerial images;3D environment model reconstruction;UAV;optical flow approach;interest image points;robust estimation;camera locations;image data;3D model reconstruction;unmanned aerial vehicle;adjacent image reconstruction;structure-from-motion algorithm;features from accelerated segment test;Three-dimensional displays;Solid modeling;Image reconstruction;Two dimensional displays;Cameras;Feature extraction;Optical imaging;3D Model reconstruction;Aerial images;Structure from motion;Unmanned aerial vehicle}, doi={10.23919/ELMAR.2017.8124461}, ISSN={}, project={NRTP,ICARUS}, address = {Zadar, Croatia}, publisher={IEEE}, url={https://ieeexplore.ieee.org/document/8124461}, month={Sep.}, unit= {meca-ras} } - H. Balta, “Spatial registration of 3D data from aerial and ground-based unmanned robotic systems," PhD Thesis, 2017.
[BibTeX] [Abstract]
Robotic systems are more and more leaving the protected laboratory environment and entering our daily lives. These robotic entities can come in the form of aerial systems (drones), ground robots or unmanned maritime systems. Each of these robots gathers data about its environment for analysis and reasoning purposes. As more and more robotic systems are deployed, the amount of environmental data gathered by these systems also increases tremendously. This gives rise to a new problem: how to coherently combine the environmental information acquired by different robotic systems into one representation that is both accurate and easy to use by human end-users? In this thesis, we introduce novel methodologies to solve this data fusion problem, by proposing a novel framework for combining heterogeneous 3D data models acquired by different robotic systems, operated in unknown large unstructured outdoor environments into a common homogeneous model. The first proposed novelty of the research work is a fast and robust ground-based 3D map reconstruction methodology for large-scale unstructured outdoor environments. It is based on an enhanced Iterative-Closest- Point algorithm and an iterative error minimization structure, as well as the fast and computational very efficient method for outlier analysis and removal in 3D point clouds. The second proposed novelty of the research work is a registration methodology combining heterogeneous data-sets acquired from unmanned aerial and ground vehicles (UAV and UGV). This is accomplished by introducing a semi-automated 3D registration framework. The framework is capable of coping with an arbitrary scale difference between the point clouds, without any information about their initial position and orientation. Furthermore, it does not require a good initial overlap between the two heterogeneous UGV and UAV point clouds. Our framework strikes an elegant balance between the existing fully automated 3D registration systems (which often fail in the case of heterogeneous data-sets and harsh-outdoor environments) and fully manual registration approaches (which are labour-intensive). A special and defining aspect of this PhD. work was that we did not only focus on investigating scientific and technical innovations but that we also concentrated on bringing these innovations to the terrain in real operational environments in the security context. As an example, we deployed the technological tools developed in the framework of this research work to the field for demining and crisis relief operations in an actual crisis situation. This operational deployment was highly successful, based upon the feedback provided by the end-users.
@PHDTHESIS {phdbalta, author = "Haris Balta", title = "Spatial registration of 3D data from aerial and ground-based unmanned robotic systems", school = "Royal Military Academy of Belgium", year = "2017", project={NRTP,ICARUS,TIRAMISU}, abstract = {Robotic systems are more and more leaving the protected laboratory environment and entering our daily lives. These robotic entities can come in the form of aerial systems (drones), ground robots or unmanned maritime systems. Each of these robots gathers data about its environment for analysis and reasoning purposes. As more and more robotic systems are deployed, the amount of environmental data gathered by these systems also increases tremendously. This gives rise to a new problem: how to coherently combine the environmental information acquired by different robotic systems into one representation that is both accurate and easy to use by human end-users? In this thesis, we introduce novel methodologies to solve this data fusion problem, by proposing a novel framework for combining heterogeneous 3D data models acquired by different robotic systems, operated in unknown large unstructured outdoor environments into a common homogeneous model. The first proposed novelty of the research work is a fast and robust ground-based 3D map reconstruction methodology for large-scale unstructured outdoor environments. It is based on an enhanced Iterative-Closest- Point algorithm and an iterative error minimization structure, as well as the fast and computational very efficient method for outlier analysis and removal in 3D point clouds. The second proposed novelty of the research work is a registration methodology combining heterogeneous data-sets acquired from unmanned aerial and ground vehicles (UAV and UGV). This is accomplished by introducing a semi-automated 3D registration framework. The framework is capable of coping with an arbitrary scale difference between the point clouds, without any information about their initial position and orientation. Furthermore, it does not require a good initial overlap between the two heterogeneous UGV and UAV point clouds. Our framework strikes an elegant balance between the existing fully automated 3D registration systems (which often fail in the case of heterogeneous data-sets and harsh-outdoor environments) and fully manual registration approaches (which are labour-intensive). A special and defining aspect of this PhD. work was that we did not only focus on investigating scientific and technical innovations but that we also concentrated on bringing these innovations to the terrain in real operational environments in the security context. As an example, we deployed the technological tools developed in the framework of this research work to the field for demining and crisis relief operations in an actual crisis situation. This operational deployment was highly successful, based upon the feedback provided by the end-users.}, unit= {meca-ras} }