Our Client:

A defence company involved in the deployment of drones in GNSS-denied environments

Situation:

Traditional methods of aerial surveying and mapping rely heavily on GPS for localization, which can be inaccurate in urban canyons or non-existent in hostile environments where jamming is widely used to render drones inoperable. Our client sought an innovative solution to enhance their drone localization reliability, especially in areas where GPS reliability is compromised. The challenge was to develop a method for precise localization without relying on GPS data, using visual cues from the environment.

Task:

The primary task was to design and implement a deep learning model capable of matching real-time drone-captured images with pre-existing satellite images for accurate localization. This task involved several key steps: data capture, training of a deep learning image matching system, and deployment on a prototype platform for validation.

In order to capture enough data to train the model, we developed a custom image capture rig that combines a high-resolution camera with a Raspberry Pi, mounted on a fixed-wing Goblin drone. This setup allowed us to collect a diverse dataset of aerial images across different environments, weather conditions and times of the day.

Once data was acquired in combination with different satellite and drone image datasets, we trained a convolutional neural network based models to perform image matching. We could then localize the drone by matching camera image with satellite images. The model was trained on a dataset of paired drone and satellite images, enabling it to learn distinctive features and landmarks for accurate matching.

The final step involved integrating the trained model into a compact and efficient Jetson Nano module. This module was then mounted on a quadcopter, allowing for on-board processing and immediate localization feedback during flight missions.

Result:

The system demonstrated high accuracy in matching drone-captured images with satellite images, achieving flights without GPS data. The model successfully handled various scenarios, including urban areas, forests, and coastal regions with a high degree of precision.

During testing, we identified edge cases where the model's performance was affected by factors such as poor weather conditions, low-light environments, and high-density urban areas with repetitive patterns.

This project represents a significant step forward in the field of aerial localization and mapping, offering a scalable and accurate solution that reduces dependence on GPS technology.