Data Acquisition

The backbone and heart of successful computer vision algorithms is data. It is the driving force of today's automated intelligent systems. Especially the data-driven field of deep learning relies on large amounts of realistic and diverse data. On the other hand, the data acquisition is one of the most complicated steps in the whole process. This is particularly true for the maritime domain, which introduces further challenges regarding safety, regulations, environmental aspects and more.
In the following two sections, we provide some background of our data acquisition missions with regards to organization and logistics difficulties, regulations, general problems and more.

Lake Constance Mission

From the 25th to the 30th of August, we carried out the Lake Constance data acquisition mission. Lake Constance, Germany, has the valuable advantage of being smooth enough to carry out first experiments to gather knowledge yet also large enough to simulate the open sea.
Beforehand, we had to request several regulatory authorizations and permissions and had to coordinate with several institutions. In particular,

• Flight permission from the state agency Baden-Württemberg
• Ascent permission from the sailboat marina of Meersburg
• Event authorization from the waterways and shipping office Friedrichshafen
• Flying authorization from the nature conservancy department Bodenseekreis
• Informing the airport Friedrichshafen
• Informing the local police station in Überlingen
• Hygienic concept to be adhered to by all participante due to Covid-19

These authorizations and precautionary measures helped us have a smooth and safe data acquisition missions thoroughgoing. Owing to the restrictions from many different parties, we could narrow down the number of deployment sites to a single location:


47°39'53.1"N 9°16'20.3"E

In particular, it was outside of any natural habitat, outside of any no-flight zone and off the ship's waterways. This location was also close enough to have our fixed wing plane Trinity F90+ start and land from the mainland location as depicted here:


47°41'11.6"N 9°17'03.1"E

See the start

and the landing


We prepared flight plans to let this UAV automatically traverse a pre-defined area, which was around the GPS coordinate as given above. It shot RGB and multi-spectral bird's eye view images in intervals of one second with cameras as described in the original paper.

The other UAVs were quadcopters, which were launched from boats that we rented from the boat rental Hagnau.



The starting and landing of the UAVs actually turned out to be quite difficult due to the water movement and the limited space. We needed to allow for a few minutes to complete the landing of the individual quadcopters.



We also had pre-planned trajectories for the quadcopters, setting parameters, such as GPS coordinates, altitude, viewing angle and others. However, due to the current the boats and swimmers drifted off the original location. In the beginning, we tried readjusting the location of the boats and swimmers. However, the current was too strong to adjust in time which is why we decided to manually steer the quadcopters to allow for maximum data harvesting given the limited time. The larger plane was not affected by this offset too much as it covered a larger area while flying higher.

Considering that we were flying with multiple UAVs at the same time, we made sure that we had an altitude separation of all UAVs at all times.

Another major point in the planning was concerned with the subjects of interest. We hired 20 students that were physically fit to withstand several consecutive minutes in cold water and were athletic enough to be able to swim in open water. Furthermore, to ensure the safety of all participants we had a lifeguard and had a strict assignment of groups of two where only ever one person in water while the other is keeping track of the person. Furthermore, we had the participants fill out forms that informed them about the plans and where they ensured that they fulfill the physical requirements. We also had forms where they transferred the rights to their images to us.



The swimmers were offered life jackets and wetsuits and could get out of the water at any time. We instructed the participants to vary several variables, such as wearing a life jacket, a wetsuit, different swimming clothes, swimming versus floating, waving their arms and more.

Initially, we planned to capture data on five days but strong winds prevented us from going on the water on two days. During the other three days, we tried to vary the time of going out to have different lighting conditions.

North Sea Mission

Finished. Description coming soon...