Drones have been widely used in the entertainment industry (such as television program/movie production), amateur photography, and even become popular fun toys. With the ability to reach complex areas, the application of drones is gradually expanding to professional scenarios such as industrial inspection, logistics distribution, and security monitoring. But did you know that the core component supporting the operation of drones is the visual system? Before delving deeper into this topic, we will first clarify the definition of drones, sort out their diverse application scenarios, and analyze the logic behind their rapid popularity. Finally, we will explore how onsemi can leverage technological innovation to drive the upgrade of drone visual systems.

Types and Applications

Drones belong to unmanned aerial vehicles (UAVs), also known as unmanned aerial systems (UAS), and in rare cases, also referred to as remotely piloted aircraft (RPA). They do not require manned driving and can operate autonomously through various systems.

Drones are divided into three types: fixed wing drones, single rotor/multi rotor drones, and hybrid rotor drones. The purpose of each type of model is different and highly compatible with their respective expected application scenarios.

Fixed wing drones are typically used for heavy payload transportation and long endurance flight missions, with deployment scenarios including intelligence, surveillance, and reconnaissance (ISR) missions, combat operations, patrol deployments, mapping, and scientific research activities.

Single rotor/multi rotor unmanned aerial vehicles are the most widely used, with industrial application scenarios covering conventional warehousing, equipment inspection, and even logistics distribution. Due to the diverse application scenarios of these models, highly optimized electromechanical solutions are required to meet different needs.

The hybrid rotary wing drone combines the advantages of the two types of aircraft mentioned above, with vertical takeoff and landing (VTOL) capabilities, making its application scenarios more flexible, especially suitable for areas with limited space. So it's not difficult to understand that most logistics delivery drones will choose this type.

Drone motion and navigation system

Drones are equipped with various sensors for motion and navigation, including accelerometers, gyroscopes, magnetometers (collectively known as inertial measurement units, IMUs), barometers, and more. They use various algorithms and technologies, such as optical flow (with the help of depth sensors), synchronous localization and mapping (SLAM), and visual odometry. Although these sensors can perform well, they often struggle to achieve the required accuracy and precision within a reasonable cost and optimal size range. This problem will be further exacerbated during long endurance flights, leading to the need for expensive batteries or shortened flight duration due to battery charging and discharging cycle limitations.

UAV Vision System

Image sensors provide functional supplements to the aforementioned sensors, bringing significant performance enhancements and making drones highly accurate and precise devices. The visual system mainly consists of two types of components: gimbal (usually also known as payload) and visual navigation system (VNS).

Yuntai * - provides first person perspective (FPV); Usually integrated with multiple image sensors, covering a wide electromagnetic spectrum range (including ultraviolet sensors in special cases, conventional CMOS image sensors cover the 300nm-1000nm band, short wave infrared (SWIR) sensors extend to 2000nm, while medium wave infrared (MWIR) and long wave infrared (LWIR) sensors cover bands above 2000nm).

Visual Navigation System (VNS) - used to provide navigation guidance, target recognition, and obstacle avoidance functions; Usually composed of low-cost and low resolution image sensors, combined with IMU and other sensor data, a complete autonomous navigation scheme is constructed through computer vision technology.

The Importance of Visual System

As mentioned earlier in terms of usage and application, drones can operate in indoor and outdoor environments. These scenes are often full of challenges, not only with large-scale changes in lighting, but also with limited visibility in dust, fog, smoke, and pitch black environments. Unmanned aerial vehicle systems require the use of a large number of artificial intelligence (AI) and machine learning (ML) algorithms to process image data, while utilizing the data provided by the aforementioned technologies. The premise of all of this is to enable this highly optimized device to operate at low power consumption and achieve the goal of long-distance or long endurance operations.

The data input into these algorithms must have high fidelity and rich details, but in some usage scenarios, only necessary information needs to be provided to achieve efficient processing. The training time of AI/ML needs to be shortened, and the inference process needs to be fast and have high accuracy and precision. Regardless of the environment in which the drone operates, it is necessary to ensure image quality to meet the above requirements.

Sensors that can only capture scenes and submit information for processing are far from sufficient to support the high-quality operation of these devices, and in most cases, may even fail to achieve deployment goals. An ideal sensor should have the following capabilities: achieving miniaturization while preserving complete details of the area of interest; Having a wide dynamic range to cope with the brightness and darkness of light in the same frame of the image; Minimize or eliminate parasitic effects in images to the greatest extent possible; Resolve visibility issues caused by dust, mist, and smoke; Using high depth resolution to assist image processing. Such sensors will bring great benefits to unmanned aerial vehicles becoming highly optimized devices.

These capabilities can minimize the resource scale required for image reconstruction, analysis, and decision acceleration processes, including processing cores, graphics processing units (GPUs), on-chip or off chip memory, bus architecture, and power management. This also reduces the Bill of Materials (BOM) cost of the entire system, especially considering that today's drones can easily carry more than 10 image sensors. In addition, with the same resource allocation, deeper analysis and more complex decision support algorithms will be achieved, allowing drones to form a differentiated advantage in highly competitive fields.

Ansenmei is a technology leader in the field of perception, not only bringing significant innovation to visual system solutions, but also providing a comprehensive range of image sensors to meet the needs of gimbal and VNS. The product series of Hyperlux LP, Hyperlux LH, Hyperlux SG, Hyperlux ID, and SWIR integrate a large number of technologies and features, which can fully meet the needs of unmanned aerial vehicle vision systems. Nowadays, drone manufacturers can obtain various visual sensors they need from Ansenmi in one place and comply with NDAA requirements.

About The Author

This is reported by Top Components, a leading supplier of electronic components in the semiconductor industry

They are committed to providing customers around the world with the most necessary, outdated, licensed, and hard-to-find parts.

Media Relations

Name: John Chen

Email: salesdept@topcomponents.ruThis is reported by Top Components, a leading supplier of electronic components in the semiconductor industry. They are committed to p with the most necessary, outdated, licensed, and hard-to-find parts.

Media Relations Name: John Chen

Email: salesdept@topcomponents.ru