AI Makes Drones Smart, Easy for Photographers

A certified drone pilot and artificial intelligence expert explain how technology innovations are making drones smarter, more capable and easier to fly.

Difficult-to-fly, remote control consumer drones from just a few years ago are being superseded by smart, autonomous aerial robots. Powered by cutting edge computer vision and artificial intelligence (AI) technologies, these new drones can see, think and react to their owner automatically, and experts say this is making drones easier and safer for almost anyone to fly.

Drone innovation is skyrocketing as more sophisticated technologies are making drones smarter and increasingly capable, according to Kara Murphy, a photographer turned drone fanatic and a certified Part 107 pilot licensed to fly small unmanned aircraft (UAS) for commercial uses.

"As someone who started at the very beginning, drone technology has come a long way in a short period of time," Murphy said.

Murphy, a contributing writer for Drone360 Magazine and a consultant for companies like DroneDeploy, is involved with the annual Flying Robot International Film Festival. She said drones are evolving to give people more control over flying as well as opening new photographic experiences.

A few years ago, battery life was only about six minutes at most. Today, drone batteries can last up to 30 minutes. She's seeing more drones, like the new DJI Spark, that come equipped with built-in AI for facial recognition and object detection to avoid crashes. The technology allows drones to follow their owner like a welcome aerial paparazzi, avoid objects because they're context aware and react to simple hand gestures.

"It's easier to pilot and keep track of drones today," said Murphy. "They've become almost idiot-proof."

Smart Flying Drones

Spark, the first mini drone released this year by DJI, uses an array of cameras and sensors feeding into AI and deep learning algorithms running on a Movidius Myriad 2 vision processing unit (VPU).

This onboard vision system detects and avoids objects, generates 3D maps, establishes contextual awareness, and even recognizes a pilot's face and reacts to hand gestures. The vision sensors fitted inside the underbelly of the drone detect and identify what's below to assist with a safe landing, even on a pilot's outstretched hand.

"I can signal it to take a selfie from the air, then wave it away or gesture for it to come back home," said Murphy, describing some of the Spark's AI-powered automation features.

"The fact that I don't need a remote to control this drone is mind-blowing. It just shows how far drones have come in a matter of years."

Murphy said collision or object avoidance, powered by computer vision and intelligent algorithms, is becoming more common in new drones, and it can be a drone lifesaver.

"It is supremely helpful, because sometimes you are flying in narrow spaces, and you're not sure if you have enough room, so having these sensors is really key to avoid damaging collisions," she said.

These capabilities make it easier to fly because pilots don't have to stay glued to a remote control and screen, she said. It allows them to become aware of their surroundings and focus on capturing that perfect shot.

The compact Spark is built with technologies that were previously only available in larger, more expensive drones. In particular, it has chips and software designed specifically for bringing on-device AI to so-called "edge devices," which includes almost anything that computes and connects to the internet.

Seeing Clearly

The Spark's Movidius Myriad 2 VPU enables the drone to think, learn, and act quickly and simultaneously, according to Cormac Brick, director of embedded machine intelligence at Movidius, an Intel company.

While central processing units (CPUs) — the brains used in computers or computing devices — can perform a wide variety of workloads, Brick said the VPU is tailored for one very specific vision workload, so it has fast performance using low power.

Cormac Brick points out how built-in AI makes mini drones ideal for getting the best shot.

"The VPU allows the drone to use both traditional geometric vision algorithms and deep learning algorithms so it can be spatially and contextually aware," he said.

"It enables the device to recognize where it is, where you are, where your hand is, and plot a course to safely hover and then soft land into the palm of your hand."

As soon as a Spark lifts off from a person's hand, the cameras immediately look for recognizable features in the environment to build a digital map. All the while, the drone recognizes the user's face, always keeping that person in frame.

Future of Intelligent Drones

Brick said the Spark indicates how AI is changing the drone market, and he sees the technology getting better all the time.

His team's just-released Movidius Myriad X is the first VPU with a dedicated neural compute engine, which will allow device makers much more compute performance than what's currently available. That means drones will become smarter, fly more safely and allow people to capture more footage fully autonomously.

"In the future, you'll be able to take a drone out of your pocket, throw it up in the air and let it fly around your backyard for the afternoon while you're having a barbecue," Brick said.

"An hour later, it could send your phone a 45-second video clip or the 10 best shots so you can share on social media."

Building AI into drones is helping make them easier and safer to fly, but Brick said the technology has the potential to unlock all kinds of new automated camera and navigation capabilities.

Murphy believes that drone popularity will increase as drones become more autonomous and simpler to use in capturing life's moments.

"This trend will continue as drones get easier and more fun for people to use," Murphy said.

Editor's note: Myriad 2 AI technology used in the DJI Spark was recently made available to product developers, researchers and makers in a new plug-and-play form. The Movidius Neural Compute Stick, the first USB-based deep learning inference kit and self-contained AI accelerator, brings dedicated deep neural network processing capabilities to a wide range of edge devices.

Source: AI Makes Drones Smart, Easy for Photographers