News | Sep 24, 2019

Algorithm corrects for video latency, delivers precision control of unmanned ground vehicles

Your company can license it now from the U.S. Army lab that invented it

The Army's prototype Wingman command-and-control vehicle, left, and the unmanned Wingman vehicle work together to engage targets.

Keith Briggs/U.S. Army

An Army laboratory in Michigan was issued U.S. Patent 10,425,622 on Tuesday for a predictive video display algorithm that enables high-speed operation of unmanned ground vehicles, a technology area known as teleoperation.

Unmanned ground vehicles can be used in convoys to haul fuel, water, and equipment.

And unmanned ground vehicles can also help foot soldiers by carrying heavy weapons. A Raytheon-Lockheed Martin team recently fired an anti-tank missile from a small unmanned ground vehicle built by QinetiQ and Milrem Robotics. Named the Titan, the unmanned ground vehicle has eight cameras and a top speed of 15+ MPH.

But it’s tough to drive fast if you know your camera feed is traveling on a network with significant latency, too easy to miss a corner, get stuck in a ditch, hit a tree or your own troops.

According to Dr. Mark Brudnak from the Army’s CCDC Ground Vehicle Systems Center, network latency causes operators to adopt a “move and wait” style of driving, which is too slow to be useful in many situations.

Previous fixes to the operator’s video feed used graphics or video manipulation, which Brudnak described as too complicated and difficult to implement. So, that’s why Brudnak invented his new system.

In a conference paper (linked below), Brudnak explained how his algorithm splits the video image, transforms the perspective, and stitches it back together to give an accurate point of view to the operator “in the most natural way.”

Simulation illustration via Mark Brudnak/Army

“Preliminary experimental results were presented in which the predictive display was shown to be an effective method for the mitigation of latency by increasing achieved speed and by reducing the path deviation and the heading error significantly,” Brudnak wrote. “By implementing predictive displays as a mitigation of latency in teleoperation, a minimally invasive approach to teleoperation was developed which has the potential for broad application to several UGV types and missions.”

Through technology transfer, private businesses can leverage the Army’s research and development work to integrate this and other military inventions into their own products and services.

In this case, a patent license agreement with the Army would enable the algorithm and related tech developed by Brudnak to be transferred and transitioned into use by companies selling products and services to the military or commercial markets. A cooperative research and development agreement may also be appropriate to further the technical readiness level.

Dr. Brian Metzger, senior technology manager at TechLink, is an expert in invention licensing and is assisting the Army lab in Michigan with its technology transfer efforts.

Metzger said Tuesday that he plans on visiting the lab in mid-October to discuss its patent portfolio and other technologies that have commercial potential.

Companies working on self-driving cars, like Designated Driver, might be interested in learning and licensing teleoperation technologies developed by the Army, Metzger said. (Some states are requiring unmanned vehicles to be capable of remote control.)

“We’re regularly reviewing the military’s patent applications,” Metzger said. “And now that the patent has issued, the technology has tangible commercial value and licensing potential.”


Business inquiries can be sent to Dr. Brian Metzger at brian.metzger@montana.edu or reach him by telephone at 406-994-7782.