ASU startup's breakthrough in explainable AI secures Air Force contract for reliable transparent models

Business professor leveraged brain theory to patent explainable AI methodology.

Professor of Information Systems Asim Roy is not an expert in computer vision — using artificial intelligence (AI) to analyze images or videos to extract specific, contextual information. And that's why his most recent project has been successful.

Roy and Teuvonet Technologies, an ASU and Arizona Board of Regents faculty startup he founded, invented a computer vision method using explainable AI (XAI) last year which ASU and ABOR have applied to patent. XAI uses frameworks that help stakeholders understand its predictions and decision-making processes, and by showing its work, XAI instills confidence in users that the AI understands a prompt and has come to the correct conclusion. The XAI technology for computer vision led to an Air Force/Space Force Small Business Technology Transfer (STTR) Phase I contract to study the feasibility of automated, real-time imagery processing onboard satellites, a technology that does not currently exist.

Teuvonet Technologies' XAI method for computer vision comes years after DARPA's call for XAI technology for deep learning AI models. Although the military uses many deep-learning computer vision models, it requires more reliable, transparent AI models for high-risk defense applications. However, DARPA funding over the years did not produce the desired form of XAI, which includes object recognition based on an object's individual parts.

"We moved into computer vision out of curiosity," says Roy. "Why can't computer vision do what DARPA wants? That's where we started."

Using brain theory to teach AI

Roy's curiosity outside the information systems field ultimately led to his computer vision work, and he credits his XAI success to his interest in neuroscience.

"It always pays to be intellectually curious, and my brain theory debate came in handy and provided some insights on how to do XAI," says Roy.

Thirty years ago, Roy began studying the concept cell theory, which hypothesizes that the brain has grandmother cells, or highly abstract cells representing complex things, and neurons that identify simple objects. For example, a grandmother cell might recognize a car, while less complex neurons recognize unique car traits like windows, wheels, or an engine.

Teuvonet Technologies primarily focuses on developing machine-learning systems for hardware platforms and only began experimenting with computer vision three years ago while Roy was working with several Master of Science in Business Analytics (MS-BA) students interested in doing research with him to get more experience in machine learning and AI. Building off his brain theories, Roy and his MS-BA students first taught the AI to identify whole objects before teaching it to identify specific object parts.

"Initially, babies learn about a whole object: A baby can recognize a car or a bicycle. We learn whole object detection early on, and then we build on top of that," says Roy. "First, we taught a system to recognize an image — a cat, dog, or bird. Then we cut out different animal parts and taught the system to recognize the legs, body, tail, and so on."

Roy and his MS-BA students then taught the AI to identify and differentiate between objects using increasingly specific criteria. Eventually, they automated the AI application to recognize the difference between different dog breeds while providing information to the user that justified its decision.

"If the AI detects unique cat parts like whiskers, claws, and so on, that gives us more confidence to trust what it's saying," says Roy.

After developing the XAI method, Teuvonet Technologies joined the National Security Academic Accelerator and began presenting their findings at military conferences. The company partnered with the defense companies Lockheed Martin and Raytheon Missiles and submitted an Air Force contract proposal through Open Topic with a strong support letter from Space Force.

In his recent paper published in the Military Operation Research Society's Phalanx magazine (Winter issue, 2023, p. 18-23), Roy describes how DARPA initiated an XAI program for high-risk defense applications in 2012. Deep learning computer vision models often incorrectly identify manipulated or complex objects in images that a person could otherwise recognize correctly. For example, when presented with a photo containing several planes, buildings, and streets, current deep-learning technology cannot infer that the image depicts an airport. Since Roy's automated XAI methodology recognizes parts of objects, it teaches the AI to infer the object's identity based on the available information. Overall, Roy's technology enhances the explainability and transparency of deep learning models and helps to automate the AI application with confidence and trust, a potential benefit for the Air Force and Space Force that can now streamline their manual review process of satellite imagery.

During the three-month Phase l contract, Teuvonet Technologies provided Space Force with a proof of concept for their XAI methodology. Phase ll will include developing XAI models to detect various objects on the ground from satellite imagery, rigorous model testing on various hardware, and locating Air Force/Space Force end-users and customers. In addition, a satellite company — which has over a hundred low-orbit working satellites — will support actual satellite testing of the XAI models, and Raytheon Missiles is interested in testing the XAI models on its drones. Lockheed Martin will also collaborate in various ways during Phase II.

While Roy and Teuvonet Technologies still have a long road ahead before the defense sector can adopt their models and methods, Roy is proud of W. P. Carey's role in creating the XAI technology.

"Nobody could figure out how to do these things, and it came out of a business school," says Roy. "Can you imagine that?"