$1M DARPA Grant for Intelligent Diagnosis for Machine and Human-Centric Adversaries
ECE Assistant Professor Xue “Shelley” Lin, in collaboration with Michigan State University Professors Xiaoming Liu and Sijia Liu, received $1M funding from DARPA for their project titled Intelligent Diagnosis for Machine and Human-Centric Adversaries. The project will build a scalable learning system for reverse engineering of deception (RED), which can automatically recover and index attack toolchain signatures in both machine-centric and human-centric attacks, with targets to fool machine learning decisions and human decisions, respectively.
Making Machine Learning Safer for More Applications
Artificial intelligence and machine learning are two of the most exciting developing technologies in the world. From self-driving cars, to robotics, to healthcare, their potential to help people is virtually unlimited.
As with many technologies, however, it comes with potential security issues. And until we can solve those issues, it’s difficult to be truly comfortable using it in many of these applications.
The security threats for machine learning come in many forms, but can be broken down into machine-centric and human-centric attacks. As you might guess, machine-centric attacks target machine learning decisions, and human-centric attacks aim to fool humans making decisions. Both types of attacks aim at “information deception” — manipulating the input data on machine learning models or producing falsified media and other information with machine learning models.
Countering these attacks is at the heart of ECE Assistant Professor Xue “Shelley” Lin’s, in collaboration with Michigan State University Professors Xiaoming Liu and Sijia Liu, project titled Intelligent Diagnosis for Machine and Human-Centric Adversaries. The project, which recently received $1M in funding from DARPA, will build a scalable learning system for reverse engineering of deception (RED). It aims to develop and scale technology that can automatically recover and index attack toolchain signatures in both machine-centric and human-centric attacks.
Before getting into solutions, let’s see what these attacks can look like.
One of the major attack types involves falsified media. Adversaries can use ML algorithms to produce fake media, like a fake piece of news or fake images. The attack’s goal is to fool humans, for example, making it look like they’re saying something that isn’t true.
Adversaries can also produce data to fool machine learning algorithms. “For example, we can add very minor manipulations to a panda image,” Lin explains. “The image still looks like a panda to people, but the machine learning algorithm will identify it as another animal.”
These types of attacks involve the execution phase of a machine model. The model itself isn’t changed, but because of the machine learning algorithms, it’s possible to find images to fool the machine learning models.
Another type of attack is implemented during the training of ML models. In this phase, developers use training data to build a model that will execute tasks. Adversaries execute these attacks by manipulating the training data itself.
“The training phase attack can be done by replacing a small portion of the training data — as little as five or 10 percent,” Lin details. “We can, for example, put a very small cross in a corner of the images, and that will change the label of the image to the wrong label. Then when the model is trained, whenever an image has a cross in the corner, the model will attach the wrong label to the information.”
The project will develop ways to identify these attacks, and others, and also index them for future use.
“What we want to do is design an intelligence that detects both of these adversaries and can detect falsified data,” Lin explains. “For example, if it’s a machine-cetric attack, we need to know the adversary’s goal and their knowledge. If it’s a human-centric attack, we want to figure out the particular type of model being used to produce falsified data.”
The project has two main phases. This first is to build and evaluate the tool, then comes the indexing, scaling, and adaptability.
“Our project is the first one to develop a unified attack toolchain that covers a broad range of attacks, both human-centric and machine-centric,” Lin says. “And then to index each attack, we have different families of attacks, and even reverse engineer the data to extract the unique adversary signature, and also the supervised attack classifier.”
While Lin’s PhD study was in computer systems, she didn’t start focusing on artificial intelligence and machine learning until she joined Northeastern. At that time, ML was gaining popularity and the importance of these security issues was starting to gain notice. Northeastern’s support, and unique environment, has been a major plus, not just with this project, but with Lin’s other work.
“We get a lot of support from the department, and in other projects we have broad collaborations with many researchers with different expertise — so it’s complementary to my work and very helpful,” Lin says. “Also, I really like my PhD students and they do a lot of great work.”
In the end, this project should help make ML safer to use for a wider variety of tasks.
“Machine learning can be powerful, but there are too many uncertainties right now holding us back from using it more widely,” Lin says. “We need to gain more understanding of it and the potential security issues to use it more confidently. This project is about helping more us in the fundamental direction of being able to use it more.”