During their time at the Allen School, recent alumni Maarten Sap (Ph.D., ‘21) and Ivan Evtimov (Ph.D., ‘21) tackled some of the thorniest issues raised by emerging natural language processing and machine learning technologies — from endowing NLP systems with social intelligence while combating inequity and bias, to addressing security vulnerabilities in the convolutional neural networks that fuel state-of-the-art computer vision systems. Recently, the faculty honored both for their contributions with the William Chan Memorial Dissertation Award, which was named in memory of the late graduate student William Chan to recognize dissertations of exceptional merit. Evtimov earned additional recognition for his work from the Western Association of Graduate Schools and ProQuest as the recipient of the WAGS/ProQuest Innovation in Technology Award, which recognizes distinguished scholarly achievement at the master’s or doctoral level.
Sap — who is currently a postdoctoral/young investigator at the Allen Institute for AI (AI2) — worked with Allen School professors Yejin Choi and Noah Smith. His dissertation, “Positive AI with Social Commonsense Models,” advanced new techniques for making NLP systems more human-centric, socially aware and equity-driven.
“Maarten’s dissertation presents groundbreaking work advancing social commonsense reasoning and computational models serving equity and inclusion. More specifically, his work presents technical and conceptual innovations that make deep learning methods significantly more equitable,” said Choi and Smith, both of whom are also senior research managers at AI2. “Maarten’s research steers the field of NLP and its products toward a better future.”
One example is ATOMIC, a large-scale social commonsense knowledge graph Sap and collaborators created to help machines comprehend day-to-day practical reasoning about events, causes and effects. To create equity-driven NLP systems, he also helped develop PowerTransformer, a controllable text rewriting model that helps authors mitigate biases in their writing, particularly biases related to how the public describes people of different genders. Sap also tackled the problem of detecting biases and toxicity in language by identifying issues with the current hate speech detectors that lead to racial biases. His work introduced Social Bias Frames, a linguistic framework for explaining the biased or harmful implications in text. The papers supporting this, The Risk of Racial Bias in Hate Speech Detection and Social Bias Frames: Reasoning about Social and Power Implications of Language were nominated for a Best Short Paper Award by the Association for Computer Linguistics in 2019 and won the Best Paper Award at the West Coast NLP Summit in 2020, respectively. Sap was also a member of the team that won the first Amazon Alexa Prize for a conversational chatbot called Sounding Board that engages with users about current topics.
TechCrunch, Forbes, Fortune and Vox have all covered Sap’s research. After completing his postdoc with AI2’s MOSAIC team, he will join Carnegie Mellon University’s Language Technology Institute as a professor in the fall.
Evtimov’s dissertation, “Disrupting Machine Learning: Emerging Threats and Applications for Privacy and Dataset Ownership,” makes significant contributions to the security of adversarial machine learning. His research as a member of the Allen School’s Security & Privacy Research Lab focused particularly on the vulnerabilities of convolutional neural networks (CNN) that allow maliciously crafted inputs to affect both their inference and training. Evtimov said that understanding new technologies in terms of security and privacy is important in order to think ahead of adversarial actors.
“Ivan’s dissertation is highly innovative, and contributed significant results to the field of real-world attacks against computer vision algorithms. His work is of fundamental importance to the field,” Allen School professor and lab co-director Tadayoshi Kohno said. “Computer vision is everywhere — in autonomous cars, in computer authentication schemes, and more. Ivan’s dissertation helps the field develop secure computer vision systems and also provides foundations for helping users protect their privacy in the face of such systems.”
Evtimov’s work shows that the vulnerabilities of CNNs exhibit a duality when it comes to security and privacy. For example, he found the computer algorithms for cameras reading traffic signs in autonomous cars could be tricked by an object as simple as a sticker on a stop sign. The sticker could fool the cameras into reading the stop sign as a speed limit sign. In the case of autonomous driving, it is critical to identify anything that could be exploited by malicious parties in such a safety-critical setting. Machine learning, Evtimov found, can also be used in an unauthorized manner. Take, for example, a search engine for facial recognition. To protect privacy, Evtimov studied the conditions in which people could flood a database full of photos gathered from the public without permission with decoys. He proposed FoggySight, a tool that involves community users uploading modified photos — for instance, labeling photos of Madonna as photos of Queen Elizabeth — to poison the facial search database and throw off searches in it. He also found ways to protect visual data released for human consumption from misuse through machine learning, including developing a protective mechanism that can be applied to the information contained in datasets before public release to prevent unauthorized parties from training their own models using the data.
Evtimov’s research has been covered by Ars Technica, IEEE Spectrum and more. He previously won a Distinguished Paper Award at the Workshop on Adversarial Machine Learning in Real-World Computer Vision Systems and Online Challenges for his work examining the vulnerability of combined image and text models to adversarial threats. After graduating from the Allen School, Evtimov joined Meta as a research scientist.
Congratulations to Maarten and Ivan!