Allen School News

Allen School Ph.D. students Vivek Jayaram and John Thickstun have been named 2020 Qualcomm Innovation Fellows for their work in signal processing, computer vision and machine learning using the latest in generative modeling to improve source separation. In their paper, “Source Separation with Deep Generative Priors,” published at the 2020 International Conference on Machine Learning, the team addresses perceptible artifacts that are often found in source separation algorithms. Jayaram and Thickstun are one of only 13 teams to receive a fellowship out of more than 45 finalists across North America. 

Thickstun and Jayarum have been working with their advisors, Allen School professors Sham Kakade, Steve Seitz, and Ira Kemelmacher-Shlizerman and adjunct faculty member Zaid Harchaoui, a professor in the UW Department of Statistics, on this research. Potential applications include separating reflections from an image, voices of multiple speakers or instruments from an audio recording, brain signals in an EEG and telecommunication technologies from Code Division Multiple Access (CDMA). The team’s work introduces a new algorithmic idea for solving source separation problems using a Bayesian approach. 

“In contrast to source separation models, modern generative models are largely free of artifacts,” said Thickstun. “Generative models continue to improve and one goal of our proposal is to find a way to use the latest advances in generative modeling to improve source separation results.” 

Employing a cutting-edge generative model is a powerful tool for source separation and can be applied to different data domains. Using the Bayesian approach and Langevin dynamics, Thickstun and Jayaram can decouple the source separation problem from the generative model, achieving a state-of-the-art performance for separation of low resolution images. 

By combining images and using the algorithm to then separate each of them, the team was able to illustrate how their theory works.

“Our algorithm works on mixtures of any number of components without retraining,” Jayaram said. “The only training is a generative model of the original images themselves; we never train it on mixtures of a fixed number of sources.”

Audio separation proved to be more challenging, but the two implemented Stochastic gradient Langevin dynamics to speed up the process and make it more practical. Their approach can be modified to tackle many different strains of optimization problems by modifying the reconstruction objective.

“John and Vivek’s work takes a fundamentally new and promising approach that leverages the power of deep networks to help separate out signals,” Kakade said. “The reason this approach is so exciting is that deep learning methods have already demonstrated remarkable abilities to model distributions, and their work looks to harness these models for the classical signal processing problem of source separation.”

Jayaram and Thickstun have each published additional papers in advancing source separation at the 2020 IEEE Conference on Computer Vision and Pattern Recognition (Jayaram), which focuses on background matting in images, and the 2019 International Society for Music Information Retrieval (Thickstun), which investigates end-to-end learnable models for attributing composers to musical scores.

Since 2009, the Qualcomm Innovation Fellowship program has recognized and supported innovative graduate students across a broad range of technical research areas. Previous Allen School recipients include Vincent Lee and Max Willsey (2017), Hanchuan Li and Alex Mariakakis (2016), Carlo del Mundo and Vincent Lee (2015), Vincent Liu and Vamsi Talla (2014) and Adrian Sampson and Theirry Moreau (2013). Learn more about the 2020 Qualcomm Fellows here. 

Congratulations, Vivek and John! 

Our latest Allen School student spotlight features Jenny Liang, a Kenmore, Washington native and recent UW graduate who majored in computer science and informatics. Liang was named among the Husky 100 and earned the Allen School’s Undergraduate Service Award for her leadership, compassion, and focus on developing technology for social good in her work with the Information and Communication Technology for Development Lab (ICTD). 

This summer, Liang started an internship at the Allen Institute for Artificial Intelligence (AI2) after being awarded the 2020 Allen AI Outstanding Engineer Scholarship for Women and Underrepresented Minorities. The scholarship is designed to encourage diversity and equity in the field of artificial intelligence while strengthening the connection between academic and industry research. She previously held internships at Microsoft, Apple and Uber.

Allen School: Congratulations on the AI2 Scholarship! What makes this scholarship special, and who should apply?

Jenny Liang: The AI2 scholarship is an opportunity for folks in underrepresented communities in technology. Its aim is to combat the lack of representation currently seen in the tech industry and academia. As part of the scholarship, students receive one year of tuition covered by AI2 along with an internship. The winners have coincidentally been women in the past couple of years, but I’d like to emphasize that anyone who identifies with any underrepresented identity is qualified to apply. I would encourage any Allen School student who belongs to any minoritized identity to take advantage of this opportunity. 

Allen School: How has the experience been? 

JL: It’s been a positive career-changing experience, and my time at AI2 has been really awesome so far. I currently work on the MOSAIC team headed by the Allen School’s own Yejin Choi, where I’m building a research demo using cutting-edge computer vision and natural language processing (NLP) models. The exciting thing is it’ll be released soon to the public. I’m also dabbling with conducting my own research on building NLP models to detect social bias in language, as well as interpreting the predictions of these models. The goal of this is to illuminate how and why these models behave the way they do, and whether they can be improved to be more than just black boxes that predict complex phenomena in natural language. This provides more context in how these models interact with society, which ultimately has real-life consequences on people. Both the engineering and research aspects of this internship are all very new and challenging experiences for me. It’s been my first time working with computer vision and NLP deep learning models, which has given me a new perspective into challenges that developers face. I feel like this has pushed me to learn and adapt as a budding researcher, and provided me with lots of tools and skill sets I’ll be using in the future.

Allen School: What initially interested you in computer science and informatics? 

JL: At the time of choosing my major, I loved software engineering, and I still do. This meant I was interested in both the theoretical and applications of technology. The theory is so important to understand what makes technology systems work and why. But, understanding how technologies are applied is equally important in building software that is usable and performant and serves people in fair and ethical ways. To me, the Allen School taught me the theoretical foundations of computer science, while the iSchool provided the ability to build technology applications. Being in both CSE and INFO has allowed me to become a well-rounded technologist, where I can both build technologies quickly but also understand the complicated theoretical underpinnings of these systems.

Allen School: You have had a lot of industry experience with your internships. Do you plan to continue on that path to a career in industry?

JL: In the past year, I’ve decided to switch to academia after working in the industry. So I am applying to Ph.D. programs this fall. I’ve always enjoyed software engineering, but after a while, I found the engineering work I did in industry personally unfulfilling since I wanted to learn the fundamental properties of what makes software “good” or “bad” and why, especially as software scales. I didn’t think my trajectory in industry would quite allow me to gain that expertise because of its focus on building new technologies. Thanks to some outstanding and involved mentorship from iSchool professor Amy Ko, postdoc Spencer Sevilla in the Allen School’s ICTD Lab, and AI2 researchers Chandra Bhagavatula and Swabha Swayamdipta, I’ve been slowly convinced that academia is the space for me to do that.

Allen School: What is the best thing about being a student in the Allen School?

JL: To me, the best thing is the breadth of high-quality opportunities this school has to offer. I’m really grateful and feel so privileged for the opportunities I’ve been given because I’m in CSE. For the past five years, I’ve known I wanted to work with technology after I taught myself to code my freshman year and totally loved it. What has not been clear is how and to what capacity I’d like to do that. Because of the many opportunities the Allen School provides, I’ve really been able to find my own fulfilling niche in tech. I’m really fortunate to have developed my career as a software engineer, but also quickly pivot to a career in academia. Due to the school’s industry connections, I’ve been able to work on the world’s largest technology systems and with the best engineers; thanks to the opportunities I’ve had to do undergraduate research, serve as a TA, and take graduate-level courses, I’ve gotten a taste of what it’s like to be a Ph.D. student and really enjoyed it. Most importantly, though, my connections with the school’s faculty and staff have supported so much of my growth, and I would be nowhere without them.

Allen School: What interested you in becoming a member of the Allen School’s Student Advisory Council and continuing to serve in it? 

JL: Working with the SAC has been important to my CSE experience. I struggled a lot my first several years at UW with my mental health, which also compromised my academics for a while. Without the support of my friends and professors in the Allen School, I would not be the same person I am today. Being involved with SAC is my way of giving back to the community that supported me, as well as deriving meaning from my painful experiences. Because I understand what it’s like to struggle while being a CSE student, I’m committed to finding the ways in which CSE as a system could improve in supporting the undergraduate experience and advocating for change.

I’ve stayed with SAC because through listening to my peers’ diverse experiences and struggles, I’ve realized this work really matters. Although change can be slow-going and allyship is hard, the work we do allows students to be more successful academically, builds community within the Allen School, and creates a welcoming environment where everyone can thrive. This has been especially important in response to the tumultuous current events, and I’m really proud of how all the other student groups are committed to this mission too.

Thank you for all that you’ve done for the Allen School and UW community, Jenny — and good luck with those grad school applications! 

Researchers at the University of Washington will lead a team of universities in creating the Institute for Foundations of Data Science (IFDS) to tackle important theoretical and technical questions in the field. Supported by a $12.5 million, five-year grant from National Science Foundation, IFDS is one of two institutes nationwide to receive funding from the latest phase of the agency’s Transdisciplinary Research in Principles of Data Science (TRIPODS) program.

The IFDS — a collaboration between UW and the Universities of Wisconsin-Madison, California Santa Cruz and Chicago — brings together theoretical computer scientists, mathematicians and statisticians to continue the study of complex and entrenched problems in data science. Together, they aim to create scalable and robust algorithmic tools that can read the unprecedented growth of large datasets, ultimately accelerating the pace of science and engineering. 

“As data science is increasingly incorporated in all facets of our lives, its success is uncovering pressing challenges that call for new theories,” said lead principal investigator Maryam Fazel, a professor in the UW Department of Electrical & Computer Engineering and adjunct professor in the Allen School, in a UW News release. “We need the expertise of all core disciplines to understand the mysteries and to address the pitfalls of data science and artificial intelligence algorithms.”

In 2017, Fazel and professor Sham Kakade, who holds a joint appointment in the Allen School and Department of Statistics, led the UW’s successful proposal to establish the Algorithmic Foundations of Data Science Institute (ADSI). They received a $1.5 million award from NSF’s TRIPODS. Three UW teams subsequently earned TRIPODS-X grants in 2018 designed to expand their original work to broader areas of science, engineering and mathematics.

“Phase II is a scaled-up version of phase one,” said co-principal investigator Yin Tat Lee, a professor in the Allen School’s Theory group. “It involves more students and more PIs, which will foster more collaborations between facilities in different areas. It supports us to visit other TRIPODS partners, to get new kinds of research going.”

UW researchers have already hosted workshops and hackathons to recruit more diverse participants to the field. Their continued mission is to improve accuracy and decrease bias in algorithmic decision making processes, as well as methods to cope with ever-changing data that may be corrupted by noise or even malicious intent.

“One thrust of the TRIPODS phase II program that I’m particularly excited about aims to advance the understanding and practice of closed-loop learning. In this paradigm of data science, data collection and inference feed off of each other so that inferences on past data inform what data should be collected next,” said co-principal investigator and Allen School professor Kevin Jamieson. “A well-executed closed-loop learning protocol can often accomplish data science tasks using just a small fraction of the data necessary for traditional methods. This can accelerate the discovery of novel materials or medicines, and our ability to learn useful machine learning models to predict things like health outcomes from a given treatment.”

Jamison added that even small advances in collecting data more efficiently and smarter could have huge aggregated benefits across many fields because the algorithms ADSI develops are broadly applicable across disciplines. 

“While the last decade has enjoyed an embrace and democratization of machine learning and AI across fields like biology, chemistry, and even physics, tools for helping these practitioners collect dataset are essentially nonexistent,” he said. “One of the aims of the close-loop data thrust is to develop these tools.” 

In addition to Fazel, Kakade, Lee and Jamieson, participants in the UW IFDS include professor Dmitriy Drusvyatskiy of the UW Department of Mathematics, Zaid Harchaoui, professor of statistics and adjunct faculty member in the Allen School and Abel Rodriguez, who recently arrived from UC Santa Cruz to serve as diversity liaison for the IFDS.

Read the NSF’s announcement here and the UW News release here. 

Allen School postdoctoral researcher Joseph Jaeger and visiting researcher Nirvan Tyagi, a Ph.D. student at Cornell Tech, received the Best Paper by Early Career Researchers Award at the 40th Annual International Cryptology Conference (Crypto 2020) organized by the International Association for Cryptologic Research (IACR). Jaeger and Tyagi, who have been working with professor Stefano Tessaro of the Allen School’s Theory and Cryptography groups, earned the award for presenting a new approach to proving multi-user security in “Handling Adaptive Compromise for Practical Encryption Schemes.” 

Jaeger and Tyagi set out to explore a classic problem in cryptography: How can the security of multi-party communication be assured in cases where an adversary is able to adaptively compromise the security of particular parties? In their winning paper, the authors aim to answer this question by presenting a new, extensible framework enabling formal analyses of multi-user security of encryption schemes and pseudorandom functions in cases where adversaries are able to adaptively compromise user keys. To incorporate an adversary’s ability to perform adaptive compromise, they expanded upon existing simulation-based, property-based security definitions to yield new definitions for simulation-based security under adaptive corruption in chosen plaintext attack (SIM-AC-CPA) and chosen ciphertext attack (SIM-AC-CCA) scenarios. Jaeger and Tyagi also introduced a new security notion for pseudorandom functions (SIM-AC-PRF), to simulate adaptive compromise for one of the basic building blocks of symmetric encryption schemes. This enabled the duo to pursue a modular approach that reduces the complexity of the ideal model analysis by breaking it into multiple steps and splitting it from the analysis of the high-level protocol — breaking from tradition in the process.

“Traditional approaches to formal security analysis are not sufficient to prove confidentiality in the face of adaptive compromise, and prior attempts to address this gap have been shown to be impractical and error-prone,” explained Jaeger. “By employing idealized primitives combined with a modular approach, we avoid the pitfalls associated with those methods. Our framework and definitions can be used to prove adaptive security in a variety of well-studied models, and they are easily applied to a variety of practical encryption schemes employed in real-world settings.”

One of the schemes for which they generated a positive proof was BurnBox, a system that enables users to temporarily revoke access from their devices to files stored in the cloud to preserve their privacy during compelled-access searches — for example, when an agent at a border crossing compels a traveler to unlock a laptop or smartphone to view its contents. In another analysis, the authors applied their framework to prove the security of a commonly used searchable symmetric encryption scheme for preserving the confidentiality of data and associated searches stored in the cloud. In both of the aforementioned examples, Jaeger and Tyagi showed that their approach produced simpler proofs while avoiding bugs contained in previous analyses. They also discussed how their framework could be extended beyond randomized symmetric encryption schemes currently in use to more modern nonce-based encryption — suggesting that their techniques will remain relevant and practical as the use of newer security schemes becomes more widespread.

“Joseph and Nirvan’s work fills an important void in the cryptographic literature and, surprisingly, identifies important aspects in assessing the security of real-world cryptographic systems that have been overlooked,” said Tessaro. “It also defines new security metrics according to which cryptographic systems ought to be assessed, and I can already envision several avenues of future research.”

Read the full research paper here.

Congratulations to Joseph and Nirvan!

The University of Washington is among the recipients of a five-year, $100 million investment announced today by the National Science Foundation (NSF) aimed at driving major advances in artificial intelligence research and education. The NSF AI Institute for Foundations of Machine Learning (IFML) — one of five new NSF AI Institutes around the country — will tap into the expertise of faculty in the Allen School’s Machine Learning group and the UW Department of Statistics in collaboration with the University of Texas at Austin, Wichita State University, Microsoft Research, and multiple industry and government partners. The new institute, which will be led by UT Austin, will address a set of fundamental problems in machine learning research to overcome current limitations of the field for the benefit of science and society.

“This institute tackles the foundational challenges that need to be solved to keep AI on its current trajectory and maximize its impact on science and technology,” said Allen School professor and lead co-principal investigator Sewoong Oh in a UW News release. “We plan to develop a toolkit of advanced algorithms for deep learning, create new methods for coping with the dynamic and noisy nature of training datasets, learn how to exploit structure in real-world data, and target more complex and real-world objectives. These four goals will help solve research challenges in multiple areas, including medical imaging and robot navigation.”

Oh is part of a group led by UW colleague Sham Kakade that will collaborate on the development of a toolkit of fast and efficient algorithms for training neural networks with provable guarantees. The group also aims to eliminate human bottlenecks associated with training machine learning models by constructing a new theoretical and algorithmic framework for neural architecture optimization (NAO). The latter has received minimal attention from researchers despite a broad range of potential applications, including the deployment of energy-efficient networks for edge computing and the Internet of Things, more transparent interpretable models to replace so-called blackbox predictions, and automated, user-friendly systems that enable developers to apply deep learning to real-world problems.

“The lack of science around NAO is a structural deficit within machine learning that makes us reliant on human intervention for hyper-parameter tuning, which is neither scalable nor efficient,” explained Kakade, who holds a joint appointment in the Allen School and the Department of Statistics. “Using techniques from mathematical optimization and optimal transport, we will automate the process to speed up the training pipeline while significantly reducing its carbon footprint to meet the growing need for academic and commercial applications. Our work will also provide a rigorous theoretical foundation for driving future advances in the field.”

In addition to making progress on NAO and other core machine learning problems, IFML researchers are keen to demonstrate how the results of their work can have real-world impact. To that end, they will apply the new tools and techniques they have developed to multiple use cases where machine learning holds the potential to advance the state of the art, including video compression and recognition, imaging tools for medical applications and circuit design, and robot navigation. The latter effort, which will be spearheaded by Allen School professor Byron Boots, seeks to overcome current limitations on the ability of robots to operate in unstructured environments under dynamic conditions while simultaneously reducing the training burden.

“Room layouts vary, objects can be moved, and humans are generally unpredictable. These conditions pose a challenge to the safe and reliable operation of robots alongside the many users, co-workers, and random passers-by who may share the same space,” noted Boots. “We need to broaden our concept of what constitutes a robot perception task, from one of pure recognition to one where the robot is capable of viewing the environment in the context of goals shaped by interaction and intention. I’m looking forward to working with this team to translate our foundational research into practical solutions for supporting this new paradigm.”

On the human side, a major goal of the IFML is the broadening of participation in AI education and careers to meet expanding workforce needs and to ensure that the field reflects the diversity of society. Institute members will focus their education and workforce development efforts along the entire pipeline, from K-12 to graduate education. Their plans include development of course content for high school students who currently lack access to AI curriculum, the launch of a new initiative aimed at engaging more undergraduate students in AI research, and the build-out of a multi-state, online Master’s program that will leverage faculty from all three member institutions. Allen School professor Jamie Morgenstern, whose research focuses on the social impacts of machine learning, will lead the charge to implement Project 40×24, which aims to increase the number of women participating in AI to represent at least 40% of the field by the year 2024.

“Given the skyrocketing demand for expertise in AI across academia and industry, it should be a national priority to give students and working professionals access to high-quality educational opportunities in this field,” Morgenstern said. “We need to prepare more people from diverse backgrounds to actively participate in shaping the technologies that will have a growing impact on everyone’s lives. And we have a responsibility to ensure that new knowledge and economic opportunities generated by innovations in machine learning are broadly accessible to all.”

Zaid Harchaoui, a professor in the Department of Statistics and an adjunct faculty member in the Allen School, rounds out the UW team.

The IFML is one of two NSF AI Institutes announced today with UW involvement. The other is the NSF AI Institute for Research on Trustworthy AI in Weather, Climate, and Coastal Oceanography led by the University of Oklahoma in collaboration with UW’s Evans School of Public Policy & Governance and other academic and industry partners. 

Each of the five inaugural NSF AI Institutes will receive $20 million over five years. NSF has cast today’s announcement as the start of a longer term commitment, as the agency anticipates making additional institute announcements in future. The initiative, which represents the United States’ most significant federal investment in AI research and education to date, is a partnership between NSF and the U.S. Department of Agriculture, U.S. Department of Homeland Security, and U.S. Department of Transportation.

Read the NSF announcement here, the UW News release here, and UT Austin’s IFML press release here. Learn more about the NSF AI Institutes here.

In July, the Allen School kicked off its inaugural AI4ALL summer program, online. Created to introduce artificial intelligence (AI) to underrepresented pre-college students, AI4ALL is a national program that works to diversify AI by recruiting students who identify with other groups underrepresented in AI. The University of Washington’s debut this summer is the first instance of AI4ALL to focus on students with disabilities and their representation in AI.

The University of Washington joined the program this year, offering a free, two-week data science and AI workshop. Organized by the Allen School’s Taskar Center for Accessible Technology (TCAT), directed by Anat Caspi, the program shows students how to understand, analyze, interpret and discuss real-world applications of data science and machine learning with the ultimate goal of understanding the impact and being comfortable pursuing further work in data science and machine learning.

“The camp aims to increase diverse youth representation in computer science and to promote fair practices among data scientists. Our specific focus at the UW instance of AI4ALL is on fairness and non-ableism in AI,” said Dylan Cottrell, a UW alum, a content writer at the Taskar Center, and a teaching assistant in AI4ALL. “Students learn about artificial intelligence, with a specific emphasis on creating accessible technologies with non-biased data that is fit to serve the diverse community of people who use AI in their daily lives.” 

Students enrolled in the two-week camp had the opportunity to practice using tools in data science and machine learning while making connections with computer scientists and exploring the impacts and ethical implications of AI. Initially, it was intended to be a day camp for local students. But because the COVID-19 pandemic forced it to go online, many students around the country plus one from Germany, were able to participate. were able to participate. In addition to reaching more students, Caspi found other benefits to the online program as well.

“By offering the AI4ALL program virtually this year, TCAT was able to avoid some of the logistic difficulties of getting students to the UW campus, whether due to geographic distance, travel disadvantage, or home situations that wouldn’t allow them to be away from home. We were able to include students with disabilities and students from other diverse backgrounds from both the East and West Coasts,” Caspi said. “I feel that working concurrently through multiple virtual platforms, including teleconferencing, team communication technologies, collaborative programming environments and more, we were able to encourage active learning among students who may not have felt comfortable — or able — to speak in a classroom setting. Instead, they had a choice of voicing their knowledge, questions, opinions or concerns via one of the many platforms we were using. Importantly, I believe every single student had their voice heard, knew they were valued, and understood that there was a spot for them at the broad ‘AI table’.”

Some of the planned camp  activities that we did not get to use virtually were intended to integrate tactile, tangible technologies for students to work with, removing the primary dependency on vision that many demos rely on, Caspi explained. But she hopes that next year the program will be in-person so that they can show students a robot demonstrating clustering algorithms. Such demonstrations would make the learning outcomes perceptible through multiple modes- involving visual, auditory and motion outputs — making one format accessible if another was not. Since it was virtual this year, they were unable to integrate those elements into the course.

Caspi said that while some participants had some exposure to programming and FIRST Robotics, this was most likely the first time many were exposed to more formal principles of  AI. For campers like Sophia Lin, a junior high school student from Bellevue, Washington, AI4ALL was a great opportunity to learn more about it.

“AI4ALL is an extremely fast-paced and challenging data science program. Throughout the first week, I gained insight into various conceptual aspects of the process of training a machine to classify based on interpretation of training data, and was introduced to rigorous programming libraries and data collections that visualize data,” Lin said. “I have been able to open my mind to a wide range of fields that machine learning and data visualization can be applied to, and appreciate having the opportunity to expand my analytical mindset and critical thinking skills. I hope to apply the knowledge and programming skills to future projects that address people with disabilities, to enable them to access and utilize products and services.”

Krithika Subramanian, a high school junior from Oviedo, Florida, thought the program gave her exposure to an area of computer science that she had never considered before.

“My experience in AI4ALL has been thought-provoking, overwhelming, and fun. It has shown me an infinite amount of ways on how AI can be used and the problems it can solve. AI4ALL has opened my eyes to the other sides of the computer science world,” Subramanian said. “Even with coding experience, it has pushed my boundaries, expanding them to encompass an unknown world. A world where plotting a regression line does not seem so hard or using large data sets to classify a small almost insignificant point. Overall, this is one of the most enjoyable classes I have taken this summer and AI4ALL has exposed me to one of the most interesting fields ever.”

This summer the UW’s AI4ALL introduced 18 underrepresented students to data science and machine learning. The camp will continue next year, learn more about the program and how to enroll on the website. 

A team of researchers that includes Allen School professor Carlos Guestrin, Ph.D. student Tongshuang Wu and alumnus and affiliate professor Marco Tulio Ribeiro (Ph.D. ’18) of Microsoft captured the Best Paper Award at the 58th annual meeting of the Association for Computational Linguistics (ACL 2020). In the winning paper, “Beyond Accuracy: Behavioral Testing of NLP Models with CheckList,” they and former Allen School postdoc Sameer Singh, now a professor at the University of California, Irvine, introduce a methodology for comprehensively testing models for any natural language processing (NLP) task.

“In recent years, many machine learning models have achieved really high accuracy scores on a number of natural language processing tasks, often coming very close to or outperforming human accuracy,” Guestrin said. “However, when these models are evaluated carefully, they show critical failures and a basic lack of understanding of the tasks, indicating that these NLP tasks are far from solved.” 

Much of lead author Ribeiro’s prior work is on analysis and understanding of machine learning and NLP models. As the driving force of the project, he began testing and found critical failures in several state-of-the-art models used for research and commercial purposes — despite the fact that these models out-performed humans on standard benchmarks. To remedy this problem, Ribeiro and his colleagues created a process and a tool called CheckList, which provides the framework and tooling necessary to enable developers to break their tasks into different “capabilities” and then write tests for these using a variety of test types. 

People in several industries found new and actionable bugs to fix by using CheckList, even in systems that had been extensively tested in the past. In fact, NLP practitioners found three times more bugs by using CheckList than compared to existing techniques. 

“I did a case study with a team at Microsoft and they were very happy to discover a lot of new bugs with CheckList”,” Ribeiro said. “We also tested models from Google and Amazon and we’ve received positive feedback from researchers in both companies.”

Including CheckList, University of Washington researchers earned three out of the five recognized paper awards at ACL this year. The second of these, a Best Paper Honorable Mention, went to a group of researchers that included professor Noah Smith, Ph.D. student Suchin Gururangan, and postdocs Ana Marasović and Swabha Swayamdipta for their work on “Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks.” The team — which also included applied research scientists Kyle Lo and Iz Beltagy of AI2 and Northwestern University professor Doug Downey — explored whether they could apply a multi-phased, adaptive approach to pretraining to improve the performance of language models that are pretrained on text from a wide variety of sources, from encyclopedias and news articles, to literary works and web content. Using a method they referred to as domain-adaptive pretraining (DAPT), the researchers showed how tailoring a model to the domain of a specific task can yield significant gains in performance. Further, the team found that by adapting the model to a task’s unlabeled data — an approach known as task-adaptive pretraining (TAPT) — they could boost the model’s performance with or without the use of DAPT.

In addition, Allen School adjunct professor Emily Bender of the UW Department of Linguistics and Saarland University professor Alexander Koller were awarded  Best Theme Paper for “Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data,” in which they draw a distinction between form and meaning in assessing the field’s progress toward natural language understanding. While recognizing the success of large neural models like BERT on many NLP tasks, the authors assert that such systems cannot capture meaning from linguistic form alone. Instead, Bender and Koller argue, these models have been shown to learn some reflection of meaning into the linguistic form that has proven useful in various applications. With that in mind, the authors offer some thoughts on how members of their field can maintain a healthy — but not overhyped — optimism with respect to communicating about their work with language models and the overall progress of the field. 

Congratulations to all on an outstanding showing at this year’s conference!

Allen School professor emeritus Richard Ladner, a leading researcher in accessible technology and a leading voice for expanding access to computer science for students with disabilities, has been named the 2020 recipient of the Public Service Award for an individual from the National Science Board (NSB). Each year, the NSB recognizes groups and individuals who have made significant contributions to the public’s understanding of science and engineering. In recognizing Ladner, the board cited his exemplary science communication, diversity advocacy, and well-earned reputation as the “conscience of computing.”

A mathematician by training, Ladner joined the University of Washington faculty in 1971. For much of his career, he focused on fundamental problems underpinning the field of computer science as one of the founders of what is now the Allen School’s Theory of Computation research group. After making a series of significant contributions in computational complexity and optimization — and later, branching out into algorithms and distributed computing — his career would take an unexpected but not altogether surprising turn toward accessibility advocacy and research.

Ladner enrolled in an American Sign Language course at a local community college, a move that represented a “return to his roots” after growing up in a household where both parents were deaf. That experience spurred him to begin volunteering in the community with people who were deaf and blind and to occasionally write about accessibility issues.

Then, in 2002, Ladner began working with Ph.D. student Sangyun Hahn at the UW. Hahn, who is blind, related to Ladner how he was having trouble accessing the full content of his textbooks; mathematical formulas had to be read aloud to him or converted into Braille, while graphs and diagrams had to be manually traced, labeled in Braille, and printed on an embosser. His student’s frustration was the impetus for Ladner and Hahn to launch the Tactile Graphics project, which automated the conversion of textbook figures into an accessible format. Ladner followed that up with MobileASL, a collaboration with Electrical & Computer Engineering professor Eve Riskin to enable people who are deaf to communicate in American Sign Language using mobile phones. Ladner also mentored many Ph.D. students in accessibility research — among them Anna Cavender (Ph.D., ‘10), who developed technology to consolidate a teacher, display screen, sign language interpreter, and captioning on a single screen; Jeffrey Bigham (Ph.D., ‘09), who developed a web-based screen reader that can be used on any computer without the need to download any software; Information School alumnus Shaun Kane, who developed technology to make touchscreen devices accessible to people who are blind; and Shiri Azenkot (Ph.D., ‘14), who developed a Braille-based text entry system for touchscreen devices. 

Ladner’s approach to accessibility research is driven by the recognition that to build technology that is truly useful, you have to work with the people who will use it. It’s a lesson he took from his earlier experience as a volunteer, and one that he has emphasized with every student who has worked with him since. During his career, Ladner has mentored 30 Ph.D. students and more than 100 undergraduate and Master’s students — many of whom followed his example by focusing their careers on accessible technology research. 

“I visited Richard’s lab at the University of Washington just over 10 years ago. While I did get to see Richard, he was most interested in my meeting his Ph.D. students — and I could see why,” recalled Vicki Hanson, CEO of the Association for Computing Machinery. “Richard had provided an atmosphere in which his talented students could thrive. They were extremely bright, enthusiastic, and all involved in accessibility research. I spent the day talking with his students and learning about their innovative work.

“All were committed to developing technology that would overcome barriers for people with disabilities. Sometimes there are barriers in being able to use technology – in other cases, however, the use of technology actually provides opportunities to remove barriers in various aspects of daily living,” Hanson continued. “Richard’s students were working on both of these aspects of accessibility. The collegial and inspiring interactions among his students would serve as a model of research collaboration for computing labs everywhere.”

Ladner’s impact on students extends far beyond the members of his own lab. In addition to his research contributions and mentorship, Ladner has been a prominent advocate for providing pathways into computer science for students with disabilities. To that end, he has been a driving force behind multiple initiatives designed to engage a population that, until recently, was often overlooked in technology circles.

“When we think about diversity, we must include disability as part of that,” Ladner noted. “The conversation about diversity should always include disability.”

To that end, Ladner has been a leading voice for the inclusion of people with disabilities in conversations around improving diversity in technology. He served as a founding member of the board of the national Center for Minorities and People with Disabilities in Information Technology (CMD-IT). The organization hosts the ACM Richard Tapia Celebration of Diversity in Computing, which attracts an estimated 1,500 attendees of diverse backgrounds and abilities each year. Ladner was also a member of the steering committee that established the Computing Research Association’s Grad Cohort Workshop for Underrepresented Minorities and Persons with Disabilities (URMD) for beginning graduate students. In discussions leading up to the program’s launch, Ladner was instrumental in making sure that the “D” made it into the name and scope of the workshop.

Ladner has also worked directly with colleagues and students around the country to advance diversity in the field. The longest-running of these initiatives is the Alliance for Access to Computing Careers (AccessComputing), which he co-founded with Sheryl Burghstahler, Director of the UW’s DO-IT Center, with funding from National Science Foundation’s Broadening Participation in Computing program. AccessComputing and its 60 partner institutions and organizations support students with disabilities to successfully pursue higher education and connect with career opportunities in computing fields. Since its inception in 2006, that initiative has served nearly 1,000 high school and college students across the country. For seven consecutive years, Ladner also organized the annual Summer Academy for Advancing Deaf and Hard of Hearing in Computing to prepare students to succeed in computing majors and careers.

More recently, Ladner partnered with Andreas Stefik, a professor at the University of Nevada, Las Vegas, on AccessCSForAll. That initiative is focused on developing accessible K-12 curricula for computer science education along with professional development for teachers. The duo also partnered with Code.org to review and modify the Computer Science Principles Advanced Placement course to ensure that online and offline course activities met accessibility standards for students with disabilities. This included developing accessible alternatives to visually-based unplugged activities as well as making interactive tools that would work with screen readers. Ladner and his collaborators on the project earned a Best Paper Award at last year’s conference of the ACM’s Special Interest Group on Computer Science Education (SIGCSE 2019) for their efforts.

This past spring, Ladner was one of nine researchers to co-found the new Center for Research and Education on Accessible Technology and Experiences (CREATE) at the UW. The mission of CREATE is to make technology accessible and to make the world accessible through technology. The center, which was established with an inaugural $2.5 million investment from Microsoft, consolidates the efforts of faculty from the Allen School, Information School, and departments of Human Centered Design & Engineering, Mechanical Engineering, and Rehabilitation Medicine who work on various aspects of accessibility. 

“Richard is a gifted scientist and mentor who really helped to put UW on the map when it comes to accessible technology,” said professor Magdalena Balazinska, Director of the Allen School. “As a staunch advocate for innovation that serves all users, his impact on computing education and research cannot be overstated.”

Since his retirement in 2017, Ladner has remained engaged with the Allen School community and continues to invest his time and energy in accessible technology research and increasing opportunities for students with disabilities in computing fields. In accepting this latest accolade — one in a long line of many prestigious awards he has collected during his career — Ladner expressed optimism that accessibility’s importance is recognized by an increasing number of his peers.

“I am honored to receive this recognition from the National Science Board and heartened that the scientific community is rising to the important challenge of supporting students with disabilities,” Ladner said.

Read the NSB press release here, and learn more about Ladner’s career and contributions in a previous Allen School tribute here.

Allen School professor and alumnus Michael Ernst (Ph.D., ‘00) has had a distinguished research career that spans more than two decades and includes multiple, influential contributions to programmer productivity through software analysis, testing and verification. In recognition of his impressive body of work, Ernst was recently recognized with the 2020 Outstanding Research Award from the Association for Computing Machinery’s Special Interest Group on Software Engineering (ACM SIGSOFT). 

Since he returned to his alma mater in 2009 to take up a faculty position with the Allen School’s Programming Languages and Software Engineering (PLSE) group, Ernst has advanced the state of the art in the field on multiple fronts and earned many accolades along the way. The latest of these, SIGSOFT’s most prestigious award, recognizes his enduring impact on the theory and practice of software engineering.

“Michael Ernst is one of the most accomplished software engineering researchers. His work has led to many tools that are applied daily to real life software products,” said SIGSOFT Chair Tom Zimmermann, Senior Principal Researcher at Microsoft. “His Daikon tool was revolutionary and an elegant solution that impacted the research community in testing, verification, and programming languages.”

Daikon is a software system that dynamically detects likely program invariants. Ernst and his co-authors first presented Daikon in the 1999 paper, “Dynamically discovering likely program invariants to support program evolution,” while he was a Ph.D. student with fellow graduate student Jake Cockrell (M.S., ’99), Allen School alumnus and UCSD professor William G. Griswold (Ph.D., ’91), and the late Allen School professor David Notkin. In the paper, Ernst and his collaborators described novel techniques, such as tracing variables of interest, for enabling programmers to easily identify program properties that must be preserved when modifying code. The team’s approach initiated an important new line of research, earned the 2013 Impact Paper Award, and continues to be relevant more than two decades later.

Another research highlight of Ernst’s impressive career so far was a paper he co-authored with Carlos Pacheco of Google, and Shuvendu Lahiri and Thomas Ball of Microsoft Research called “Feedback-directed random test generation” in 2007. In the paper, Ernst and his colleagues presented a test generation technique and corresponding tool, Randoop, which generates tests for programs written in object-oriented languages such as Java and .NET. The technique put forward by the team generates one test at a time, executes the test, and classifies it as a normal execution, a failure, or an illegal input. Based on this information, the technique biases the subsequent generation process to extend good tests and avoid bad tests. Ernst and the team earned the Most Influential Paper Award from the International Conference on Software Engineering (ICSE) in 2017, and Randoop remains the standard benchmark against which other test generation tools are measured.

A year after his ICSE recognition, Ernst received a 2018 Impact Paper Award from the International Symposium on Software Testing and Analysis (ISSTA) for the 2008 paper, “Practical pluggable types for Java.” With this project, Ernst and his co-authors set out to address a well-known verification problem for programmers working in Java: while type checking could prevent many bugs, it couldn’t prevent enough bugs. To address this, Ernst and his colleagues developed the Checker Framework, a scalable tool for creating expressive and concise pluggable type systems for Java. The Checker Framework incorporated a type system component for defining type qualifiers and their semantics, and a compiler plug-in that enforces the semantics. It is also backward-compatible and integrates easily with standard development environments and tools, making it useful to programmers seeking to write bug-free code as well as type-system designers looking to evaluate and deploy their type systems. At the time of publication, Ernst was a faculty member at MIT, working with researcher Jeff H. Perkins, graduate student Matthew M. Papi, and undergraduates Mahmood Ali and Telmo Luis Correa Jr.

Among his many other accolades, Ernst earned a 2019 ISSTA Impact Paper Award for “HAMPI: a solver for string constraints,” a 2011 European Conference on Object-Oriented Programming (ECOOP) Best Paper Award and a total of nine ACM Distinguished Paper Awards. He was the recipient of the inaugural John Backus Award (2009) — created by IBM to honor mid-career university faculty members — and was elected a Fellow of the ACM in 2014. 

“Michael is an exceptional software-engineering researcher and his continual momentum to produce research has advanced the field and has made some incredible real-world impacts,” said professor Magdalena Balazinska, director of the Allen School. “In addition to his research, he’s been an admirable influence on our next generation of computer scientists and engineers. We’re grateful he is a part of the Allen School faculty.”

In 2018, Ernst was formally recognized for his work as an educator and mentor with the CRA-E Undergraduate Research Faculty Mentoring Award from the Computing Research Association. The award recognizes faculty who provide exceptional mentorship and support to student researchers.

“I’m passionate about mentoring because doing research as an undergraduate changed the course of my career and my life,” Ernst said at the time. “I love making new discoveries — and I get a vicarious thrill from helping others to experience that same wonder.”

Congratulations, Mike!

No one could accuse Ph.D. student Vikram Iyer of just winging it. Since his arrival at the University of Washington, Iyer has advanced ground-breaking innovations in low-power wireless communication and computation to expand the Internet of Things, from 3D-printable wireless objects capable of storing and transmitting data, to insect-scale platforms that provide a bug’s eye view of the world. As a sign of just how his ideas have taken flight, today Iyer was named one of three recipients of the Paul Baran Young Scholar Award by the Marconi Society.

“By creating low cost, mobile IoT devices that can help answer questions and solve problems in any environment, Vikram’s work supports the Marconi Society’s mission of bringing the opportunity of the network to everyone,” said internet pioneer Vint Cerf, Chair of the Marconi Society, in a press release. “We are proud to welcome him to the Marconi family.”

The award recognizes innovative young engineers who show extraordinary technical acumen, creativity and promise for creating tomorrow’s information and communications technologies to support a digitally inclusive society. The honor came as no surprise to Iyer’s advisor, Allen School professor Shyam Gollakota, who recognized early on that his student would be a highflier.

”Vikram is a one-of-a-kind creative interdisciplinary researcher who is also humble,” said Gollakota. “He develops creative solutions that are at the intersection of hardware, software and biology. In so doing, he transforms what was once science fiction into reality.”

Iyer, a student in the UW Department of Electrical & Computer Engineering, began working with Gollakota in the Networks & Mobile Systems Lab in 2015. Among their first projects together was a collaboration with Allen School and ECE professor Joshua Smith of the Sensor Systems Laboratory on an ultra-low power system to provide wireless connectivity for implantable devices. Interscatter — short for intertechnology backscatter — employs a technique called backscatter communication to convert Bluetooth transmissions to WiFi and ZigBee signals over the air using commodity devices. As part of that project, Iyer and his collaborators created the first prototype contact lens antenna and an implantable neural recording interface capable of communicating directly with smartphones and smart watches. The team earned a Best Paper Award from the Association for Computing Machinery’s Special Interest Group on Data Communications (ACM SIGCOMM).

More recently, Iyer, Gollakota and their colleagues teamed up with professor Sawyer Fuller of the UW Mechanical Engineering Department’s Autonomous Insect Robotics (AIR) Lab to enable new wireless robotic technologies to take flight. The result was Robofly, the world’s first wireless fly-sized drone to achieve liftoff. Unlike previous insect-scale drones, Robofly does not require a wire to the ground to supply power and control signals — a significant achievement on the path toward autonomous robot flight. The team’s bio-inspired design featured dual flapping wings driven by a pair of piezoelectric actuators and directed by a lightweight microcontroller, which issues a series of pulses mimicking the action of a biological fly’s wings. An onboard photovoltaic cell converts light from a laser beam into electricity to power the onboard components without the need for heavy batteries, while the first sub-100 milligram boost converter and piezo driver sufficiently boosts the voltage to enable RoboFly’s ascent. 

While news of Robofly’s exploits took off, Iyer recognized that there are limitations to what a drone insect could do. For one thing, robotic liftoff was difficult to achieve. And commercial drones are limited in how long they can fly uninterrupted.

“This made me wonder, rather than building a system that mimics an insect, could we augment live insects with sensing, computing and communication functionalities to create a mobile IoT platform?” Iyer explained. “We could use this platform to study micro-climates on large farms, answer questions about insects’ behavior or collect air quality data at a more granular level than by using a handful of stationary sensors.”

To explore the idea, Iyer set up an amateur beekeeping operation in a room in the Paul G. Allen Center on campus. The result was Living IoT, a mobile platform that combines sensing, computation, and communication packaged into a tiny wireless backpack light enough to be carried by a bumblebee. The entire system — antenna, envelope detector, sensor, microcontroller, backscatter transmitter, and rechargeable battery — weighed in at just 102 milligrams, or around half a bumblebee’s potential payload. Because the system did not need to power flight, only data collection, the team could keep the weight down by designing the system to transmit data and recharge the battery when the bee returned to the hive each day.

For his latest project, which was recently published in Science Robotics, Iyer’s insect subjects kept their feet on the ground. Building off of the previous work with bees, Iyer and his collaborators created a new wireless backpack containing a tiny, steerable video camera operated via Bluetooth. This time, they fitted their system on two species of beetle to demonstrate the potential for insect-scale robotic vision. Dubbed “BeetleCam,” the system emulates a real bug’s energy-efficient approach to gathering visual information, which relies on head motion independent of its body, while a built-in accelerometer prolongs the battery life by allowing the system to capture images only when the beetle is in motion. Weighing in at a mere 250 milligrams, or roughly half the payload the insects can carry, the system enables the beetles to freely navigate terrain and climb trees.

The team used what it learned to design the world’s smallest power-autonomous terrestrial robot with vision — proving, once again, that good things really do come in small packages.

“This is the first time that we’ve had a first-person view from the back of a beetle while it’s walking around. There are so many questions you could explore, such as how does the beetle respond to different stimuli that it sees in the environment?” Iyer said in a UW press release. “But also, insects can traverse rocky environments, which is really challenging for robots to do at this scale. So this system can also help us out by letting us see or collect samples from hard-to-navigate spaces.”

Iyer is the second UW student — and second from Gollakota’s lab — to earn this prestigious award. His labmate and frequent collaborator, Rajalakshmi Nandakumar (Ph.D. ‘19), now a faculty member at Cornell Tech, was honored in 2018 for her work on mobile apps for detecting life-threatening health issues. In addition to Iyer, the Marconi Society recognized two other researchers with 2020 Young Scholar Awards: Yasaman Ghasempour at Rice University (soon joining the Princeton University faculty) for her work on efficient, ultra-high speed network connections for next-generation IoT, and Piotr Roztocki at Canada’s Institut National de la Recherche Scientifique (INRS) for his work on scalable quantum resources for “future-proofing” telecommunications network security. The honorees were selected by an international panel of engineers drawn from leading universities and companies.

“Our Young Scholars are the braintrust that will put the speed, security and applications of next generation networks into the hands of billions,” said Cerf.

View Iyer’s Marconi Society profile here, and learn more about the 2020 Young Scholar Awards here. Watch a conversation between Iyer and Marconi Fellow Brad Parkinson here, and check out Iyer’s Geek of the Week profile on GeekWire here.

Congratulations, Vikram!