Allen School News

The Institute of Electrical and Electronics Engineers (IEEE) recently honored three members of the Allen School community for advancing the field of computing: professor and alumnus Michael Ernst (Ph.D., ‘00), affiliate professor Thomas Zimmermann of Microsoft Research, and alumnus Jeffrey Dean (Ph.D., ‘96) of Google. Ernst and Zimmerman were elevated to IEEE Fellows in recognition of their career contributions in software engineering, while Dean earned the organization’s prestigious John von Neumann Medal for advancing internet-scale computing and artificial intelligence. 

Each year, the IEEE Fellows Program recognizes members with extraordinary records of accomplishment in computing, aerospace systems, biomedical engineering, energy, and more. With 420,000 members in 160 countries, a maximum of only one-tenth of one percent of the total voting membership can be elevated to Fellow status in a given year. The John von Neumann Medal, which is among the highest honors IEEE bestows upon members of the engineering community, recognizes individuals who have made outstanding contributions in computer hardware, software or systems that have had a lasting impact on technology, society, and the engineering profession.

Michael Ernst, IEEE Fellow

After earning his Ph.D. from the Allen School working with the late professor David Notkin, Michael Ernst spent eight years on the faculty of MIT. He returned to the Pacific Northwest in 2009 to take up a faculty position in the Allen School, where he is part of the Programming Languages and Software Engineering (PLSE) group. Since rejoining his alma mater, Ernst has continued to make significant contributions through research and mentorship that have had an enduring impact on the field. IEEE recognized that impact by electing him Fellow for “contributions to software analysis, testing, and verification.”

One of Ernst’s earliest contributions, which he made while a Ph.D. student at UW, was Daikon, a tool that enables programmers to easily identify program properties that must be preserved when modifying code. The novel techniques he developed with his collaborators — Notkin, Jake Cockrell (M.S., ’99), and Allen School alumnus William G. Griswold (Ph.D., ’91), now a professor at University of California, San Diego — offered a revolutionary solution that went on to impact the research community in testing, verification and programming languages. The team’s 1999 paper presenting Daikon, “Dynamically discovering likely program invariants to support program evolution,” went on to earn the 2013 Impact Paper Award from the Association for Computing Machinery’s Special Interest Group on Software Engineering (ACM SIGSOFT). 

Ernst later created Randoop, an instrumental tool for generating tests for programs written in object-oriented languages such as Java and .NET, with Carlos Pacheco of Google and Shuvendu Lahiri and Thomas Ball of Microsoft Research while he was on the faculty of MIT. Randoop generates and executes one test at a time and classifies it as a normal execution, a failure, or an illegal input; it then uses that information to find biases in the subsequent generation process to extend good tests and avoid bad tests. A decade later, Ernst and the team earned the Most Influential Paper Award from the International Conference on Software Engineering (ICSE) for their work. Even today, Randoop remains the standard benchmark against which other test generation tools are measured.

For programmers working in Java, Ernst and his colleagues developed the Checker Framework, a system for easily and effectively developing special-purpose type systems that can prevent software bugs. The team presented its work in a 2008 paper that later earned a 2018 Impact Paper Award from the International Symposium on Software Testing and Analysis (ISSTA). The following year, Ernst earned his second ISSTA Impact Paper Award for “HAMPI: a solver for string constraints.” HAMPI is designed for constraints generated by program analysis tools and automated bug finders. Given a set of constraints, HAMPI outputs a string that satisfies all the constraints, or reports that the constraints are unsatisfiable, at a faster pace than similar tools. 

During his career, Ernst has received a total of nine ACM Distinguished Paper Awards in addition to a Best Paper Award from the European Conference on Object-Oriented Programming (ECOOP). Last year, Ernst’s extraordinary career contributions to the field of software engineering earned him the ACM SIGSOFT Outstanding Research Award. He previously received the CRA-E Undergraduate Research Faculty Mentoring Award in 2018, in recognition of his exceptional support for student researchers, and the inaugural John Backus Award — created by IBM to honor mid-career university faculty members — in 2009. Ernst was elected a Fellow of the ACM in 2014. 

Ernst’s election as an IEEE Fellow brings the total number of current or former Allen School faculty members who have earned this distinction to 17.

Thomas Zimmermann, IEEE Fellow

Thomas Zimmermann, who has been an affiliate faculty member in the Allen School since 2011, is a member of the Software Analysis and Intelligence (SAINTES) group at Microsoft Research. He focuses on the development of solutions that increase programmer productivity, including tools for the systematic mining of version archives and bug databases. His goal is to help developers and managers learn from past successes and failures to create better software. IEEE named Zimmerman a Fellow based on his “contributions to data science in software engineering, research and practice.”

Zimmermann earned his Ph.D. from Saarland University in Saarbrücken, Germany and joined Microsoft Research in 2008. During his career, he has earned a total of seven Most Influential Paper or Test of Time Awards and five ACM SIGSOFT Distinguished Paper Awards for his work. Last year, Zimmermann shared the IEEE CS TCSE New Direction Award with Ahmed Hassan of Queen’s University for their leadership in establishing the field of mining software repositories (MSR) and a successful conference series with the same name, which advanced the use of analytics and data science in software engineering and led the field in new directions. In addition to his research, Zimmermann serves as co-editor in chief for the journal Empirical Software Engineering and current chair of ACM SIGSOFT.

Jeffrey Dean, John von Neumann Medal

Allen School alumnus Jeffrey Dean was honored with the 2021 John von Neumann Medal in recognition of his “contributions to the science and engineering of large-scale distributed computer systems and artificial intelligence systems.” Dean, who completed his Ph.D. working with then-professor Craig Chambers on the development of whole-program optimization techniques for object-oriented languages, is currently a Google Senior Fellow and senior vice president of Google Research and Google Health.

Dean joined Google in 1999, roughly a year after its founding — making him one of the company’s longest-serving employees. During his tenure, Dean led the conception, design and implementation of core elements of Google’s search, advertising and cloud infrastructure that would transform the internet and computing as we know it. His contributions included five generations of the company’s crawling, indexing, and query serving systems. Dean was also responsible for the initial development of Google’s AdSense for Content, which revolutionized online advertising by enabling content creators to monetize their websites.

Dean’s subsequent contributions to internet-scale data storage and processing, in collaboration with Google colleague Sanjay Ghemawat, helped propel the company to the forefront of cloud computing. He played a leading role in the design and implementation of MapReduce, a system for simplifying the development of large-scale data processing applications. To date, the paper presenting MapReduce has garnered more than 30,000 citations and inspired future advances in distributed computing. Dean and Ghemawat were also central figures in the development of BigTable, a semi-structured data storage system designed for flexibility and high performance while scaling to petabytes of data across thousands of commodity servers. BigTable underpins a variety of Google products and services, including web indexing, Google Earth, and Google Finance. Along the way, Dean also contributed to the development and implementation of Google News, Google Translate, and a variety of other projects that enhanced Google’s data management, job scheduling, and code search infrastructure.

More recently, Dean has accelerated Google’s leadership in artificial intelligence as co-founder of the Google Brain team focused on the fundamental science of machine learning as well as projects aimed at infusing the company’s products with the latest developments in the field. The team has been responsible for advancing the technology of deep learning and its impact on computer vision, speech, machine translation, natural language processing, and a variety of other applications. One of Dean’s crowning achievements in this area is TensorFlow, an open-source platform for the large-scale training and deployment of deep learning models. TensorFlow offered unprecedented flexibility to application developers and to support experimentation with novel training algorithms and optimizations. 

Dean is a Member of the National Academy of Engineering and a Fellow of both the ACM and the American Academy of Arts and Sciences. Together with his collaborator, Ghemawat, he earned the ACM SIGOPS Mark Weiser Award from the ACM’s Special Interest Group on Operating Systems in 2011 and the ACM–Infosys Foundation Award — now known as the ACM Prize in Computing, one of the highest honors bestowed by the organization — in 2012 for introducing revolutionary software infrastructure that advanced internet-scale computing.

Congratulations to Michael, Thomas and Jeff! 

Tal August, a Ph.D. student working with Allen School professors Katharina Reinecke and Noah Smith, has been named a 2021 Twitch Research Fellow for his work in creating writing tools to accommodate different online audiences. His current research focuses on strategies used to help moderate these communities in order to shape them into vibrant, supportive, online spaces.

August is one of five Twitch Fellows selected and aims to use his Fellowship to automate tools that support conversations in domains like science communication and in online communities like Twitch and Reddit. 

“Online communities can be such vibrant places, and that is often reflected in their language,” he said. “The goal of this work is to build tools that tap into the language of a community, guiding newcomers and moderators to communicate more effectively.” 

Using LabintheWild, a site that tests people’s abilities and preferences so that researchers can improve users’ experience when interacting with technology, August studied the effects of language styles that influence user behavior. This led to his work on building writing tools tailoring language to better engage different audiences.

Currently, most tools for online moderation are used for policing posts to eliminate profanity and toxic behavior. August aims to make moderation tools more welcoming. Working at the intersection of human computer interaction (HCI) and natural language processing (NLP), he has found that newcomers in online communities face barriers by not understanding the norms of the community. Based on his research, he will create tools to identify ways for moderators to encourage positive contributions for more inclusive engagement and discourse. 

With the help of machine learning, August’s tool will analyze newcomer remarks and moderator responses in order to suggest responses that are more likely to encourage future positive contributions by the newcomer. Many online forums list rules or frequently asked questions for new users to read before engaging in the conversation. August would build a separate tool to arm newcomers with more information by giving them just-in-time recommendations for making positive posts.

“Tal’s research at the intersection of HCI and NLP will help encourage more positive and thoughtful conversations in online communities such as Twitch,” Reinecke said.

August’s tools will be open-sourced for moderators to use in their online communities. They will help them respond to newcomers in a more meaningful way and foster a more welcoming online space for everyone. 

In addition to his Twitch Fellowship, August received a University of Washington Endowed Fellowship in Computer Science and Engineering in 2018 and has published nine refereed conference papers, including the Conference on Empirical Methods in Natural Language Processing and the Association of Computing Machinery’s Conference on Human Factors in Computing Systems. In addition to being a student and researcher, he is a teaching assistant for NLP and HCI capstone classes. 

Congratulations, Tal! 

Joy He-Yueya applies data science techniques to measures of patient behavior to assess how they might predict the onset of schizophrenia symptoms. Meanwhile, Ximing Lu uses machine learning to improve cancer diagnosis and explores how neural language models can advance commonsense reasoning. Parker Ruth builds mobile sensing systems to detect and monitor a variety of health conditions, while Jenny Liang develops programming techniques to support developer education and collaboration. And Emily Bascom contributes to research aimed at promoting user privacy and improving patient care.

For producing results with the potential for real-world impact, each of these University of Washington students — three nominated by the Allen School, two by the Information School — recently earned national recognition as part of the Computing Research Association’s 2021 Outstanding Undergraduate Researcher Awards. Each year, the CRA competition highlights a select group of undergraduates at universities and colleges across North America who are already advancing the field of computing and making an impact through their work. The achievements of the five outstanding student researchers honored in this year’s competition are a testament to UW’s commitment to undergraduate research; they are also proof that it’s never too early to begin using computing in the service of social good.

Joy He-Yueya (Awardee – Allen School)

CRA award recipient Joy He-Yueya is a senior majoring in computer science in the Allen School who has engaged in a variety of research projects related to health and education. Last year, He-Yueya began working with professor Tim Althoff, leader of the Allen School’s Behavioral Data Science Group, and UW Medicine faculty Benjamin Buck and Dror Ben-Zeev on a project seeking to mine the vast quantities of data generated by passively sensed measures of behavioral stability to support mental health. Their goal was to use  data science techniques to understand the relationship between patients’ routines and the onset of schizophrenia symptoms. He-Yueya, who took the lead on data preparation, analysis and visualization as well as algorithmic development for the project, was first author of the paper describing the team’s findings that recently appeared in the journal npj Schizophrenia. 

“What sets Joy apart as a student researcher is her independence to lead a research project herself and to collaborate with clinical researchers to connect innovations in computing and measurement to clinical goals,” said Althoff. “She was also very impressive at handling the complexity of a project that involved significant experimentation and seeing a project through from the first ideas to writing and to publication.”

He-Yueya recently contributed to a project at the Max Planck Institute for Software Systems in Saarbrücken, Germany that applies reinforcement learning to generate personalized curricula for students learning to code. She also has been working with researchers at Seattle-based Giving Tech Labs to develop methods for identifying relationships between voice and emotions and between voice and aging. In addition to her research, He-Yueya has served as a teaching assistant for the Allen School’s Introduction to Algorithms and Data Structures and Parallelism courses and has volunteered her time to a number of tutoring and peer mentoring roles — including leading workshops to help her fellow undergraduates get their own start in research.

He-Yueya’s entrée to academic research was working with iSchool professor Katie Davis in the Digital Youth Lab, where she focused on digital incentives for students to pursue science and engineering-related education. She earned a Mary Gates Research Scholarship from UW last year for her work. He-Yueya plans to pursue a Ph.D. following her graduation from the Allen School next spring.

Ximing Lu (Runner-up – Allen School)

Ximing Lu, who is majoring in computer science and statistics, was named a runner-up in the CRA competition for her work in machine learning and natural language processing. In less than three years, Lu has contributed to four major papers in submission — three of them as lead author. Her first foray into undergraduate research was working on a project with professor Linda Shapiro, who holds a joint appointment in the Allen School and Department of Electrical & Computer Engineering, that applies machine learning to improve the speed and accuracy of breast cancer diagnosis by reducing the uncertainty that stems from subjective human interpretation. The system they designed, Holistic ATtention Network (HATNet), is capable of learning representations from clinically relevant tissue structures without explicit supervision to classify gigapixel-sized biopsy images with the same level of accuracy as human pathologists.

Since last year, Lu has collaborated with Allen School professor Yejin Choi and colleagues in the Allen Institute for AI’s MOSAIC group on multiple projects seeking to advance the state of the art in natural language processing and visual commonsense reasoning. Among Lu’s contributions is TuBERT, a new multi-modal neural network capable of commonsense reasoning about the temporal relationship between visual events using a combination of YouTube video content and clues from their accompanying text narratives. Since its introduction, TuBERT has achieved state-of-the-art results on multiple commonsense reasoning tasks by outperforming substantially larger, commercially funded neural networks. Lu has also worked on Reflective Decoding, an approach for enabling pre-trained language models to complete paraphrasing and abductive text-infilling tasks without supervision, and the NeuroLogic decoding algorithm for controlling neural text generation models through logical constraints.

“Ximing is one of the most creative and innovatives undergraduate students I have had the pleasure to work with,” said Choi. “She has an impressive ability to rapidly synthesize new technical ideas based on seemingly disconnected pieces. Everyone in my lab has been eager to collaborate with her.”

Last fall, Lu received the Lisa Simonyi Prize honoring an Allen School student who exemplifies excellence, leadership and diversity. She was also named a Levinson Emerging Scholar by the UW in recognition of her accomplishments in research. After graduation, Lu plans to continue her studies next fall as a student in the Allen School’s fifth-year master’s program.

Parker Ruth (Finalist – Allen School)

Parker Ruth, a computer engineering and bioengineering major advised by professor Shwetak Patel in the Allen School’s UbiComp Lab, was named a finalist by CRA for his cross-disciplinary work on mobile health sensing technologies and computational tools for supporting population health. During his more than three years as an undergraduate researcher, Ruth has contributed to multiple projects aimed at enabling early identification and monitoring of symptoms and risk factors associated with a variety of  medical conditions. His efforts have included the development of non-invasive, smartphone-based tools for measuring bone density to screen for osteoporosis, tracking cardiovascular disease through real-time measurement of pulse transit time, and detecting cough to facilitate diagnosis and monitoring of respiratory illness.

Most recently, Ruth has led the design of a smartphone-based exercise sensing system that employs ultrasonic sonar to measure physical activity as part of the Exercise Rx initiative in collaboration with The Sports Institute at UW Medicine. He is also developing algorithms to screen for risk of stroke by measuring blood flow in videos of the face. In addition, Ruth has contributed to the development of a wearable pulse sensing system for detecting a rare but serious cardiovascular condition known as postural orthostatic tachycardia syndrome. In response to the current pandemic, Ruth has worked on environmental sampling and viral detection protocols for screening air filtration systems in public transit and, in collaboration with bioengineering professor Barry Lutz, built image processing software that powers a smartphone-based tool for streamlining molecular assays for the virus in order to speed up diagnosis.

“Parker is advancing how we think about digital health and how commodity devices can play a role in democratizing health care and increasing access for everyone. He has demonstrated unprecedented work ethic, creativity, rigor, and an unique ability to present his work to a general audience,” said Patel. “He already shows the maturity of a graduate student and the capacity to define broad research agendas. On top of that, he is the most humble and selfless person you will ever meet.”

Ruth previously was named a Goldwater Scholar and recognized as a member of the Husky 100. He is a student in the University’s Interdisciplinary Honors Program and Lavin Entrepreneurship Honors Program and has been active in outreach to K-12 students, including helping to oversee the UbiComp Lab’s high school mentorship program.

Jenny Liang (Honorable Mention – iSchool)

Jenny Liang is well-known to the Allen School community, as she majors in both computer science and informatics. Her research spans software engineering, human-computer interaction, and applied machine learning. Liang earned an Honorable Mention from the CRA for her work with iSchool professor and Allen School adjunct professor Amy Ko in the Code & Cognition Lab. 

Liang collaborated with Ko and partners at George Mason University on the development of HowTooDev, a searchable knowledge base of strategies for solving hard programming problems that has the potential to transform programmer productivity and reshape computer science education. Liang’s contributions to the project included the development and testing of multiple search-interface prototypes and a classification ​system for​ various programming activities. She combined the latter with semantic text search that leverages natural-language strategy notations to build the front and back end of a robust strategy search engine.

“Jenny is a force,” said Ko. “She is the kind of force that we don’t find often in academia — the kind that pushes the boundaries of our knowledge, and leads.”

Previously, Liang worked with postdoc Spencer Sevilla and professor Kurtis Heimerl in the Allen School’s Information and Communication Technology for Development (ICTD) Lab on the development of LTE-based community-owned networks to connect rural and developing communities to the internet. Liang co-authored a paper presenting the team’s solution, dubbed CoLTE, that appeared at the 25th International Conference on Mobile Computing and Networking (Mobicom 2019). The recipient of the 2020 Allen AI Outstanding Engineer Scholarship for Women and Underrepresented Minorities, since last summer Liang has been an intern with AI2’s MOSAIC team working on toxic language classification. She previously was honored with the Allen School’s Undergraduate Service Award and recognized as a member of the 2020 class of the Husky 100.

Emily Bascom (Honorable Mention – iSchool)

Informatics major Emily Bascom earned an Honorable Mention from CRA for her work on user privacy and information technology tools for improving patient outcomes. Bascom, who is pursuing a concentration in human-computer interaction, spent two years working with iSchool professor Alexis Hiniker in the User Empowerment Lab. There, she focused on a project examining the privacy risks associated with ubiquitous audio recording capabilities of smartphones, smart speakers, and other devices. Her contributions included helping to design the study protocol, leading design workshops with study participants, and analyzing data generated by design sessions and interviews. 

“It was apparent throughout the project that Emily is a very talented scholar with an exciting career ahead of her,” said Hiniker, who is also an adjunct professor in the Allen School. “Her design insights and intellectual contributions far exceeded my expectations, and I can’t wait to see her translate those kinds of contributions into social change in the future.”

Bascom also collaborated with iSchool professor Wanda Pratt in the iMed research group on a project to understand how best to support patients and caregivers in acting as safeguards for hospital care — including improving communication between providers and patients to reduce medical errors. The researchers developed a tool, NURI, that enables patients and caregivers to record audio and semi-automatically transcribe their interactions with physicians and help them to understand the information they were given during those interactions. Bascom’s contributions included qualitative analysis of the user studies and preparation of the manuscript detailing the team’s findings and related work. She subsequently contributed to all aspects of a project led by Dr. Ari Pollack of UW Medicine and Seattle Children’s Hospital to develop tools to support pediatric kidney transplant patients, including protocol development, qualitative data analysis, and manuscript preparation.

Read more about Liang’s and Bascom’s work in a related iSchool story here.

Congratulations to these five exceptional researchers on their achievements!

Allen School alumnus Aditya Vashistha (Ph.D., ‘19) received the 2020-2021 WAGS/ProQuest Innovation in Technology Award from the Western Association of Graduate Schools for his doctoral dissertation on “Social Computing for Social Good in Low-Resource Environments.” The award recognizes a graduate thesis or dissertation that presents an innovative technology with a creative solution to a significant problem.

Vashistha, now a computing and information science professor at Cornell University, completed his Ph.D. working with professor Richard Anderson in the Information & Communication Technology for Development (ICTD) Lab on social computing technologies for underserved communities in low-resource environments. His dissertation focused on designing, building and evaluating new computer technologies to include people who are often excluded from social computing platforms because they are too poor to afford smartphones, too remote to access the internet or too low-literate to navigate all the text on the internet. 

“Being born and raised in India where illiteracy, poverty, and social ills were just a stone’s throw away, I witnessed how non-reading people, low-income women, and people with disabilities struggled with digital inequity and social injustice,” Vashistha said. “These experiences have profoundly motivated me to build computing technologies that include people of all backgrounds in the information revolution, particularly marginalized communities who are often neglected by the designers and builders of mainstream technologies.”

The first step in his journey began while working with Bill Thies of Microsoft India. Vashistha created IVR Junction, a system that uses interactive voice response (IVR) technology to enable people with basic phones to participate in voice-based social networks. Deployed in remote regions of Somalia, Mali and India, IVR Junction gave communities a way to share news, call attention to rights violations and report lack of services. Given the explosive growth of these systems, Vashistha then focused on Sangeet Swara, an interactive voice forum that enables people in these rural areas to moderate and manage the content generated in local languages which are yet unsupported by advances in natural language processing. The paper presenting Sangeet Swara earned a Best Paper Award at the 2015 ACM Conference on Human Factors in Computing Systems. Vashistha subsequently earned an Honorable Mention at the CHI 2017 for his work on Respeak, a voice-based speech transcription that relies on crowd-sourcing and speech recognition to transcribe audio files while providing additional earning opportunities to low-literate people without access to smartphones and internet connectivity. 

In addition to building voice forums for all people, Vashistha studied how technology amplifies existing sociocultural norms and values in society, including its strengths, shortcomings and biases. For example, he found that while Sangeet Swara transformed the lives of low-income blind people in rural regions, it also exposed that women who were marginalized due to patriarchy-driven abuse and hate speech. His dissertation advances the discourse on the benefits and pitfalls of social computing, highlights new challenges and big frontiers in building social good applications in low-resource environments, and offers solutions to make computing technologies more diverse, inclusive, and impactful. 

In addition to the WAGS/ProQuest recognition, Vashsitha received multiple accolades for his work, including the Allen School’s William Chan Memorial Dissertation Award in 2019, a 2020 Google Faculty Research Award  to combat online harassment of marginalized women using human-centered artificial intelligence approaches, the UW College of Engineering’s 2017 Graduate Student Research Award, a Facebook Graduate Fellowship in 2016, and a Best Student Paper award at ASSETS 2015 for his analysis of social media use by low-income blind people in India. 

Previous recipients with an Allen School connection include Vamsi Talla in 2017, advised by professor Shyam Gollakota, Sidhant Gupta in 2015, advised by professor Shwetak Patel and Tapan Parikh in 2009, advised by professor Ed Lazowska and emeritus professor David Notkin.

Congratulations, Aditya! 

The University of Washington’s Undergraduate Research Program has recognized six Allen School students for excelling in their area of research. Skyler Hallinan, Raida Karim and Ximing Lu were selected as Levinson Emerging Scholars and Jerry Cao, Jakub Filipek and Millicent Li were named Washington Research Foundation (WRF) Fellows.

Both the Levinson Scholars, funded by Art (UW ‘72) and Rita Levinson, and the WRF Fellows are chosen for their innovative research in bioscience and other related fields. The scholarships will enable them to pursue their projects while continuing to be supported by their mentors and lab colleagues. 

Hallinan is a senior majoring in computer science, bioengineering, and applied and computational mathematical sciences. He works with bioengineering professor Paul Yager.

Hallinan’s research focuses on combating chronic kidney disease (CKD), a disease that affects millions globally. People with CKD often accumulate indoxyl sulfate, a uremic toxin normally filtered out by healthy kidneys, which can cause major illness and lead to death. Hallinan is working to develop a method to effectively remove indoxyl sulfate from CKD patients’ blood via an orally ingestible hydrogel. He is currently modeling, prototyping and testing different candidate hydrogels to engineer a substitute for a functional kidney.

Karim is a senior studying computer science, working with Allen School professor Maya Cakmak in the Human-Centered Robotics Lab. 

Karim’s research focuses on measuring stress levels in teens and creating therapeutic, intervening techniques for them with a social robot called EMAR (Ecological Momentary Assessment Robot). Using EMAR, Karim collects stress level measurement data from high school students, and offers interventions that draw from dialectic behavioral therapy and acceptance and commitment therapy. Through this robot interaction experience, Karim aims to help teens be more mindful and present in the moment while developing a robot with a heightened level of sensitivity.

Lu is a computer science and statistics major working with professor Linda Shapiro, who holds a joint appointment in the Allen School and Department of Electrical & Computer Engineering, and with Allen School professor Yejin Choi as a research intern at the Allen Institute for AI. 

Lu’s current research with Choi centers around natural language processing and commonsense reasoning. With Shapiro, she focuses on creating a computer-aided biopsy classification system to reduce cancer diagnosis uncertainties. The system, HATNet: An End-to-End Holistic Attention Network for Diagnosis of Cancer Biopsy Images, streamlines the histopathological image classification pipeline and shows how to learn representations from gigapixel size images end-to-end. HATNet can learn representations from clinically relevant tissue structures and match the classification accuracy of pathologists.

Cao is a junior studying computer science and is in the University’s Honors Program. He works with Allen School professors Jennifer Mankoff in the Make4all Lab and Shwetak Patel in the Ubiquitous Computing Lab. 

Cao is currently working on the development of a non-invasive, continuous blood pressure monitor to help patients with cardiovascular conditions. In particular, his work aims to help people with postural orthostatic tachycardia syndrome (POTS), a condition where the body cannot properly regulate blood vessels that causes lightheadedness, fainting, and spikes in heart rate. The device will collect the pulse transit time (PTT), the time for a pulse wave to travel between two points. PTT correlates with blood pressure, which is a known predictor of adverse POTS symptoms. In addition to helping predict symptoms, the device will also provide physicians with a dataset that will make conditions like POTS easier to diagnose.

Filipek is a senior computer science major enrolled in the Allen School’s B.S./M.S. program. He works with physics professor Shih-Chieh Hsu.

Filipek’s research is in quantum machine learning (QML). Quantum computers have the potential to produce results in simple artificial intelligence algorithms to sophisticated neural networks better than their classical counterparts. However, these models are dealing with the bottleneck issue of a limited number of quidbits (the basic unit of quantum information) in near-term quantum devices. FIlipek is working on a hybrid neural network that functions by sandwiching any QML algorithm between two classical neural networks. This allows for automatic scaling of quantum algorithms to inputs and outputs of any size, addressing the bottleneck issue while provisioning an easy way of comparing classical algorithms to quantum ones.

Li is a senior in computer science working with Patel in the Ubiquitous Computing Lab. 

Li’s research is in improving the communication abilities of people with speech impairments by developing a silent speech interface that can facilitate communication between two people or with smart devices by outputting speech that is imagined but not spoken. Li aims to use a combination of neural signals from the brain to sense the user’s intent in their day-to-day lives and to provide speech accordingly, without the need of facial movement, by using optical signals like functional near-infrared spectroscopy and electrical signals like electroencephalogram.

Congratulations to these outstanding Allen School scholars who are making an impact through undergraduate research! 

Our latest undergraduate student spotlight features Sammamish, Washington native Joe Spaniac, a second year computer science major admitted under the Allen School’s expanded Direct to Major admission pathway who is also majoring in drama. During his first quarter at the Allen School, he knew he wanted to be a part of Q++, a student organization for LGBTQIA+ members studying in the Allen School. He has served as a member of the board since then and currently co-chairs the group with Lavinia Dunagan. Despite the challenges of meeting and holding events virtually this year, Spaniac enjoys being a part of the group and all that it does for students in the Allen School.

Allen School: What is the mission of Q++?

Joe Spaniac: Overall, our mission with Q++ is to raise awareness of the issues that LGBTQIA+ members face in the field of computer science and provide a place that anyone and everyone can feel welcome and supported regardless of gender expression or sexual orientation. This year especially, we are really focused on the second half of that mission statement due to the many social challenges that come with, seemingly, an entirely online school year. Very frequently, members of the community really look forward to the typical “college experience” as an opportunity to reinvent themselves and sometimes even escape households that might not accept them as their true selves. Although Q++ may not be able to entirely replace this experience, we hope to be a good intermediate option while we all wait for things to return to normal.

Allen School: Why do you think it’s important to have an organization like Q++ at the Allen School?

J.S.: Speaking from personal experience, the UW is massive, and it’s very easy to feel lost and isolated without supportive friends. For this reason, social organizations like Q++ are vital in providing students, especially ones that might struggle to immediately befriend a group of peers, a community. Additionally, Q++ has the potential to amplify and empower the voices of many LGBTQIA+ students within the Allen School itself. As a group, we can help advocate for meaningful changes that have the potential to better the student experience in the school for years to come. Together, there’s the added benefit that if someone is struggling, someone else in Q++ has likely faced a similar issue before. In this way, we can help guide each other through many of the common challenges we might experience at the UW.

Allen School: Why did you choose to major in computer science?

J.S.: I took my first computer science class my junior year of high school and from then on, I was hooked. When I was programming, however cliché it might sound, things seemed to click. I found that I wanted to continue coding after school, and sometimes looked forward to the homework assignments. Having never experienced this level of enthusiasm for any of my other studies, I thought that it had to mean something, so I applied to the Allen School. Now, having more knowledge about the field as a whole, I’ve decided to continue studying computer science because of how widespread software is. These days it seems like everything relies on computers and programming, meaning the opportunity is there to contribute to something that positively impacts the lives of people all around the world. I truly hope that by pursuing an education in computer science, I’ll get one of those opportunities to make a meaningful mark.

Allen School: What do you find most enjoyable about being an Allen School student? 

J.S.: Personally, although there’s a whole lot that I’ve really enjoyed about being an Allen School student so far, I think that the overall community I’ve interacted with has really been the highlight. From Discord study groups to my grading parties with my fellow teaching assistants, everyone I’ve met has been open, welcoming, and more than willing to help you out if you get stuck on a tricky 311 proof. At the same time, the wealth of opportunities available at the Allen School makes being a student here all the better. As I mentioned above, I’m currently an undergraduate TA but I know people who are exploring research opportunities, participating in hack-a-thons, and contributing to codebases or working on personal projects. No matter what, there is always something to do at the Allen School if you take the time to look for it!

Another great thing about the school is that although I am studying computer science, the flexibility of the major has also allowed me to explore another of my passions: theatre. Even though there are many challenges with being a double degree student, I’m extremely grateful that I’ve been able to study two wildly different fields while at the UW.

Allen School: Who or what inspires you in the Allen School?

J.S.: This year especially, I think the resilience and perseverance of everyone in the Allen School is extremely inspiring. Sure, there have been issues and we’ve all faltered at some point, but the fact that everyone has worked through all the adversity that has surrounded this quarter is insanely impressive. Both the faculty and student body deserve some praise for how many adaptations have been made to make this quarter as normal as possible.

I also mentioned them earlier, but the TA community has been another force of inspiration throughout my time here in the Allen School. I’ve met countless peers who all share my passion for educating and spreading our knowledge of programming to fellow undergraduates, many of whom have no prior coding experience. The fact that so many within the Allen School are more than happy to share their understanding and unique perspective of computer science never ceases to amaze me and keeps me coming back quarter after quarter.

Thank you for your leadership and for supporting your fellow Allen School students in and out of the virtual classroom, Joe! 

As a student at Seattle’s Hamilton International Middle School, Leo Maddox Schneider demonstrated early mastery of mathematics and languages, was an avid gamer and athlete, and carved out a reputation as a budding conservationist. Enthusiastic about learning from an early age, Leo had already taken to heart his mother Sylvia Bolton’s advice to find something that he loved and was passionate about and to make that his profession. As she relayed to her son at the time, “it will bring fulfillment and a lot of happiness.”

What Leo loved was computer coding and Lego design; what he was passionate about were environmental causes. He might have pursued both at the University of Washington if not for the injuries he sustained in an automobile accident. Four and a half months later, on January 12, 2019, Leo passed away from those injuries and related complications. Nearly two years after that tragic loss, the foundation established by Leo’s mother to honor her son’s memory will give Allen School students the opportunity to fulfill their own dreams and carry on his legacy through the Leo Maddox Foundation Scholarship in Computer Science & Engineering.

”Leo loved computer science,” Bolton explained. “He and his friend Lennox shared a dream of attending a university that excelled in computer science so they could build their own company and make a difference in the world.”

Even at the tender age of 13, Leo was already well on his way toward making that difference. He forged enduring friendships with Lennox and Judson while playing Minecraft and Fortnite, which helped spark his interest in coding. He was already three years ahead of his grade level in mathematics and conversant in both Spanish and Bulgarian. His enthusiasm for the outdoors led Leo to champion environmental causes; he once convinced his mother to enter into one of their “non-negotiable” agreements permitting him to collect garbage for recycling. (Another of their non-negotiable agreements stipulated that he would eat his vegetables at dinner.) Leo was particularly passionate about the ocean, learning to swim with dolphins and developing a love of boat building craftsmanship inspired in part by his mother’s work as a luxury yacht designer.

“Everyone knew Leo as having a big, sweet soul and people just loved him. Losing him turned our world upside down into complete darkness,” recalled Bolton. “But we do not want the tragedy of Leo’s passing to define him. Leo was and will always be remembered as the smart, kind and compassionate kid who was gifted at math and science, loved the outdoors, and was a friend to many. With so much life ahead of him.”

To that end, Bolton established the Leo Maddox Foundation as a way to ensure that Leo’s legacy and aspirations for the future would live on in others. The Foundation supports a variety of initiatives designed to help promising young students with financial need to fully achieve their academic and creative potential, from assisting Rainier Scholars to go to college, to “Love, Leo” genius grants inspired by their namesake’s creative, can-do approach to solving problems he saw in the world. The new Leo Maddox Foundation Scholarship in Computer Science & Engineering will support Allen School undergraduate students in covering the cost of tuition and other educational expenses based on academic merit and financial need.

“We are heartbroken that Leo will never get the chance to apply to the Allen School and our hearts and prayers are with his family. We are deeply appreciative of the scholarship established by the Foundation in his name,” said professor Magdalena Balazinska, director of the Allen School. “This scholarship will touch many lives. It will promote the success of many talented students who need support to fulfill their dreams.”

In deference to her son’s twin loves, in addition to the Allen School scholarship Bolton also created the Leo Maddox Foundation Scholarship in Oceanography to support students in the School of Oceanography engaged in climate-related studies. The university’s preeminence in both disciplines and focus on student support convinced the Foundation to entrust it with Leo’s memory.

“As important as it is for the Leo Maddox Foundation to support young adults, it is equally important that we do so with the leaders in both fields,” said Vivian Ho, creator of the Leo Maddox Foundation. “In conducting our due diligence, it was clear that the University of Washington had a lot to offer in both areas of study and in shaping support for student scholarships. They created the perfect vehicles for our founder, Sylvia Bolton, to make the impactful difference she was seeking for Leo’s legacy.”

Learn more about the Leo Maddox Foundation Scholarships here and Leo’s life and legacy here.

Last spring, Newport High School student Sophia Lin of Bellevue, Washington, was eager to start coding. Having applied to several summer coding programs, she was ready to learn. Unfortunately, as the pandemic spread, they were all canceled. Seeing her younger sister’s disappointment, Allen School senior Elizabeth Lin decided to be her coding teacher. With the help of her twin sister Christin, a senior in the University of Washington’s Department of Electrical & Computer Engineering, the two created a virtual summer coding program. Then they invited 300 kids from around the world to join them, free of charge

“As a teaching assistant in the Allen School’s Intro to Programming course, I had already taught hundreds of students how to code so I knew I could teach her, too,” said Elizabeth. “But knowing I had a busy summer with an internship at Microsoft as a software engineer, I decided to record the lessons. That way, I could teach others, too.”

The two sisters reached out to their contacts from area high schools and advertised on Facebook to recruit 6th through 12th grade students interested in their new initiative, STEM League Developer Program, an offshoot from an outreach program the two started in high school. 

“As big advocates for STEM education, we previously co-founded STEM League, an organization we started in high school with the mission to promote STEM opportunity and awareness to students in our local community,” said Christin. “Through STEM League, we hosted a series of STEM outreach programs at local libraries, facilitated Seattle STEM company tours, and volunteered for STEM events at local elementary schools. We wanted to continue that mission, therefore we launched the STEM League Developer Program.”

And within two weeks of recruiting students, the Lins had students from around the world signed up for the program with sponsorship from T-Mobile.

“The exponential interest in the Seattle area and around the world amazed us, but also brought some concerns,” said Elizabeth. “We worried about making sure every student had a quality experience. Learning to code can be daunting by yourself, and we wanted to ensure each student had support throughout the 8 weeks.”

The Lins once again returned to social media to recruit mentors — experienced high school and college programmers willing to work with the students. In about a month, the two screened and interviewed 50 mentors. By the end of June, they enrolled 300 students. Each mentor was assigned seven students to help, in various time zones across the world.

On June 29, they launched the program. Elizabeth said that Allen School courses such as CSE 490A: Entrepreneurship led by Allen School professor Ed Lazowska and Greg Gottesman, managing director and co-founder of Pioneer Square Labs, and CSE: 160 Data Programming with Allen School professor Ruth Anderson, inspired the structure of the 8-week coding program. 

“Greg and I are thrilled when our interdisciplinary, hands-on entrepreneurship course contributes in at least some small way to giving students the inspiration and courage to head out and do remarkable things,” said Lazowska. “And what Elizabeth and her team have accomplished is truly remarkable.”

From Harry Potter-themed coding competitions, to a two-week program-wide hackathon where students created their own projects, to using Discord so everyone could stay in touch easily, the Lins and their collaborators found new and exciting ways to engage students and make coding more relevant to them. 

“I thought the STEM League Developer Program was really fun, and it was definitely the highlight of my summer. I learned many important coding skills that I can now leverage for my future,” said one 10th grade participant. “Not only did I grow as a developer, I grew as a person, as I learned life skills like tenacity, motivation, and perseverance. The Discord community and the mentors were very supportive of everyone’s needs and helped create a positive learning environment where everyone could grow.”

Parents noticed how much their kids were enjoying the program. Several wrote in to thank the Lins and the mentors for teaching their kids during a particularly hard summer.

“My kids have been doing some other coding classes on a regular basis. But what they have learned at STEM League is of far better quality and better retained than any of those outside classes,” noted the parent of a student in Bellevue. “On top of it all they are having a lot of fun and they look forward to the next lesson. I don’t even have to keep nagging them to do it.”

According to Christin, the twins always wanted to host coding classes for students. They intend to continue the program next summer after they graduate.

Both sisters and five of the mentors in the program are alumni of the UW College of Engineering’s  Washington State Academic RedShirt (STARS) program, a two-year specialized curriculum designed to build learning skills and strengthen academic preparation for core math and science prerequisites.

“This coding program idea sparked back in 2018, but we never found the time to do it,” she said. “However, this summer, we decided to launch it because we are both interested in entrepreneurship and experiencing something created by ourselves from scratch.” 

Visit the STEM League Developer Program website to learn more and access the free curriculum. 

While the internet is so critical for employment, education and communication, millions of Americans in rural and urban areas still do not have access to affordable connections. This lack of access further contributes to digital and economic inequality, especially during a pandemic when many schools and jobs have been moved online. A team of University of Washington researchers, led by professor Kurtis Heimerl of the Allen School’s Information & Communications Technology for Development Lab and his graduate student, Esther Jang are helping to address this problem right here in Washington.

With support from the Public Interest Technology University Network (PIT-UN), the group which includes UW Tacoma urban studies professor Emma Slager, and Jason Young, a senior research scientist in the iSchool, is deploying networks that will bring new, inexpensive, community-owned connectivity to marginalized communities in Seattle and Tacoma. The UW is one of 25 universities to receive a PIT-UN grant, which was created to fund critical research and build an inclusive career pipeline to advance the field of public interest technology. 

Community cellular networks are owned and operated by the community they serve with the help of public and local organizations such as schools, nonprofits, community centers, makerspaces, libraries, small businesses, and tiny house villages. Leveraging their expertise in working with community cellular networks internationally, the team deployed a new network with a local connectivity non-profit, the Tacoma Community Network (TCN), to bring inexpensive, community-owned connectivity to the Hilltop community in the third largest city in Washington. 

Bee Ivey, TCN’s volunteer executive director, said that as a nonprofit cooperative, TCN can focus on great speed and service without worrying about pleasing shareholders. To be sustainable, TCN needs about 20-25 members per “gateway,” which is an access point from which internet connectivity is distributed to the community. The partnership with the UW will empower TCN to connect more individuals at a lower price, which in turn allows them to get even more low income and extremely low income households online. Although the project is focused on making digital connections, it turns out that a personal touch was critical to making it work.

“We did a lot of canvassing prior to the pandemic, which allowed us to really connect with residents of Hilltop. One of the great things about being face-to-face is that you get to know people and hear their stories about how the internet affects their lives,” said Ivey. “It was truly eye-opening for us to meet so many people who didn’t have internet and had no way to access it, and definitely brought the statistics and research we’d seen to life. 

Up to a quarter of all urban residents don’t have internet access, according to some studies, Ivey said, and it was made very clear the ways in which people are held back from living full, productive and satisfying lives when they lack internet access. Hearing these stories definitely strengthened the team’s resolve to continue the work connecting everyone to the internet. Now with the pandemic, they are focused on social media and networking, along with mailers to help reach more people who need internet access.

In Tacoma, one LTE network deployed in November contains eight households and is growing. Althea, the software company that makes mesh networking technology in which TCN’s routers use blockchain-based micro-payments to pay each other for traffic forwarding is supporting the project.  It has set up community wireless mesh networks all over the world, and is interested in integrating with LTE. 

“Although we had a modest start, it represents a 30 percent adoption rate among the houses we were immediately able to reach,” said Ivey. “With the University of Washington’s help, we will be able to expand the number of households within our reach, as well as offer different types of internet connections — both typical wireless ISP equipment and LTE, the same data network used by cell phones. While there are many fantastic community-based internet networks out there, this particular type has never been deployed before in the United States to my knowledge, and it will make it far easier for individuals to access the internet.”

UW spun out the Local Connectivity Lab to deploy the LTE networking technology, powered by open-source software and operating in the Citizen’s Band Radio Service frequency spectrum, which is open enough to allow unlicensed devices to transmit in much of Seattle and Tacoma. This will allow the researchers to run open-sourced cellular networks in the U.S. on a small community scale. 

“Cellular networks, with their higher-power access points, more favorable spectrum, and more efficient waveforms, have a much wider coverage area and user capacity than typical WiFi networks, and are also designed for user mobility like cell phones,” said Esther Jang, an Allen School graduate student leading the project. “Some initial line-of-sight link performance tests from our test deployment at UW yielded 60 megabits per second down and eight Mbps up with Consumer Premises Equipment, like a stationary user device with our SIM card at 1.3 miles away, from a backhaul connection around 150 Mbps.”

The team’s work is a part of the ICTD Lab’s goal to eventually create a cellular network that will allow people or organizations to deploy their own networks as easily as they do WiFi routers, where each network can come together to provide mutual roaming, which they call “cooperative cellular.” They are currently looking for non-profit organizations to help launch in Seattle, in addition to a King County Equity and Social Justice they recently received.

In addition to creating this open-sourced software and deploying it in communities that need it most, the group will also develop a STEM course called Community Networking. The course will give students an opportunity to explore the research, development and practice of access-related PIT and the partners and communities that are demonstrating an alternative viable path for a career in technology. 

“Most of us take Internet access for granted to the point that, when the internet goes down, we struggle with continuing to get our work done,” said Allen School Director Magdalena Balazinska. “Yet some people, here in the United States, do not have such access. As computer scientists, we should always strive to solve important societal and world problems. I’m very excited about the way this project is using computer science to have a profound, positive impact on society.”

Learn more about the project on the Information School’s website.

Deep learning has been immensely successful in recent years, spawning a lot of hope and generating a lot of hype, but no one has really understood why it works. The prevailing wisdom has been that deep learning is capable of discovering new representations of the data, rather than relying on hand-coded features like other learning algorithms do. But because deep networks are black boxes — what Allen School professor emeritus Pedro Domingos describes as “an opaque mess of connections and weights” — how that discovery actually happens is anyone’s guess.

Until now, that is. In a new paper posted on the preprint repository arXiv, Domingos gives us a peek inside that black box and reveals what is — and just as importantly, what isn’t — going on inside. Read on for a Q&A with Domingos on his latest findings, what they mean for our understanding of how deep learning actually works, and the implications for researchers’ quest for a “master algorithm” to unify all of machine learning.

You lifted the lid off the so-called black box of deep networks, and what did you find?

Pedro Domingos: In short, I found that deep networks are not as unintelligible as we thought, but neither are they as revolutionary as we thought. Deep networks are learned by the backpropagation algorithm, an efficient implementation for neural networks of the general gradient descent algorithm that repeatedly tweaks the network’s weights to make its output for each training input better match the true output. That process helps the model learn to label an image of a dog as a dog, and not as a cat or as a chair, for instance. This paper shows that all gradient descent does is memorize the training examples, and then make predictions about new examples by comparing them with the training ones. This is actually a very old and simple type of learning, called similarity-based learning, that goes back to the 1950s. It was a bit of a shock to discover that, more than half a century later, that’s all that is going on in deep learning!

Deep learning has been the subject of a lot of hype. How do you think your colleagues will respond to these findings?

PD: Critics of deep learning, of which there are many, may see these results as showing that deep learning has been greatly oversold. After all, what it does is, at heart, not very different from what 50-year-old algorithms do — and that’s hardly a recipe for solving AI! The whole idea that deep learning discovers new representations of the data, rather than relying on hand-coded features like previous methods, now looks somewhat questionable — even though it has been deep learning’s main selling point.

Conversely, some researchers and fans of deep learning may be reluctant to accept this result, or at least some of its consequences, because it goes against some of their deepest beliefs (no pun intended). But a theorem is a theorem. In any case, my goal was not to criticize deep learning, which I’ve been working in since before it became popular, but to understand it better. I think that, ultimately, this greater understanding will be very beneficial for both research and applications in this area. So my hope is that deep learning fans will embrace these results.

So it’s a good news/bad news scenario for the field? 

PD: That’s right. In “The Master Algorithm,” I explain that when a new technology is as pervasive and game-changing as machine learning has become, it’s not wise to let it remain a black box. Whether you’re a consumer influenced by recommendation algorithms on Amazon, or a computer scientist building the latest machine learning model, you can’t control what you don’t understand. Knowing how deep networks learn gives us that greater measure of control.

So, the good news is that it is now going to be much easier for us to understand what a deep network is doing. Among other things, the fact that deep networks are just similarity-based algorithms finally helps to explain their brittleness, whereby changing an example just slightly can cause the network to make absurd predictions. Up until now, it has puzzled us why a minor tweak would, for example, lead a deep network to suddenly start labeling a car as an ostrich. If you’re training a model for a self-driving car, you probably don’t want to hit either, but for multiple reasons — not least, the predictability of what an oncoming car might do compared to an oncoming ostrich — I would like the vehicle I’m riding in to be able to tell the difference. 

But these findings could be considered bad news in the sense that it’s clear there is not much representation learning going on inside these networks, and certainly not as much as we hoped or even assumed. How to do that remains a largely unsolved problem for our field.

If they are essentially doing 1950s-style learning, why would we continue to use deep networks?

PD: Compared to previous similarity-based algorithms such as kernel machines, which were the dominant approach prior to the emergence of deep learning, deep networks have a number of important advantages. 

One is that they allow incorporating bits of knowledge of the target function into the similarity measure — the kernel — via the network architecture. This is advantageous because the more knowledge you incorporate, the faster and better you can learn. This is a consequence of what we call the “no free lunch” theorem in machine learning: if you have no a priori knowledge, you can’t learn anything from data besides memorizing it. For example, convolutional neural networks, which launched the deep learning revolution by achieving unprecedented accuracy on image recognition problems, differ from “plain vanilla” neural networks in that they incorporate the knowledge that objects are the same no matter where in the image they appear. This is how humans learn, by building on the knowledge they already have. If you know how to read, then you can learn about science much faster by reading textbooks than by rediscovering physics and biology from scratch.

Another advantage to deep networks is that they can bring distant examples together into the same region, which makes learning more complex functions easier. And through superposition, they’re much more efficient at storing and matching examples than other similarity-based approaches.

Can you describe superposition for those of us who are not machine learning experts?

PD: Yes, but we’ll have to do some math. The weights produced by backpropagation contain a superposition of the training examples. That is, the examples are mapped into the space of variations of the function being learned and then added up. As a simple analogy, if you want to compute 3 x 5 + 3 x 7 + 3 x 9, it would be more efficient to instead compute 3 x ( 5 + 7 + 9) = 3 x 21. The 5, 7 and 9 are now “superposed” in the 21, but the result is still the same as if you separately multiplied each by 3 and then added the results.

The practical result is that deep networks are able to speed up learning and inference, making them more efficient, while reducing the amount of computer memory needed to store the examples. For instance, if you have a million images, each with a million pixels, you would need on the order of terabytes to store them. But with superposition, you only need an amount of storage on the order of the number of weights in the network, which is typically much smaller. And then, if you want to predict what a new image contains, such as a cat, you need to cycle through all of those training images and compare them with the new one. That can take a long time. With superposition, you just have to pass the image through the network once. That takes much less time to execute. It’s the same with answering questions based on text; without superposition, you’d have to store and look through the corpus, instead of a compact summary of it.

So your findings will help to improve deep learning models?

PD: That’s the idea. Now that we understand what is happening when the aforementioned car suddenly becomes an ostrich, we should be able to account for that brittleness in the models. If we think of a learned model as a piece of cheese and the failure regions as holes in that cheese, we now understand better where those holes are, and what their shape and size is. Using this knowledge, we can actively figure out where we need new data or adjustments to the model to fix the holes. We should also improve our ability to defend against attacks that cause deep networks to misclassify images by tweaking some pixels such that they cause the network to fall into one of those holes. An example would be attempts to fool self-driving cars into misrecognizing traffic signs.

What are the implications of your latest results in the search for the master algorithm?

PD: These findings represent a big step forward in unifying the five major machine learning paradigms I described in my book, which is our best hope for arriving at that universal learner, what I call the “master algorithm.” We now know that all learning algorithms based on gradient descent — including but not limited to deep networks — are similarity-based learners. This fact serves to unify three of the five paradigms: neural, probabilistic, and similarity-based learning. Tantalizingly, it may also be extensible to the remaining two, symbolic and genetic learning.

Given your findings, what’s next for deep learning? Where does the field go from here?

PD: I think deep learning researchers have become too reliant on backpropagation as the near-universal learning algorithm. Now that we know how limited backprop is in terms of the representations it can discover, we need to look for better learning algorithms! I’ve done some work in this direction, using combinatorial optimization to learn deep networks. We can also take inspiration from other fields, such as neuroscience, psychology, and evolutionary biology. Or, if we decide that representation learning is not so important after all — which would be a 180-degree change — we can look for other algorithms that can form superpositions of the examples and that are compact and generalize well.

Now that we know better, we can do better.

Read the paper on arXiv here.