Allen School News

The Allen School has a proud tradition of nurturing undergraduate student researchers whose work has the potential for real-world impact. This year, three of those students — Jerry Cao, Mike He and Yu Xin — earned honorable mentions from the Computing Research Association (CRA) as part of its 2022 Outstanding Undergraduate Researcher Awards competition for their contributions in health sensing and fabrication, programming languages and machine learning, and building robust computer systems.

Jerry Cao

The CRA recognized senior Jerry Cao, who is majoring in computer science and applied mathematics, for his research in health sensing and fabrication. Advised by professors Jennifer Mankoff and Shwetak Patel, his work aims to apply computing and fabrication to improve individuals’ quality of life. To reduce the burden of health monitoring and make it easier for users to prototype custom tools that fit their personalized needs, Cao is creating a wearable device in compression sleeve-form for the leg that records changes in the blood volume in the body’s superficial tissue. This can help predict the onset of adverse symptoms throughout the day for conditions such as Postural Orthostatic Tachycardia Syndrome (POTS) where blood flow is improperly regulated throughout the body.

Cao is also working on a project to rapidly prototype physical objects. He aims to reduce the number of iterations — currently requiring several to reach the final product — by reconfiguring a model to support real-time iteration. He is developing a pipeline to take a parametric model and produce a reconfigurable prototype where each parameter can be adjusted up to a specified and allowed range. Users can more easily change the size of the physical model this way and record all the necessary measurements to fabricate a final version. For example, when building a cabinet, builders must ensure it fits in its designated space. The reconfigurable prototype will limit the iterations and allow users to explore different configurations of the object, then create the final version using actual materials.

Mike He

Mike He, a senior studying computer science, was acknowledged for his work in programming languages, formal verification, compilers and machine learning systems. Advised by professor Zachary Tatlock, He worked with the Allen School’s PLSE group on Dynamic Tensor Rematerialization (DTR), an algorithm that trains deep learning models under constrained memory budgets. Since deep learning models use up a lot of GPU memory while training, He and his colleagues created DTR to train these models under restricted memory budgets. DTR removes restrictions on classic compilers and when memory fills up, DTR evicts the oldest, stalest, cheapest-to-recompute tensor to make room for the next allocation. If the training loop later tries to access a previously evicted tensor, DTR recomputes it on demand by tracking operator dependencies. 

In addition to his contributions to DTR, He led the push to develop new flexible accelerator matching compiler techniques to easily target new hardware accelerators in deep learning frameworks. To do so, the team is enabling devices to be more easily incorporated into an existing DL framework and, in principle, for formal functional verification down to the hardware implementation. The project, 3LA, has a built-in pattern-matching algorithm that can find accelerator supported workloads in deep learning models using equality saturation. The project addressed the mapping gap between deep learning models represented in high-level domain-specific languages and specialized accelerators using instruction-level abstraction as the software-hardware interface.

Yu Xin 

Yu Xin, a senior studying computer science and applied and computational mathematical science, was honored by the CRA for his work with Allen School professor Arvind Krishnamurthy in building effective and robust computer systems. In particular, Xin worked to develop a scheduler for serving deep learning inference tasks. Applications using cloud-based deep learning models, when deployed on a large scale, tend to flood data center GPU clusters, slowing down the time it takes to respond and causing delays and extra expense. To help with the cost and speed, Xin and his collaborators created Symphony, a centralized dispatcher to satisfy requests within a latency bound, ensuring load-balance across GPUs and maximizing their efficiency by using appropriate dynamically-sized batches of inference requests. By loading dozens of deep learning models on each GPU, Symphony enables burst amortization across models and has the potential to eliminate the need for overprovisioning. Enabling multiple dispatchers for better scalability, Xin designed an algorithm to partition the model space into many disjoint subsets in which each dispatcher handles one of the models. The algorithm finds the partitioning scheme that minimizes the deviation between partitions in terms of total request rates and model sizes by generating and solving a Mixed Integer Linear Programming (MILP) problem.

Xin’s previous work includes developing tools for analyzing images of proteins generated from a cryo-electron microscope. For example, filtering out high-frequency noises by generating an artificial image based on a mathematical model and comparing it against every patch of the image to see if there is a match and then output all the matched results. This approach saves researchers time while increasing their effectiveness by directing their attention to the most relevant sites.

Congratulations to Jerry, Mike and Yu! 

Read more →

During their time at the Allen School, recent alumni Maarten Sap (Ph.D., ‘21) and Ivan Evtimov (Ph.D., ‘21) tackled some of the thorniest issues raised by emerging natural language processing and machine learning technologies — from endowing NLP systems with social intelligence while combating inequity and bias, to addressing security vulnerabilities in the convolutional neural networks that fuel state-of-the-art computer vision systems. Recently, the faculty honored both for their contributions with the William Chan Memorial Dissertation Award, which was named in memory of the late graduate student William Chan to recognize dissertations of exceptional merit. Evtimov earned additional recognition for his work from the Western Association of Graduate Schools and ProQuest as the recipient of the WAGS/ProQuest Innovation in Technology Award, which recognizes distinguished scholarly achievement at the master’s or doctoral level.

Sap — who is currently a postdoctoral/young investigator at the Allen Institute for AI (AI2) — worked with Allen School professors Yejin Choi and Noah Smith. His dissertation, “Positive AI with Social Commonsense Models,” advanced new techniques for making NLP systems more human-centric, socially aware and equity-driven.

“Maarten’s dissertation presents groundbreaking work advancing social commonsense reasoning and computational models serving equity and inclusion. More specifically, his work presents technical and conceptual innovations that make deep learning methods significantly more equitable,” said Choi and Smith, both of whom are also senior research managers at AI2. “Maarten’s research steers the field of NLP and its products toward a better future.”

One example is ATOMIC, a large-scale social commonsense knowledge graph Sap and collaborators created to help machines comprehend day-to-day practical reasoning about events, causes and effects. To create equity-driven NLP systems, he also helped develop PowerTransformer, a controllable text rewriting model that helps authors mitigate biases in their writing, particularly biases related to how the public describes people of different genders. Sap also tackled the problem of detecting biases and toxicity in language by identifying issues with the current hate speech detectors that lead to racial biases. His work introduced Social Bias Frames, a linguistic framework for explaining the biased or harmful implications in text. The papers supporting this, The Risk of Racial Bias in Hate Speech Detection and Social Bias Frames: Reasoning about Social and Power Implications of Language were nominated for a Best Short Paper Award by the Association for Computer Linguistics in 2019 and won the Best Paper Award at the West Coast NLP Summit in 2020, respectively. Sap was also a member of the team that won the first Amazon Alexa Prize for a conversational chatbot called Sounding Board that engages with users about current topics.

TechCrunch, Forbes, Fortune and Vox have all covered Sap’s research. After completing his postdoc with AI2’s MOSAIC team, he will join Carnegie Mellon University’s Language Technology Institute as a professor in the fall.

Evtimov’s dissertation, “Disrupting Machine Learning: Emerging Threats and Applications for Privacy and Dataset Ownership,” makes significant contributions to the security of adversarial machine learning. His research as a member of the Allen School’s Security & Privacy Research Lab focused particularly on the vulnerabilities of convolutional neural networks (CNN) that allow maliciously crafted inputs to affect both their inference and training. Evtimov said that understanding new technologies in terms of  security and privacy is important in order to think ahead of adversarial actors. 

“Ivan’s dissertation is highly innovative, and contributed significant results to the field of real-world attacks against computer vision algorithms. His work is of fundamental importance to the field,” Allen School professor and lab co-director Tadayoshi Kohno said. “Computer vision is everywhere — in autonomous cars, in computer authentication schemes, and more. Ivan’s dissertation helps the field develop secure computer vision systems and also provides foundations for helping users protect their privacy in the face of such systems.”

Evtimov’s work shows that the vulnerabilities of CNNs exhibit a duality when it comes to security and privacy. For example, he found the computer algorithms for cameras reading traffic signs in autonomous cars could be tricked by an object as simple as a sticker on a stop sign. The sticker could fool the cameras into reading the stop sign as a speed limit sign. In the case of autonomous driving, it is critical to identify anything that could be exploited by malicious parties in such a safety-critical setting. Machine learning, Evtimov found, can also be used in an unauthorized manner. Take, for example, a search engine for facial recognition. To protect privacy, Evtimov studied the conditions in which people could flood a database full of photos gathered from the public without permission with decoys. He proposed FoggySight, a tool that involves community users uploading modified photos — for instance, labeling photos of Madonna as photos of Queen Elizabeth  — to poison the facial search database and throw off searches in it. He also found ways to protect visual data released for human consumption from misuse through machine learning, including developing a protective mechanism that can be applied to the information contained in datasets before public release to prevent unauthorized parties from training their own models using the data. 

Evtimov’s research has been covered by Ars Technica, IEEE Spectrum and more. He previously won a Distinguished Paper Award at the Workshop on Adversarial Machine Learning in Real-World Computer Vision Systems and Online Challenges for his work examining the vulnerability of combined image and text models to adversarial threats. After graduating from the Allen School, Evtimov joined Meta as a research scientist. 

Congratulations to Maarten and Ivan! 

Read more →

Allen School professor emeritus Richard Ladner has been elected a Fellow of the American Association for the Advancement of Science (AAAS) for his “distinguished contributions to the inclusion of persons with disabilities in the computing fields.” One of 26 leading scientists in the organization’s Information, Computing & Communications section to attain the rank of Fellow this year, Ladner has devoted the past two decades to research and advocacy aimed at making computing education and careers more accessible while designing technologies that empower all users.

A mathematician by training, Ladner helped establish the University of Washington’s theoretical computer science group shortly after joining the faculty in the early 1970’s. At the time, Ladner’s interest in disability issues was personal, having been raised by two parents who were deaf. Later, after he completed an American Sign Language course at a local community college, Ladner began doing volunteer work with people who were deaf and blind as well as writing about accessibility issues. Having worked on several accessibility projects in the 1980s and 1990s, his first full-time foray into accessible technology development — an experience that would alter the course of his career in terms of both research and advocacy — would not come until 2002.

That year, Ladner met Sangyun Hahn, a graduate student who was blind. Hahn related to his new advisor his frustration at being unable to easily access certain content in his textbooks, such as mathematical formulas and diagrams. Their discussions led to the launch of the Tactile Graphics project to automate the conversion of textbook figures into an accessible format. A series of accessibility projects followed, including MobileASL, a collaboration between Ladner and UW Electrical & Computer Engineering professor Eve Riskin to enable people to communicate using American Sign Language via a mobile phone; WebAnywhere, a non-visual platform enabling people who are blind to navigate the web using any browser, on any device, with Jeffrey Bigham (Ph.D., ‘09); Perkinput, a Braille-based text entry system for touchscreen devices, with Shiri Azenkot (Ph.D., ‘14) and iSchool professor Jacob Wobbrock; and Blocks4All, an accessible, touchscreen-based blocks environment for children who are blind to learn programming, with Lauren Milne (Ph.D., ‘18). 

As a student with a disability, Hahn was a relative rarity in computer science graduate programs at the time he and Ladner met. When the latter turned his attention full-time from exploring the theoretical underpinnings of computing to making computing more accessible to all users, he recognized that one of the obstacles was the lack of pathways for more people with disabilities to pursue computer science and bring their perspectives to the development of new technologies. This led him to partner with Sheryl Burgstahler, director of UW’s DO-IT Center, to establish the Alliance for Access to Computing Careers, or AccessComputing, in 2006 with support from the National Science Foundation’s Broadening Participation in Computing program. AccessComputing helps high school, undergraduate and graduate students to build skills and connections with mentors and professional opportunities in the computing fields. So far, the program has directly served more than 2,400 students with disabilities through a range of activities, from academies and workshops to research and work-based internships.

Ladner subsequently teamed up with Andreas Stefik at the University of Nevada, Las Vegas to launch AccessCSforAll, an initiative aimed at providing accessible curriculum and resources to engage students with disabilities in K-12 computer science education. That work led Code.org and the Computer Science Teachers Association to name Ladner, Stefik and the Quorum programming team 2018 Computer Science Champions. A year later, Ladner and Stefik were again recognized — this time alongside collaborators William Allee and Sean Mealin — with a Best Paper Award from the Association for Computing Machinery’s Special Interest Group in Computer Science Education at the SIGCSE 2019 conference. In the winning paper, “Computer Science Principles for Teachers of Blind and Visually Impaired Students,” the team presented the results of its partnership with Code.org to review and revamp the Advanced Placement CSP curriculum and tools for accessibility.

In 2020, the National Science Board recognized Ladner with its Public Service Award for his exemplary science communication and diversity advocacy — the latest in a long line of previous accolades for his leadership on accessible technology and education that includes the Strache Leadership Award from the Center on Disabilities at University of California, Northridge, the Award for Outstanding Contributions to Computing and Accessibility from the ACM Special Interest Group on Accessible Computing (SIGACCESS), the Richard A. Tapia Achievement Award for Scientific Scholarship, Civic Science and Diversifying Computing from the Center for Minorities and People with Disabilities in Information Technology (CMD-IT), and more. Along the way, Ladner also earned the 2019 Harrold and Notkin Research and Graduate Mentoring Award — named in part for the late David Notkin, former chair of what was then known as the UW Department of Computer Science & Engineering — from the National Center for Women and Information Technology (NCWIT) for his long-standing efforts to advance gender diversity in computing.

Even after he officially attained emeritus status at the Allen School in 2017, Ladner remained active in research and mentoring students in addition to advocacy and program leadership. Over the course of his career, he has supervised or co-supervised 30 Ph.D. students and more than 100 undergraduate researchers — many of whom sought Ladner out for his focus on accessibility before that line of research entered the mainstream. Some of those same students later established the Richard E. Ladner Endowed Professorship, currently held by his faculty colleague Jennifer Mankoff, and the Richard Ladner Endowed Fund for Graduate Student Support in his honor.

Ladner also continues to build on his legacy of advocacy for engaging people with disabilities in technology development. The same year he was recognized for making the K-12 computer science curriculum more accessible, he helped establish the UW’s Center for Research and Education on Accessible Technology and Experiences (CREATE) alongside eight colleagues from multiple UW departments with an inaugural investment from Microsoft. Under the slogan “making technology accessible and making the world accessible through technology,” CREATE supports transformational, multidisciplinary research that will translate into real-world impact while building expertise in accessible technologies and increasing representation in the field for people with disabilities.

“Richard is truly a pioneer in the field of accessible computing,” said professor Magdalena Balazinska, director of the Allen School. “He understood the importance of fully including people with disabilities long before the rest of the field recognized this challenge and he continues to innovate today. He’s an inspiration to all of us.”

Ladner was previously elected a Fellow of the ACM and of the IEEE. He is one of four UW faculty members recognized in the 2021 class of AAAS Fellows, including Emily Carrington, who was honored in the Biological Sciences, and Julia A. Kovacs and Stefan Stoll, who were both honored in Chemistry. Founded in 1848, AAAS is the world’s largest general scientific society.

Learn more about the newly elected AAAS Fellows here.

Congratulations, Richard! Read more →

Mohit Shridhar, a Ph.D. student working with Allen School professor Dieter Fox, has been named a 2022-2023 NVIDIA Graduate Fellow for his research in building generalizable systems for human-robot collaboration. Shridhar’s work is focused on connecting language to perception and action for vision-based robotics.

Shridhar aims to use deep learning to connect abstract concepts to concrete physical actions with long-term reasoning to develop robot butlers. The Fellowship will help him continue his work in building robots that learn through embodied interactions rather than from static datasets. Using his own creation CLIPort, a language-conditioned imitation-learning agent, will advance precise spatial reasoning and learning generalizable semantic representations for vision and language. Shridhar’s framework combines two-streams with semantic and spatial pathways, where the semantic stream uses an internet pre-trained vision language model to bootstrap learning. This end-to-end framework can solve a variety of language-specified tabletop tasks, from packing unseen objects to folding clothes with centimeter-level precision.

“Mohit’s CLIPort work is the first to show the power of combining general language and image understanding models with fine-grained robot manipulation capabilities,” said Fox, who leads the Allen School’s Robotics & State Estimation Lab and is senior director of robotics research at NVIDIA..

In order to communicate with the butlers, Shridhar developed the Action Learning From Realistic Environments and Directives dataset (ALFRED). This is a dataset for agents to learn mapping from natural language instructions and egocentric vision to sequences of actions for household tasks. ALFRED consists of 25,000 natural language directives, including high-level instructions like “rinse off a mug and place it in the coffee maker” and lower-level language directions like “walk to the coffee maker on the right.” Tasks given to ALFRED are more complex in terms of sequence length, action space and language than previous vision-and-language task datasets.

Taking the next step beyond communicating tasks to the robots, Shridhar wants the robots to think about long-term actions without directly dealing with the complexities of the physical world. An example he gives is telling an agent to make an appetizer with sliced apples. Without any physical interactions, ALFWorld, a simulator that enables agents to learn abstract, “textual” policies in an interactive TextWorld, will train the robot to check the fruit bowl for apples and look in the drawers for a knife to make the appetizer. Before ALFWorld, agents did not have the infrastructure necessary for both reasoning abstractly and executing concretely. 

Shridhar intends to deploy ALFRED-trained models in household environments where a mobile manipulator can be commanded to perform tasks such as putting two plates on the dining table.

“I hope to build collaborative butler robots that aid and better human living,” Shridhar said.

Before coming to the Allen School, Shridhar received his Bachelor’s in Engineering from the National University of Singapore. He has interned at Microsoft Research, NVIDIA and an augmented reality startup. 

Shridhar is only one of 10 students recognized by the Graduate Fellowship Program based on their innovative research in Graphics Processing Unit (GPU) computing. Previous Allen School recipients of the NVIDIA Fellowship include Anqi Li (2020) and Daniel Gordon (2019).

Read more about the 2022-2023 NVIDIA Graduate Fellowship awards here.

Congratulations, Mohit! Read more →

“You are what you eat,” as the saying goes. But not everyone has the same degree of choice in the matter. An estimated 19 million people in the United States live in so-called food deserts, where they have lower access to healthy and nutritious food. More than 32 million people live below the poverty line — limiting their options to the cheapest food regardless of proximity to potentially healthier options. Meanwhile, numerous studies have pointed to the role of diet in early mortality and the development of chronic diseases such as heart disease, type 2 diabetes and cancer.

Researchers are just beginning to understand how the complex interplay of individual and community characteristics influence diet and health. An interdisciplinary team of researchers from the University of Washington and Stanford University recently completed the largest nationwide study to date conducted in the U.S. on the relationship between food environment, demographics, and dietary health with the help of a popular smartphone-based food journaling app. The results of that five-year effort, published today in the journal Nature Communications, should give scientists, health care practitioners and policymakers plenty of food for thought. 

“Our findings indicate that higher access to grocery stores, lower access to fast food, higher income and college education are independently associated with higher consumption of fresh fruits and vegetables, lower consumption of fast food and soda, and less likelihood of being classified as overweight or obese,” explained lead author Tim Althoff, professor and director of the Behavioral Data Science Group at the Allen School. “While these results probably come as no surprise, until now our ability to gauge the relationship between environment, socioeconomic factors and diet has been challenged by small sample sizes, single locations, and non-uniform design across studies. Different from traditional epidemiological studies, our quasi-experimental methodology enabled us to explore the impact on a nationwide scale and identify which factors matter the most.”

Althoff ‘s involvement in the study dates from when he was a Ph.D. student at Stanford working with professor and senior author Jure Leskovec and fellow student and co-author Hamed Nilforoshan. Together with co-author Dr. Jenna Hua, a former postdoctoral fellow at Stanford University School of Medicine and founder and CEO of Million Marker Wellness, Inc., the team analyzed data from more than 1.1 million users of the MyFitnessPal app — spanning roughly 2.3 billion food entries and encompassing more than 9,800 U.S. zip codes — to gain insights into how factors such as access to grocery stores and fast food, family income level, and educational attainment contribute to people’s food consumption and overall dietary health. 

The team measured the association of the aforementioned input variables with each of four dietary outcomes: fresh fruit and vegetable consumption, fast food consumption, soda consumption, and incidence of overweight or obese classified by body mass index (BMI). To understand how each variable corresponded positively or negatively with those outcomes, the researchers employed a matching-based approach wherein they divided the available zip codes into treatment and control groups, split along the median for each input. This enabled them to compare app user logs in zip codes that were statistically above the median — for example, those with more than 20.3% of the population living within half a mile of the nearest grocery store — with those below the median.

Among the four inputs the team examined, higher educational attainment than the median, defined as 29.8% or more of the population with a college degree, was the greatest positive predictor of a healthier diet and BMI. All four inputs were found to positively contribute to dietary outcomes, with one exception: high family income, defined as income at or above $70,241, was associated with a marginally higher percentage of people with a BMI qualifying as overweight or obese. But upon further investigation, these results only scratched the surface of what is a complex issue that varies from community to community.

“When we dug into the data further, we discovered that the population-level results masked significant differences in how the food environment and socioeconomic factors corresponded with dietary health across subpopulations,” noted Nilforoshan.

As an example, Nilforoshan pointed to the notably higher association between above-median grocery store access and increased fruit and vegetable consumption in zip codes with a majority of Black residents, at a 10.2% difference, and with a majority of Hispanic residents, at a 7.4% difference, compared to those with a majority of non-Hispanic white residents, where he and his colleagues found only a 1.7% difference. These and other findings indicate that factors such as proximity to grocery stores or higher income, on their own, are not sufficient for people to bypass the drive-thru or kick the (soda) can to the curb — and that future attempts to address dietary disparities need to take variations across zip codes into account.

“People assume that if we eliminate food deserts, that will automatically lead to healthier eating, and that a higher income and a higher degree lead to a higher quality diet. These assumptions are, indeed, borne out by the data at the whole population level,” explained Hua. “But if you segment the data out, you see the impacts can vary significantly depending on the community. Diet is a complex issue! While policies aimed at improving food access, economic opportunity and education can and do support healthy eating, our findings strongly suggest that we need to tailor interventions to communities rather than pursuing a one-size-fits-all approach.”

Althoff believes that both the team’s approach and its findings can guide future research on this complex topic that has implications for both individuals and entire communities.

“We hope that this study will impact public health and epidemiological research methods as well as policy research,” said Althoff. “Regarding the former, we demonstrated that the increasing volume and variety of consumer-reported health data being made available due to mobile devices and applications can be leveraged for public health research at unprecedented scale and granularity. For the latter, we see many opportunities for future research to investigate the mechanisms driving the disparate diet relationships across subpopulations in the U.S.”

Read the paper in Nature Communications here. Access the publicly available data and code associated with the study here. Read more →

As the world watched COVID-19 grow from a mysterious virus in far-off places to a planetary pandemic, news outlets worked hard to keep the world informed on how, where and why it was spreading. At the start of the outbreak, Matthew Conlen, a Ph.D. student in the Allen School’s Interactive Data Lab, was working as a graphic/multimedia editor for the New York Times helping with their elections forecasting application, also known as “The Needle.” He switched gears to contribute to the paper’s coverage of the novel coronavirus as it enveloped the globe — work that contributed to stories that earned the New York Times a Pulitzer Prize in Public Service for, in part, filling “a data vacuum that helped local governments, healthcare providers, businesses and individuals to be better prepared and protected.” 

Conlen’s forte is creating interactive data visualizations that do precisely that: help the public to comprehend what is happening in the country and throughout the world. In this particular case, he led a data collection effort on COVID-19 in nursing homes, and he also worked with epidemiologists and modelers to give readers an understanding of what could happen in different scenarios when schools reopened, as the vaccine rolled out and when the U.S. could reach herd immunity.

“Data journalism can provide a valuable perspective on our world, complementing traditional narrative reporting with additional context, more comprehensive accounts and increased audience engagement,” Conlen’s advisor and Allen School professor Jeffrey Heer said. “Interactive visualizations rank among the most visited and revisited pieces published by major news outlets. Though data visualization on the web has largely ‘come of age,’ a major remaining challenge is empowering more journalists — as well as educators and others  —  without an extensive technical background to author and collaborate on interactive articles.”

Conlen’s work, Heer said, leads on all of these fronts: publishing data-driven news at multiple major outlets and via Parametric Press, which Conlen co-founded, while simultaneously researching at the Allen School, which has resulted in new open-source languages and tools that make these articles easier to create. 

“It’s a virtuous cycle of research and practice,” Heer said.

After earning dual bachelor’s degrees in computer science and applied mathematics, Conlen began working on a big data analytics platform at an advertising technology, or ad tech, company. Despite interesting technical challenges, he found more fulfilling work in journalism, using digital news tools at the Huffington Post, FiveThirtyEight, The New Yorker and NASA’s earth science communications team. He became interested in data visualization because it combines math and statistics with tough programming challenges and a creative design aspect. This combination of technical and creative elements is, he notes, hard to come by in other fields.

“I’m interested in computer science generally because I think computers can be tremendously empowering tools,” Conlen said. “I want to develop systems that enable people to do things that were otherwise out of reach. It’s like giving someone superpowers but all you have to do is write some code.”

He said he saw his pursuit of a Ph.D. as the next phase of a career oriented around data visualization and digital publishing.

“Within the world of academic research I can spend more time understanding how people learn from data visualizations and interactive graphics and what makes certain designs effective, and I can engage with rich fields like human-computer interaction, or HCI, to better understand how to build effective digital tools for journalists and others.”

Combining his journalism and research, Conlen could see that visual forms are effective for communicating complex data sets. As a journalist, he understands the real-world constraints that HCI needs to account for in order to be useful in practice. He and Heer created Idyll, a toolkit that reduces the amount of effort and custom code required to make it easier to author and publish interactive articles, based on the challenges Conlen observed in the newsrooms in which he worked. The interactive capabilities of Idyll are seen in Unraveling the JPEG by graphics programmer Omar Shehata. Conlen explained that by using the interactive capabilities of Idyll, Shehata constructed a narrative walkthrough of the JPEG compression algorithm that connected with a big audience online — an audience that might not be interested to learn about that topic if not for the graphics that he made. 

“It wouldn’t have been possible to create this system without the practical knowledge that I gained as a journalist or without the space and time to think deeply and build ambitious research systems that the Allen School affords,” Conlen said.

In addition to Idyll, Conlen published the Beginner’s Guide to Dimensionality Reduction, which earned a Best Paper Honorable Mention in 2018 at the IEEE Visualization and Visual Analytics Workshop on Visualization for AI Explainability. The article used interactive graphics to introduce a complex technical topic to new readers in a gentle and engaging way.

“I’m regularly impressed by the resilience of my students,” Heer, who leads the Interactive Data Lab said. “Matt’s ability to bridge the worlds of professional journalism and academic research is a standout example, animated by Matt’s commitment to a more just and better-informed society.”

After spending a year at The New York Times, Conlen returned his focus to academic research. In October he presented his paper, “Idyll Studio: A Structured Editor for Authoring Interactive & Data-Driven Articles,” at the Association for Computing Machinery’s Symposium on User Interface Software and Technology. Idyll Studio is a new graphical interface for writing interactive and data-driven stories. 

“Think Microsoft Word but you can create documents that are dynamically driven by databases and include interactive visualizations and graphics,” Conlen said.

Conlen defended his dissertation last week and is currently working on a short-term contract with the New York Times. In early 2022 he will continue his work on the Idyll ecosystem — the open source project received a donation from venture capitalist Albert Wegner that will allow Conlen to put more time into improving the core project and refining Idyll Studio. He will continue doing data journalism and building tools to support that work. 

To view more of Conlen’s work combining journalism and data visualization, check out his collection of published articles on his website. 

Read more →

Earlier this month, the Allen School held a virtual celebration showcasing efforts to increase diversity in computing and honoring members of our community who have demonstrated their commitment to diversity, excellence and leadership. An annual tradition, the event this year also offered Allen School leaders an opportunity to share highlights from its five-year strategic plan to increase diversity, equity, inclusion and access (DEIA), spanning curriculum and programs, professional development, policies and procedures, internal community engagement, external outreach and budget. Allen School professor and director Magdalena Balazinksa had the happy task of introducing two undergraduate scholarship winners: senior Nayha Auradkar, recipient of the Allen AI Outstanding Engineer Scholarship for Women and Underrepresented Minorities from the Allen Institute for Artificial Intelligence (AI2), and senior Caiwei Tian, recipient of the Lisa Simonyi Prize. 

Auradkar, who is enrolled in the Allen School’s B.S./M.S. program, exemplifies the goal of the AI2 scholarship to encourage students from underrepresented groups to excel in computer science and engineering and become leaders and role models in their fields. Finding a passion for machine learning and human computer interaction, Auradkar used it to conduct accessibility research in the Make4All lab with Allen School professor Jennifer Mankoff. As an undergraduate she published two papers, one aimed at analyzing the features of personal protective equipment in response to the pandemic and the other focused on automating the process of creating complex textured knitting objects to make it easier for people with mobility-related disabilities to knit. Auradkar said that as someone with a disability, accessibility research has deep personal value to her and enables her to use her skills to help other people with disabilities. 

Auradkar isn’t focused solely on academics, though; she’s determined to make a difference on campus through leadership, too. She is the chair of the ACM-W, founded and leads the Allen School affinity group Ability, founded and leads Huskies Who Stutter and served as the outreach director for the Society of Women Engineers. In these roles she teaches middle school girls introductory engineering, cultivates a strong community of women in tech, promotes disability community and accessibility awareness and supports other UW students who stutter.

“This scholarship will enable me to learn from and collaborate with top research scientists, which will allow me to grow my research skills as I transition in my graduate degree,” Auradkar said. “It will also provide me with extra support in my DEIA work.”

The Lisa Simonyi Prize was established by longtime Allen School supporters Lisa and Charles Simonyi. The couple created the scholarship to recognize and support students who exemplify excellence, leadership, and diversity. This year’s recipient, Tian, is a double major in computer science and applied and computational mathematical sciences. She added the former after a data structure and algorithm course inspired a newfound interest in using programming as a tool to turn complex ideas into practice and discussing algorithms and the tradeoff between runtime and memories. Tian works in the Allen School’s UbiComp Lab with professor Shwetak Patel on developing a generalized deep learning model that uses video signals from smartphones to measure blood oxygen saturation (SpO2) levels, a crucial test in modern medicine. This work focuses on building a unified platform-agnostic model that works on all major smartphone systems.

Tian also has worked as a software development engineering intern at Amazon, a research assistant in the Make4All Lab and a research assistant at Fred Hutch. Tian co-founded a Chinese student choir group, MotE, a 50-member group that performs at festivals. She also assisted underprivileged students and provided academic support and encouragement as a math and science tutor for students at Licton Spring K-8 Public School.

“I’m really excited and honored to get this scholarship,” Tian said. “Knowing nothing about computer science before coming to UW and now graduating with this award, I think it is an evidence of my hard-work and my growth at the Allen School. It also encourages me to go further and keep learning.”

Thanks to AI2 and the Simonyis for supporting diversity and excellence, and thanks to everyone who logged on to celebrate the people who are making our school and our field a more welcoming destination for all. And congratulations to Nayha and Caiwei! 

For more about the Allen School’s efforts to advance diversity in computing, please visit our website. 

Read more →

Researchers affiliated with the Allen School took home all three Best Paper Awards at the Association for Computing Machinery’s 28th Symposium on Operating Systems Principles (SOSP). Current Ph.D. student Jacob Van Geffen, recent alumnus James Bornholt (Ph.D.,’19), former postdoc and incoming professor Simon Peter and affiliate professor Daniel Berger contributed to the winning papers that presented new advances in debugging, distributed computing and caching. 

In the paper, “Using Lightweight Formal Methods to Validate a Key-Value Storage Node in Amazon S3,” Van Geffen and Bornholt, now a professor at the University of Texas, Austin, present ShardStore, a new storage backend for Amazon Simple Storage Service (S3). Built on 40,000 lines of Rust code, ShardStore optimizes disk IO efficiency and currently stores hundreds of petabytes of customer data. The paper describes how ShardStore is resilient, resilient and crash-safe, and how AWS uses formal methods to catch and fix bugs early. Additional authors of the paper include Vytautus Astrauskas, a Ph.D. student at ETH Zurich and a team of researchers from Amazon Web Services that included Rajeev Joshi, Brendan Cully, Bernhard Kragl, Seth Markle, Kyle Sauri, Drew Schleit, Grant Slatton, Serdar Tasiran and Andrew Warfield.

Focused on building automated mechanisms to help engineers ensure the correctness of every change they make, ShardStore was developed to employ techniques like property-based testing and model checking with far lower overhead than traditional provable correctness. According to the team, this lightweight formal method prevented a number of issues like crash consistency and concurrency problems, before reaching production. The team plans to continue improving their techniques for cloud-based data storage. 

Peter, who is currently a professor at the University of Texas, Austin and will join the Allen School faculty in January, co-authored “LineFS: Efficient SmartNIC Offload of a Distributed File System with Pipeline Parallelism,” about fitting the high demands of a distributed file system (DFS) onto smart network interface cards (SmartNICs). In the paper, the team presents LineFS, a SmartNIC-offloaded, high-performance DFS with support for client-local persistent memory. LineFS moves CPU-intensive tasks to a SmartNIC, improving latency in LevelDB — a fast, key-value store — up to 80%. Korea Advanced Institute of Science and Technology researchers Jongyul Kim, Insu Jang, Jaeseong Im and Youngjin Kwon, along with Waleed Reda and Dejan Kostic from the KTH Royal Institute of Technology and Emmett Witchel also at UT Austin, contributed to the paper.

In “Kangaroo: Caching Billions of Tiny Objects on Flash,” Berger, a researcher at Microsoft and UW,  and his co-authors present a new flash cache that enables more efficient caching of tiny objects — often in social media and IoT services — called Kangaroo. Kangaroo overcomes challenges in existing flash cache designs such as minimizing main memory usage, which is expensive and energy hungry, and reduces load on back-end storage systems. Additionally, Kangaroo reduces flash memory wear out, extending flash cache lifetimes by multiple years. This also helps cost and sustainability. The paper was written with Carnegie Mellon University researchers Sara McAllister, Benjamin Berg, Julian Tutuncu-Macias, Juncheng Yang, Nathan Beckman and Gregory Ganger and Facebook researchers Sathya Gunasekar and Jimmy Lu.

Congratulations to all! 

Read more →

Professor Joshua R. Smith, who holds a joint appointment in the Allen School and the Department of Electrical & Computer Engineering, was elected into the 2021 class of Fellows of the National Academy of Inventors for his impactful creations in the fields of wireless power, communication, sensing and robotics. Smith, who leads the Sensor Systems Lab, is one of only five University of Washington faculty members to have received this prestigious award that highlights the prolific spirit of innovation in academic inventors. The NAI Fellows program was created to recognize inventors and their contributions to society, which stimulate the economy, improve and save lives, and make the world a better place. It is the highest professional distinction given solely to academic inventors. 

The NAI Fellow selection process considers inventions that have been licensed or commercialized. Smith holds 48 U.S. patents and 16 international patents, 44 of which are licensed by companies. His inventions have led to hundreds of millions of dollars in product revenues, bolstering the economy and the creation of approximately 70 full-time jobs, according to Suzie Pun, a professor in the Department of Bioengineering and another UW faculty member who is an NAI Fellow. 

Smith’s first six patents, developed while he was a graduate student at MIT, pioneered mutual capacitance sensing and led to the creation of a smart airbag system that was included in every Honda car between 2000 and 2015. Before his arrival at UW, Smith spent five years at Intel Research Seattle, creating new capabilities in wireless power, wireless sensing and robotics. He led the creation of the Wireless Identification and Sensing Platform (WISP), the first fully programmable platform for wireless, battery-free sensing and computation powered by radio waves.

Soon after, he developed Wireless Resonant Energy Link (WREL), which uses magnetically coupled resonators to efficiently transfer wireless power even as range, orientation and load vary. With the help of a heart surgeon from Yale, Smith was able to power a ventricular assist device designed for implantation in the human body without requiring a cable through the patient’s chest, called the Free-range Resonant Electrical Energy Delivery System (FREED). This wireless power work at UW is commercialized by WiBotic, a company Smith co-founded with ECE alumnus Benjamin Waters (Ph.D., ‘15). The UW patents are also licensed for implanted heart pumps by Corisma.

“Among the many outstandingly inventive engineers at Intel Research Seattle, we were especially excited that Josh joined our faculty, he is extraordinary in every imaginable respect,” said Ed Lazowska, professor and Bill & Melinda Gates Chair Emeritus at the Allen School. “He is an academic inventor and entrepreneur of the highest caliber and in the finest tradition.”

In 2013, Smith, together with Allen School professor Shyam Gollakota and a team of graduate students, developed Ambient Backscatter using existing wireless signals to provide power and communication for low-power sensing and computing devices. This next led to the creation of Passive-Wi-Fi, bringing low-power Wi-Fi to transmissions. They also invented Interscatter, using wireless transmissions over the air from one technology to another for internet-connected implanted devices. Smith also co-led the UW team behind the world’s first battery‐free phone, as well as a series of ultra-low-power battery-free wireless cameras that communicate via backscatter.

The team’s research is being commercialized by Jeeva Wireless, a UW spinout co-founded by Smith, Gollakota, and ECE alumni Vamsi Talla (Ph.D., ‘16) and Aaron Parks (Ph.D., ‘17).

“Josh has a consistent record of impactful inventions,” said Pun. “I have gotten to know him through a research collaboration to develop touchscreen-based sensors for detection of pathogens such as SARS-CoV-2. Josh devised a creative method to improve detection sensitivity for the virus; he is in the process of testing this idea in his laboratory. If successful, his design could be applied for next generation biosensing devices.”

Smith also co-founded Proprio, which provides surgical visualization and navigation, together with UW neurosurgeon Sam Browd, Allen School graduate student Jim Youngquist, UW Foundation board member Ken Denman, and Michael G. Foster School of Business alumnus Gabe Jones (MBA, ‘14). Smith served on advisory councils and task forces for the United States Postal Service and the Smithsonian Institution and is an IEEE Fellow. His work has earned multiple Best Paper Awards, and he is known for his dedicated mentorship of student researchers.

“I feel so privileged to collaborate with my outstanding UW faculty and student co-inventors,” said Smith, who holds the Milton and Delia Zeutschel Professorship in Entrepreneurial Excellence in ECE. “And invention is just one part of a long process to bring new things into the world.

“I am very grateful to the many people who have worked so hard to take these inventions from the lab to the world, including UW CoMotion, many patent attorneys, and most of all the co-founders and employees at the companies making these technologies real.”

Read the NAI announcement here, and the full list of 2021 Fellows here.

Congratulations, Josh! 

Read more →

There are many options for mapping and route planning on a smartphone, but one thing they all have in common is their car-centric nature. Those apps that do support pedestrian navigation tend to make assumptions about a user that are at best inaccurate, and at worst dangerous.

For residents and visitors in three western Washington cities, that changes today with the release of the AccessMap mobile app. The app, which was developed by the Taskar Center for Accessible Technology housed at the Paul G. Allen Center of Computer Science & Engineering at the University of Washington and is based off of the web-based tool of the same name, enables users of Android and iOS in the cities of Seattle, Bellingham and Mount Vernon to generate customized walking directions on the go based on their own mobility needs and preferences. The app’s release coincides with the International Day of Persons with Disabilities, an annual observance initiated by the United Nations to promote the rights and well-being of persons with disabilities in all spheres of society, including political, social, economic and cultural life.

“Many apps offer some semblance of pedestrian directions, but those directions assume a user profile that ignores the lived experience of a vast number of people,” explained Anat Caspi, director of the Taskar Center. “They make assumptions about the steepness of hills you can take, how speedy you will be, and what constitutes a safe walking or rolling environment. Some apps don’t even provide basic information like whether or not there is a sidewalk. Rather than this one-size-fits-all approach, AccessMap empowers people by giving them the ability to tailor a route that suits them as individuals. 

“This is significant not only because of the departure from the mass-produced apps,” she continued. “Even apps that are ‘accessibility-minded’ tend to lump together specific device use, like putting all manual wheelchair users or power wheelchair users in one category. Our research shows dramatic variability even within the same device user group, and this is also addressed by AccessMap.”

The ability to personalize their navigation options tied to real-world constraints, informed by a series of user studies during the development phase, will open up new avenues of accessibility for people with and without mobility impairments. For example, a user can request a route that takes them through intersections with “curb cuts,” which make it easier for people with strollers as well as wheelchairs to cross the street. Users can also specify a path that will enable them to avoid steep hills. The app — which is screen-reader accessible for people who are blind or low-vision — comes pre-loaded with three pedestrian profiles tailored to users who rely on a manual wheelchair, power wheelchair, or cane, along with the option to customize settings like maximum incline according to personal preference. Like other popular navigation apps, AccessMap includes color-coded maps that enable users to see at a glance which streets would be most favorable for travel; unlike those other apps, the emphasis here is on sidewalk traversability rather than automobile traffic.

Another aspect of AccessMap that sets it apart from other navigation tools: the sensitive nature of users sharing details of their personal mobility needs. For this reason, each user’s profile and preferences — for example, requests for wheelchair-accessible routes or maximum incline settings — is stored locally on their phone.

The process of turning the original web-based tool into a mobile app began in earnest in the summer of 2020 and further ramped up this past summer, when Caspi and Allen School postdoctoral researcher Nick Bolten assembled a team of undergraduate students to take on the task of translating the existing AccessMap into a user-friendly — and accessible — mobile interface. In addition to being a technical achievement, the project represented a unique learning opportunity for student researchers Xianxian Cheng, Jay Lin, and Eric Yeh. 

For Yeh, AccessMap was an opportunity to combine his desire to have a positive impact on the community with the prospect of doing something he had never done before: build a mobile app from scratch. 

“The main roadblock for me was having no development experience other than coursework at the time. Although I ran into a lot of bugs and issues along the way, those struggles helped me to understand how mobile apps are built and compiled,” recalled Yeh, a senior computer science major in the Allen School who is also pursuing a minor in neural engineering. “I had the opportunity to implement a large range of features such as the map, accessibility settings, and user feedback. I did not expect my very first project to expand to the scale that it is now, and it’s amazing to see how it has evolved from a barebones map to a fully functional application.”

Lin became interested in mobile app accessibility after taking the Allen School’s Interaction Programming course and making the connection between accessibility issues associated with commonly used apps and other disciplines. 

“After taking classes in the Disability Studies department, I realized that there’s quite a lot of overlap between issues in technology and with legal/social justice,” explained Lin, a fourth-year student who splits their time between UW Bothell’s computer science program and the aforementioned disability studies at UW Seattle. “What’s common in these two topics is that disability rights and accessibility are often treated as an afterthought, yet this impacts a large part of your audience or user base.”

Support from the U.S. Department of Transportation paved the way for AccessMap’s move to mobile through a multi-year grant awarded to the Taskar Center and its partners in the Transportation Data Equity Initiative. Today’s release represents a step toward one of USDOT’s primary goals for that grant: to promote multimodal accessibility that includes transportation options such as bus and rail.

“Multimodal accessibility is key to providing a more useful pedestrian trip planner, as strictly pedestrian travel is usually just a subset of an overall trip,” noted Bolten, who co-led development of the original AccessMap while earning his Ph.D. from the UW Department of Electrical & Computer Engineering. “Other trip segments may involve boarding a bus, boarding a train, or driving a car, all of which must then interface with the pedestrian network. For example, choosing the right bus stop will surely depend on the accessibility of the surrounding pedestrian network, so picking the best bus and pedestrian routes aren’t independent things: they should be co-planned.

“In addition, every pedestrian trip planning question can also be thought of as an analysis of the network itself: how accessible is a city, a neighborhood, or all paths to a grocery store?” he continued. “A multimodal network will also ‘upgrade’ those questions and enable data-based conversations about public transportation and infrastructure.”

The three cities currently covered by the app are just the beginning. Caspi and her team have plans to extend coverage to the eastern side of Lake Washington, along the Bellevue-Redmond corridor, as well as to the city of Austin, Texas. The group is also collaborating with municipalities, transit agencies and advocacy groups to collect data for new cities, including Los Angeles, California and four Latin American cities as part of a partnership with international nonprofit G3ict and Microsoft’s AI4Accessibility initiative, to extend the app’s coverage even further — plans that align with the Taskar Center’s mission of designing for the fullness of the human experience, which will enable people to fully participate in civic life. 

That goal is one that AccessMap’s student developers will carry with them into their future work. 

“Working on AccessMap has helped me to see how people see, hear, feel, touch, and perceive the world in their own ways. There are some experiences we may take for granted, but we should always remember there may be constraints that are invisible to our eyes which might harm others who have different ways of interacting with the world on a daily basis,” said Cheng, a senior studying informatics and user experience designer for the AccessMap app. “I feel honored that I could be a part of this digital translation that leverages accessible user experience.”

The AccessMap mobile app is available now from Apple’s App Store (iOS) and Google Play (Android). Government agencies, municipalities, institutions and community organizations interested in having their city included in the next release of AccessMap may submit their request to the team via email at opensidewalks@gmail.com. Read more →