Skip to main content

Allen School researchers discover medical AI models rely on “shortcuts” that could lead to misdiagnosis of COVID-19 and other diseases

Chest x-ray
Source: National Institutes of Health Clinical Center*

Artificial intelligence promises to be a powerful tool for improving the speed and accuracy of medical decision-making to improve patient outcomes. From diagnosing disease, to personalizing treatment, to predicting complications from surgery, AI could become as integral to patient care in the future as imaging and laboratory tests are today. 

But as Allen School researchers discovered, AI models — like humans — have a tendency to look for shortcuts. In the case of AI-assisted disease detection, such shortcuts could lead to diagnostic errors if deployed in clinical settings.

In a new paper published in the journal Nature Machine Intelligence, a team of researchers in the AIMS Lab led by Allen School professor Su-In Lee examined multiple models recently put forward as potential tools for accurately detecting COVID-19 from chest radiography (x-ray). They found that, rather than learning genuine medical pathology, these models rely instead on shortcut learning to draw spurious associations between medically irrelevant factors and disease status. In this case, the models ignored clinically significant indicators in favor of characteristics such as text markers or patient positioning that were specific to each dataset in predicting whether an individual had COVID-19. 

According to graduate student and co-lead author Alex DeGrave, shortcut learning is less robust than genuine medical pathology and usually means the model will not generalize well outside of the original setting.

Portrait of Alex Degrave
Alex DeGrave

“A model that relies on shortcuts will often only work in the hospital in which it was developed, so when you take the system to a new hospital, it fails — and that failure can point doctors toward the wrong diagnosis and improper treatment,” explained DeGrave, who is pursuing his Ph.D. in Computer Science & Engineering along with his M.D. as part of the University of Washington’s Medical Scientist Training Program (MSTP). 

Combine that lack of robustness with the typical opacity of AI decision-making, and such a tool could go from potential life-saver to liability.  

“A physician would generally expect a finding of COVID-19 from an x-ray to be based on specific patterns in the image that reflect disease processes,” he noted. “But rather than relying on those patterns, a system using shortcut learning might, for example, judge that someone is elderly and thus infer that they are more likely to have the disease because it is more common in older patients. The shortcut is not wrong per se, but the association is unexpected and not transparent. And that could lead to an inappropriate diagnosis.”

The lack of transparency is one of the factors that led DeGrave and his colleagues in the AIMS Lab to focus on explainable AI techniques for medicine and science. Most AI is regarded as a “black box” — the model is trained on massive data sets and spits out predictions without anyone really knowing precisely how the model came up with a given result. With explainable AI, researchers and practitioners are able to understand, in detail, how various inputs and their weights contributed to a model’s output.

Portrait of Joseph Janizek
Joseph Janizek

The team decided to use these same techniques to evaluate the trustworthiness of models that had recently been touted for what appeared to be their ability to accurately identify cases of COVID-19 from chest radiography. Despite a number of published papers heralding the results, the researchers suspected that something else may be happening inside the black box that led to the models’ predictions. Specifically, they reasoned that such models would be prone to a condition known as worst-case confounding, owing to the paucity of training data available for such a new disease. Such a scenario increased the likelihood that the models would rely on shortcuts rather than learning the underlying pathology of the disease from the training data.

“Worst-case confounding is what allows an AI system to just learn to recognize datasets instead of learning any true disease pathology,” explained co-lead author Joseph Janizek, who, like DeGrave, is pursuing a Ph.D. in the Allen School in addition to earning his M.D. “It’s what happens when all of the COVID-19 positive cases come from a single dataset while all of the negative cases are in another.

“And while researchers have come up with techniques to mitigate associations like this in cases where those associations are less severe,” Janizek continued, “these techniques don’t work in situations you have a perfect association between an outcome such as COVID-19 status and a factor like the data source.” 

The team trained multiple deep convolutional neural networks on radiography images from a dataset that replicated the approach used in the published papers. They tested each model’s performance on an internal set of images from that initial dataset that had been withheld from the training data and on a second, external dataset meant to represent new hospital systems. The found that, while the models maintained their high performance when tested on images from the internal dataset, their accuracy was reduced by half on the second, external set — what the researchers referred to as a generalization gap and cited as strong evidence that confounding factors were responsible for the models’ predictive success on the initial dataset. The team then applied explainable AI techniques, including generative adversarial networks (GANs) and saliency maps, to identify which image features were most important in determining the models’ predictions. 

Figure from paper showing three x-ray images paired with color-coded saliency maps indicating weight of factors in model prediction
The team used explainable AI to visualize the image factors that influenced neural network models’ predictions of COVID-19 status based on chest radiography. Here, saliency maps reveal the models’ tendency to emphasize diagnostically irrelevant features such as laterality tokens, image corners or the patient’s diaphragm in addition to — or instead of — the lung fields when making their predictions.^

When the researchers trained the models on the second dataset, which contained images drawn from a single region and was therefore presumed to be less prone to confounding, this turned out to not be the case; even those models exhibited a corresponding drop in performance when tested on external data. These results upend the conventional wisdom that confounding poses less of an issue when datasets are derived from similar sources — and reveal the extent to which so-called high-performance medical AI systems could exploit undesirable shortcuts rather than the desired signals.

Despite the concerns raised by the team’s findings, DeGrave said it is unlikely that the models they studied have been deployed widely in the clinical setting. While there is evidence that at least one of the faulty models – COVID-Net – was deployed in multiple hospitals, it is unclear whether it was used for clinical purposes or solely for research.

“Complete information about where and how these models have been deployed is unavailable, but it’s safe to assume that clinical use of these models is rare or nonexistent,” he noted. “Most of the time, healthcare providers diagnose COVID-19 using a laboratory test (PCR) rather than relying on chest radiographs. And hospitals are averse to liability, making it even less likely that they would rely on a relatively untested AI system.”

Janizek believes researchers looking to apply AI to disease detection will need to revamp their approach before such models can be used to make actual treatment decisions for patients.

“Our findings point to the importance of applying explainable AI techniques to rigorously audit medical AI systems,” Janizek said. “If you look at a handful of x-rays, the AI system might appear to behave well. Problems only become clear once you look at many images. Until we have methods to more efficiently audit these systems using a greater sample size, a more systematic application of explainable AI could help researchers avoid some of the pitfalls we identified with the COVID-19 models,” he concluded.

Janizek, DeGrave and their AIMS Lab colleagues have already demonstrated the value of explainable AI for a range of medical applications beyond imaging. These include tools for assessing patient risk factors for complications during surgery, which appeared on the cover of Nature Biomedical Engineering, and targeting cancer therapies based on an individual’s molecular profile, as described in a paper published in Nature Communications.

Su-In Lee holding pen with coffee mug and laptop
Su-In Lee

“My team and I are still optimistic about the clinical viability of AI for medical imaging. I believe we will eventually have reliable ways to prevent AI from learning shortcuts, but it’s going to take some more work to get there,” Lee said. “Going forward, explainable AI is going to be an essential tool for ensuring these models can be used safely and effectively to augment medical decision-making and achieve better outcomes for patients.”

The team’s paper, “AI for radiographic COVID-19 detection selects shortcuts over signal,” is one of two from the AIMS Lab to appear in the current issue of Nature Machine Intelligence. Lee is also the senior and corresponding author on the second paper, “Improving performance of deep learning models with axiomatic attribution priors and expected gradients,” for which she teamed up with Janizek, his fellow M.D.–Ph.D. student Gabriel Erion, Ph.D. student Pascal Sturmfels, and affiliate professor Scott Lundberg (Ph.D., ‘19) of Microsoft Research to develop a robust and flexible set of tools for encoding domain-specific knowledge into explainable AI models through the use of attribution priors. Their framework supports the widespread adoption of techniques that will improve model performance and increase computational efficiency in AI for medicine and other areas of applied machine learning. 

Also see a related GeekWire article here.

* Image sourced from the National Institutes of Health (NIH) Clinical Center and used with permission: Wang, X. et al. “ChestX-ray8: Hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017. https://nihcc.app.box.com/v/ChestXray-NIHCC

^ Figure 2a (bottom) adapted with permission from Winther, H. et al. COVID-19 image repository. figshare. https://doi.org/10.6084/m9.figshare.12275009

Read more →

Anna Karlin receives ACM Paris Kanellakis Theory and Practice Award

Collage of award recipients' portraits with ACM logo
Top, from left: Yossi Azar, Andrei Broder and Anna Karlin; bottom, from left: Michael Mitzenmacher and Eli Upfal

Professor Anna Karlin and a team of collaborators have received the ACM Paris Kanellakis Theory and Practice Award from the Association for Computing Machinery for their work on balanced allocations, also known as the “power of two choices.” Karlin, a member of the Allen School’s Theory of Computation group, first introduced the balanced allocations paradigm with co-authors Yossi Azar of Tel Aviv University, Andrei Broder of Google Research, and Eli Upfal of Brown University at the 1994 ACM Symposium on Theory of Computing (STOC) in what the ACM has described as “elegant theoretical results” that continue to have a demonstrable impact on the practice of computing. 

The team’s paper considers a basic “balls-in-bins” problem: It has long been known that when n balls are tossed one at a time into n bins, independently and uniformly at random, the expected value of the maximum load is about log(n)/loglog(n). Karlin and her colleagues showed that balancing the allocation of the balls by first selecting not one, but two bins at random, and then placing the ball in the bin that has the lesser load between the two reduces the expected maximum load across all the bins to loglog(n) — an exponential improvement.

Fellow honoree Michael Mitzenmacher of Harvard University later significantly extended these initial results.

“Since bins and balls are the basic model for analyzing data structures, such as hashing or processes like load balancing of jobs in servers, it is not surprising that the power of two choices that requires only a local decision rather than global coordination has led to a wide range of practical applications,” the ACM observed in a press release. “These include i-Google’s web index, Akamai’s overlay routing network, and highly reliable distributed data storage systems used by Microsoft and Dropbox, which are all based on variants of the power of two choices paradigm.”

Karlin’s Allen School colleague, Paul Beame, recalls that it was “stunning and unexpected” that such a simple change in the algorithm would yield such a dramatic improvement — one that has proven to have an enduring impact within the field.

“One can easily ensure balanced allocations if one has prior knowledge about the pattern of requests or some form of centralized control but, without that, finding good balance becomes difficult. The typical solution before the work of Anna and her co-authors allocated each request based on a single random choice of a resource. However, unless the system is vastly over-provisioned, this will result in a somewhat bad overall imbalance,” explained Beame. “What Anna and the team showed is that a simple adjustment — making two random choices rather than just one, and keeping the better of the two — results in an exponential improvement in the overall balance, effectively at most a small constant for all reasonable data sizes.

“In the last couple of decades, their paper has led to much follow-on research,” Beame noted. “The ‘power of two choices’ has since become one of the essential new paradigms in the design of dynamic data structures and algorithms and is regularly taught in graduate courses worldwide.”

Prior to winning the ACM Paris Kanellakis Award, Karlin recently became the first Allen School faculty member to be elected to the National Academy of Sciences for career contributions to algorithms and algorithmic game theory. She is one of two Allen School faculty members to be recognized with ACM technical awards this week, as Shyam Gollakota received the ACM Grace Murray Hopper Award for his work on wireless sensing and communication. 

Read the ACM announcement here, and learn more about the Paris Kanellakis Theory and Practice Award here.

Congratulations, Anna!

Read more →

Shyam Gollakota wins ACM Grace Murray Hopper Award

Portrait of Shyam Gollakota

Allen School professor Shyam Gollakota received the ACM Grace Murray Hopper Award from the Association for Computing Machinery for “contributions to the use of wireless signals in creating novel applications, including battery-free communications, health monitoring, gesture recognition, and bio-based wireless sensing.” Each year, the ACM Grace Murray Hopper Award honors an early-career professional in computing who has made a major technical or service contribution to the field before the age of 35. 

As director of the Allen School’s Networks & Mobile Systems Lab, Gollakota advances big ideas in compact, energy-efficient form factors to expand the Internet of Things (IoT). Among his earliest contributions was ambient backscatter, a groundbreaking technique he pioneered with Allen School colleague and electrical engineering professor Joshua Smith. Backscatter essentially produces power out of thin air by harvesting television, WiFi, and other wireless signals to enable battery-free computation and communication. The team later refined and expanded their work to enable transmissions over greater distances and via embedded devices with long-range backscatter, and even produced a prototype of the world’s first battery-free cellphone. Gollakota also showed how backscatter communication could be accomplished without electronics with the introduction of 3D printed smart objects. The researchers started a venture-backed company, Jeeva Wireless, to commercialize their work.

Gollakota has also tapped into the sensing capabilities of smartphones to develop a series of health screening and monitoring tools in collaboration with clinicians at UW Medicine. These include a non-invasive app for detecting fluid in the ear — a symptom of ear infection and one of the most common reasons for visits to the pediatrician — and one for detecting obstructive sleep apnea that was subsequently commercialized by ResMed. He also showed how smartphones can be powerful tools for tackling broader public health crises with Second Chance, an app that employs sonar to monitor a person’s breathing and movements for signs of a potential opioid overdose. Gollakota and his colleagues have since explored how other smart devices, such as Amazon’s Alexa smart speaker, can be used for home health monitoring. To that end, he and his team have developed new smart speaker skills to detect if a person has an irregular heart rhythm or is experiencing a cardiac emergency and also to monitor a baby’s breathing during sleep. He co-founded two companies, Sound Life Sciences and Wavely Diagnostics, to commercialize this work.

A common thread running throughout Gollakota’s research is his creativity in addressing real-world problems while pushing the limits of what was previously thought possible through technology.

“Simply put, Shyam is amazing — he is easily the most creative person I have ever met,” said Allen School professor Thomas Anderson. “He repeatedly invents and builds prototypes that, before you see them demonstrated, you would have thought impossible. I do not know of any junior faculty member, in any area of computer science, whose work has had greater practical impact on our understanding of how to build useful systems.”

Lately, those systems have bridged the digital and natural worlds in Gollakota’s efforts to enable wireless sensing and computation to take off — literally as well as figuratively. In a series of projects that can be described as “living IoT,” Gollakota and his colleagues attached sensors to bees to demonstrate a system for wireless data transmission with applications in agriculture and environmental monitoring; outfitted a beetle with a steerable robotic camera that can be used to track moving objects and live-stream images to a smartphone; and developed a lightweight sensor that can be transported to hard-to-reach locations by moths in a step towards creating the Internet of Biological Things. Gollakota also has taken inspiration from nature to build insect-sized robots capable of wireless flight in a collaboration with mechanical engineering professor Sawyer Fuller.

“His work has revolutionized and re-imagined what can be done using wireless systems and has a feel of technologies depicted in science fiction novels,” the ACM said of Gollakota in a press release.

The ACM Grace Murray Hopper Award is accompanied by a monetary prize of $35,000, which Gollakota has opted to donate in support of LGBTQIA+ students in the Allen School.

“We are extremely proud of Shyam and his research group. They produce incredibly creative and innovative technology that also strives to address fundamental environmental and societal problems,” said Magdalena Balazinska, professor and director of the Allen School. “On a personal level, I am proud to call Shyam a colleague and am very happy to see him recognized with this award not only for his technical contributions, but also for his service as a mentor and role model for future innovators in our field.”

Gollakota is the second Allen School faculty member to receive the Grace Murray Hopper Award. The ACM previously recognized professor Jeffrey Heer in 2017 for his work on leading-edge visualization tools for exploring and understanding data.

Read the ACM announcement here, and learn more about the Grace Murray Hopper Award here. In addition, see our story on professor Anna Karlin receiving the ACM Paris Kanellakis Theory and Practice Award, also announced today, for her work on balanced allocation or the “power of two choices.”

Congratulations, Shyam!

Read more →

UW recognizes three Allen School undergraduates in the Husky 100

Husky 100 banner

Allen School undergraduates Nayha Auradkar, Melissa Birchfield and Raida Karim have been selected for the 2021 class of the Husky 100. Each year the program honors 100 University of Washington students across its three campuses who are making the most of their time as Huskies to have a positive impact on the UW community. 

Nayha Auradkar

Nayha Auradkar

Nayha Auradkar is a junior from Sammamish, Washington, majoring in computer science with a minor in neural computation and engineering. She aims to create technology that supports accessibility and inclusion for people with disabilities. Her work with Jennifer Mankoff in the Make4all Lab in human-computer interaction (HCI) and accessible technology complements her goals. Outside of the lab, Auradkar is the chair of the UW chapter of the Association for Computing Machinery for Women (ACM-W), working to cultivate a strong, supportive community of women in the Allen School, and is president of Huskies Who Stutter. She previously served as the outreach director of the Society of Women Engineers

“Technology can benefit the world in limitless and profound ways — even more so when everyone’s voice is heard,” Auradkar said. “Throughout my time as a Husky I have worked towards advocating for more equitable communities in tech and beyond.”

Melissa Birchfield

Melissa Birchfield

Melissa Birchfield is a senior also from Sammamish, Washington, majoring in computer science with UW Interdisciplinary Honors. Birchfield worked as a teaching assistant for several CS and Human Centered Design & Engineering (HCDE) courses and has led outreach projects through Alternative Spring Break. She has contributed to research as part of the Programming Languages & Software Engineering (PLSE) group and worked with Ph.D. student Eunice Jun  in the Interactive Data Lab to facilitate statistical analysis for end-users. She has also explored the intersection of HCI and machine learning in HCDE’s Inclusive Design Lab. She is an anti-human trafficking advocate and author of “Data for Dignity: Leveraging Technology in the Fight Against Human Trafficking” examining the role of data in cross-sector collaboration to combat human trafficking.

“My time at UW has challenged me to embrace new experiences while clarifying my heart for outreach and innovation,” Birchfield said. “As an educator, researcher, author and anti-trafficking advocate, I pursue ways to bridge gaps in communities by leveraging technology to open unexplored opportunities.”

Raida Karim

Raida Karim

Raida Karim is a fourth year computer science major from Dhaka, Bangladesh. She’s a researcher working with Allen School professor Maya Cakmak in the Human-Centered Robotics Lab. Karim is focused on meeting critical human needs and achieving social good, which has inspired her to create technologies like a robot that measures stress levels in teens and develops some therapeutic, intervening techniques to help them. She served as a program lead in Mentor Power for Success in the Office of Minority Affairs & Diversity, a student leader in the UW chapter of the Association for Computing Machinery, and worked as a teaching assistant in the Allen School’s direct admit seminar. Off campus, she was an intern at Cisco and is currently an X-Force Fellow sponsored by the National Security Innovation Network. 

“My personal background as a woman of color and an immigrant allows me to embrace all differences in humanity,” Karim said. “I hope my visibility and persistence demonstrates that hard work, determination and self-worth are keys to tackling odds in engineering, when there are no familiar faces or legacies.”

View the 2021 Class of the Husky 100 here.

Congratulations, Nayha, Melissa and Raida! 

Read more →

Allen School alumni and friends honored with College of Engineering Diamond Awards

Dadgar, Hashimoto, Israel
Left to right: Dadgar, Hashimoto, Israel

Allen School alumni Armon Dadgar (B.S. ‘11) and Mitchell Hashimoto (B.S. ‘11) and long-time Allen School friend and University of Washington alumnus Allen Israel (B.S. ‘68 Mechanical Engineering, MBA ‘71, J.D. ‘78) have been honored with 2020 Diamond Awards from the College of Engineering. Dadgar and Hashimoto were recognized with the Early Career Achievement Award, given to outstanding graduates who have made exceptional professional contributions to engineering through research, teaching or service within the first 10 years of their careers. Israel earned the Dean’s Award for extraordinary contributions to the advancement of engineering.

Dadgar and Hashimoto are the co-founders of HashiCorp, which helps companies to address fundamental challenges related to infrastructure, applications, networking and security in the cloud. The company’s software offerings, including Consul, Terraform and Vault, have become hugely popular tools for automating processes that are used by organizations worldwide. The early vision for HashiCorp — now a $5.2 billion company — was originally hatched when the two were Allen School undergraduates.

Dadgar and Hashimoto initially met while collaborating on a research project. Afterward, Dadgar reached out to Hashimoto to see if he wanted to work on something fun outside of the classroom.

“We were passionate about creating something people needed,” Hashimoto said. “What makes our company unique is that it’s open-source and free. But we never expected to make a career out of it, let alone start a successful company. We are still surprised at how far we’ve come.”

Hashimoto said they loved the idea of automating repetitive tasks. Their plan was to build a program based around cloud infrastructure that would enable developers to eliminate some of the more tedious manual processes needed in cloud computing through a self-service automation tool.

“When Mitchell and I started HashiCorp, our mission was really focused on building great products that we ourselves would enjoy using,” Dadgar said. “It was about solving an immediate problem that we had personally experienced, and we didn’t have a clear business plan going into it. It has been an incredible journey at HashiCorp, especially figuring out how to transition from making tools people like into being a company that organizations depend on.”

Their creation grew from two students working in a basement on campus in their spare time, to young college graduates working from IKEA desks in Dadgar’s living room, to a business with more than 1,300 employees that serves some of the biggest companies in the world. The duo had agreed to give the business a year to take off before selling the desks and taking traditional jobs as software engineers. There was no need to sell the desks.

“I think it’s safe to say it has exceeded our wildest imaginations. From an adjustment phase, I like to joke that every quarter I need to figure out what my job is again,” Dadgar said. “Given our growth, our roles continue to evolve dramatically. In the beginning, Mitchell and I would write code full time – we personally authored many of our initial products. I don’t think I’ve written any software for at least three years now, but instead spend much more time on hiring, customers, partners, and running the business.”

The developers-turned-business-leaders took a lot of risks in the beginning to see their vision for HashiCorp come to fruition.

“It wasn’t a smooth transition, we had a lot of ups and downs,” Hashimoto acknowledged. “We had stages where the company was doing well and not doing well. Personally, we’ve had to adapt and learn to recognize where we’re operating smoothly and providing value, and where we don’t enjoy and don’t provide any value.”

Hashimoto said he still enjoys the work and he’s driven by a desire to identify other areas where automated tools will make people’s lives easier and to continue building new products. Dadgar is also still 100%  committed to HashiCorp; with the company’s continued success, he and his husband, Josh Kalla, are interested in pursuing more philanthropic endeavors. They’ve already given more than $3.5 million to the UW’s Office of Minority Affairs & Diversity for student support.

“With any big success like HashiCorp, there is always a mix of hard work, opportunity, and luck that is necessary. We’ve been so fortunate in our lives, that now it is our turn to help create opportunities for others,” Dadgar said. “The OMA&D really was a great fit for our interests, because they focus on first-generation, underrepresented, and financially challenged students. Those are the groups we feel need to have more luck and opportunities, and so working with OMA&D to create a scholarship was a great fit.”

Dan Grossman, professor and vice director of the Allen School, said the combination of business and technical skills from such young alumni is especially notable. “Mitchell and Armon have demonstrated a unique combination of applying deep computer science, customer focus, and business skills to create a series of technologies, spanning a wide range of distributed systems solutions, that are pragmatic and loved by customers,” Grossman said. “And they’ve managed in a very short time to create a successful business based on these developments.”

While Dadgar and Hashimoto are in the early stages of their professional and philanthropic journeys, their fellow Diamond Award honoree, Allen Israel, has had a lifetime of impact on the lives of others.

After completing his UW undergraduate degree in Mechanical Engineering, Israel worked as an engineer at Boeing Commercial Airplanes before returning to his alma mater to earn his MBA followed by his J.D. After law school, Israel joined Foster Garvey PC, where he has spent more than 40 years practicing law, primarily in business and mergers and acquisitions. In 2013, he was named “Lawyer of the Year” in Seattle by Best Lawyers M&A.

In the course of his work, Israel represented nonprofit and individual clients in corporate, business, real estate and contract law. One of those individuals was Paul G. Allen, who Israel served as personal attorney from the early 1980s until Mr. Allen’s death in 2018. Among many other activities, Israel represented Mr. Allen in making many transformative gifts to the University of Washington — including the naming gift for the Paul G. Allen Center for Computer Science & Engineering, and the gift that enabled the creation of the Paul G. Allen School of Computer Science & Engineering.

“Allen has been instrumental in helping us to expand our leadership in the field and our impact around the world,” said Magda Balazinska, Professor and Director of the Allen School. “He played a central role in the construction of the Allen Center, our first permanent home, and in the creation of the Allen School on the occasion of our 50th anniversary. He also represents the Allen School on the University of Washington Foundation Board.”

Throughout his career, Israel has made time to champion and provide leadership in many areas of his alma mater. For nearly 25 years he served as a member of the College of Engineering Dean’s Visiting Committee, providing strategic counsel to four successive leaders. He also has served on the UW Law School Dean’s Advisory Committee and the Law School Foundation Board.

“Allen Israel has been a tremendous friend of the Allen School, the College of Engineering, and the University of Washington for many decades,” said Ed Lazowska, Professor and Bill & Melinda Gates Chair Emeritus in the Allen School. “He is an engineer, an MBA, a lawyer, and a role model.”

The College formally honored the 2020 Diamond Award recipients in a virtual celebration last week after having postponed the event due to the pandemic.

Congratulations to Allen, Mitchell and Armon, and thank you for your continued friendship to the Allen School and our students!

Read more →

Shayan Oveis Gharan receives EATCS Presburger Award for groundbreaking contributions to the Traveling Salesperson Problem

Portrait of Shayan Oveis Gharan

Professor Shayan Oveis Gharan, a member of the Allen School’s Theory of Computation group, earned the 2021 Presburger Award for Young Scientists from the European Association for Theoretical Computer Science (EATCS) for his research on the Traveling Salesperson Problem (TSP). Each year, the EATCS bestows the Presburger Award on an early-career scientist who has made outstanding contributions in the field of theoretical computer science. In its unanimous selection of Oveis Gharan for this year’s honor, the award committee heralded his “creative, profound, and ambitious” work on a fundamental problem that has advanced scientists’ understanding of the design and analysis of algorithms.

“Shayan is a leader in the application of algebraic and spectral methods to classical problems in combinatorial optimization, and he’s the architect of a series of surprising and profound developments in the theory of algorithms,” said Allen School professor James Lee. “He exhibits a remarkably consistent ability to make progress on important problems that had remained open for decades.”

That progress began when Oveis Gharan was a Ph.D. student at Stanford University, where he and his collaborators produced an approximation algorithm that offered the first asymptotic improvement on TSP in the asymmetric case in three decades. It has culminated — so far, at least — in the first performance improvement on metric TSP in nearly half a century. In between, Oveis Gharan also contributed to the first improvement over Christofides’ 3/2-approximation for the symmetric graph case of TSP, first put forward in 1976, in what the Presburger Award committee describes as a “remarkable tour de force.” Over the course of his career, Oveis Gharan has continued to develop and expand the concept of “negative dependence” between the presence of edges in a certain distribution on random spanning trees of a graph — a tool he first applied to great effect in his initial contribution to TSP as a student — to push the field forward. 

For his latest milestone, Oveis Gharan worked with Allen School Ph.D. student Nathan Klein and faculty colleague Anna Karlin to devise an approximation algorithm capable of returning a solution that surpasses 50% of the optimum for the very first time. The team will receive a Best Paper Award at the Association for Computing Machinery’s upcoming Symposium on the Theory of Computing (STOC 2021) for their groundbreaking achievement. According to Karlin, Oveis Gharan stands out not only for his technical contributions, but also for the way he approaches his research.

“Shayan is an exceptionally brilliant, ambitious and fearless researcher,” said Karlin. “What blows my mind is that, on top of all of that, he is also one of the kindest, most generous people I know. Every meeting with Shayan and our students lifts my spirits because he brings such enthusiasm, warmth and positivity to his work.”

The central question of TSP — how to determine the shortest and most efficient route between multiple destinations and back to the starting point — is more than a theoretical problem. It has multiple real-world applications across a variety of domains, from planning and scheduling, to supply chain logistics, to microchip manufacturing. It also has provided an ideal vehicle for Oveis Gharan to apply his expertise in analysis, probability and combinatorics to push the theoretical limits of computation and enable progress in other fields while providing the inspiration and the tools for other young researchers to follow his lead.

“Shayan’s enthusiasm for what he does is infectious, and he has helped me gain a new appreciation of computer science. He communicates to his students a strong sense that we are not here just to solve problems but to learn, grow, and discover new ideas,” Klein said. “He also has a great ability to zoom out and synthesize. For example, during the TSP project I came to him with a mess of proofs of a dozen probabilistic lemmas we needed. He immediately recognized a common theme and extracted an elegant theorem that characterized all of these lemmas and more. This theorem ended up being quite useful in guiding our understanding for the remainder of the project.”

While the challenge of TSP may hold particular fascination for Oveis Gharan, it is not the only open problem on which he has made notable progress in recent years. In 2019, he earned a STOC Best Paper Award for his work with Allen School Ph.D. student Kuikui Liu and collaborators on the first fully polynomial randomized approximation scheme (FPRAS) for counting the bases of a matroid. Drawing from several seemingly unrelated areas of mathematics and theoretical computer science — namely Hodge theory for combinatorial geometries, analysis of Markov chains, and high dimensional expanders — the team applied a novel theory of spectral negative dependence to prove a conjecture by Mihail and Vazirani that had remained an open question for 30 years. As a follow-up, members of that team applied some of those same insights to sampling an independent set from the hardcore model. Their results addressed a 25-year-old open problem concerning the mixing time of Glauber dynamics by proving that, for any graph, they mix in polynomial time up to the tree uniqueness threshold.

Since his arrival at the University of Washington in 2015, Oveis Gharan has earned a Sloan Research Fellowship, a CAREER Award from the National Science Foundation, an ONR Young Investigator Award from the Office of Naval Research, and a Google Faculty Research Award. In 2016, Science News magazine named him one of “10 Scientists to Watch” for his work on TSP. Oveis Gharan will collect his latest honor virtually during the upcoming annual meeting of the EATCS, the International Colloquium on Automata, Languages and Programming (ICALP 2021), in July.

Read the EATCS announcement here, and learn more about the Presburger Award here.

Congratulations, Shayan!

Read more →

Allen School and bioengineering senior Parker Ruth awarded College of Engineering Dean’s Medal

Parker Ruth

Parker Ruth, a senior who will graduate this spring with bachelor’s degrees in computer engineering and bioengineering, has been awarded the College of Engineering’s Dean’s Medal for Academic Excellence. Each year, the college selects two students to receive this award in recognition of their academic achievement, leadership, and engagement in research and extra-curricular activities. Ruth exemplifies these qualities with departmental honors in both of his majors, his contributions to multiple research projects, his leadership roles and his work as an instructor. 

“Parker is a gifted student with remarkable compassion and intellect,” said Magdalena Balazinska, professor and director of the Allen School. “Parker shines inside and outside of the classroom and is without a doubt one of the finest students that the Allen School has ever seen, and he is incredibly selfless and humble.”

In addition to pursuing a dual degree, Ruth is a student in the University Honors Program and the Lavin Entrepreneurship Program. The combination of his two majors has enabled him to build skills in biosignal processing, embedded systems, circuit design and fabrication. He seeks to improve the quality and accessibility of healthcare and has already been doing so as an undergraduate, working in the UbiComp Lab with Allen School professor Shwetak Patel

“Parker’s research has looked at developing new health sensing solutions that are easy to deploy outside of the clinical setting including new ways to measure pulse transit time, stroke screening, osteoporosis screening, and supporting rehab — to name just a few,” Patel said. “His background and training in bioengineering coupled with computer science gives him both a strong intuition on the biological underpinnings of these projects while having exceptional computational, software, and hardware knowledge to research, build and deploy these unique solutions. Parker has demonstrated unprecedented work ethic, creativity, rigor, and a strong ability to present his work to a general audience.”

Ruth has supported research on more than 15 projects in the UbiComp Lab, multiple projects with bioengineering professor Barry Lutz, synthetic biology lab automation with electrical & computer engineering professor and chair Eric Klavins, and several other projects outside of either lab. From a mobile heart monitor prototype that takes electrocardiograms, to VaderNet, a deep neural network that can classify Star Wars characters, Ruth’s side projects are impressive on their own, but with Patel and Lutz, his research exceeds what is expected of most undergraduates.

In the Ubicomp Lab, Ruth built a smartphone app called OsteoApp that can screen for osteoporosis. The app, which works with Android, infers bone density from the resonant frequency of the tibia as measured by the smartphone’s accelerometer. With Lutz, Ruth built image processing algorithms to quantify results from rapid diagnostic tests. He first implemented his work to help improve rapid HIV diagnosis with 99% accuracy. He then switched gears during the pandemic and adapted his algorithm to quantify fluorescence results from point-of-care SARS-CoV-2 assays — expanding access to testing in low-resources settings.

Outside of research, Ruth created the Quantum Spot Academy for people interested in exploring physics. For several years he also worked as a teaching assistant in Allen School and Bioengineering courses and has served on Bioengineering’s curriculum committee since the fall of 2018. 

Ruth’s work ethic, talent, and initiative illustrates his willingness to work hard as a leader. He founded the Bioengineering Journal Club, where students share their passions for cutting-edge research in bioengineering. From there, he evolved that club into Bioexplore, an outgrowth of the club that increased its scope and scale of events as well as the diversity of the community it serves. He’s also a volunteer with the Pacific Northwest Brain Bee, which inspires high school students to study neuroscience. 

In addition to the Dean’s Medal, Ruth has earned multiple university and external honors. Earlier this year, he received a National Science Foundation Graduate Research Fellowship and was named a finalist in the Computing Research Association Outstanding Undergraduate Researcher Award competition for the second year in a row. Parker also earned a 2021 graduate fellowship from Tau Beta Pi, the engineering honor society. He previously was named a Barry Goldwater Scholar and a member of the Husky 100, which recognizes students who are making the most of their time at the University of Washington.

Ruth will enter the computer science Ph.D. program at Stanford University in the fall, where he will join his sister Kimberly, who graduated from the Allen School last year. Kimberly was a Dean’s Medalist, received a National Research Foundation Graduate Research Fellowship, received a Computing Research Association Outstanding Undergraduate Researcher Award, was named a Barry Goldwater Scholar, and was a member of the Husky 100.

Congratulations, Parker! 

Read more →

GeekWire recognizes Allen School’s Lauren Bricker as STEM Educator of the Year

Portrait of Lauren Bricker

Allen School teaching professor and alumna Lauren Bricker (Ph.D., ’98) received a STEM Educator of the Year Award from GeekWire for her leadership in advancing computer science education. Bricker is one of three local education leaders to be recognized with the award, which GeekWire created this year to honor innovative educators in the Pacific Northwest who are inspiring students to achieve more in the areas of science, technology, engineering and math.

“If Pacific Northwest computer science education was a solar system, Lauren Bricker could arguably play the role of the sun,” GeekWire’s Lisa Stiffler wrote in an article highlighting the honorees. “Her efforts to reach students from K-12 to college have shone a light into far-reaching and diverse stretches of the educational system.”

Bricker’s orbit extends far beyond the classroom to encompass the design of computer science courses for the Washington State Academic Redshirt (STARS) program at the University of Washington, outreach to K-12 classrooms through the UW in the High School and Allen School ambassadors programs, curriculum development for Code.org, and leadership and advocacy in her role as President of the Puget Sound Computer Science Teachers Association (CSTA). According to Bricker’s colleague Dan Grossman, professor and vice director of the Allen School, only a fraction of her role fits into the conventional mold of a UW teaching professor; in fact, he says, Bricker joined the faculty in 2017 in part to be a major conduit between the Allen School and K-12 educators across Washington state.

“Lauren is an unheralded hero of computer science education in the Pacific Northwest, filling a role that improves high-school computing education, improves university computing education, and connects the two in totally unique and crucial ways,” said Grossman. “What’s more, Lauren approaches all of her work through the lens of diversity, equity, and inclusion. To her, access to computing courses is but the starting point, where the real goal is for everyone — particularly those who never saw themselves as computer scientists — to thrive.”

To that end, one of Bricker’s signature contributions has been her leadership of multiple efforts aimed at expanding support at UW for students from low-income, first-generation, and underserved backgrounds to be successful in computer science and other engineering-related disciplines. Bricker developed and teaches the introductory computer science course for the College of Engineering’s STARS program, which supports incoming engineering and computer science students by strengthening academic preparation for core mathematics and science prerequisites as well as building the skills and support systems they need to excel in college-level work. Bricker also works closely with students in the Allen School’s Startup program. Each year, Startup provides ongoing support to incoming freshmen who were admitted directly into the computer science or computer engineering major and who are from underserved communities and/or have had limited programming experience.

As the faculty lead for Startup, Bricker has been instrumental in crafting the curriculum for the program’s intensive, four-week pre-autumn course that immerses students in learning foundational computer science concepts, building their critical thinking and problem solving skills, and preparing them for the transition from high school to college. She also has contributed to workshops designed to reinforce core concepts for students during their freshman year. Bricker, whose own journey to higher education included plans to enroll in medical school before she eventually became “hooked” on computers, is motivated in part by a recognition that many students do not enjoy the same advantages she had growing up.

“I lived in a college town. I went to an academically focused high school. I had access to computers and advanced math classes,” Bricker explained. “I also had access to a once-a-week outreach event to encourage women in the medical fields, and my parents encouraged me in the areas I wanted to study. I realize what a privilege that was and how not everyone gets those privileges. I just want to share the good fortune I had.”

A self-described “geek generator,” Bricker brings a personal touch — not just a pedagogy — that has made a difference for students just starting out in computer science. Said one former Startup student of Bricker’s, “I’d go to office hours sometimes and she’d explain the concept to me in a different way so I could understand it better. I really appreciated having her as my professor, she’s the best I’ve had!”

Another former student who later served as a teaching assistant for Startup noted that Bricker’s willingness to meet students where they are, with lessons and examples that are personally meaningful to them, has been especially important in the context of remote learning during the Covid-19 pandemic.

“I’ve had the opportunity to work with Lauren as a TA during online learning and seen her adapt in a way that keeps students engaged,” sophomore Karman Singh said. “She takes the time to walk through exercises so that everyone can understand. She lets students control their learning by having them participate in interactive activities, lead group projects, and create demonstration videos.”

Group of smiling students posing on concrete stairwell with trees and the Paul G. Allen Center in the background
Students in the Allen School’s 2019 Startup cohort on the UW Seattle campus

According to Singh’s classmates Kashish Aggarwal and Jessica Louie, Bricker helps students do more than grasp the material — she also instills a level of comfort and confidence that they can carry with them throughout the remainder of their time at the UW. 

“I was constantly reminded of how skilled Professor Bricker was in her field, but also how incredible she was with working with students,” recalled Aggarwal. “Always encouraging and working with her students, she always had a smile on her face, never giving up on anyone. She helped me gain more confidence in my abilities and resilience in my work.”

“She is great at breaking down difficult topics to digestible and easy-to-learn parts,” Louie said of Bricker’s teaching style. “She is also super approachable and down to earth, which helped me overcome a fear of speaking with professors. I admire her knowledge and ability to make students feel comfortable around her.”

Startup co-instructor Leslie Ikeda, an academic adviser in the Allen School, credits Bricker with playing a key role in the growth and success of the program since the latter first became involved in 2018.

“Lauren is a fierce innovator and educator, and she is relentless in her commitment to supporting students who have been historically underrepresented in computing,” Ikeda said. “Lauren fosters inquiry and creativity through her curriculum and thinks intentionally about how our Startup students can bring themselves — their identities, experiences, values and interests — into their technical projects. Her enthusiasm and passion for teaching is felt by each of her students, and I am grateful to have had the opportunity to work alongside Lauren and witness her incredible work.”

Sonya Cunningham, executive director of the STARS program, agreed, noting that students are inspired by Bricker in part because they can tell she genuinely cares about them as individuals.

“If a student really wants to learn about computer science, Lauren never gives up on them. She will spend as much extra time as needed to help them succeed,” said Cunningham. “I have seen her do this several times with STARS students who were really struggling. As a result of her patience and willingness to go the extra mile, I’ve witnessed amazing turnarounds in our students’ confidence as well as their enthusiasm and love for computer science.

“By teaching students how to learn, how to embrace change, and how to remain flexible and adaptable, Lauren inspires our students to work hard and bring their best,” Cunningham continued. “Under her guidance, STARS students do really well in the computer science gateway courses. We feel lucky to have Lauren as a part of the STARS team!”

In addition to her teaching and K-12 outreach on behalf of the Allen School, Bricker has co-led multiple teacher training workshops for Code.org and contributed to the development of the organization’s Computer Science Discoveries curriculum currently in use by tens of thousands of middle school students around the world. Before joining the Allen School faculty, Bricker spent 20 years in industry in various software engineering, management and consulting roles and 10 years as a teacher at Seattle’s Lakeside School, where she developed and taught honors-level computer science courses to students in grades 9–12 as well as advanced courses in mobile, web and Arduino development and 3D printing and modeling. While at Lakeside, Bricker also spent three years restructuring and teaching a course that introduced students to programming in the 7th grade. 

Bricker herself has learned some valuable lessons over the years when it comes to strategies for engaging more underrepresented students in STEM education and careers.

“We should ‘decenter’ the curriculum by allowing students’ voices to be heard through projects that engage with their own interests. Moreover, make that curriculum relevant by connecting it to real-world problems,” Bricker explained. “We also need to expand representation in the classroom. It’s critically important that we encourage teachers from diverse backgrounds to learn how to teach STEM fields and to feel confident in teaching them.”

Bricker and fellow STEM Educator of the Year honorees Cathi Rodgveller and Kim Williams will be recognized during the 2021 GeekWire Awards celebration on May 20th. Rodgveller founded the organization IGNITE Worldwide to engage more women and girls in STEM education and careers. Williams is a science teacher who serves as the department head and Science Club faculty advisor at Cougar Mountain Middle School in the Bethel School District.

“I am grateful to know Lauren as a student, teaching assistant, and as someone I can connect with at any time,” Singh said. “She puts so much effort into her teaching and making sure all students can understand various concepts. She takes inclusion, equity, individual learning, and growth mindset to heart.

“I can honestly say that she is one of the best teachers I have met.”

Read more about the STEM Educator of the Year Awards and listen to a podcast featuring the honorees discussing the future of learning courtesy of GeekWire.

Way to go, Lauren!

Read more →

Allen School professors Emina Torlak and Xi Wang receive Amazon Research Award for their advancements in critical infrastructure software

Photos of Wang and Torlak
Wang and Torlak

Allen School professors Emina Torlak and Xi Wang, co-founders of the UNSAT group, have received an Amazon Research Award for their work in automated verification for critical infrastructure software. 

The award will support Torlak and Wang’s continued work in applying cutting edge verification and synthesis technology to critical infrastructure software. Their most recent project, Jitterbug, is a tool for writing and proving the correctness of just-in-time (JIT) compilers for computers using the extended Berkeley Packet Filter (eBPF) technology. It provides a formal correctness guarantee for eBPF JITs, a security-critical component of the Linux operating system, through a precise specification of JIT correctness and an automated verification strategy that scales to practical implementations. 

“Compiler bugs can be very costly in this setting,” Torlak said, “so it’s important to catch them early.” 

Jitterbug builds on two of UNSAT’s existing projects: Serval, a framework for creating automated verifiers for systems software, and Rosette, a solver-aided programming language for synthesis and verification. The team has used the tool to find more than 30 new bugs in a number of deployed eBPF JITs and to develop new optimizations. With support from the Amazon award, the group aims to improve the usability of Jitterbug and make the integration with development better.

The UNSAT group was formed when Torlak and Wang joined the Allen School in 2014, with Wang’s expertise in operating systems complementing Torlak’s in programming languages and automated reasoning.

“It’s hard for people to get everything right when building software systems,” Wang said. “How do you build correct systems without too much effort? We created UNSAT to pull our resources together to develop solutions that make this process easier.”

Read more about Jitterbug and explore more of the group’s research here. Learn more about the Amazon Research Award program here.

Congratulations, Emina and Xi! 

Read more →

Professor Anna Karlin elected to the National Academy of Sciences

Portrait of Anna Karlin

Professor Anna Karlin of the University of Washington’s Theory of Computation group recently became the first Allen School faculty member to be elected a member of the National Academy of Sciences. Karlin, who holds the Bill & Melinda Gates Chair in Computer Science & Engineering and serves as Associate Director for Graduate Studies at the Allen School, was honored for her significant contributions to algorithms and algorithmic game theory. She joins a distinguished community of scholars elected by their peers to advise the nation on matters related to science and technology while promoting education and research.

Much of Karlin’s early research was focused on online and probabilistic algorithms and analysis. Online algorithms receive a series of inputs over time, and must make decisions as each input arrives, without the benefit of information about future inputs. Karlin and her collaborators Mark Manasse, Larry Rudolph and Daniel Sleator coined the phrase “competitive algorithm” to describe an online algorithm that achieves the provably best possible performance compared to the clairvoyant optimal result — in other words, performs nearly as well as if decisions were made with full knowledge of the future. 

A useful abstraction and simple illustrative example is the ski rental problem. “When you start out skiing, you don’t know if you’ll love it or hate it or how many times you will end up going,” Karlin explained. “And yet, each time you head out, you have to make a decision: should you rent or should you buy?

“If you rent until the total amount you’ve spent equals the cost of buying skis,” she went on, “you will never pay more than twice what the optimal clairvoyant would have paid. More interesting is that using randomization reduces this factor of 2 down to e/(e-1), which is about 1.58. And it turns out that both of these bounds are provably optimal.”

Karlin and various co-authors applied this style of analysis to achieve optimally competitive algorithms for a number of problems including memory management, load balancing, and transmission control protocol (TCP) acknowledgement. 

Her research on paging and caching naturally led her into collaborations with systems researchers on various other memory management problems. For example, in work with Pei Cao, Edward Felten (Ph.D., ‘93) and Kai Li that received a Best Paper Award at the 1995 ACM SIGMETRICS Conference, she studied optimal integrated strategies for prefetching and caching. The team complemented its theoretical analysis with simulations showing that the new strategies proposed were able to reduce the running time of applications by up to 50%. Karlin and her student Tracy Kimbrel (Ph.D., ‘97) subsequently extended the theoretical results to prefetching and caching from multiple parallel disks. In another example, Karlin worked with Alec Wolman (Ph.D., ‘02), Geoff Voelker (Ph.D., ‘00), Nitin Sharma (M.S., ‘98), Neal Cardwell (M.S., ‘00) and faculty colleague Hank Levy to demonstrate the limitations of web proxy cache sharing among organizations.

A favorite topic of Karlin’s throughout her career has been probabilistic algorithms and analysis. One of her most impactful results in this area emerged from a collaboration with Yossi Azar, Andrei Broder and Eli Upfal on “Balanced Allocations,” which has come to be known as the “power of two choices.” This paper considers a basic “balls-in-bins” problem: It has long been known that when n balls are tossed one at a time into n bins, independently and uniformly at random, the expected value of the maximum load is about log(n)/loglog(n). Karlin and her co-authors showed that if, when placing each of the n balls into a bin, two random bins are selected and then the ball is placed in the less loaded bin, this maximum load drops to loglog(n) — an exponential improvement. 

“Balls-in-bins problems are ubiquitous in computer science,” said Karlin, “and the ‘power of two choices’ paradigm has turned out to be useful in a variety of applications, from hashing and load balancing to network and data center management.”

In 1999, Karlin became excited by work on “Competitive Auctions and Digital Goods” that her student Jason Hartline (Ph.D., ‘03) had done with Andrew Goldberg and Andrew Wright during a summer internship at InterTrust Technologies’ STAR Lab. That work, which recently earned the 2021 SIGECOM Test of Time Award for “initiating a long and fruitful line of work in approximately revenue-optimal auction design in prior free settings,” inspired Karlin to turn her attention to auction theory and, more generally, the field of mechanism design. Mechanism design applies the tools of game theory to design systems in which rational participants intent on maximizing their own self-interest will also achieve the designer’s intended goal. Potential applications run the gamut from network traffic routing and scheduling tasks in the cloud, to ecommerce and search engine advertising — all of them enabled by the rise of the internet.

“Nowadays, pretty much any real algorithmic problem that arises in a distributed setting is in fact a mechanism design problem,” Karlin noted.

The implications of Karlin’s work in this area extends to multiple industries in which the provision of goods and services depends on finding the ideal balance between pricing and profit given available resources and consumer motivations. For example, in joint work with Shuchi Chawla, Nikhil Devanur and Balasubramanian Sivan, she considered the design of “pay-per-play” pricing of songs, apps, video games and other software for consumers whose value for such a digital good evolves as they use it according to a stochastic process known as a martingale. In contrast to the standard approach to selling goods, where the buyer purchases an item and then can use it an unlimited number of times, in this form of pricing the buyer pays a little bit each time they want to use it. This enables a seller to extract different amounts of money from different consumer types based on how long and to what extent they remain interested in the product, as opposed to the one-price-fits-all model. 

“We showed that using a free trial period followed by a fixed posted price for each use yields near-optimal profit for such digital goods — and one which is, generally speaking, significantly higher than that obtained using other pricing methods,” Karlin said. “Pay-per-play has the added advantage of being risk-free to the consumer, since they can stop buying that app, game or song whenever they lose interest.”

In another paper, Karlin worked with Kira Goldner (Ph.D. ‘19), Amos Fiat and Elias Koutsoupias to solve the “FedEx problem,” wherein the seller has to balance maximizing revenue with customer expectations related to service quality and cost. Sellers tend to have a variety of options available to them for responding to price sensitivity and other customer-driven factors, including market segmentation, differential pricing, or lower-cost (and presumably lower-quality) versions of a product or service. Karlin and her collaborators came up with the optimal approach for the seller to extract maximum revenue in instances where customer values such as desired delivery time or other factors are drawn from known probability distributions.

Despite its name, like many of the questions Karlin has tackled in her career, the results on the FedEx problem are more broadly applicable to a variety of domains. This also holds true for her work on novel algorithmic mechanisms for maximizing social welfare in the pricing of goods with the prospect of resale with Goldner, Fiat, Alon Eden and Michal Feldman. In this case, the team addressed a scenario known as the interdependent value setting, which captures the fact that information one prospective buyer has about goods being sold may significantly affect the value other buyers have for those goods. The researchers used the example of auctioning off oil drilling rights on a piece of land, where geologic surveys done by one prospective buyer contain information that could significantly sway the interests of another prospective buyer. 

Anna Karlin and Kira Goldner, the latter wearing Ph.D. regalia
Karlin with newly-minted Ph.D. Kira Goldner at the Allen School’s 2019 graduation celebration

The decades-old economics literature on this problem had yielded strong impossibility results, even in extremely simple scenarios such as selling a single item. Karlin and her co-authors were able to circumvent these negative results using a combination of randomization and approximation, obtaining the first positive results for a broad class of combinatorial auctions — and earning the award for Best Paper with a Student Lead Author at the 20th Conference on Economics and Computation (EC ‘19)

Other problems Karlin has considered include pricing and resource allocation in the cloud, sponsored search auction design and analysis, minimizing cost in procurement auctions such as “buying” a shortest path in a network or a spanning tree, strategic behavior of Bitcoin miners, and the design of auctions that achieve near-optimal performance in the presence of a Bayesian prior without actually knowing the prior. Inspired by the Mechanism Design for Social Good initiative, initially founded by her former student Goldner and Rediet Abebe, going forward, Karlin aims to build on past work and apply her expertise in mechanism design and modern matching markets to domains such as health care, energy, and the gig economy.

Along with advancing new algorithms for mechanism design, Karlin is also interested in pushing the field forward by addressing long-standing problems that underpin the theory and practice of computing. In what is likely to be the most influential result of her career, last year she worked with current Ph.D. student Nathan Klein and faculty colleague Shayan Oveis Gharan to achieve the first improved approximation algorithm for the Traveling Salesperson Problem (TSP) in over 40 years. TSP — which aims to find the most efficient route across multiple points and back to the start — is a fundamental question with real-world implications ranging from transportation and logistics, to genomics and circuit board design. TSP is also a cornerstone problem in theoretical computer science, the study of which has led to the development of many foundational algorithmic techniques.

“TSP is NP-complete, so the holy grail has been to design an efficient approximation algorithm that is guaranteed to get a nearly optimal solution,” Karlin explained. “In the 70s, researchers discovered an algorithm that always guarantees a solution whose cost is never more than 50% higher than optimum. Nathan, Shayan and I were finally able to surpass the 50% threshold, which hopefully will pave the way for even greater progress down the road.”

According to Oveis Gharan, Karlin stands out not only for being a gifted theoretician but also for her eagerness to embrace unfamiliar ideas and learn alongside her students.

“I have worked with a few senior researchers in my career, but Anna is perhaps the only one who gets so excited and spends a significant amount of time learning new ideas and new techniques outside of her comfort zone,” Oveis Gharan said. “This is a great motivation for graduate students who are just starting out to learn new techniques together with their advisor and to build their self-confidence as researchers.”

Karlin’s enthusiasm for helping students to explore new ideas inspired her to co-author the book Game Theory, Alive with mathematician Yuval Peres. Published by the American Mathematical Society in 2017, the book presents the mathematical concepts that underpin game theory in an approachable and engaging way, including anecdotes, illustrations, and profiles of the individual mathematicians, statisticians, and economists credited with advancing the field.

“Game theory’s influence is felt in a wide range of disciplines, and the authors deliver masterfully on the challenge of presenting both the breadth and coherence of its underlying world-view,” Cornell professor Jon Kleinberg wrote. “The book achieves a remarkable synthesis, introducing the reader to the blend of economic insight, mathematical elegance, scientific impact, and counter-intuitive punch that characterizes game theory as a field.”

Cover of Game Theory, Alive!

Karlin’s research career began at Stanford University, where she earned her bachelor’s degree in applied mathematics before earning her Ph.D. in computer science. Following a postdoc at Princeton University, Karlin spent six years as a principal scientist at DEC’s Systems Research Center. While it became clear early on that she would be a “rock star” in the field of computing, Karlin was also an actual rock star wannabe; in 1993, she took part in the first musical performance to be live-streamed on the internet with the band Severe Tire Damage — although, as Karlin herself says with a smile, “just because we were first doesn’t mean we were any good.” 

Shortly thereafter, she left Palo Alto for Seattle to join the UW as a visiting professor in 1994. She made the move permanent two years later.

“I’m so honored to receive this recognition,” said Karlin. “All I can say is that my good fortune is due to the many brilliant colleagues and students I’ve had the pleasure of collaborating with during my career.”

Karlin is one of 120 leading scholars and scientists to be elected full members of the NAS and among a record-high 59 women recognized in the 2021 class. The Academy elected another 30 non-voting international members, bringing the total honorees to 150.

“We are incredibly proud of Anna for being elected to the National Academy of Sciences,” said Magdalena Balazinska, professor and director of the Allen School. “Anna is a tremendously talented researcher. She’s also a dedicated colleague and deeply caring teacher and mentor. She’s an inspiration to all of us.”

Karlin was previously elected Fellow of the American Academy of Arts & Sciences and of the Association for Computing Machinery. She is the fifth Allen School faculty member to be elected a member of the National Academies, joining professors Tom Anderson, Ed Lazowska, and Hank Levy and professor emeritus Susan Eggers — all of whom were inducted into the National Academy of Engineering.

Read the NAS announcement here.

Congratulations, Anna!

Read more →

« Newer PostsOlder Posts »