Allen School News

It can feel lonely being the first in your family to pursue a four-year degree.

How do you apply? How will you pay for it? What major should you choose? How will you navigate your new surroundings, not to mention make new friends? If you run into difficulty, where do you turn for help?

And what are “office hours,” anyway?

Nearly one-third of the more than 43,000 undergraduates enrolled at the University of Washington are first-generation. So while it may seem lonely at times, they are not alone. To remind them of this fact — and to remind everyone at UW of the many ways in which first-gen students enrich our campus community — each year on November 8th the University participates in the National First-Generation College Celebration. To highlight the Allen School’s diverse first-gen community, we asked students to share what it means to them to “be the first” and any wisdom they have for those who have yet to embark on their first-gen journey.

Zander Brumbaugh: Turning a hobby into an opportunity for connection and empowerment

While Zander Brumbaugh knew from a young age that he wanted to be a scientist, he didn’t know he wanted to be a computer scientist. Growing up in Tumwater, Washington, Brumbaugh turned his fascination with the inner workings of various systems into a hobby making video games in high school and, eventually, the beginnings of a career in computing. Currently pursuing his master’s degree in the Allen School’s combined B.S./M.S. program, Brumbaugh has refined his career goals to focus on artificial intelligence research — an “immensely important field” in which he hopes to make a positive impact. To start, he is writing a book on how to adapt and use language models effectively for specific needs as a way of promoting public literacy around these rapidly emerging technologies.

What does it mean to you and/or your family to be the first to pursue a bachelor’s degree?

My family has always been very supportive of the work that I do and my decision to pursue higher education. Both of my parents were unable to attend college due to financial limitations. Because of the scholarships I received, I was able to overcome this and be the first in my family to earn a degree, for which I am eternally grateful. In short, my degree is a source of empowerment; it gives me the ability to create new opportunities for myself and to connect with like-minded individuals who share similar goals.

What has been the most challenging aspect of being a first-gen student?

College is quite different from high school or anything else most students are likely to have encountered in their early academic careers. Finding the groove in my first few quarters wasn’t easy, especially with the start of the COVID-19 pandemic less than halfway into my first year. I made a group of close friends who came from many different backgrounds, and I found my experience improved greatly. Being a first-gen student, I didn’t have anyone at first to help me navigate student life, but we ultimately found our way through it together. 

And the most rewarding?

By far the most rewarding part is simply being able to call home and tell my parents what I’ve been up to. Both of my parents are retired and enjoy hearing the details of my classes, activities with friends, and my research — though they say most of it goes over their heads! My father is my biggest promoter; oftentimes I’ll receive messages online from people who work as cashiers at stores or waitstaff at restaurants whom he’s cheerfully told about my books and games. My parents’ pride inspires me to be the best version of myself — and also thankful for the opportunities I’ve been given, knowing it’s something they didn’t have.

What motivated you to continue on and get your master’s?

As it was always my goal to become a researcher, I began looking for undergraduate research opportunities during my sophomore year. I first worked in AI for vision and language for creative applications and eventually found intersections with robotics that greatly interested me. I joined the ARK lab led by professor Noah Smith during my senior year and started my journey with natural language processing (NLP) research. Wanting to continue my research and eventually pursue a Ph.D. in the future, I applied to the B.S./M.S. program and was accepted. So far, the experience has been everything I imagined; the program provides an environment where I’m immersed in intriguing research, exchanging ideas with others both in and outside of my field and developing projects across various topics.

What advice would you give to other first-gen students?

Heading off for college is an exciting time, full of new experiences. Even if you have family or friends who went to college, it can be difficult to find advice on how exactly you should be approaching different problems — and if you don’t, it can be even more so. While everyone’s experience may be different, finding even a small, close group of friends can help to make a support system. You can help each other navigate your classes, work, or simply your social lives. I would encourage you to check out a club meeting, be outgoing whenever you can, and try to look for others with whom you might share something in common (or not!). There are also mentorship programs offered by multiple groups affiliated with the Allen School that may be helpful in getting you started.

Daniel Campos Zamora: Making a career out of making change and helping people at scale

Daniel Campos Zamora followed what he calls a “long and winding road” to computer science that extends back to his birthplace of Costa Rica. Growing up in New Jersey, Campos Zamora had always been interested in making interesting things; he just never considered making a career out of it. That changed after he began an interdisciplinary degree in psychology and art at Carnegie Mellon University. There, he discovered programming tools like Arduino and Processing that made him realize how powerful computing could be as a medium for change. After earning his bachelor’s, Campos Zamora worked for a professor of human-computer interaction, and later, for Disney Research; that combination of experiences caused him to realize that he wanted to do HCI research himself. The road eventually led him to pursue a Ph.D. in the Allen School’s Makeability Lab working professor Jon Froehlich — and to tap into his first-gen experience in his roles as a reviewer for the school’s Pre-Application Mentorship Service (PAMS) and faculty recruiting.

What does it mean to you and/or your family to be the first to pursue a bachelor’s degree?

I was raised by a single mom who immigrated to the U.S. because she had high hopes for us to get an education and get ahead. She always instilled in me and my siblings that she wanted us to go to college, but she didn’t really understand what that entailed. College didn’t really feel real to us; no one in our family had gone to college, and we didn’t know a lot of people in this country who had college degrees. My mom maybe took one class back in Costa Rica before she had to drop out to have my brother. So it meant the world to her that her three kids were able to get degrees. After graduating, I framed my diploma and gave it to her as a Christmas gift, because I knew how much it meant to her. And if you don’t know, the CMU diploma is gigantic!

What was the most challenging aspect of being a first-gen student?

I think the whole experience of college is really different when you’re first generation. I was the only one of us to move away for college and be away from family and live in the dorms. When you do that, you don’t have a support system, and you don’t know what kind of support is available at school. You don’t know about office hours; you may know they exist, but you don’t know what they actually mean, and you don’t know what any of the offices on campus mean or do. You don’t know what you don’t know yet — you’re dealing with “unknown unknowns.” I struggled because I didn’t identify with a lot of people at my institution, and it was really hard finding support.

How did you navigate those “unknown unknowns”?

I went back to what I knew: I can just work on the classes and do my best. I think I stumbled through some of the other parts, like eating, taking care of yourself, your mental health. You don’t realize, when you’re away from your family, it requires extra work to do that. The saving grace for me was that I met someone on my floor who has a similar background but who knew the school better after doing a summer program there. Through him I got involved in a minority organization and found a support system. And they led me to the campus resource center that assists minoritized students, including first-gen and low income. Through their counselors, I got a lot of support — but it took two years before I even knew that office was there.

What was the most rewarding aspect of being a first-gen student?

I feel like I’m in a much better place to help younger family members thinking about college to understand what it actually means to go down this path. I can take them to look at colleges and help them understand the processes and that there is so much more to the college experience than just getting good grades.

For me personally, what was most rewarding was being exposed to really talented people and really exciting ideas, and to be able to take advantage of resources once I knew they were available. I think it opened up a lot of doors for me, and I would not be at the Allen School if it was not for that experience at CMU. Also, the friends and connections that I made there — I know I have those relationships for a lifetime.

What advice would you give to aspiring first-gen students?

Advocate for yourself, but also be able to admit that you don’t know stuff. I feel that when you get to college, the imposter syndrome — that feeling like, “I don’t belong here” — is so aggravated because you are the first one. So I think it’s knowing that you do belong, but you might need help. Also, usually people who make it to these schools have done well academically, and they might not have struggled too much up until that point. And because so much importance is placed on going to college and getting good grades, when you do run into roadblocks, it’s so disorienting and discouraging.

It takes a lot of courage to acknowledge that you’re struggling and to ask for help. I think that’s very tough for first-gen students who may not even know that they are struggling. I wish someone had told me that it’s okay to ask for help. I’ve had to ask for help here doing my Ph.D. — I’m the first in my family to go to grad school, so that’s a totally new thing. The more you ask for help, and the earlier you do that, the better off you’ll be.

Ha Vi Duong: Choosing her own path while embracing the power of creative problem solving

As a high school student in Moses Lake, Washington, Ha Vi Duong envisioned a career in medicine. While she soon realized that she wanted to do something else with the rest of her life, she wasn’t sure what that something was. A conversation with an advisor — and an encounter with programming through Girls Who Code — helped her see how computer science would allow her to exercise her creativity while solving real-world problems. Duong entered the Allen School as part of the 2021-22 cohort of Allen Scholars. She later took on the role of chair of GEN1, a student group dedicated to empowering and guiding first-gen students, and joined the Vietnamese Student Association’s VSAUW Dance Team. Throughout her time at UW, she has been determined to work hard not just for her own future, but also for that of her parents — in appreciation for the sacrifices they’ve made.

What does it mean to you and your family to be among the first to pursue a bachelor’s degree?

My parents always emphasized the importance of education and believed it should be a top priority in life. I remember them sharing stories about their own experiences when they had to help provide for their families instead of continuing their education. For them, education didn’t always come first. They made the selfless decision to come to America in search of a better life, especially for their children. They’ve worked tirelessly to make this happen, running a restaurant that demands long hours and hard work. 

One summer, I got a firsthand look at what they go through when I helped out at their restaurant. It was an eye-opener, and I gained a deep appreciation for what my parents do every day. When I talked to them about how tough it is, they told me something that has stuck with me ever since: “You should work hard to not have a job like ours. We didn’t know what else to do.” It made me realize how lucky I am. I have the chance to pursue higher education, to explore many career options, and choose my own path. This awareness has made me incredibly grateful.

What has been the most challenging aspect of being a first-gen student at the Allen School?

The most challenging aspect has been dealing with imposter syndrome. The rigorous coursework often makes it feel as though I’m behind compared to my peers. To this day, I still can’t believe that I am able to be where I am. However, amidst this struggle, I’ve discovered a valuable support system within the Allen School community and an understanding that we are all on our own paths and are here for a reason, which has been key in overcoming these challenges. 

What about the most rewarding?

The most rewarding part of my experience has been the sense of accomplishment that comes from successfully completing those demanding courses. It’s immensely satisfying to see the progress I’ve made and to be able to connect the material learned in one class to another and eventually apply this knowledge in the real world. This interconnectedness between academic learning and practical application makes the educational journey at the Allen School both challenging and deeply fulfilling.

Any advice for other first-gen students at UW?

For first-gen students, it’s essential to remember that UW offers fantastic programs and a wide range of groups and organizations. While it may initially feel overwhelming, it’s all about the effort you put into discovering resources and building connections to support you in your college journey. Don’t hesitate to step out of your comfort zone and make connections; you never know where it might lead! It’s normal to feel a bit lost, but remember that everyone has their unique path and pace.

What are you hoping to do after graduation?

I’m currently in the process of exploring my post-graduation options, and I believe that finding a career where I can witness the tangible impact of my work is crucial. While I don’t have a specific plan in place just yet, I’m actively seeking opportunities that align with my interests and values. My goal is to pursue a path that allows me to make a meaningful difference in the world and see the results of my efforts come to life.

Derrik Petrin: Rediscovering his love of computing by leaving the lab and entering the arena

Derrik Petrin went all the way to Yale University to earn a bachelor’s degree in biochemistry before he realized that he did not, in fact, enjoy working in a lab. As a middle-school student in Issaquah, Washington, he had taken a programming class and liked it; he also liked the original Magic the Gathering card game by Wizards of the Coast. After he returned to this coast, Petrin eventually parlayed both into a position with the company as a software development engineer after spending some time as a freelance software consultant. Lately, Petrin has been working on the team that produces the digitized version of the game, Magic the Gathering Arena. When he’s not getting paid to play during work hours, he’s advancing his own story arc by pursuing a graduate degree in computer science through the Allen School’s flexible Professional Master’s Program (PMP) — which offered Petrin not only the opportunity to obtain a computer science degree but also to explore what his next chapter might be. He also has connected with his roots through his involvement in the local Hungarian-American Association.

What did it mean to you and/or your family to be among the first to pursue a bachelor’s degree?

It’s funny — I didn’t really start thinking of myself as first-generation until near the end of undergrad. I went to school on the Eastside, on the plateau, and didn’t really appreciate the differences. Some of my aunts and uncles went to college, but my dad’s parents didn’t, my dad didn’t, my mom emigrated from Hungary. So it didn’t really start dawning on me until I was in undergrad, when I realized that all of my classmates’ parents were professionals — lawyers, doctors, engineers and mathematicians — and mine weren’t. I started noticing how my parents didn’t have any advice they could give me. I always thought I would get a degree; I think my parents always expected that, too. So it was hard to imagine me not doing that. 

What was the most challenging aspect of being a first-gen student?

My parents are not academically inclined. By late middle school and high school, I felt really comfortable in an academic setting and was used to planning and deciding everything myself. One of the reasons I ended up going to Yale was that it had the most generous financial aid package. I remember in my senior year seeing a statistic about the percentage of students who received no financial aid at all — and that’s a pretty high income cut-off — and it was a large number of students. And then I noticed that a lot of the classmates that I had formed close friendships with were also on some form of financial aid. So it dawned on me that this stuff tends to organically group us together without us realizing it.

I also had some pretty severe mental health struggles. Yale has had some publicity in recent years about their poor handling of student mental health. So it was not the easiest environment to not have parental support, but also I did not realize that that’s something that was making things more difficult. It was pretty overwhelming.

How has that experience shaped your career path since?

If I had not been first-generation, it’s more likely I would have continued straight into applying to Ph.D. programs. But also after college, I probably would have taken a less winding path than I have taken. And there are some benefits and disadvantages to that. One of the benefits is, when I ended up doing freelance consulting for a while, I dropped out of being in a cohort after spending most of my life in a cohort in an institutional setting. And I got used to being okay with doing things that are not necessarily the typical way to do them. For example, even though the PMP is typically an evening program, during one quarter the programming languages course was taught in parallel with the “normal” morning one. I had the flexibility to take that morning class instead and spend time with the Ph.D. students. One of the professors then invited me to spend time in the Programming Languages & Software Engineering (PLSE) lab, so I again got to interact with Ph.D. students there. It’s put me into a mentality where I think less in terms of a structured path. 

That flexibility is useful, but sometimes it would be nice to have more structure. Another challenge is that if you don’t have this very clear box to show to people, they’re not sure what to make of you. So, for instance, maybe you’re interested in doing research — but people aren’t sure even logistically how that would work. The PMP is not a research program; but at the same time, it’s this great, very broad survey program. So as I’m taking courses in these different areas, I’m thinking about which one sparks the most interest. But one area where the first-gen experience comes in is, I don’t know how to take that and follow through to make an ongoing connection. A lot of students do PMP for professional development, and that’s what it is primarily set up for, but there are also students, like me, who have that intellectual itch and this is the most accessible foot back in the door.

What advice would you give to other first-gen college students?

Something that I think is good advice for undergrads in general is to go to office hours, which as an undergrad, I did not do. Coming back to school and paying for the classes myself — and being really excited about them — face to face time with the professors is so important. Go to office hours even if you are behind on the assignment, or don’t have questions about the assignment, just to listen to what other students are asking. Even if you don’t have anything prepared, some conversation will happen. That has been really helpful for me coming back to the PMP. 

Some people who are first-gen students are very aware of it; it’s part of their identity right from the get-go. But for others, like me, we don’t realize right away how much being first-gen impacts our experience. So keep in mind that you are carrying a lot more weight than other students are. If it seems harder, if it seems things are not coming as easily to you, that’s not surprising. It’s also not your fault, so practice self-compassion.

Nicole Sullivan: Advancing science and sustainability while assisting others in their journey

Nicole Sullivan first began to consider a career in computing-related research as a high school student in Cerritos, California. After enrolling in a computer science course in the 11th grade, she became fascinated with the field’s potential to address environmental challenges such as nature conservation, climate change, agriculture and more. She found further inspiration as a Karsh STEM Scholar and undergraduate researcher at Howard University, an experience she credits with setting her on the path to earning a Ph.D. She followed that path across the country to the UW, where Sullivan is making meaningful contributions to data science and sustainability working alongside professor Magdalena Balazinska in the Allen School’s Ph.D. program. She is also helping to inspire a new generation of researchers by mentoring underrepresented minority students hoping to follow in her footsteps.

What did it mean to you and/or your family to be the first to pursue a bachelor’s degree?

I’m grateful for the opportunity to pursue higher education, which my parents fully support. They were proud of my independence during my undergraduate studies and are ecstatic about my pursuit of a Ph.D.  

What was the most challenging aspect of being a first-gen student, and what was the most rewarding?

While it was overwhelming and challenging to figure out scholarship and college applications on my own, I’ve gained a solid understanding of the process. That has equipped me with valuable insights that enable me to assist others in their journey. 

How have you applied your experience to assist others?

I am currently a graduate mentor for A Vision for Electronic Literacy and Access (AVELA). Before that, while I was at Howard University, I was a National Society of Black Engineers (NSBE) Jr. Mentor and a Microsoft Code Academy (MCA) Lead Learner. As an AVELA mentor, I create and teach original STEM content for Black, Brown, and Indigenous middle and high school students throughout greater Seattle. In NSBE Jr., I supported two teams on their way to the NSBE national robotics competition. Additionally, through MCA, I spent every other weekend teaching programming fundamentals to Black students in kindergarten through 5th grade.

What advice would you give to aspiring first-gen college students?

I highly recommend participating in programs like AVELA and NSBE. Engaging with AVELA, which offers free courses in coding basics, machine learning, hardware, and more, can provide an excellent foundation in various areas of study. Not only will you acquire valuable knowledge, but you’ll also have compelling experiences to highlight in your college application essays and add to your resumes. Furthermore, NSBE extends scholarships to high school students and organizes an annual conference where you might discover field-related opportunities and gain hands-on experience during your high school years. While the NSBE conference isn’t free, consider contacting your high school counselor or local NSBE chapter to explore potential funding options.

What do you hope to do after earning your Ph.D. from the Allen School?

Although I’m not exactly sure what will happen after I graduate, I know I will choose a path either in academia or in industry research. And I intend to continue mentoring underrepresented minority students to foster their enthusiasm for STEM Ph.D. programs and higher education in general.

Learn more about the First Generation College Celebration at UW here.

Read more →

Each year, Google recognizes approximately 75 exceptional graduate students from around the world through its Google Ph.D. Fellowship Program. The students, who come from a variety of backgrounds, are selected based on their potential to influence the future of technology through their research in computer science and related fields. As part of its 2023 class of Fellows, the company selected two future leaders from the Allen School: Miranda Wei in the Security and Privacy category and Mitchell Wortsman in Machine Learning.

Wei joined the Allen School in 2019 to work with professors Tadayoshi Kohno and Franziska Roesner, co-directors of the Security and Privacy Research Lab. Now in the fifth year of her Ph.D., Wei seeks to empower people and mitigate harms from emerging technologies through her dissertation research that explores how factors like gender affect people’s experiences with technological security and privacy. Her work has already contributed to an important new subfield that centers on sociotechnical factors in security and privacy research.

“Miranda’s focus on assessing conditions of disempowerment and empowerment, and then developing mechanisms to help users improve their computer security and privacy, is truly visionary,” said Kohno, who also serves as the Allen School’s Associate Director for Diversity, Equity, Inclusion and Access. “Miranda has not only identified important work to do, but she has identified a strategy and the key components for moving the whole field forward.”

Grounding her research in critical and feminist theories, Wei explores the (dis)empowerment of users through security and privacy measures in several contexts. Her work draws from the social sciences and multiple fields of computer science, including human-computer interaction and information and communication technologies for development in addition to security and privacy. In recent work, Wei has applied both quantitative and qualitative approaches, including case studies and participant interviews, to examine topics such as gender-based stereotypes and computer security and the connection between digital safety and online abuse. 

“This research sets the foundation for learning from experiences of marginalization to understand broader sociotechnical systems,” explained Wei. “This enables equitable improvements to security and privacy for all online users.”

Wei’s academic journey began as an undergraduate student at the University of Chicago, where she earned a degree in political science with a minor in computer science. In addition to having published over a dozen peer-reviewed papers, Wei volunteers her time to support new and prospective graduate students through the Allen School’s Pre-Application Mentorship Service (PAMS) and Care Committee. She is also active with DUB as a student coordinator and participates in the University of Washington’s graduate application review process as an area chair.

“Miranda has great insight for research problems at the intersection of computer security and privacy and society, and she pursues this vision passionately and independently,” said Roesner. “At the same time, she is a wonderful collaborator and community member who looks out and advocates for others.”

Wortsman, who is also in his fifth year at the Allen School, earned a Google Ph.D. Fellowship for his work with professors Ali Farhadi, co-director of the Reasoning, AI and VisioN (RAIVN) Lab and CEO of the Allen Institute for AI, and Ludwig Schmidt, who is also a research scientist in the AllenNLP group at AI2. Wortsman has broad interest in large-scale machine learning spanning deep learning, from robust and accurate fine-tuning to stable and low-precision pre-training. His dissertation work seeks to improve large pre-trained neural networks as reliable foundations in machine learning.

“One of my main research goals is to develop computer vision models that are robust, meaning that their performance is less degraded by changes in the data distribution,” explained Wortsman. “This will enable the creation of models which are useful and reliable outside of their training distribution.”

With the progress in pre-training large-scale neural networks, machine learning practitioners in the not-so-distant future could potentially spend most of their time fine tuning these networks. Wortsman studies the loss landscape of large pretrained models and explores creative solutions for fine tuning with the goal of improving accuracy and robustness. Wortsman wants his models to be useful to society at large and not exclusively for academic and commercial applications. One of his ongoing projects includes a collaboration with the UW School of Medicine.

Wortsman is first author on over nine peer-reviewed publications, several of which he co-authored as a predoctoral young investigator at AI2, and collaborated on the development of an open source reproduction of OpenAI’s CLIP model. He has also served as a teaching assistant in the Allen School and as a reviewer for PAMS.

“Michell’s work has laid the foundations for many open models that let computers understand and generate images,” said Schmidt. “Mitchell is one of the core developers of OpenCLIP, which is downloaded several thousand times per day and has become part of many AI projects. Every time someone uses Stable Diffusion, one of Mitchell’s models provides the text guidance for the image generation process.”

Learn more about the Google Ph.D. Fellowship program here.

Read more →

In the late 1990’s, members of the Allen School faculty experimented with a new — some would say unorthodox — way to mark the conclusion of Visit Days, the annual pilgrimage made by prospective graduate students to computer science programs around the country. To commemorate the visitors’  time in Seattle, professors in what was then the Department of Computer Science & Engineering would cheerfully send them on their way with a surprise parting gift: a palm-sized chunk of concrete.

The concrete in question had, without any human intervention, become dislodged from the crumbling facade of Sieg Hall — the building that, should the recipients choose the University of Washington, would become a home away from home for the duration of their Ph.D. 

“The souvenir definitely made us memorable, and it helped our cause when it came to recruitment,” Allen School professor Ed Lazowska, who chaired the department at the time, recalled wryly. “One student emailed that they just couldn’t say ‘no’ to us after we literally gave them a piece of our building. But giving out chunks of the building, like the building itself, was a joke. We were woefully behind other top computing programs when it came to facilities.”

While outside the building was crumbling, inside it was cramped — so much so that, as a prank, someone set up a “graduate student office” on a ledge in the stairwell, complete with a handy rope ladder for access. More than two decades after it first housed UW’s burgeoning Computer Science & Engineering program, Sieg was no longer fit for the purpose. In 1999, the department stepped up its campaign for a new, permanent home.

Lazowska and local technology industry leaders led the charge, forging a public-private partnership that was unprecedented in UW’s history. All told, they raised $42 million in private funds — substantially more than half the project’s cost — from more than 250 donors. Lazowska’s faculty colleague Hank Levy oversaw the design and construction of the building in tandem with LMN Architects and general contractor M.A. Mortenson. He saw to it that the funds were put to good use.

“Our goal was to create a warm and welcoming environment that would facilitate teaching, research and collaboration,” said Levy. “Every aspect of the building — the materials, the artwork, the abundant natural light, the open spaces that encourage people to gather and exchange ideas — were intentional choices made with this goal in mind.”

Those choices were supported in large part by leadership gifts from the building’s namesake, the late Paul G. Allen, along with the Bill & Melinda Gates Foundation and Microsoft. Completion of the 85,000 square-foot facility, which was dedicated on October 9, 2003, tripled the program’s available space and set off a chain of events that made the Allen School into the powerhouse it is today. 

Allen himself understood at the time that he was investing in something more meaningful than bricks and mortar.

“I’m proud to have supported this beautiful and unique facility, but what really sets UW’s computer science program apart are the people,” Allen observed during the grand opening celebration. “The faculty here is unparalleled, and the undergrad and graduate students are dedicated and inspiring.”

Allen’s faith would inspire a period of expansion that no one — including Lazowska, who has been the program’s most vocal cheerleader over the years — could have foreseen in 2003. 

“I cannot stress enough the importance of the Allen Center to the trajectory of our program,” he said. “It provided us with competitive space for the first time in our history. It was the spark that set us on a path to triple our degree production, ramp up our ability to deliver computer science education to students across campus, and attract the brightest researchers in the field to Seattle.

“And in the midst of all that,” Lazowska added, “we became a full-fledged school!”

On move-in day in the summer of 2003, fewer than 40 faculty members unpacked boxes in their shiny new offices; two decades later, that number is approaching 100. And faculty recruiting has barely kept pace with the explosive growth in student interest, with the Allen School the most requested major among freshman applicants to the University for several years running. It now serves roughly 2,800 students across its degree programs — and thousands more who take one or more courses as non-majors each year.

As the program grew in size, it also grew in stature, thanks in no small part to its new and improved laboratory space.

“Computer Science & Engineering at the University of Washington is an engine of opportunity,” Allen had said at the time, “and I want to ensure it’s an even more cutting-edge resource for the coming generation.”

That engine has been going full throttle ever since. One high-profile example of how the move to the Allen Center greased the wheels of innovation is UW’s emergence as a center for mobile health. By tapping into the built-in sensing capabilities of smartphones coupled with advances in machine learning, Allen School researchers, in conjunction with UW Medicine clinicians, have developed a range of mobile tools for screening and monitoring of a variety of health conditions spanning fever, pre-diabetes, sleep apnea, infant jaundice, reduced lung function, ear infection, newborn hearing loss and more. All got their start in the Allen Center’s labs, and several led to the creation of Allen School spinout companies.

The collaborations don’t stop there, as the Allen Center provided a launch pad for multiple cross-campus initiatives, some supported by significant federal and/or private investment. These include efforts to advance accessible technologies and more accessible communities, data science for discovery and innovation, neurotechnologies for people with spinal cord injury, stroke or other neurological disorders, next-generation cloud computing infrastructure, computing for environmental sustainability and more. In the past five years alone, the Allen School has secured more than $200 million in grants and contracts to support its research. Along the way, the school has strengthened its leadership in core areas such as systems, architecture and theoretical computer science even as it has expanded its expertise to encompass new areas, including cryptography, molecular programming, quantum computing and natural language processing.

And that list is by no means exhaustive. 

“We took Paul’s words to heart, and the impact of the community’s investment continues to be felt today far beyond the Allen Center’s walls,” said Magdalena Balazinska, director of the Allen School and Bill & Melinda Gates Chair in Computer Science & Engineering. “It is felt through the graduates we’ve mentored, the technologies we’ve developed, the companies we’ve started, the opportunities we’ve created, and the leadership we’ve provided.”

The growth sparked by the Allen Center eventually led UW to break new ground in computing literally as well as figuratively; nearly 16 years later, with its first building now bursting at the seams, the Allen School dedicated its second building, the Bill & Melinda Gates Center, which doubled its physical space.

That additional space came just in time, too. Thanks to advocacy by the University and additional investments from the state legislature, the school is currently on track to award 820 degrees annually and has cemented its place in the top echelon of computer science programs in the nation.

“I said back then that the true measure of this building will be what we do inside to take our programs to the next level of excellence,” said Levy. “I’d like to think that we lived up to that promise, and then some.”

For more on the Allen Center’s history, see the Allen Center dedication brochure, a special pre-dedication insert in the Most Significant Bits newsletter, and the dedication issue of MSB from fall 2003.

Read more →

After a two-year hiatus, the University of Washington’s Bonderman Travel Fellows are back, independently traveling the world and benefitting from the monumental growth that comes with immersing oneself in unfamiliar spaces. Since its inception in 1995, the fellowship has supported over 300 UW students on their travels based on their curiosity, openness, resilience and creativity. 

Soon, it will be Allen School Ph.D. student Gus Smith’s turn to hit the road, along with seven other graduate students who were named 2023 Bonderman Fellows. Smith, who is in his fifth year at the Allen School co-advised by professors Luis Ceze and Zach Tatlock, focuses his research on using programming language tools to automatically generate compilers for custom hardware.

With support from the fellowship, he will have an opportunity to explore a different kind of language, far from the computing lab.

“I will use my Bonderman journey to bridge gaps between myself and my international friends and colleagues, not only by experiencing their home countries directly, but by challenging myself to experience the feeling of being an outsider in new and unfamiliar countries,” explained Smith. “In the process, I hope to gain more empathy for what it is like to live so far from your country of origin.”

Inspired by the international and first-generation American friends and colleagues he has encountered during his time at UW, Smith proposed to visit at least six countries over a period of five months following graduation. His itinerary — which is still taking shape — will span the continent of Asia from Taiwan in the east to Israel and Jordan in the west. Other highlights will include two months in India, a brief stay in Singapore and an exploration of Chiang Mai and Bangkok in Thailand. 

Along the way Smith hopes to connect with the cultures of the people he has known to develop a better understanding of their backgrounds. He also anticipates personal growth that will facilitate deeper connections with his friends, fellow computer scientists and other people he meets throughout his life.

“During my travels, I’ll seek to understand how people across the world engage with the search for happiness and how they cope with the knowledge that what they seek may be elusive or impermanent,” said Smith. “If I meet a thousand people, I’ll find a thousand different answers to that question.”

Learn more about the Bonderman Fellowship here. Read more →

On a cool afternoon at the heart of the University of Washington’s campus, autumn, for a few fleeting moments, appears to have arrived early. Tiny golden squares resembling leaves flutter then fall, switching from a frenzied tumble to a graceful descent with a snap. 

Aptly named “microfliers” and inspired by Miura-fold origami, these small robotic devices can fold closed during their descent after being dropped from a drone. This “snapping” action changes the way they disperse and may, in the future, help change the way scientists study agriculture, meteorology, climate change and more. 

“In nature, you see leaves and seeds disperse in just one manner,” said Kyle Johnson, an Allen School Ph.D. student and a first co-author of the paper on the subject published in Science Robotics this month. “What we were able to achieve was a structure that can actually act in two different ways.” 

When open flat, the devices tumble chaotically, mimicking the descent of an elm leaf. When folded closed, they drop in a more stable manner, mirroring how a maple leaf falls from a branch. Through a number of methods — onboard pressure sensor, timer or a Bluetooth signal — the researchers can control when the devices transition from open to closed, and in doing so, manipulate how far they disperse through the air. 

How could they achieve this? By reading between the lines. 

“The Miura-ori origami fold, inspired by geometric patterns found in leaves, enables the creation of structures that can ‘snap’ between a flat and more folded state,” said co-senior author Vikram Iyer, an Allen School professor and co-director of the Computing for the Environment (CS4Env) initiative. “Because it only takes energy to switch between the states, we began exploring this as an energy efficient way to change surface area in mid-air, with the intuition that opening or closing a parachute will change how fast an object falls.”

That energy efficiency is key to being able to operate without batteries and scale down the fliers’ size and weight. Fitted with a battery-free actuator and a solar power-harvesting circuit, microfliers boast energy-saving features not seen in larger and heavier battery-powered counterparts such as drones. Yet they are robust enough to carry sensors for a number of metrics, including temperature, pressure, humidity and altitude. Beyond measuring atmospheric conditions, the researchers say a network of these devices could help paint a picture of crop growth on farmland or detect gas leaks near population centers. 

“This approach opens up a new design space for microfliers by using origami,” said Shyam Gollakota, the Thomas J. Cable Endowed Professor in the Allen School and director of the school’s Mobile Intelligence Lab who was also a co-senior author. “We hope this work is the first step towards a future vision for creating a new class of fliers and flight modalities.”

Weighing less than half a gram, microfliers require less material and cost less than drones. They also offer the ability to go where it’s too dangerous for a human to set foot. 

For instance, Johnson said, microfliers could be deployed when tracking forest fires. Currently, firefighting teams sometimes rappel down to where a fire is spreading. Microfliers could assist in mapping where a fire may be heading and where best to drop a payload of water. Furthermore, the team is working on making more components of the device biodegradable in the case that they can’t be recovered after being released. 

“There’s a good amount of work toward making these circuits more sustainable,” said Vicente Arroyos, another Allen School Ph.D. student and first co-author on the paper. “We can leverage our work on biodegradable materials to make these more sustainable.”

Besides improving sustainability, the researchers also tackled challenges relating to the structure of the device itself. Early prototypes lacked the carbon fiber roots that provide the rigidity needed to prevent accidental transitions between states. 

Collecting maple and elm leaves from outside their lab, the researchers noticed that while their origami structures exhibited the bistability required to change between states, they flexed too easily and didn’t have the venation seen in the found foliage. To gain more fine-grained control, they took another cue from the environment. 

“We looked again to nature to make the faces of the origami flat and rigid, adding a vein-like pattern to the structure using carbon fiber,” Johnson said. “After that modification, we no longer saw a lot of the energy that we input dissipate over the origami’s faces.” 

In total, the researchers estimate that the development of their design took about two years. There’s still room to grow, they added, noting that the current microfliers can only transition from open to closed. They said newer designs, by offering the ability to switch back and forth between states, may offer more precision and flexibility in where and how they’re used. 

During testing, when dropped from an altitude of 40 meters, for instance, the microfliers could disperse up to distances of 98 meters in a light breeze. Further refinements could increase the area of coverage, allowing them to follow more precise trajectories by accounting for variables such as wind and inclement conditions. 

Related to their previous work with dandelion-inspired sensors, the origami microfliers build upon the researchers’ larger goal of creating the internet of bio-inspired things. Whereas the dandelion-inspired devices featured passive flight, reflecting the manner in which dandelion seeds disperse through the wind, the origami microfliers function as complete robotic systems that include actuation to change their shape, active and bi-directional wireless transmission via an onboard radio, and onboard computing and sensing to autonomously trigger shape changes upon reaching a target altitude.

“This design can also accommodate additional sensors and payload due to its size and power harvesting capabilities,” Arroyos said. “It’s exciting to think about the untapped potential for these devices.” 

The future, in other words, is quickly taking shape. 

“Origami is inspired by nature,” Johnson added, smiling. “These patterns are all around us. We just have to look in the right place.”

The project was an interdisciplinary work by an all-UW team. The paper’s co-authors also included Amélie Ferran, a Ph.D. student in the mechanical engineering department, as well as Raul Villanueva, Dennis Yin and Tilboon Elberier, who contributed as undergraduate students studying electrical and computer engineering, and mechanical engineering professors Alberto Aliseda and Sawyer Fuller.

Johnson and Arroyos, who co-founded and currently lead the educational nonprofit AVELA – A Vision for Engineering Literacy & Access, and their teammates have done outreach efforts in Washington state K-12 schools related to the research, including showing students how to create their own bi-stable leaf-out origami structure using a piece of paper. Check out a related demonstration video here, and learn more about the microflier project here and in a related UW News release and GeekWire story.

Read more →

In 2020, a group of researchers unveiled a tool called Face Depixelizer that would take a low-resolution image as an input and, with the help of a generative machine learning model called StyleGAN, produce a high-resolution image in its place. But the model, which was not designed to “fix” the original low-quality image but instead generate an imaginary replacement, had a tendency to predominantly imagine white people — even when the original image depicted someone of another race.

The following year, a group of web developers and accessibility experts signed an open letter urging website owners to avoid using accessibility overlays on their sites. The signatories had become alarmed by the growing reliance on these automated tools, which are marketed under the guise of helping website owners improve the user experience while avoiding potentially costly litigation, when it became apparent that they can actually make the experience worse for people with screen readers — to the point of making a site unusable. To date, nearly 800 individuals have added their names to the letter.

These are just two examples of how technology can have unforeseen, and ostensibly unintended, negative consequences in the real world. Spurred on by these and other cautionary tales, a team of researchers at the Allen School want to assist their colleagues in anticipating and mitigating the consequences of their own work. With support from a five-year institutional transformation grant through the National Science Foundation’s Ethical and Responsible Research (ER2) program, the team hopes their project will usher in a new paradigm in computing-related research not just at the University of Washington, but across the field.

One member of the team, Allen School Ph.D. student Rock Yuren Pang, already had begun thinking about how society increasingly bears the brunt of unintended consequences from new technologies. After enrolling in a graduate-level computer ethics seminar taught by professor Katharina Reinecke, he began to fully appreciate the difficulties researchers face in attempting to understand, let alone mitigate, what those might be.

“Emerging technologies are being used for a growing range of applications that directly impact people’s lives — from how communities are policed, to which job applicants are called for an interview, to what content someone sees online,” Pang said. “As a young Ph.D. student, I thought the question of how we as researchers might think about the downstream impacts of our work to be a really important problem. But I also felt overwhelmed and didn’t know how to even begin tackling it.”

In a new white paper, Pang, Reinecke and Allen School professors Dan Grossman and Tadayoshi Kohno offer a potential starting point. Dubbed PEACE — short for “Proactively Exploring and Addressing Consequences and Ethics” — their proposal offers a vision for empowering researchers to anticipate those consequences “early, often, and across computer science.” 

The latter is important, Reinecke notes; while artificial intelligence may dominate the headlines at the moment, these issues extend throughout the field.

“We can’t just point fingers at AI; every technology, no matter how seemingly benign, has the potential to have undesirable impacts,” said Reinecke, the PI on the NSF grant whose research in the Allen School’s Wildlab includes investigating how people relate to technology differently across languages, cultures and abilities. “When we interviewed researchers across multiple subfields, they generally acknowledged the importance of trying to anticipate the consequences of innovation. But to translate that into practice, they need some scaffolding in place.”

To that end, Reinecke and her co-authors propose a holistic approach that would weave such considerations into the school’s teaching and research while making it easier for researchers to tap into existing resources for assistance in anticipating and mitigating undesirable impacts. Two of the resources the team intends to explore as part of the NSF grant, the Tarot Cards of Tech and guided stakeholder analysis, have Seattle roots. The latter is a pillar of Value Sensitive Design, co-conceived by UW iSchool professor and Allen School adjunct faculty member Batya Friedman, that engages individuals or groups who could be directly or indirectly affected by technology. As part of the process, researchers could save the results of their analysis in the form of a PEACE report that could be shared with collaborators on a particular project and updated anytime.

Researchers will also have the option to share their PEACE reports with an ethics board comprising faculty colleagues from across campus with expertise in areas such as law, bioethics, science and technology studies, and gender, women and sexuality studies. Members of this group will act as a sounding board for researchers who wish to follow up on the results of their exploration — and help them think through how they could address any potential unintended consequences they’ve identified in the process.

As with other elements of the proposed PEACE process, consultation with the ethics board would be entirely voluntary.

“We want to give researchers a low-friction, low-stakes mechanism for seeking diverse perspectives on how a technology might be used or misused. This could help surface potential implications we may not think of on our own, as computer scientists, that can inform how we approach our work,” Reinecke said. “We aren’t saying ‘don’t do this risky piece of research.’ What we’re saying is, ‘here’s a way to anticipate how those risks might manifest’ in order to mitigate potential harm.”

In his role as co-director of the Allen School’s Security and Privacy Research Lab and the Tech Policy Lab at UW, Kohno has had ample opportunity to analyze the harm that can result when researchers haven’t thought ahead.

“Many times during my career have I wondered if the original researchers or developers could have prevented a problem before deployment,“ said Kohno. “For years, I and my colleagues have encouraged the people who build new technologies to apply a security and privacy mindset from the start rather than having to fix vulnerabilities later, after damage has been done. That’s essentially what we’re suggesting here — we’re asking our colleagues to apply a societal mindset, and to front-load it in the research process instead of relying on hindsight, when it may be too late.”

Grossman is vice director of the Allen School and often teaches the undergraduate computer ethics seminar, which the school began offering to students on a quarterly basis in 2020. He sees an opportunity for the PEACE project to eventually transform computing education and research on a massive scale. 

“We are in a position to guide the future leaders of our field toward thinking not only about the technical aspects of computing, important as they are, but also the ethical ones — to train future researchers and technologists how to rigorously consider the potential ramifications socially, politically, environmentally, economically or any combination thereof,” said Grossman. “We need the people who understand their proposed technology to grapple with these issues as well as to learn how to interact with non-technologists, such as public-policy experts, who have complementary expertise.”

The team will deploy and evaluate the PEACE project within the Allen School to start, with plans to extend access to other academic units on campus in later years. Eventually, Pang and his colleagues plan to distill the findings from their evaluation of the UW deployment into detailed design guidelines that can be adapted by other institutions and companies.

“I want to create the go-to place for UW researchers to learn, anticipate and bounce ideas off other researchers about the potential consequences of our work,” Pang said. “But I hope this initiative encourages a broader culture in which computer scientists are unafraid to think critically and openly about these issues. And I believe we can do it in a way that supports, not stifles, innovation.”

Read the team’s white paper here. This work is supported by National Science Foundation award #2315937. Read more →

Allen School professor Dieter Fox will be honored at the 32nd International Joint Conference on Artificial Intelligence (IJCAI) with the 2023 John McCarthy Award. The award is named for the eponymous scientist, widely regarded as one of the founders of the field of artificial intelligence (AI), and recognizes established researchers who have built up a distinguished track record of research excellence in AI. Fox will receive his award this week and give a presentation on his work at the conference held in Macao, S.A.R.

“Receiving the John McCarthy Award is an incredible honor, and I’m very grateful for the truly outstanding students and collaborators I had the pleasure to work with throughout my career,” Fox said. “I also see this award as a recognition of the importance the AI community places on building intelligent systems that operate in the real world.” 

Fox has made a number of key contributions to the fields of AI and robotics, developing powerful machine learning techniques for perception and reasoning, as well as pioneering Bayesian state estimation and the use of depth cameras for robotics and activity recognition. 

His research focuses on systems that can interact with their environment in an intelligent manner. Currently, most robots lack the intelligence to perceive and understand changing environments over time. They move objects in a set, programmable way. In a factory, where conditions are tightly controlled, this is a strength. Everywhere else, it’s a problem.

During his time as a Ph.D. student at the University of Bonn, Fox’s work on Markov localization tackled a fundamental problem in robotics and is now considered a watershed moment for the field. Near the start of the 21st century, researchers concentrated on the problem of tracking, giving a robot a map and its initial location. But these robots lacked true autonomy. They were unable to estimate their location and recover from mistakes out in the field — traits, importantly, displayed by a human pathfinder. 

Fox and his collaborators developed grid-based and sampling-based Bayes filters to estimate a robot’s position and orientation in a metric model of the environment. Their work produced the first approach that allowed a robot to reorient itself and recover from failure in complex and changing conditions. Fox’s pioneering work in robotics touches virtually every successful robot navigation system, be it indoors, outdoors, in the air or on streets. 

Fox’s contributions go beyond core robotics. Using a variety of data sources, including GPS, Wi-Fi signal strength, accelerometers, RFID and geospatial map information, Fox developed and evaluated hierarchical Bayesian state estimation techniques to solve human activity recognition problems from wearable sensors. With his collaborators, he demonstrated that a person’s daily transportation routines could be gleaned from a history of GPS sensor data. The work was motivated by the aim to help people with cognitive disabilities safely navigate their community without getting lost. Trained on GPS data, the wearable system assists users who get off track by helping them find public transportation to reach their intended destination. This influential work earned an Association for the Advancement of Artificial Intelligence 2004 (AAAI-04) Outstanding Paper Award, a 2012 Artificial Intelligence Journal (AIJ) Prominent Paper Award and a Ubicomp 2013 10-Year Impact Award.

In 2009, Fox began a two-year tenure as director of Intel Labs Seattle. There, he and his collaborators developed some of the very first algorithms for depth camera-based 3D mapping, object recognition and detection. Back at the University of Washington, Fox and his colleagues set an additional precedent with a separate study on fine-grained 3D modeling. Called DynamicFusion, the approach was the first to demonstrate how depth cameras could reconstruct moving scenes and objects, such as a person’s head or hands, with impressive resolution in real time. The work won a Best Paper Award from the Conference on Computer Vision and Pattern Recognition (CVPR) in 2015. 

For Fox, the McCarthy Award represents another milestone in a journey that began in his youth. As a high school student in Germany, he stumbled upon the book “Gödel, Escher, Bach: An Eternal Golden Braid” by Douglas Hofstadter. The pages, he found, flew by. When he finally closed its cover, he was spellbound. 

“From the book, I was fascinated by the ideas behind logic, formal reasoning and AI,” Fox said. “I learned that by studying computer science, I’d be able to continue to have fun investigating these ideas.”

Fox currently shares his time between the UW and NVIDIA, joining the company in 2017. He directs the UW Robotics and State Estimation Laboratory and is senior director of robotics research at NVIDIA. His work at NVIDIA stands at the cutting edge of deep learning for robot manipulation and sim-to-real transfer, bringing us ever closer to the dream of smart robots that are useful in real world settings such as factories, health care and our homes

Among his many honors, he is the recipient of the 2020 RAS Pioneer Award presented by the IEEE Robotics & Automation Society, and multiple best paper awards at AI, robotics and computer vision conferences. Fox, who joined the UW faculty in 2000, was also named a 2020 Association for Computing Machinery (ACM) Fellow, an IEEE Fellow in 2014 and a 2011 Fellow of the AAAI. Read more →

With nearly a billion unique monthly users, Wikipedia has become one of the most trusted sources of information worldwide. But while it’s considered more reliable than other internet sources, it’s not immune to bias. 

Last year, a team led by Allen School professor Yulia Tsvetkov developed a new methodology for studying bias in English Wikipedia biographies, and this spring won the 2023 Wikimedia Foundation Research Award of the Year for its efforts. The team first presented its findings at The Web Conference 2022. 

“Working with Wikipedia data is really exciting because there is such a robust community of people dedicated to improving the platform, including contributors and researchers,” Tsvetkov said. “In contrast, when you work with, for example, social media data, no one is going to go back and rewrite old Facebook posts. But Wikipedia editors revise articles all the time, and prior work has encouraged edit-a-thons and other initiatives for correcting biases on the platform.”

For the continuously evolving site, the research fills crucial content gaps in its data and how it is ultimately used. In the past, related studies focused mainly on one variable, binary gender, and lacked tools to isolate variables of interest, limiting the conclusions that could be drawn. For example, previous research involved comparing the complete sets of biographies for women and men in order to determine how gender influences their portrayals in these bios.

Tsvetkov’s team developed a matching algorithm to build more comprehensive and comparable sets, targeting not just gender but also other variables including race and non-binary gender. For instance, given a set of articles about women, the algorithm builds a comparison set about men that matches the initial set on as many attributes as possible (occupation, age, nationality, etc.), except the target one (gender).

The researchers could then compare statistics and language in those two sets of articles to conduct more controlled analyses of bias along a target dimension, such as gender or race. They also used statistical visualization methods to assess the quality of the matchings, supporting quantitative results with qualitative checks.

As a result, the researchers saw a significant difference when analyzing articles with and without their matching approach. When the approach was implemented, they found data confounds decreased — a boon for better evaluating bias in the future. 

“We did a lot of data curation to be able to include analyses of racial bias, non-binary genders, and intersected race and gender dimensions,” said lead author Anjalie Field, a professor at Johns Hopkins University who earned her Ph.D. from Carnegie Mellon University working with Tsvetkov. “While our data and analysis focus on gender and race, our method is generalizable to other dimensions.”

Future studies could further build upon the team’s methodology, targeting biases other than gender or race. The researchers also pointed to shifting the focus from the data sets to the natural language processing (NLP) models that are deployed on them. 

“As most of our team are NLP researchers, we’re also very interested in how Wikipedia is a common data source for training NLP models,” Tsvetkov said. “We can assume that any biases on Wikipedia are liable to be absorbed or even amplified by models trained on the platform.”

The study’s co-authors also included Chan Young Park, a visiting Ph.D. student from Carnegie Mellon University, and Kevin Z. Lin, an incoming professor in the University of Washington’s Department of Biostatistics. Lin earned his doctorate from Carnegie Mellon University and was a postdoc at the University of Pennsylvania when the study was published. 

Learn more about the Wikimedia Research Award of the Year here, and Tsvetkov’s research group here. Read more →

The Allen School’s Gabriel Ilharco and Ashish Sharma are among 13 students across the U.S. and England to receive 2023 J.P. Morgan AI Ph.D. Fellowships. The fellowships are part of the J.P. Morgan AI Research Awards Program, which advances artificial intelligence (AI) research to solve real-world problems.  

Ilharco, a fourth-year Ph.D. student in the Allen School’s H2Lab, is advised by professors Ali Farhadi and Hannaneh Hajishirzi. His research focuses on advancing large multimodal models as reliable foundations in AI. 

“I believe the next-generation of models will push existing boundaries through more flexible interfaces,” Ilharco said. “There is much progress to be made towards that vision, both in training algorithms and model architectures, and in understanding how to design better datasets to train the models. I hope my research will continue to help in all of these directions.”

During his fellowship, Ilharco said he hopes to continue his work in building more reliable machine learning systems. While models such as GPT-4, Flamingo and CLIP have demonstrated impressive versatility across applications, there is still room for growth. In the past decade, machine learning systems have become highly capable, particularly when performing specific tasks such as recognizing objects in images or distilling a piece of text. Yet their abilities can advance further, Ilharco said, with the end goal being a single model that can be deployed across a wider range of applications. 

To meet this challenge, Ilharco is targeting dataset design. A recent project, DataComp, acts as a benchmark for designing multimodal datasets. Ilharco was part of the research team that found smaller, more stringently filtered datasets can lead to models that generalize better than larger, noisier datasets. In their paper, the researchers discovered that the DataComp workflow led to better training sets overall. 

Ilharco and his collaborators will host a workshop centered around DataComp at the International Conference on Computer Vision 2023 (ICCV23) in October. 

“DataComp is designed to put research on datasets on rigorous empirical foundations, drawing attention to this understudied research area,” Ilharco said. “The goal is that it leads to the next generation of multimodal datasets.”

Another project introduced a framework for editing neural networks and appeared at the 11th International Conference on Learning Representations (ICLR 2023) this spring. Ilharco co-authored the paper that investigated how the behavior of a trained model could be influenced for the better using a technique called task arithmetic. In one example, the team showed how the model could produce less toxic generations when negating task vectors. Conversely, adding task vectors improved a model’s performance on multiple tasks simultaneously as well as on a single task. Ilharco and his collaborators also found that combining task vectors into task analogies boosted performance for domains or subpopulations in data-scarce environments. 

Their findings allow users to more easily manipulate a model, expediting the editing process. Because the arithmetic operations over task vectors involve only adding or subtracting model weights, they’re more efficient to compute compared to alternatives. Additionally, they result in a single model of the same size, incurring no extra inference cost. 

“We show how to control the behavior of a trained model — for example, making the model produce less toxic generations or learning a new task — by operating directly in the weight space of the model,” Ilharco said. “With this technique, editing models is simple, fast and effective.”

For Ilharco, the next wave of multimodal models is fast-approaching. He wants to be at the center of it. 

“I hope to be a part of this journey,” he said.

Sharma, also a fourth-year Ph.D. student, is advised by professor Tim Althoff in the Allen School’s Behavioral Data Science Lab. He studies how AI can support mental health and well-being. 

“I’m excited to be selected for this fellowship which will help me further my research on human-AI collaboration,” he said. “AI systems interacting with humans must accommodate human behaviors and preferences, and ensure mutual effectiveness and productivity. To this end, I am excited to pursue my efforts in making these systems more personalized.”

Sharma’s long-term goal focuses on developing AI systems that empower people in real-world tasks. His research includes work on AI to assist peer supporters to increase empathy in their communications with people seeking mental health support and exploring how AI can help users regulate negative emotions and intrusive thoughts.

Both put the user — the human being — at the center. 

“Effectively supporting humans necessitates personalization,” Sharma said. “Current AI systems tend to provide generalized support, lacking the ability to deliver experiences tailored to the specific needs of end-users. There is a need to put increased emphasis on developing AI-based interventions that provide personalized experiences to support human well-being.”

Sharma’s work with mental health experts and computer scientists was among the earliest efforts to demonstrate how AI and natural language processing-based methods could provide real-time feedback to users in making their conversations more empathetic.

At The Web Conference 2021, he and his co-authors won a Best Paper Award for their work on PARTNER, a deep reinforcement learning agent that learns to edit text to increase “the empathy quotient” in a conversation. In testing PARTNER, they found that using the agent increased empathy by 20% overall and by 39% for those struggling to engage empathetically with their conversational partners. 

“PARTNER learns to reverse-engineer empathy rewritings by initially automating the removal of empathic elements from text and subsequently reintroducing them,” Sharma said. “Also, it leverages rewards powered by a new automatic empathy measurement based on psychological theory.”

Earlier this year, Sharma was also lead author on a paper introducing HAILEY, an AI agent that facilitates increased empathy in online mental health support conversations. The agent assists peer supporters who are not trained therapists by providing timely feedback on how to express empathy more effectively in their responses to support seekers in a text-based chat. HAILEY built upon Sharma’s work with PARTNER. 

In addition, Sharma and his collaborators recently won an Outstanding Paper Award at the 61st annual meeting of the Association for Computational Linguistics (ACL 2023) for developing a framework for incorporating cognitive reframing, a tested psychological technique, into language models to prompt users toward healthier thought processes. With cognitive reframing, a person can take a negative thought or emotion and see it through a different, more balanced perspective. 

With a focus on people and process, Sharma sees how his research area can continue to grow. He said he hopes to advance AI’s ability to personalize to the user, while also remaining safe and secure. 

“Utilizing my experience in designing and evaluating human-centered AI systems for well-being, I will investigate how such systems can learn from and adapt to people’s contexts over time,” Sharma said. “I’ve always been fascinated by technological efforts that support our lives and well-being.” Read more →

Allen School researchers took home multiple Best Paper and Outstanding Paper Awards from the 61st Annual Meeting of the Association for Computational Linguistics (ACL) held in Toronto last month. Their research spanned a number of projects aimed at enhancing the performance and impact of natural language models, including how artificial intelligence (AI) processes humor, the impact of built-in political biases on model performance, AI-assisted cognitive reframing to support mental health, identifying “WEIRD” design biases in datasets and how to imbue language models with theory of mind capabilities. Read more about their contributions below.

Best Paper Awards

Do Androids Laugh at Electric Sheep? Humor ‘Understanding’ Benchmarks from The New Yorker Caption Contest

Allen School professor Yejin Choi and her collaborators earned a Best Paper Award for their study exploring how well AI models understand humor, challenging these models with three tasks involving The New Yorker Cartoon Caption Contest.

The tasks included matching jokes to cartoons, identifying a winning caption and explaining why an image-caption combination was funny. For an AI model, it’s no joke. Humor, the authors point out, contains “playful allusions” to human experience and culture. Its inherent subjectivity makes it difficult to generalize, let alone explain altogether. 

“Our study revealed a gap still exists between AI and humans in ‘understanding’ humor,” said Choi, who holds the Wissner-Slivka Chair at the Allen School and is also senior research manager for the Allen Institute for AI’s MOSAIC project. “In each task, the models’ explanations lagged behind those written by people.” 

The team applied both multimodal and language-only models to the caption data. Compared to human performance, the best multimodal models scored 30 accuracy points worse on the matching task. 

Even the strongest explanation model, GPT-4, fell behind. In more than two-thirds of cases, human-authored explanations were preferred head-to-head over the best machine-authored counterparts. 

Future studies could focus on other publications or sources. The New Yorker Cartoon Caption Contest represents only a “narrow slice” of humor, the authors note, one that caters to a specific audience. New research could also explore generating humorous captions by operationalizing feedback produced by the team’s matching and ranking models. 

The study’s authors also included Jack Hessel and Jena D. Hwang of AI2, professor Ana Marasović of the University of Utah, professor Lillian Lee of Cornell University, Allen School alumni Jeff Da (B.S., ‘20) of Amazon and Rowan Zellers (Ph.D., ‘22) of OpenAI and Robert Mankoff of Air Mail and Cartoon Collections. 

From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models

Allen School professor Yulia Tsvetkov, Ph.D. student Shangbin Feng and their collaborators earned a Best Paper Award for their work focused on evaluating pretrained natural language processing (NLP) models for their political leanings and using their findings to help combat biases in these tools. 

To do this, they developed a framework based on political science literature for measuring bias found in pretrained models. Then they analyzed how these biases affected the models’ performance in downstream social-oriented tasks, such as measuring their ability to recognize hate speech and misinformation. 

They found that bias and language are difficult to separate. Both non-toxic and non-malicious data, they note, can cause biases and unfairness in NLP tasks. If political opinions are filtered from training data, however, then questions arise concerning censorship and exclusion from political participation. Neither is an ideal scenario.

“Ultimately, this means that no language model can be entirely free from social biases,” Tsvetkov said. “Our study underscores the need to find new technical and policy approaches to deal with model unfairness.”

The consistency of the results surprised the team. Using data from Reddit and several news sources, the researchers found that left-leaning and right-leaning models acted according to form. The left-leaning models were better at detecting hate speech towards minority groups, while worse at detecting hate speech towards majority groups. The pattern was reversed for right-leaning models. 

In evaluating data from two time periods — before and after the 2016 U.S. presidential election — they also discovered a stark difference between the levels of political polarization and its attendant effect on the models’ behavior. With more polarization comes more bias in language models. 

Future studies could focus on getting an even more fine-grained picture of the effect of political bias on NLP models. For example, the authors note that being liberal on one issue does not preclude being conservative on another. 

“There’s no fairness without awareness,” Tsvetkov said. “In order to develop ethical and equitable technologies, we need to take into account the full complexity of language, including understanding people’s intents and presuppositions.”

The study’s co-authors also included Chan Young Park, a visiting Ph.D. student from Carnegie Mellon University, and Yuhan Liu, an undergraduate at Xi’an Jiaotong University.  

Outstanding Paper Awards

Cognitive Reframing of Negative Thoughts through Human-Language Model Interaction

Ph.D. students Ashish Sharma and Inna Wanyin Lin and professor Tim Althoff, director of the Allen School’s Behavioral Data Science Group, were part of a team that won an Outstanding Paper Award for their project investigating how language models can help people reframe negative thoughts and what linguistic attributes make this process effective and accessible. 

Working with experts at Mental Health America, the team developed a model that generates reframed thoughts to support the user. For example, the model could produce reframes that were specific, empathic or actionable — all ingredients for a “high-quality reframe.” The study was the first to demonstrate that these all make for better reframes, Althoff said, and the team illustrated this with gold standard randomized experiments and at scale. 

The research has already seen real-world impact. Since its introduction late last year, the team’s reframing tool has had more than 60,000 users. 

“The findings from this study were able to inform psychological theory — what makes a reframe particularly effective?” Sharma said. “Engaging with real users helped us assess what types of reframes people prefer and what types of reframes are considered relatable, helpful and memorable.”

Those “high-quality reframes” could be particularly helpful for those who lack access to traditional therapy. The team pointed out several obstacles to care, including clinician shortages, lack of insurance coverage and stigmas surrounding mental health, that served as motivations for the study. 

The model can also be integrated into existing therapy workflows, Sharma added, helping both clients and therapists in the process. For example, therapists often assign “homework” to clients, asking them to practice cognitive reframing, a technique by which a person can picture a negative thought through a different, more balanced perspective. 

But many clients report having difficulty in applying those techniques following a session. Sharma and Althoff said the team’s reframing tool can provide support in those moments. 

“It turns out that often our thoughts are so deep-rooted, automatic and emotionally triggering that it can be difficult to reframe thoughts on our own,” Althoff said. “This kind of research not only helps us improve our intervention itself, but could also inform how clinicians teach these skills to their clients.”

The study’s co-authors also included Kevin Rushton, Khendra G. Lucas and Theresa Nguyen of Mental Health America, Allen School alum David Wadden (Ph.D., ‘23) of AI2 and Stanford University professor and clinical psychologist Adam S. Miner.

Minding Language Models’ (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker

Choi and Tsvetkov worked with Allen School Ph.D. students Melanie Sclar and Peter West and collaborators on SymbolicToM, an algorithm that improves large language models’ abilities to reason about the mental states of other people that earned the team an Outstanding Paper Award. The team also won the Outstanding Paper Award at the ToM Workshop at the 2023 International Conference on Machine Learning (ICML) for this work. 

Theory of mind (ToM), or the ability to reason about others’ thoughts and intentions, is a key part of human intelligence. But today’s AI models lack ToM capabilities out of the box. Prior efforts at integrating ToM into language models required training, with existing reading comprehension datasets used for ToM reasoning remaining too simplistic and lacking diversity. 

“This implied that models trained solely on this data would not perform ToM reasoning,” Sclar said, “and rather only mimic these skills for the simplistic data they were trained on.”

Enter SymbolicToM. Without requiring ToM-specific training, the decoding-time algorithm takes a divide-and-conquer approach. SymbolicToM splits a given problem into subtasks, Sclar said, solving each with off-the-shelf large language models. The result is a better, more robust model. 

“We knew from the get-go that our approach needed to focus on having good generalization capabilities, and thus would benefit from not requiring training,” Sclar said. “SymbolicToM is to the best of our knowledge the first method for theory of mind reasoning in natural language processing that does not require any specific training whatsoever.”

The team tasked SymbolicToM with answering reading comprehension questions based on a story featuring multiple characters. They tracked each character’s beliefs, their estimation of others’ beliefs and higher-order levels of reasoning through graphical representations. In doing so, the models could reason with more precision and interpretability.  

“Our method in particular is not focused on training neural language models, but quite the opposite: given that we have imperfect language models trained with other objectives in mind, how can we leverage them to dramatically improve theory of mind performance?” Tsvetkov said. “This is key because data with explicit theory of mind interactions are scarce, and thus training directly is not a viable option.”

Sclar pointed to potential avenues for future applications, including education and business. For example, AI agents with ToM reasoning skills could assist in tutoring applications, providing a deeper understanding of students’ knowledge gaps and designing tests based on their mental model of each student. 

Another instance involves negotiation strategy. If AI agents can intuit what each party hopes to achieve and how much they value certain aspects of a deal, Sclar said, they can provide support in reaching a fair consensus. 

“Imbuing neural language models with ToM capabilities would improve these models’ potential on a wide range of applications,” Sclar said, “as well their understanding of human interactions.”

The study’s authors also included visiting Ph.D. student Sachin Kumar of Carnegie Mellon University and professor Alane Suhr of the University of California, Berkeley. 

NLPositionality: Characterizing Design Biases of Datasets and Models

Ph.D. student Sebastin Santy, professor Katharina Reinecke and their collaborators won an Outstanding Paper Award for devising a new framework for measuring design biases and positionality in NLP datasets that provides a deeper understanding of the nuances of language, stories and the people telling them. 

“Language is a social phenomenon,” Santy said. “Many in the NLP field have noticed how certain datasets and models don’t work for different populations, so we felt it was the right time to conduct this large-scale study given these gaps and with the right kind of platform.” 

That platform, LabintheWild, provides more reliable data from a more diverse set of users. Reinecke, one of the platform’s co-founders and director of the Wildlab at the Allen School, noted that as opposed to Mechanical Turk, a popular paid crowdsourcing site, LabintheWild collects results from a greater pool of countries. 

With LabintheWild, the personal is emphasized over the pecuniary. After completing a study, users can see personalized feedback and compare their results with others’ performance on the platform.

This feedback is eminently shareable, Reinecke added, increasing its reach. The researchers’ recent study collected 16,299 annotations from 87 countries — one of the first NLP studies to reach that scale. They applied their framework, called NLPositionality, to LabintheWild’s vast participant pool, implementing users’ annotations from existing datasets and models for two tasks: social acceptability and hate speech detection. 

Their findings aligned with Reinecke’s previous work, which shows that technology is often designed for people who are Western, Educated, Industrialized, Rich and Democratic, or “WEIRD.” 

“WEIRD bias is well-known in psychology and our hypothesis was that we might find similar results in AI as well, given most of the recent advances make use of mostly English data from the internet and filter for ‘high-quality,’ ” said Reinecke, who holds the Paul G. Allen Career Development Professorship. “While we had a feeling that there would be Western bias because of how most of the datasets are curated in the Western Hemisphere, we did not expect it to be this pronounced.”

The study’s co-authors also included Allen School alumni Maarten Sap (Ph.D., ‘21) and Jenny Liang (B.S., ‘21), now professor and Ph.D. student, respectively, at Carnegie Mellon University, and Ronan Le Bras of AI2.  Read more →