Earlier this month, the Allen School commemorated Women’s Research Day, an annual event that celebrates the research contributions of women and nonbinary people in the school and the greater Seattle area. While this year’s event was compelled to move from the Allen Center atrium to Zoom due to COVID-19, the online format didn’t dampen participants’ enthusiasm for the program. Allen School professor emerita Susan Eggers, professor and director Magdalena Balazinska, and a virtual poster session in which undergraduate and graduate students shared their latest research, still took place as planned.
“Overall, things went fairly smoothly aside from a few technology hiccups on the day of,” said Allen School Ph.D. student and organizer Emily Furst. “We had a lot of great participation and questions during all of the sessions, and I think it might have been one of our highest turnouts ever. One great thing was that we had a really high turnout of the high school seniors who have received direct admission into the major this year. We try to invite them most years, but due to the event being virtual, they were able to attend from all over.”
During her questions and answers sesssion, Balazinska was asked about her background, advice on research, and advice to students just starting in the CSE major.
“When you’re choosing which classes to take, choose the ones you think will be the most exciting and important,” she said. “Never shy away from classes because you think they’re too hard. Now is the time to learn because it’s easier in the classroom than finding other ways. Be brave, take those classes, don’t just do the bare minimum to get by.”
Participants also heard from one of the leading researchers in computer architecture. Eggers, who joined the University of Washington faculty in 1989, delivered the keynote lecture in which she spoke about her work and what it was like to be among the roughly one percent of computer architects who were female when she entered the field. During her talk, Eggers shared her experience as one of the lead developers of the first commercially viable multithreaded architecture, Simultaneous Multithreading, which was adopted by Intel, IBM, Sun and others and earned her team both the 2010 and 2011 Test of Time Awards from the International Symposium on Computer Architecture (ISCA).
Susan Eggers
“I’ve had a lot of wonderful mentors but I also made sure to pass it on,” Eggers said about being a mentor herself. “As far as I can tell, women in academia do this, mentor women behind us in computer science. We pay it back, we don’t just take it.”
As the first woman ever to receive the Association for Computing Machinery and IEEE Computer Society’s Eckert-Mauchly Award, in 2018, Eggers told her virtual audience that during her acceptance speech she thanked the committee for breaking another professional glass ceiling. She spoke of her research highlights, then gave the high school students, undergrads, graduate students and industry attendees, advice on working in a field predominantly populated by men.
Eggers’ talk fit in with the purpose of the day, which as Furst explained, is to give women and non-binary people in the school community an opportunity to make connections with other researchers and learn from them.
“Whether that be undergrads finding grad students to work with or grad students making more connections with industry researchers,” she said. “It’s also a great event for the direct admits and undergrads to learn more about research in general and the different areas of research within computer science.”
While Furst hopes that next year’s event will once again be held in person, she also thinks future programs could incorporate more virtual aspects based on the success of this year’s event.
“Regardless of the exact format, our goal will always be to make the event a welcoming space where everyone feels comfortable participating and asking questions,” she said.
Videos of the 2020 Women’s Research Day can be viewed on the Allen School’s YouTube channel here.
Jungo Kasai, a Ph.D. student working with Allen School professor Noah Smith on natural language processing (NLP), has been named a 2020 IBM Ph.D. Fellow. Kasai, who is one of only 24 students from a total of 140 universities around the world to be selected for a fellowship, was recognized in the “Artificial Intelligence/Cognitive Computing” category for his focus on the problem of cross-lingual transfer.
Deep learning has made incredible gains for NLP, but most of the research efforts have been focused on the English language. Real-world applications of NLP need to include a diverse set of languages. Kasai’s work questions whether or not we can exploit annotated data for “rich” languages like English to improve the accuracy of NLP components for other languages as well.
“My fundamental hypothesis is that different natural languages manifest similar characteristics which can be exploited by deep learning models and their distributed representations,” said Kasai. “I want to provide further support for this hypothesis by improving representation learning for diverse languages and ultimately to make contributions toward massively multilingual processing in the real world.”
So far, his research — which he is pursuing in collaboration with Allen School Ph.D. student Phoebe Mulcaire — is proving that hypothesis to be true. The team has worked to develop methods to produce multilingual representations from a large amount of monolingual data without extra annotation. They then created a multi-language program that models probability distribution over diverse languages and uses the distributed representations for any downstream task.
Kasai and Mulcaire published two papers last year showing that artificial intelligence models can be effectively expanded to many languages with little or even no labeled training data. The first paper, which the team presented at the 2019 conference of the North American Chapter of the Association for Computational Linguistics (NAACL), described Rosita. Rosita is a method for producing multilingual contextual word representations by training a single language model on text from multiple languages, such as English and Arabic or English and Chinese. The results showed the benefits of polyglot learning, in which representations are shared across multiple languages.
“Jungo and Phoebe’s paper delves into parsing and shows substantial gains in truly low-resource scenarios, including zero-shot parsing — i.e., learn only from English or another helper language — test it on sentences in a language with no training treebank at all,” said Smith. “The team’s work goes well beyond most publications in our field these days, which reveal new state-of-the-art scores on established benchmark tasks, and introduced an innovative technique for probing why the model works. I’m very proud of this work and Jungo’s contributions, which I believe will generate a lot of excitement and follow-on by others.”
Kasai is looking forward to meeting the other Ph.D. fellows and working with IBM.
“I believe working with industry is a great way to see my Ph.D. research on multilingual NLP and machine translation from different perspectives and to put research into practice,” he said.
Congratulations Jungo — and thanks to IBM for generously supporting student research!
Emerging technologies like augmented and mixed reality have the potential to transform the way we experience the world and interact with each other. But like most technologies that, in their infancy, opened up exciting new avenues of engagement — the web, mobile phones, social networks, and so on — mixed reality also has a potential dark side. And it’s one that is made even more fraught by the interplay between the physical and virtual world.
To encourage developers of these emerging technologies to employ a privacy and safeguard mindset early on, Allen School professors Tadayoshi Kohno and Franziska Roesner organized a summit last November that brought together representatives of academia and industry. The goal was two-fold: identify potential threats to user privacy, security and safety posed by augmented and mixed reality (AR/MR), and come up with a framework to guide designers and developers in addressing those threats.
With the birth of previous technologies, the world mostly charged full steam ahead, with security and privacy relegated to playing catch-up. This time, according to Roesner, there is widespread interest in addressing potential issues proactively rather than reactively.
“Augmented and mixed reality technologies are unique in their ability to impact a user’s perceptions of and interactions with the physical world compared to other technologies, so the associated risks are fundamentally different from those technologies, too,” explained Roesner, who co-directs the Security and Privacy Research Laboratory at the University of Washington with Kohno. “Last fall, we got together with other security researchers and industry representatives to explore a set of questions that creators of AR/MR technologies should be asking to address user privacy, security, and safety from the start.”
Last week, summit contributors released a report laying out a comprehensive set of issues that should be considered when developing mixed reality hardware, platforms and applications. The report starts by acknowledging what could go right with mixed reality technologies — primarily, the variety of desirable features and functionality that will benefit users and society by enabling people to overcome human limitations of time and space, and in ways that are accessible to everyone regardless of physical, financial, or other capacities.
But to realize this vision, the authors note, researchers and the industry have to work together to address what could go wrong. Examples range from exposure to undesirable content, to excessive or overly invasive advertising, to actual physical or psychological harm. Unlike other technologies that only capture snippets of a person’s life here and there — a credit card company knows your shopping habits, a health insurance company sees what tests your doctor runs — mixed reality platforms have the potential to build a much more complete picture of a user.
“The potential of MR devices to collect and infer incredibly sensitive information about users compels us to develop platforms that give users the ability to get the benefits of MR without having to hand over a deeply personal picture of themselves and their environment,” said co-author Blair MacIntyre, a principal research scientist at Mozilla and professor at Georgia Institute of Technology’s School of Interactive Computing. “Once this data is out there, it can’t be pulled back, and MR should be available to everyone, not just those willing to ignore the potential downsides of sharing such information.”
Industry has a strong motivation to get these issues right, even at this early stage. As the report points out, just one negative incident that results in actual harm to a user or users could represent an “existential threat” to the industry by discouraging adoption and prompting well-intentioned but overly prescriptive regulation that could stifle future innovation and growth. There are past examples to draw upon, such as the outcry against autonomous vehicles — and the companies developing them — following the death of a pedestrian. In that case, the software in one vehicle failed to correctly register the person’s presence in the road. The incident prompted a federal investigation and compelled several companies to temporarily put the brakes on testing their vehicles on public roadways.
Franziska Roesner
The authors of the report propose that designers apply a threat-modeling approach often used in security and privacy research. This enables stakeholders to systematically consider which assets require protection in the system, along with how and to which adversaries that system might be vulnerable. The report offers a “fill in the blanks” framework to aid discussion of the potential risks and harms — along with the potential benefits — of mixed reality platforms and applications. The goal is to support designers, engineers, researchers, and policymakers as they work through these issues together, to strike the right balance between user security, functionality, and the industry’s ability to innovate.
One of the big issues that the group considered using the threat-modeling approach is how to manage the interaction of virtual content overlaid on the physical world. The group considered questions surrounding the appearance of undesirable content, whether in the form of advertising plastered everywhere, content that is age-inappropriate for children, or content that is disturbing or harassing to an individual. Other questions arose regarding whether virtual content would be allowed to block real-world content in ways that the user would find disruptive, or if content from multiple sources would be permitted to interfere with each other.
There is also the matter of who has access and can alter content that someone else has created in virtual space. In one high-profile example, an augmented reality artwork titled “AR Balloon Dog” was “vandalized” — or rather, a copy was vandalized and superimposed over the original’s geolocation — by a group of artists protesting the notion of corporations monetizing digital art. The original was created by artist Jeff Koons as part of a 2017 collaboration with Snapchat. The incident raised a variety of questions regarding the ownership of virtual objects, whether they should be treated the same as their real-world analogs in physical space, and whether the act even constituted vandalism, given that the virtual object in question was a copy of another virtual object.
Members of the UW lab have explored this issue before, inspired in part by the saga of AR Balloon Dog. Last year, Roesner and Kohno worked with undergraduate researcher Kimberly Ruth to release ShareAR, a toolkit that enables developers to build fully functional, multi-user AR applications that permit secure content sharing. That was the start of a more robust conversation around how to design platforms and apps that provide users a measure of control, but the researchers acknowledge that there are issues that need to be resolved that go beyond technology.
“In the physical world, if someone vandalized a painting or structure of historical significance, that person would get arrested. A person who posts offensive or trademarked content online would have that content moderated,” noted Roesner. “But those frameworks don’t exist — not yet, anyway — in the virtual world. Many questions around jurisdiction and enforcement, along with issues regarding ownership, attribution or trademark, are yet to be answered.”
There is also the matter of translating societal norms and the types of behavior we deem acceptable in our day-to-day interactions from the physical world to the virtual one. According to Kohno, it is a significant challenge — one made all the more complicated by the fact that the virtual world transcends the physical world’s cultural and geopolitical boundaries.
“This discussion raises a host of issues around how to apply rules that are ingrained in our social fabric into virtual space,” noted Kohno. “In the physical world, it would not be acceptable for someone to put a ‘kick me’ sign on someone’s back. That would violate our notion of personal space and autonomy. But how do we deal with that in a virtual world?”
In this and similar scenarios, Kohno explained, designers might consider what limitations or tools they can provide to prevent offensive content altogether and/or enable users to remove content they find offensive when it appears.
Tadayoshi Kohno
“Perhaps I am not allowed to alter another person’s avatar, because we have decided that is a core tenet of virtual space. Or perhaps we are friends and so you give me permission to alter your avatar, but the platform’s built-in controls allow you to remove alterations that you don’t like,” Kohno mused. “At the summit we discussed a ‘Bill of Rights’ governing digital spaces, to articulate what people can expect to be able to do and expect to have done to them. What we’ve done with this report is try to provide the scaffolding for the industry to consider the privacy and security issues raised by these platforms and to make conscious decisions about what rules and safeguards they need to incorporate into their design.”
A Bill of Rights, usable controls, and the aforementioned threat modeling are just a few of the potential solutions participants considered during the summit. Other avenues for exploration include early identification of trustworthy and responsible entities, the development of industry standards, support for application developers along the lines of ShareAR that make it easy for them to build security into their products, and — when the time is right — thoughtful regulatory and policy frameworks that will underpin user trust while reflecting the richness and complexity of augmented and mixed reality.
“The summit provided an opportunity for stakeholders to come together and have a collaborative conversation and forge a common language in which we can discuss these issues,” Roesner said. “With this report, we are taking a first step toward enabling a more holistic approach to important questions about security and privacy within this brave, new world that is being created.”
The Security and Privacy Research Laboratory was an early proponent of mixed reality security and privacy. In 2012, Kohno and Roesner co-authored a paper with security researcher David Molnar, then at Microsoft Research, laying out the privacy and security research challenges and new opportunities associated with emerging AR technologies. Two years later, Kohno and Roesner contributed to a primer on AR released by the UW’s Tech Policy Lab that was shared widely with policy makers to inform them about the still-nascent industry, its potential benefits, and associated pitfalls in relation to privacy, distraction and discrimination. That was followed by a series of papers in which the researchers applied what they had learned to supporting AR privacy and security more directly.
“A principled approach to user privacy and security will be a catalyst for innovation and widespread adoption, not an obstacle,” concluded Roesner. “We may still be playing catch-up when it comes to more mature technologies, but with AR/MR, we have an opportunity to build a virtual world that is safe and enjoyable for everyone almost from the ground up.”
The 2019 Industry-Academia Summit on Mixed Reality Security, Privacy, and Safety was co-funded by the UW Security and Privacy Research Lab and the UW Reality Lab. Read the full report here.
Our latest Allen School undergraduate student spotlight features Aerin Malana, a sophomore from Kent, Washington, majoring in computer science with a diversity minor. Malana has a passion for advocacy and equity, specifically in tech, and has become a leader in inclusivity at the Allen School. She serves as vice chair of the Association for Computing Machinery for Women (ACM-W) and has co-founded Gen1, an organization for first generation Allen School students. Despite the fact that the University of Washington has moved classes online due to COVID-19, Malana is still actively engaged with both groups, using video conferencing to continue programming for ACM-W and preparing for the rollout of Gen1 in the fall quarter.
Allen School: What do you enjoy about being in ACM-W and what motivated you to become vice chair?
Aerin Malana: Being in ACM-W was a great starting point for my involvement in the Allen School, as it intersected with my interest in advocacy for diversity in tech. I get to work with a great group of women; we all encourage each other to remember our mission of advocating for diversity in tech, as well as to always be the best we can be. I became an associate officer for ACM-W halfway through my freshman year. The experiences I gained and the people I met in my first year through ACM-W not only helped me to further my passion for advocacy, but also helped me to understand how amazingly complex our Allen School community is. I decided to come back this year as Vice Chair, where I could take on a bigger leadership role in the organization.
Allen School: In that leadership role, what are some goals you hope to accomplish?
AM: I hope to help ACM-W push for more awareness and increase advocacy for issues that gender minorities face in tech (women, nonbinary folx, etc.). We really aim to help community members question certain biases we’ve grown accustomed to socially and how we can undo those biases in ourselves and in our communities. I also hope to help more of our community members understand the concept of intersectionality and how that plays into each of our experiences with the world and each other. It’s important that we all understand that aspects of our identity (race, gender, sexual orientation, ability, economic class, etc.) do not operate separately, rather they intersect, overlap, and impact you, as an individual, as well as how you interact in institutions.
Allen School: How do you hope to make computer science more diverse and inclusive, and how is your diversity minor helping you to do that?
AM: While there has been an increase of women, people of color, and other marginalized communities in our institutions, there is often a lack of support for them and a lack of effort to retain them in these spaces. It’s important for all of us to ask the question of why that is. Why isn’t our department or workforce a true reflection of our people? Why aren’t the resources that lead to success truly accessible to everyone? Do the amazing things we build reflect the people that use these things and the people that will be affected by them? If the answer is no, we have to understand why that is and the many active ways we can include every single voice and identity into these institutions and amplify them.
I’m hoping to make diversity and inclusivity more of a focal point in computer science by giving opportunities for the Allen School community to understand how it impacts everything they do, from the way they write code to the way they speak with fellow CSE students. In ACM-W, we have so many events to help students understand diversity in their space: quarterly diversity discussions, allyship panels, impostor syndrome talks, etc. I’m also hoping to make diversity and inclusivity more of a focal point by helping others understand where prejudices and biases in our field come from and how we can be aware of our biases.
My diversity minor helps me in understanding the institutions our everyday life is founded on, how they are intrinsically linked to the marginalization of certain communities, and how I can apply that knowledge specifically to computer science, where our work impacts everyone.
Allen School: You’re also forming an Allen School organization for first generation students, Gen1. Why is this important to you?
AM: When I first came to UW, I was the only person in my circle of friends and acquaintances that identified as first gen; I was the first in my family to pursue a Bachelor’s degree in the U.S. It wasn’t until an entire year later, when I had a conversation with two of my friends who also identified as first gen, where I realized this community was there, it was just invisible.
We discussed the fact that there was a lack of resources and help for first generation students in a program where everyone is already connected to someone in industry or knows the ins and outs of this institution. They also felt like they were the only people in their entire social circle that were first gen, and it wasn’t until we started having active conversations surrounding this part of our identity that we realized we weren’t alone.
It’s so important for anyone to feel seen and to be heard. When you don’t have a visible community that you know is there for you and can help you succeed, it’s incredibly hard to keep up with others who don’t understand your struggle. It’s important to have a club for first generation students in the Allen School because I understand how hard it is to navigate an institution when you need to figure out everything for yourself, especially in the CSE community. There are so many others out there in the program that feel the same way but don’t know anyone else who shares their experience. With the creation of this club, we hope that we can build a visible community of amazing first gen students to share all of our experiences and support all of our ambitions.
Allen School: Why did you choose to study computer science?
AM: I didn’t necessarily choose computer science, I actually happened to stumble upon it. By the beginning of my senior year in high school, I knew I had to pick a major to apply to schools with. I had no clue what to do. I “flipped a coin” between another major and computer science, and luckily it landed on computer science. I applied to schools as a CS major very blindly, and it wasn’t until after application deadlines that I took my first programming class. Soon after, I fell in love with it. The combination of logical thinking, problem solving, and space for creativity and flexibility that CS offered interested me so much.
Allen School:What do you like most about being in the Allen School?
AM: I enjoy so many things about this school, but I would have to say that I love the people the most. The staff and faculty of this school really love what they’re doing and it shows immensely. I absolutely love getting to work closely with lecturers, who further my passion for educating our communities, and working with advisers, who help affirm and push my drive to affect positive change in the program.
I also love getting to interact with my peers in the Allen School community. I get to learn so much from fellow students with different perspectives and I have the ability to create amazing things with them, both in the classroom and in the community.
Thanks to Aerin for her advocacy and her commitment to lifting up the voices of first gen students in our school!
It was the first year that ICPC held a North American competition in between regionals and the international competition — and the first time a UW team has faced off against student programmers from outside of their own region.
“The competition in Atlanta was intense, just as we expected,” said team member Phawin Prongpaophan, a junior majoring in computer science. “Despite the fact that the rules were the same — that is, we still had five hours to solve problems — it felt a lot different from the regional competition. The problems were significantly harder and every minute really counts toward the ranking in the nationals.”
According to Porncharoenwase, the chance to participate in an event beyond the regional contest offered a lot of valuable lessons for future competitions, like how to improve the team’s time management.
At the competition in Atlanta
“We solved five problems out of 12, which is one too few from getting qualified for the world finals,” he said of his team, who finished in the middle of the pack, in 26th place. “They had an incorrect solution for the seventh problem, which, with a small tweak, would be correct. I think our team has a really high potential for next year.”
“One thing I learned from this journey is that teamwork is key to success,” said Prongpaophan. “The competition isn’t about anyone’s strength, but it is about how we deal with it together as a team. I believe we wouldn’t have made it this far without each other.”
Principal Lecturer Stuart Reges, the team’s faculty sponsor who teaches introductory programming courses in the Allen School, was thrilled by the team’s hard work and dedication. He said this year was especially exciting for both the students and coaches since ICPC added the North American component.
“This year marks the first time we have sent a team beyond our regional contest. Our region includes fierce competitors like Stanford, Berkeley, and UBC, which has kept us from qualifying for the international contest. With the addition of the new North American Championship we were finally able to break through,” said Reges. “We are grateful to Google for providing a tour of their campus to the teams that competed in the regional contest and I am personally grateful to our graduate students Sorawee Porncharoenwase and Victor Reis who did all of the work of hosting a local contest, taking teams to the regional, and helping this team attend the North American Championship.”
Another Allen School team, King Gesar, also placed in the top 10 at the Pacific Northwest regionals. Of the teams representing western Washington, UW teams occupied the top 5 spots in the regional competition.
Way to go Team Combo, Team Gesar and all of the other hardworking UW teams!
Kimberly Ruth, a senior graduating from the University of Washington this spring with bachelor’s degrees in computer engineering and mathematics, has been awarded the College of Engineering’s Dean’s Medal for Academic Excellence. Each year, the college recognizes two graduating students for academic excellence; Ruth’s combination of exemplary grades, rigorous coursework, hands-on research experience, and leadership on campus and off illustrate why she was chosen for the honor.
“We have a very strong program and many of our students are remarkable, but Kimberly stands out even from this select group,” said Allen School director and professor Magdalena Balazinska. “Her drive, leadership, undergraduate research and academic excellence are admirable, and she has only reached the beginning of her potential.”
As a freshman in the Allen School, Ruth set her sights on research right away. During her first quarter on campus, she reached out to professors Tadayoshi Kohno and Franziska Roesner, co-directors of the Security and Privacy Research Lab. Although she had not been on campus very long, Kohno and Roesner decided to interview her for a position as an undergraduate researcher anyway.
“Though we met with several other promising undergraduates that day, we knew before our meeting with Kimberly even finished that she stood out far above the rest,” recalled Kohno. “She has now been working with us since January of 2016, and her work in the past four and a half years has only strengthened that initial impression.”
Ruth’s research focuses on security and privacy for augmented reality (AR) platforms. These emerging technologies, such as Microsoft’s HoloLens, generate visual and audio feedback to change a person’s perception of the real world. They also raise new privacy and security risks for users. While working in the Security and Privacy Research Lab, Ruth played a critical role in several research projects. In one project, Ruth worked with Ph.D. student Kiron Lebeck to design an AR operating system that can protect against malicious or buggy output from applications. Ruth was second author on the resulting paper, “Arya: Operating System Support for Securely Augmenting Reality,” which appeared at the 38th IEEE Symposium on Security and Privacy and was published in the IEEE Security and Privacy magazine in 2017. Ruth followed that up by co-authoring “Securing Augmented Reality Output,” and “Towards Security and Privacy for Multi-user Augmented Reality: Foundations with End Users” the following year.
But that wasn’t quite enough for Ruth, who has made the most of her undergraduate research experience. In June of 2017, she also began leading her own project in AR security, focusing on security for multiuser AR applications like the popular game Pokémon Go. The result was ShareAR, a toolkit that helps app developers build in collaborative and interactive features without sacrificing user privacy and security. Ruth and the team published their paper, “Secure Multi-User Content Sharing for Augmented Reality Applications,” last year at the 28th USENIX Security Symposium, where she presented the results.
Ruth, presenting her research at the 28th USENIX Security Symposium
“Kimberly’s work on this project was incredible. She independently raised, explored, prioritized, and answered a range of sophisticated research questions,” said Roesner. “She worked through design questions and implementation subtleties that were not only technically but also intellectually challenging—requiring thoughtful framing of the problem space and inventing new approaches.”
Outside of the lab, Ruth is also an adept teacher, helping her fellow students to succeed as a peer tutor for the Allen School’s Foundations in Computing course last year and inspiring the next generation through Go Figure, an initiative she founded to ignite middle school students’ interest in math.
“Kimberly is wholly deserving of all of the honors she has received, and I feel so privileged to have had the opportunity to work with her in this early stage of her career,” said Roesner. “I look forward to seeing all of the great things she will do in the future, whether in computer security research or otherwise.”
That statement might hold true for a baseball field in rural Iowa — in the days before social distancing, that is — but what about when it comes to building mobile technologies to fight a global pandemic?
In the balance between individual civil liberties and the common good, there is an obvious tension between the urge to deploy the latest, greatest tools for tracking the spread of COVID-19 and the preservation of personal privacy. But according to a team of researchers and technologists affiliated with the Paul G. Allen School of Computer Science & Engineering, UW Medicine and Microsoft, there is a way to build technology that respects the individual and their civil liberties while supporting public health objectives and saving people’s lives.
In a white paper released yesterday, the team proposes a comprehensive set of principles to guide the development of mobile tools for contact tracing and population-level disease tracking while mitigating security and privacy risks. The researchers refer to these principles as PACT, short for “Privacy Sensitive Protocols and Mechanisms for Mobile Contact Tracing.”
“Contact tracing is one of the most effective tools that public health officials have to halt a pandemic and prevent future breakouts,” explained professor Sham Kakade, who holds a joint appointment in the Allen School and the UW Department of Statistics. “The protocols in PACT are specified in a transparent manner so the tradeoffs can be scrutinized by academia, industry, and civil liberties organizations. PACT permits a more frank evaluation of the underlying privacy, security, and re-identification issues, rather than sweeping these issues under the rug.”
If people were not familiar with the concept of contact tracing before, they surely are now with the outbreak of COVID-19. Public health officials have been relying heavily on the process to identify individuals who may have been exposed through proximity to an infected person to try and halt further spread of the disease. Several governments and organizations have deployed technology to assist with their response; depending on the situation, participation may be voluntary or involuntary. Whether optional or not, the increased use of technology to monitor citizens’ movements and identify other people with whom they meet has rightly sparked concerns around mass surveillance and a loss of personal privacy.
The cornerstone of the PACT framework put forward by the UW researchers is a third-party free approach, which Kakade and his colleagues argue is preferable to a “trusted third party” (TTP) model such as that used for apps administered by government agencies. Under PACT, strict user privacy and anonymity standards stem from a decentralized approach to data storage and collection. The typical TTP model, on the other hand, involves a centralized registration process wherein users subscribe to a service. While this can be a straightforward approach and is one that will be very familiar to users, it also centrally aggregates personally sensitive information that could potentially be accessed by malicious actors. This aggregation also grants the party in question — in this case, a government agency — the ability to identify individual users and to engage in mass surveillance.
The team’s white paper lays out in detail how mobile technologies combined with a third-party free approach can be used to improve the speed, accuracy, and outcomes of contact tracing while mitigating privacy concerns and preserving civil liberties. These include the outline of an app for conducting “privacy-sensitive” mobile contact tracing that relies on Bluetooth-based proximity detection to identify instances of co-location — that is, instances of two phones in proximity, via their pseudonyms — to determine who may be at risk. The team prefers co-location to absolute location information because it is more accurate than current GPS localization technologies, such as those in popular mapping and navigation apps, while affording more robust privacy protections to the user. Depending on the nature of the specific app, such a system could be useful in allowing people who test positive for the disease to securely broadcast information under a pseudonym to other app users who were in close proximity to them, without having to reveal their identity or that of the recipients.
Another example of how PACT can aid in the pandemic response include mobile-assisted contact tracing interviews. In this scenario, a person who tests positive completes a form on their smartphone listing their contacts in advance of the interview; the data remains on the person’s device until they choose to share it with public health officials. The team also describes a system for enabling narrowcast messages, which are public service messages pushed out from a government agency to a subset of the citizenry. Such communications might be used to inform people living in a specific area of local facility closures due to an outbreak, or to notify them in the event that they were at a location during the same time frame as a person who subsequently tested positive for the disease.
Illustration of the PACT tracing protocol. M Eifler
In all cases, the researchers advocate for retaining data locally on the person’s device until they initiate a transfer.
“Only with appropriate disclosures and voluntary action on the part of the user should their data be uploaded to external servers or shared with others — and even then, only in an anonymized fashion,” explained Allen School professor Shyam Gollakota. “We consider it a best practice to have complete transparency around how and where such data is used, as well as full disclosure of the risks of re-identification from previously anonymized information once it is shared.”
Gollakota and his colleagues emphasize that technology-enabled contact tracing can only augment — not entirely replace — conventional contact tracing. In fact, two out of the three applications they describe are designed to support the latter and were developed with input from public health organizations and from co-author Dr. Jacob Sunshine of UW Medicine. There is also the simple fact that, despite their seeming ubiquity, not everyone has a smartphone; of those who do, not everyone would opt to install and use a contact-tracing app.
As Allen School professor and cryptography expert Stefano Tessaro notes, all contact tracing — whether conventional or augmented with technology — involves tradeoffs between privacy and the public good.
“Contact tracing already requires a person to give up some measure of personal privacy, as well as the privacy of those they came into contact with,” Tessaro pointed out. “However, we can make acceptable tradeoffs to enable us to use the best tools available to speed up and improve that process, while ensuring at the same time meaningful privacy guarantees, as long as the people creating and implementing those tools adhere to the PACT.”
Allen School senior Parker Ruth is one of three students from the University of Washington to be named a winner as part of the 2020 Goldwater Scholarship competition sponsored by the Barry Goldwater Scholarship & Excellence in Education Foundation. The scholarship program is one of the oldest and most prestigious in the nation focused on supporting exceptional undergraduates who aim to pursue research careers in the natural sciences, mathematics, and engineering fields.
Ruth, who is majoring in computer engineering and bioengineering, plans to pursue a Ph.D. in computer science and apply computing tools to transform health care through novel hardware and software. He is a part of the university’s interdisciplinary honors program and an undergraduate researcher in the Ubicomp Lab advised by professor Shwetak Patel. Throughout his academic career, Ruth has been exploring the application of computing tools to improve the quality and accessibility of health care.
“Parker’s maturity and ability to work effectively in a research setting is simply extraordinary for such a young student. He is an undergraduate that is already showing the maturity of a graduate student in terms of unpacking a problem, diving deep into a research question, and executing high quality research studies,” said Patel. “On top of that, he is a very well-rounded student. When we have visitors come to the lab, they don’t even realize he is an undergraduate student. They just assume he is just one of the Ph.D. students given his spectacular research knowledge.”
That research involves developing new sensing and signal processing techniques for screening and diagnosing diseases using commodity and mobile technologies. Ruth has worked on a variety of projects in the lab, including the development of tools related to cardiovascular disease, osteoporosis and physical activity monitoring — to name only a few.
“In one example, Parker helped develop a robust mobile phone tool that uses the camera/flash and the phone’s gyroscope to estimate the pulse transit time to access cardiovascular risk,” Patel said. “He has also been working on an osteoporosis screening tool using sensors on a phone.”
Barry Lutz, a professor in the Department of Bioengineering, worked with Ruth on a multi-disciplinary team project creating rapid tests for HIV drug resistance. It didn’t take long for him to be impressed with Ruth’s capability, intellectual maturity, professionalism and balance of confidence and modesty.
“Parker is a top student, scientific leader, sophisticated researcher, and a kind and humble human being,” Lutz said. “He has all of the ‘typical’ star accomplishments in academics, research, and service; but in each area Parker stands out from other accomplished students due to his uncharacteristic intellectual strength.”
Ruth, who was homeschooled before attending the UW, credits his mother for instilling his drive to learn and experiment.
“My mom transformed our household into a place of experiential and self-driven learning,” he said. “At the core of my mom’s educational philosophy was a priority on imparting a love of learning; as a result, I was raised seeing learning far more like a recreation than an occupation.”
Students were provided with a kit to build their own head-mounted display, including an LCD, an HDMI driver board, an inertial measurement unit (IMU), lenses, an enclosure, and all cabling
Over a video conference presentation, Eugene Jahn showed viewers an augmented reality program he created to help aspiring Michael Jordans shoot the perfect basket, showing the best path and angle to become a better shooter. The Allen School sophomore is a student in the Virtual Reality Systems CSE 490V taught by affiliate instructor Douglas Lanman, Director of Display Systems Research at Facebook Reality Labs.
The students — five graduate students, one sophomore, two juniors and 23 seniors, all studying computer science or computer engineering — spent the winter quarter learning about virtual reality systems, building software and hardware for a complete VR headset in 10 weeks. The hands-on course introduced students to the cross-disciplinary skills needed to create their own VR systems. Jahn and his classmates presented their final projects over Zoom as part of a virtual demo day, instead of in person as originally planned, after the University of Washington moved classes online due to COVID-19.
“I gave the students the option of not doing the final project because they didn’t have as much time in the lab to build their system, but all of them chose to do it — they all have a ‘maker’ mentality,” Lanman said during his introduction to the online demo day. “Although the students planned to have more time in the maker space in the Reality Lab, they made the most of finishing up their projects and presenting them over a conference call.”
Lanman said he had long dreamed of teaching a course at UW. He saw his chance when he moved to Seattle to join Facebook Reality Labs, reaching out to Allen School faculty shortly after his arrival to discuss possible courses. He said the moment he had been waiting for came when Facebook co-funded the launch of the UW Reality Lab.
“The creation of this center positioned UW, and the greater Seattle region, at the center of augmented and virtual reality revolution,” Lanman said. “When I read about the lab, I knew it was the right time to train a new generation of students to join the growing AR/VR industry being built in our backyard.”
Lanman emailed his idea to Brian Curless and Steve Seitz, who, along with Ira Kemelmacher-Shlizerman, comprise the Reality Lab’s leadership team. They agreed to the course and Lanman began preparing the materials.
Putting the course together, Lanman said, was a major undertaking.
“The class goes through almost everything you need to know about the process of building VR headsets, the research that goes into it and actually building one over the course of the quarter,” said Andrew Wei, a graduate student who completed the course. “Then you get to go a little further and you get to explore a bit and do something in VR that you’re interested in, so it made it a great experience.”
As you might expect, the students needed to become very comfortable with computer graphics to set up the rendering engine, Lanman explained. In order to track the headsets, they also had to master a good bit of linear algebra, signal processing, and computer vision methods. Over six assignments the students programmed projects in JavaScript, WebGL, OpenCV, and Arduino frameworks. On top of all this programming, they also had to physically assemble their headsets.
“Ear Hands,” is a game to showcase the unique potential of VR to depict the position and direction of sounds correctly.
“For many, this was their first time building a piece of hardware, since most are CS students,” he said. “Learning to accept that you’ll break some, or many, parts is an important, albeit stressful, lesson. Thanks to funding from Facebook, it wasn’t an expensive lesson for the students! I’m happy to report that more than 30 students went through this course and are now the fearless hardware-software hackers I dreamed of inspiring.”
From sourcing hardware components for the headset kits to preparing comprehensive lecture notes, the course was custom-built from the ground up. Gordon Wetzstein, an electrical engineering professor at Stanford, had already built course materials over the last five years. He and Lanman have published numerous academic papers together and share an enthusiasm for augmented reality and VR display technology.
“My first call after getting the green light to bring this course to UW was to Gordon,” Lanman said. He had referred to this as an ‘experimental course’ when he first started offering it at Stanford and he said he was excited to see it ‘graduate’ to other schools.”
Building off of Wetzstein’s course, Lanman added his own spin, focusing both on computer graphics and computer vision. One unique addition was to implement a positional tracking system for the headset that used the students’ webcams. This eliminated the need for any custom hardware and meant that every student would be able to demo a fully working modern VR system to their friends and family, starting from a box of fairly simple parts. Lanman said they also spent quite a lot more time talking about the optics and display technologies behind modern AR/VR systems, “which happens to be what I work on in my day job.”
Kirit Narain was an undergraduate teaching assistant for the course. Last summer, Narain decided to try Wetzstein’s course on his own, out of an interest in VR and wanting to build his own headset. He said the two courses have some similarities and many differences. One being Lanman.
“490V leans slightly more into Doug’s expertise in displays, graphics and optics – and is more focused on understanding the cutting edge in each of these fields, including solutions still in the research stage,” Narain said. “There is also a bigger focus on augmented reality displays as well, which in my opinion makes it a more rounded class and gives students taking it an amazing insight into exactly how each of these systems works and prepared to take on their challenges.”
Lanman also added field trips and guest lectures into the course. After all, Lanman explained, Seattle is the growing center of the AR/VR universe, so how hard could it be to give students an opportunity to see the latest, greatest technologies — and meet the people behind them — at our local companies?
“As with the UW faculty, researchers and engineers at Microsoft, Valve, and Facebook Reality Labs were eager to be part of this course. This is the part of the course I’m most proud of: providing an opportunity to unite individuals passionate about AR/VR across the region and try to make something truly unique for the students,” Lanman said. “It was very exciting to see members of these companies, and others, offer guest lectures and sit in on some of the classes.”
Lanman said that building state of the art AR/VR display systems is not a theoretical or pure software exercise: his students had to get their hands dirty and yes, occasionally break things.
“I hope that this course put hardware projects on students’ radars. This course was designed for students receptive to this path: if you are scared of building a VR headset from scratch, you probably didn’t register. By the end of this course, I can see the students have really sharpened their maker skills,” Lanman said. “I told them at the beginning of the class that research is not as mysterious or difficult as it’s depicted in movies. I’m not sure they believed me. By the end, I am happy to see many students adopt the mentality of a graduate student: if you’re diligent in understanding what’s been done before, it’s not too difficult to figure out the path forward, even if it seems like walking into the unknown.”
For Narain, who was a first time TA, the experience was “unbeatable.” He was particularly keen to play a role in helping students to build a strong conceptual foundation and understanding of the numbers and algorithms, in addition to answering questions and helping them to debug their code. According to Narain, the course material was not easy, and having that maker attitude was crucial to student success. There also was a lot more nitty-gritty mathematics involved than most CSE classes, and working directly with hardware feels unfamiliar to some students.
“Having conquered these challenges over the last 10 weeks, the boost in student ambition and confidence could clearly be seen with their innovative final projects,” Narain said. “The novelty of AR/VR means there are no standard set of tools or methodologies for interaction yet. Watching the students experiment with different approaches and discovering what works and doesn’t is very interesting for me, too.”
Neil Sorens, a senior majoring in computer science, said he appreciated the way the class pulled together knowledge from many different disciplines: math, physics, computer science and various areas within computer science such as graphics and computer vision.
“Having an incredibly passionate and fun instructor was really motivating because the class was highly challenging,” Sorens said. “It was a good mix of practical and theoretical work, and we had a lot of freedom in choosing our final projects.”
During the virtual demo day, Sorens and his fellow students in the VR Systems class presented their final projects to Lanman, their TAs and each other. Allen School faculty and industry experts were also invited to drop into the Zoom meeting and see the final projects. Students worked on projects that varied, including hardware, body and hand tracking, eye tracking, rendering, audio, training and education, and even applications. In pairs or on their own, students presented a total of 19 projects as a culmination of what they learned over the quarter.
In addition to Jahn’s basketball training project, other presentations include:
360 degree vision using FOV compression (Neil Sorens): A headset with a 110 degree field of vision and equipped it with a 360 degree camera for specialized real-world applications to give users “eyes in the back of their head.”
Finger tracking using magnetometers (Alex Mastrangelo and Paul Yoo): A glove with magnetometers built in the fingertips that has high accuracy and low power for gaming.
Inverse kinematics and full-body tracking (Terrell Strong): A headset and controllers with full tracking points and animated arms and a torso to better understand a user’s sense of presence in VR experiences.
Exploring two-handed interactions (Andrew Rudasics): A task for users to manipulate a two-handed object to determine the most comfortable and accurate way to model for users.
VR Wings (Rory Soiffer and Everett Cheng): A game that allows players to fly like a bird through a virtual world by flapping their arms, using custom wing-like controllers.
Eye tracking for VR gaming (Alex Zhang): A shooting game using virtual reality by only using eye tracking techniques.
Real-time foveated ray tracing (Frank Qin and Anny Kong): Implemented foveated ray-tracing to make images more detailed and therefore look more realistic.
VR Volume Rendering (Xiao Liang, Jeffrey Tian and Nguyen Duc Duong): A 3D experience to use for medical scans in a more realistic manner.
Spatial audio for VR gaming (Thomas Hsu and Christie Zhao): A game that makes the player focus on spatial audio to navigate through the VR world.
VR batting cage (Dylan Hayre): A game that allows users to practice hitting baseballs and adjust their swing to build his or her skills.
3D drawing in VR (Daniel Lyu and Lily Zhao): A VR app that supports letter animation to enhance the learning of vocabulary.
VR galaxy tour (Natalia Abrosimova and Wenqing Lan): An experience that allows users to experience the galaxy through a VR headset.
Crime scene investigation (Zhu Lu and Weihan Lan): A 3D degree crime scene that allows users 360 degree view of the room to explore for clues.
VR dueling (Robin Schmit): A multiplayer game was created to sync real-time information from two or more headsets in order to play games with multiple participants.
We’ve all seen the images scrolling through our social media feeds — the improbably large pet that dwarfs the human sitting beside it; the monstrous stormcloud ominously bearing down on a city full of people; the elected official who says or does something outrageous (and outrageously out of character). We might stop mid-scroll and do a double-take, occasionally hit “like” or “share,” or dismiss the content as fake news. But how do we as consumers of information determine what is real and what is fake?
Freakishly large Fido may be fake news — sorry! — but this isn’t: A team of researchers led by professor Franziska Roesner, co-director of the Allen School’s Security and Privacy Research Laboratory, conducted a study examining how and why users investigate and act on fake content shared on their social media feeds. The project, which involved semi-structured interviews with more than two dozen users ranging in age from 18 to 74, aimed to better understand what tools would be most useful to people trying to determine which posts are trustworthy and which are bogus.
In a “think aloud” study in the lab, the researchers asked users to provide a running commentary on their reaction to various posts as they scrolled through their social feeds. Their observations provided the team with insights into the thought process that goes into a user’s decision to dismiss, share, or otherwise engage with fake content they encounter online. Unbeknownst to the participants, the researchers deployed a browser extension that they had built which randomly layered misinformation posts previously debunked by Snopes.com over legitimate posts shared by participants’ Facebook friends and accounts they follow on Twitter.
The artificial posts that populated users’ feeds ranged from the sublime (the aforementioned giant dog), to the ridiculous (“A photograph shows Bernie Sanders being arrested for throwing eggs at civil rights protesters”), to the downright hilarious (“A church sign reads ‘Adultery is a sin. You can’t have your Kate and Edith too’”). As the participants scrolled through the mixture of legitimate and fake posts, Allen School Ph.D. student Christine Geeng and her colleagues would ask them why they chose to engage with or ignore various content. At the end of the experiment, the researchers pointed out the fake posts and informed participants that their friends and contacts had not really shared them. Geeng and her colleagues also noted that participants could not actually like or share the fake content on their real feeds.
“Our goal was not to trick participants or to make them feel exposed,” explained Geeng, lead author of the paper describing the study. “We wanted to normalize the difficulty of determining what’s fake and what’s not.”
Participants employed a variety of strategies in dealing with the misinformation posts as they scrolled through. Many posts were simply ignored at first sight, whether because they were political in nature, required too much time and effort to investigate, or the viewer was simply disinterested in the topic presented. If a post caught their attention, some users investigated further by looking at the name on the account that appeared to have posted it, or read through comments from others before making up their own minds. For others, they might click through to the full article to check if the claim was bogus — such as in the case of the Bernie Sanders photo, which was intentionally miscaptioned in the fake post. Participants also self-reported that, outside of a laboratory setting, they might consult a fact-checking website like Snopes.com, see if trusted news sources were reporting on the same topic, or seek out the opinions of family members or others in their social circle.
The researchers found that users were more likely to employ such ad hoc strategies over purpose-built tools provided by the platforms themselves. For example, none of the study participants used Facebook’s “i” button to investigate fake content; in fact, most said they were unaware of the button’s existence. Whether a matter of functionality or design (or both), the team’s findings suggest there is room for improvement when it comes to offering truly useful tools for people who are trying to separate fact from fiction.
“There are a lot of people who are trying to be good consumers of information and they’re struggling,” said Roesner. “If we can understand what these people are doing, we might be able to design tools that can help them.”
