When you look at a knitted panel head on, all you might see is gray noise, but taking a step to the side and looking at an angle reveals another image — a hidden Mona Lisa.
This technique is called illusion knitting, where stitches of varying heights can create different images depending on what angle you view the piece. Traditionally, the intricate process of designing these illusions, by selecting stitches, choosing their colors and deciding whether they are raised or not was done by hand. Now, a team of Allen School researchers have introduced computational illusion knitting. The design framework helps automate the process by computationally generating knitting patterns for input designs, making illusion knitting more accessible and allowing for more complex and multi-view patterns that were previously impossible. The researchers presented the paper at the SIGGRAPH 2024 conference.
“If we write down the mathematical properties of the illusion knits, then we have a foundation with which to create algorithms that help the user iterate on and optimize their designs,” lead author and Allen School Ph.D. student Amy Zhu said. “With this foundation you can start to think, ‘where can I push the boundaries of what’s possible with illusion knitting?’”
To create an illusion knit object, the researchers first characterize the design’s unique microgeometry, or the small-scale geometry on the surface of a knitted piece. The goal is to then add a layer of abstraction over the complex microgeometry to streamline the design process, Zhu explained. This layer uses logical units called bumps, or raised sections made by a knit with a purl in the next row below it, that can block previous stitches, as well as flats, where the knit surface is level and is made by a knit stitch with another in the next row.
These observations led the researchers to develop various constraints that capture both the viewing behavior, or what image is seen at each angle, and the physical behavior, also known as the result from the knitted design or fabrication choices. Single image illusion knits, such as the hidden Mona Lisa, are easier to design as they only change the color between rows.
Using computational techniques, the team became the first to design and execute illusion knitting patterns that require mixed colorwork and texture between rows. The researchers first used automated methods such as gradient descent and the maximum satisfiability problem (MaxSAT) to find a compromise that satisfies the most constraints and relaxes others, Zhu explained. However, these systems are still limited by how well they can express constraints about readability, knittability as well as how accurate the theoretical model is to reality.
Instead of relying solely on the algorithm for solutions, the researchers created a user-in-the-loop framework that allows the user to easily edit the design. The user can then further simplify the design by breaking it down into tiles made of bumps and flats to fill the image.
“There’s many tricky things to consider when you’re in fabrication land that are often difficult to anticipate or capture when working theoretically — maybe the machine settings are wrong and the white stitches are smaller, making the contrast between colors lower,” Zhu said. “One reason I like our approach is that we put the fabricated reality first, so our priority is providing ways for the user to edit things that come out of the machine, so they can observe if something didn’t look right and know how to fix it.”
For example, using a knitting machine and this design framework, Zhu created a knit piece that shows artist Vincent Van Gogh’s self portrait from one angle and his sunflowers from another. The quantized input image of the self portrait had too much noise around his head which made the design unreadable when knit. For a cleaner result, all she had to do was edit the input image to remove the noise or modify the tiles.
Zhu said she hopes this framework gives users the foundation to create more innovative illusion knit designs such as clothes that display images with certain poses or knit animations. Next, Zhu is researching ways to capture and explore different fabrication plans of knit objects.
“Approaching crafts from a computer science angle allows us to expand the boundaries of what can be achieved — whether that’s developing new algorithms for machine knitting, creating novel design tools or even inventing new forms of visual art,” said senior author and Allen School professor Zach Tatlock, who co-advises Zhu alongside colleague Adriana Schulz. “It’s fascinating to see how the rigor of computer science can enhance the creativity of traditional crafts.”
Additional authors include Allen School Ph.D. students in the Graphics and Imaging Laboratory (GRAIL) Yuxuan Mei and Benjamin Jones, along with Schulz.
Read the full paper on computational illusion knitting.