Figure, an AI-powered Figma plugin
Winner: Judges' Recognition, CMU XHacks 2023
My role: As one of the team's product designers, I was responsible for creating the wireframes, prototypes, and refining the visual design of the final product.
Team: 3 designers, 2 researchers
Timeline: 48 hours
Project type: Design challenge
Project Overview
Context
XHacks is a 48-hour design challenge hosted by Carnegie Mellon University. In 2023, over 20 teams were challenged to create AI-powered solutions for user groups with unmet needs.
Problem
Designers are wasting too much time.
​
As designers ourselves, my group was all too familiar with how much time we spent on basic, tedious tasks. Designers often have to consult outside resources and backtrack to fix mistakes, leading to lost time and inconsistencies.
Opportunity
How can we leverage AI to streamline the design process and make the designers' work more consistent?
Solution
Figure is an AI-powered Figma plugin that helps designers build shippable products faster.
Through use of image recognition, personalization & recommendation capabilities, and generative AI, Figure helps designers quickly create designs that are consistent with design systems and best practices.
Figure identifies and predicts design elements via image recognition, which speeds the transition from low to mid or high-fidelity prototypes.
Figure analyzes existing designs to establish patterns, then generates new variants. Designers no longer have to create every element from scratch.
Research
Understanding the current state
I conducted semi-structured interviews with product designers working at companies of various sizes to understand their design processes.
​
In the current workflow, designers must frequently backtrack and redesign screens to fix mistakes.
Identifying pain points
Through interviews, we learned that designers encounter many of the same obstacles:
​
1. Moving from low fidelity to mid or high fidelity is tedious and time consuming. Even simple things like placing dummy text requires a lot of work when designers have to create them from scratch.
​
2. The need to consult outside resources interrupts their workflow. This can include consulting design system documentation or looking at other designs for inspiration and best practices.
​
3. Maintaining consistency is difficult. Designers using a design system may use the wrong styles or components. When working on a new design system, creating these elements from scratch requires a lot of editing, which can lead to widespread inconsistencies.
​
4. Designers working with an existing design system spend too much time searching for the right thing. Searching through endless component libraries takes up valuable time.
New proposed workflow
We saw opportunity to speed up parts of the design process and reduce backtracking by catching mistakes before they occur.
​
I created a new workflow, highlighting opportunities for AI to assist designers.
Prototypes
Concept testing
My biggest question about our idea concerned acceptability: would designers actually use an AI tool if it existed?
To answer this, I sketched the basic functionality of the tool and conducted cognitive walkthroughs with users to learn more about their feelings and interpretations.
User feedback:
"If something like this really worked, yeah I would use it all the time. Especially going from sketches to lo-fi, I could see that being really helpful."
– Product designer with 3 years of experience
"I don't know if I would use this... what if the AI makes something that I sort of like, but I want to change it a little? Could I edit those options, or do I have to start over?"
– Product designer with 5 years of experience
"I don't know if I would trust an AI to know exactly what I need. I still want to be in control, and I feel like circumventing the AI tool could be a lot more work than just sticking to my usual process."
– Product designer with 3 years of experience
The takeaway: Maintaining control is important to users.
Users wanted to easily and quickly edit AI-generated elements to better suit their needs.
Revised design: adding editability
To ensure users stay in control, I added the ability to quickly and easily edit AI-generated elements.
I created mid-fidelity prototypes focusing on the details of Figure modeless window, now with edit controls so users can modify the type of element and the visual design.
When we tested again, user acceptability had improved significantly.
"I feel like even if I did want to change the font or the color, just having the building blocks of a piece of text would save me a lot of time."
– Product designer with 4 years of experience
Final Design
With confirmation that our product would be valuable to users, we jumped into the final design.
​
We used Figma's design system to ensure Figure integrates seamlessly with the platform. This strategy also leverages users' familiarity with Figma's design patterns to make Figure more intuitive and trustworthy.
Final prototype
Figure's modeless window helps designers create screens quickly.
A closer look at editability in the final design: In order to provide users with more control and transparency, I added more information about each AI-generated element (such as font family, size, and weight) to the Figure window.
Before:
After:
Figure can surface design system components or create new ones.
Figure makes recommendations based on previous designs and existing design systems, supporting consistency and reducing the amount of time designers spend searching through component libraries.
.png)
Figure recommends a design element
Figure recommends components from the component library.
Users can generate new components consistent with their design system.
Future Opportunities
As LLMs become more advanced, Figure could continue to assist designers in new, innovative ways. These are some additional features that may be possible in the near future:
Heuristic Analysis: AI could analyze other designs found online and use pattern recognition to create a model of best practices in UI, then make recommendations to the designer.
Heuristics check would be incorporated into the Figma tool panel.
Check accessibility standards in real time: Designers currently use outside resources to check if their work meets accessibility standards. What if AI could flag violations directly in Figma?
Figure would automatically flag elements that violate accessibility standards.
Impact & Learnings
Impact
My team placed 3rd out of over 20 teams, winning the prize for Judges' Recognition.
"[Figure] is a clear and practical application of AI. That, paired with its adaptability for companies of all sizes, highlights its impressive ability to meet real-world design needs."
– XHacks Judge
Learnings & Takeaways
I spent too much time trying to test everything with a user. Given the short timeframe, I should have instead identified my riskiest assumptions and tested those immediately.
​
However, this experience did show me the value of testing early. Even with nothing but our low-fidelity sketches, I was able to get valuable feedback from users and make necessary changes early in the design process.
My team at the XHacks award ceremony