INTRODUCTION
Currently, there are 285 million visually impaired in the world with 60% of which are female. In the fast-growing makeup industry, the visually impaired are being under-served with no services that enhance the accessibility of checking makeup for them. The goal of this project is to provide an environment where doing makeup is possible, feasible, and easy for the visually impaired.
​
Alexa Demo Day Second Place
MY ROLE
User Researcher
Project Lead
SKILLS
Market Research
Contextual Inquiry
Simulation
CUI Design
Video Editing
TEAM
Zhaoqi Ge
Mengqing Jiang
Sijia Li
Ally Liu
Melissa Shi
Qian Wang
Wei Wang
Menghan Zhang
00
Beauty+ Overview
The Solution to Problems
What is Beauty + ?
An Alexa Skill hosted on Echo Look
that provides auditory assistance and feedback to the visually impaired makeup users to better streamline and support their makeup routine.
Targeted user groups
Novice visually impaired makeup users
Experienced visually impaired makeup users
What problems is Beauty + solving and how?
Users need constant feedback from others to check makeup
Provide checkup on makeup look against a reference look
Lack of tips that are tailored to visual impairments
Provide makeup tips that are tailored to visual impairments
Novice users have hard time remembering the completion of each step
Provide step-by-step walk-through of makeup routines
01
Who Are the Users?
Contextual Inquires with Real Targeted Users
Current situation...
285
million
visually impaired in the world
85%
women feel empowered by makeup
48.3
billion
makeup industry
* Research gathered from World Health Organization and Bustle.
There is a large under-served population.
Be more empathetic through simulation
To better empathize with the visually impaired, especially the blind, the team simulated what it is like to do makeup blind. Depending on members' experience, we simulated 3 different scenarios.
1) Novice users learning from Youtube tutorials
2) Novice users learning from mom or a friend
3) Experienced users with personal makeup routines
Insights:
Actionable makeup tips are crucial for novice makeup users.
Using facial structures as landmarks when communicating is more effective.
Contextual inquiries with the targeted users
Interviewing visually impaired and blind individuals on their typical makeup routine, we came to several findings that helped us with narrowing down our targeted user group.
Blind individuals has less comprehensive understanding on how to complete a desired makeup look.
Despite experience level, all visually impaired individuals desire feedback on how well their makeup looks.
Popular makeup tips are not tailored to the visually impaired's needs.
Visually impaired tends to stay with the same makeup look to guarantee the best makeup outcome
Design for the novice and experienced visually impaired makeup users
Targeted user groups
After user research, the team decided to design for the visually impaired, specifically people who have some degree of vision because the blind have less comprehensive understanding of makeup. In order to design for the blind population, an educational component will need to be incorporated.
Novice makeup users
Experienced makeup user
Needs of the targeted users
Self-fulfillment
Not let her disability prevent her from becoming her best self
Esteem needs
Feel confident about how she looks
Belongingness and love needs
Join the makeup community like the well sighed
Safety needs
Find a secure way of catching all the makeup misses
Physiological needs
Have clear instructions on makeup procedures
02
Project Objective
Define the Project Scope
To design and develop an Alexa Skill hosted on Echo Look that provides auditory assistance and feedback to the visually impaired makeup users to better streamline and support their makeup routine.
Design criteria
Must have
-
Check makeup completion
-
​Provide feedback on makeup completion
Should have
-
Auditory feedback
-
Instruction on fixing makeup
-
Analyze makeup against a reference
Nice to have
-
Provide makeup tips
-
Provide makeup walkthrough
-
Provide cosmetic recommendations
-
Help user stay trendy
-
Allow users to share their look
03
Market Potential
Who are the Players and is it Profitable?
How big is the makeup industry...?
68.7
billion
Color cosmetic makeup industry value
5.5%
Cosmetic industry growth
84%
Women feel empowered by beauty
* Research gathered from Statista and Bustle
Makeup industry is ever-growing with high market value.
Who are the players in makeup tech?
To benchmark our design against the current players in providing services in the makeup industry, I researched and compared the current competitors to understand how can our design stand out.
There are no direct competitors in providing services tailored to the needs of the visually impaired makeup users!
04
CUI Ideation + Development
Trial and Error in Solution Discovery
Dialogue flow...
Main Flow
Core Features
User Profile
Global Intents
Beauty Shop
Organize typical makeup process and tips associated
Testing the responses
Beauty+ response should be concise in content and provide users with possible next step options!
Beauty+ response should be tailored to the expertise level of the user.
1.Dip your eyeshadow brush into the light shade, then tap the brush to get rid of any excess product.
2. Apply the lighter shadow across the entire lid, starting at the lash line and ending just above the crease of your eyelid.
3. Now dip your brush into the darker color, tapping off the excess once again. Apply the color at the outer corner of your eye, just above your lash line. Sweep the darker color across your eyelid crease, just under the brow bone.
4. Stop application around the center of your eyelid, as you don’t want to darken the inner corners.
5. Take a clean shadow brush and blend the two shades together. If you’d like a more intense look, re-apply the darker shadow once more.
Before revision: Walkthrough response for expert users
Step 3, apply your eyeshadow. Let me know when you are ready for the next step.
After revision: Walkthrough response for expert users
05
Final Design
Providing Independence to the Visually Impaired Makeup Users
01
PROVIDE OVERVIEW OF MAKEUP LOOK
User can save several desired makeup looks and select from them every time they are using Beauty+ to complete a look. Upon selection, Beauty+ will provide an overview of the selected look to help novice users getting prepared to start the makeup process.
02
HELP CHECK MAKEUP COMPLETION
While completing their makeup, users can ask Beauty+ to check their look to make sure that they did not miss any spot or made any mistake. The can check their look for between steps and after full look completion which significantly decrease the likelihood of having uncaught mistakes.
03
PROVIDE MAKEUP TIPS DURING MAKEUP
While doing makeup, users can ask Beauty+ to provide tips for better completing the makeup and using makeup products. For novice users, this is a great way for them to learn the skills necessary while practicing doing makeup.
04
PROVIDE MAKEUP PROCESS WALKTHROUGH
Because novice makeup users usually have a harder time remembering all the steps to completing a look, Beauty+ offers makeup process walkthrough. The burden of remembering all the steps are eliminated for novice users, making this a much more enjoyable activity and making learning makeup easier.
05
SHARE FINAL LOOK TO DOUBLE CHECK
Since visually impaired makeup users trust their closed ones with gauging how well their makeup looks, Beauty+ incorporated this final look sharing feature where users can get a second layer of reassurance for their final makeup outcome. If the look is good, users can also save it to the database as a reference picture.
Behind the scene CUI structure
6 intents
-
StartGuide
-
NextStep
-
CheckIntent
-
CheckTips
-
ShareResult
-
EndSkill
Activate skill
"Alexa, open Beauty+."
* Underlying CUI intent model
How to identify bad makeup?
VS.
Reference image
Image of completed makeup
The main idea behind the technical structure of Beauty+ is to find a reference picture for an ideal look that users want. Every time the user request to have his/her makeup checked, Echo Look will take a picture of user's completed makeup look and compare it to the stored reference picture to make a judgment on whether user's final look is good or not.
Due to limited access to Alexa Echo Look's development features, the team had to find a workaround for taking pictures for computer vision analysis and having access to these pictures for alteration and comparison purpose.
The team got No.2 at Alexa Day!
Behind the scene technical structure
* Current technical structure and workaround
* Ideal technical structure if having access to Echo Look development features
06
Next Steps
Iterating and Refining Our Current Solution
The current solution still is not perfect and a lot of room for improvement such as the instability of performance due to the narrow threshold under different lighting condition and the low accuracy of the landmark API that we are currently using. Additionally, there are several additional features that we would like to either develop further or incorporate into Beauty+ namely
1)
Ability to receive real-time feedback from others on the shared final look;
2)
Provide cosmetic product recommendation to the visually impaired user, which also generate more revenue for Amazon;
3)
Lastly, provide updates to users on the latest makeup trends to help them stay connected with the bigger makeup community.