What is Reet?
Reet is a company that is hiring for a marketing manager position.
Reet's Business Objective: Reet understands diversity increases business development, but the company also wants Reet's hiring managers to choose candidates without the influence of unconscious biases further into the hiring process.
User Problem: Hiring managers want to feel they have hired candidates without unconscious bias
Press Play to see the user flow.
The Solution in More Detail
User Problem: Hiring managers want to feel they have hired candidates without unconscious bias
Solution: Replace a standard phone interview with Luna Connect®'s blind interview with Q AI genderless voice technology. In the interviews age, sex, gender, race, sexual orientation, disability status, location, and accent will not be demonstrated.
Why Blind Interviews?
I have a lot to say on this matter, so please read more on the background of this case study through my Medium article:
In short: through the process of redacted resumes, blind chat/emailing, an initial blind interview- the hiring manager will feel confident they have selected the final candidates with the least amount of bias as possible.
What's out there already that uses AI?
AI objectively assesses soft skills and personality traits through blind interviews with Tengai. Weeds out candidates based on the customized criteria set for the role; AI scoring.
“Every candidate gets an interview.”
Candidates text on mobile in their own time with conversational chat AI. Every candidate gets insights back after the interview.
Caveats with these programs:
-Robot interviews don’t save time because recruiters/hiring managers go back to watch the video anyways
-A person can redo their response before they actually submit- this doesn’t necessarily demonstrate a person's true communication ability (but does it since we can edit after we send?)
-It's impossible to create an unbiased AI since humans design the system
A major constraint on this project was that I did not interview the “right” people at the right times. I initially started out surveying with general questions on interviewing, as a hiring manager and applicant, to see what key takeaways there were from these surveys
I then interviewed a handful of people from my boot camp.
Of the interviews, someone emphasized their desire that their company focuses on DEI.
When asked if they would want a blind interview most said no, they would want the real 1:1 interview because soft skills are just as important as hard skills.
Most were dubious on the blind interviewing technology; how well it works and how much soft skill can come across via avatar?
Quote From Interview Participant
"I would rather know as soon as possible into the interview process if there would be discrimination. I want to be SEEN right away."
Affinity Map Based on Surveys
Mind Map Related to Live Interviews
Flowchart for My Initial MVP Idea
The idea was for the candidate to practice ahead of time for their interview with a simulator or in real-time.
You can't ask the user what they want because they don't know.
I waffled quite a bit on whether to continue with my hypothesis that the user would want a blind interview. From the people I interviewed in my bootcamp, most had said they would not want a blind interview.
So, I continued with my hypothesis.
A Typical Hiring Process:
-Submit resume and cover letter
-In-person/live video interview with hiring managers
Reet’s Blind Hiring Process Using Luna Connect®:
-Redacted resumes and cover letters
-Blind chat AI communication and emailing platform
-Blind video interview with a hiring manager (the focus of this project)
-Final in-person interview with hiring managers
Reasons Why This is a Good Approach
-This would be a better initial interview process than robot interviewing, as it’s more personal.
-The hiring manager and applicant still get a sense of each other's character while mitigating unconscious biases, compared to a phone interview where you can't see body language and can be influenced by unconscious biases.
User Persona -Who is Luna Connect® for?
Hiring Manager X
X, a hiring manager for Reet, has decided to use Luna Connect®, a blind hiring platform to hire their next candidate for Marketing Manager.
I had started off mobile-first with the hiring manager connecting via text communication.
Some wireframe examples below:
I then thought it best to move to desktop (below) via email communication as there was a lot of important information being explained to the candidate.
The initial designs included the entire flow of the candidate's job search from Google, applying on Reet.com, getting an email in Gmail from the hiring manager to schedule the interview, the candidate scheduling on Reet.com, then getting an email confirmation for the interview in Gmail with the explanation of the blind interviewing process in the email. Some example wireframes are below.
However, since this original idea explained the blind hiring process AFTER the candidate had applied, I decided it best for the candidate to know BEFORE applying about the blind hiring process. This way they could decide if this blind hiring process was for them, before putting in all the effort of applying to a job.
And, considering the assignment was to build a minimal viable product (MVP), showing the entire application process with Google search and Gmail confirmations was not needed, so I eventually removed those aspects from my final user flow, and kept the flow contained within Reet.com.
The updated flow (below) lets the user review the blind interview information and then has a conversation with a chatbot to clarify any further questions.
Some examples of the mid-fidelity wireframes that I tested.
I wanted to test to see people’s thoughts once having seen the idea for the blind interview.
The test wasn’t as much to see how well built the app could be; it was more testing the concept.
-Blind interview in lieu of phone interview; which would then follow with in-person final interview
Key Results from Testing
I gained surprising feedback on this type of blind interviewing via testing.
Question to participants:
You have gone through the demo of the blind interview process.
Would you choose a standard phone interview or a blind video interview?
Out of 15 people:
-8 would prefer a standard phone interview
-7 would prefer a blind video interview
"I would choose a standard phone interview, so I could leverage my character + tone to show interviewers what kind of personality I have. If it was a large well-known company I would still apply, but nothing less. I feel like this could make me feel like I was being scammed without talking to /seeing real people."
"It's a cool option to have an interview. I think I would prefer an initial standard phone interview because at least we can tell the personality of the potential coworkers you will be working with. If a company approaches this way of interviewing, I would still apply for the job because if the job is the position I like, then I would try it out.
"I like the process. I would definitely apply with blind interview, because as an introverted person I don't want people to have a judgemental side about my appearance or anything else except my knowledge and experience."
"Q AI is super cool to implement. It’s creating the equality in recruiting."
"I think I would like to try the blind video if there's an option. Cool idea. I think this might be more unbiased."
“If I was a hiring manager, I think it could be helpful with an unbiased decision.”
“It's an interesting concept because it's different and it promotes an equality.”
"I would be hesitant. I think that there are too many ways that AI can screw people over, such as women's voices not activating voice recognition software. And my doubts increase the more layers of tech there are. So in this case, there's the voice editing, but also the expectation that the animations will mimic one's face appropriately...no thanks."
“I think there are small things like facial expressions and intonation in voice that would be missed in the masked video interviews. So I'm not sure.”
From my initial interviews, people said they would not like blind interviews. However, after testing, it appears with a more trusted interface and technology, candidates would be keener on this idea.
I overlooked adding the Help Center in the navigation bar during the testing process, so added this in afterward after feedback from user testing.
Avatars need to not demonstrate: age, sex, gender, race, sexual orientation, disability status, location, and accent.
-As mentioned before, I did not interview the “right” people at the right times.
-Not proper demonstration of the technology I am proposing to utilize for the interview process. Ideally I would have had a simulated interview with the avatars speaking through, Q AI, the genderless voice technology.
-Not possible to have a completely unbiased influence in an interview. Even the way people talk and write is indicative of their background, whether if English is their first language or how well trained they are in writing.
One of the most valuable comments from the Reddit post that I'll always carry with me as a designer is: If you had asked people years ago whether they would have wanted Netflix, they would have said, "No way, I like going to Blockbuster."
I had learned this through my studies so far on the UX research process but really saw this fact come into play. Also, considering I was not interviewing and testing with my ideal target audience (hiring managers/recruiters), I was surprised to see that almost half of the testers would have chosen the blind video interview over the standard phone interview, after going through the prototype testing. I think more people are coming to realize how much influence bias has in our everyday interactions and choices.
To the concern from some interviewees/testers that this process seems impersonal:
Again, what is "human interaction" these days as we know it?