User-centered AI Development Framework
Project overview
The company wanted to gain a better understanding of their HR recruiters' work practices, so the development teams could improve the UX and adoption of their AI products. HumanInc also wanted to understand how their data science teams (AI development teams) worked in order to improve the management of AI development projects, as it was suspected to be the cause of poor adoption.
My Role
Led Researcher & Management Consultant
Tools
Qualtrics: For generating quantitative user research insights through surveys.
Dovetail: For analyzing qualitative user research from interviews.
Atlas.ti: For analyzing qualitative user research for ethnographic transcripts.
Duration
July 2021 - February 2023
Adopting AI for work tasks
HumanInc consultants are not adopting the AI products developed by the company, as these consultants are more focused on meeting their KPIs and recruitment quotas. In the Netherlands market, only 15% of consultants are using AI-based applications.
Business opportunities
Increase AI adoption, as the business spends a lot of money in developing AI systems in-house.
Provide sufficient information that prepares data scientists for improving AI-based products and reduces lack of trust (driving adoption).
There is a bigger goal: Change and optimize the development process of AI-based products to make them more user-centered (in this case worker-centered).
Research plan
To address the business opportunities, I conducted a 20-month ethnographic field study within HumanInc HQ in the Netherlands.
I used ethnographic research and contextual inquiry to capture the sentiment and behavior of participants in a tacit and accurate way. By spending time with recruiters, managers, and data scientists, I was able to understand their context and experiences as “one of their own”, allowing me to better comprehend their needs and pain points.
From the initial interviews I conducted during the ethnographic research, I learned that these are the most important questions I needed to solve for:
How do recruiters use AI-based products?
How do data scientists develop AI-systems and what can be done to improve the process?
Ethnographic study
During the study, I participated in over 150 company meetings and 8 shadowing sessions with the development team and recruiters.
I also took part in informal company events to observe how participants behaved in more casual, non-structured settings.
In addition, I conducted 35+ semi-structured interviews and 15 unstructured interviews at various stages of the development process, engaging with ML engineers, user researchers, data scientists, recruiters, and data engineers.
Design challenges
Throughout the research project, I identified several problems with HumanInc’s AI development process and its use by HR recruiters. These challenges were vital to ideate a design solution for the company. Some of the most important were:
Recruiters: Recruiters were not using the AI system due to lack of understanding of their work practices by the AI developers.
UX Designers: UX designers from HumanInc did not know how AI technologies work so they could not support the product adoption.
Data Scientists: Data scientists behaved as “creatives” while developing AI, which did not align with HumanInc IT management processes (mechanistic / agile). The company did not have any knowledge on how to manage IT creatives.
Solution: AI Development Framework
As a solution to the development problems I found at HumanInc, I co-designed a management framework and guidelines with the help of researchers of the KIN Center for Digital Innovation. We defined 3 Main Guidelines that managers and organizations should follow to ensure the successful development of AI systems: Create a Shared AI Vision, Build a Common AI Understanding, & Develop Complementary Abilities to Manage AI Limitations.
