BeekeeperAI
A zero-trust clean room platform accelerating development of clinical healthcare algorithms.
0-1 Product Development
Healthcare
Product Design
Multi-Engagement Project
0-1 Product Development
Product Management
Design Management
Client Management
BeekeeperAI is a zero-trust clean room platform connecting data stewards and algorithm owners (as a 2-sided marketplace) in order to collaborate for the advancement of clinical algorithm development.
I built this product as part of a consulting team via Launch Consulting in 2 engagements:
V0 — delivering a Figma prototype as a UX Designer in a team of 3 designers. Launched June 2021.
V1 — delivering a functional product and taking over as Lead UX Designer, working Product Manager, and UX Manager. Launched April 2022.
context.
A doctor's work stands between life and death. They need to pay meticulous attention at every turn while working in long shifts. An assistive tool, similar to a calculator, would make a huge difference.
AI is up and coming, and data is everywhere. What's standing in the way of developing these algorithms?
Data Quality
A viable algorithm needs to train against a lot of varied, high-quality data that fits its use case
Cost & Time
In the US, it takes approximately 3 years and costs an average $5 million for a single clinical algorithm achieve generalizability
Trust
Trust is paramount, because both algorithm developers and data holders own IP that could be easily misused
Introducing BeekeeperAI, a zero-trust(*) platform connecting data stewards and algorithm owners in order to collaborate for the advancement of clinical algorithm development.
Zero trust (ZT) is the term for an evolving set of cybersecurity paradigms that move defenses from static, network-based perimeters to focus on users, assets, and resources...
Zero trust assumes there is no implicit trust granted to assets or user accounts based solely on their physical or network location or based on asset ownership.
From the National Institute of Standards and Technology, U.S. Department of Commerce
challenge.
BeekeeperAI was built in 2 engagements:
Version 0 (3 months) is a clickable Figma prototype that the client demo'ed to raise funding & generate customer interest. The scope focused solely on matching data stewards with algorithm owners.
Upon Version 0's success, we built Version 1 (6 months) as a MVP product ("EscrowAI" now available on Azure Marketplace) that pre-contracted customers could now use to run algorithms against data and receive validation reports.
The 4-5 client Product Owners are all experts in their respective fields (medicine, cybersecurity, hardware), but they've never built software before and have big dreams.
How do I get them to all agree on a realistic vision so we can figure out what our must-have features are? In addition, how do I obtain usable product requirements when my primary knowledge sources have less than 5 hours a week to speak with me?
V1 is 6 months long, with the first 2-3 months devoted to defining scope and requirements.
Without a true product manager — how do I figure out what features to design, prioritize the right features, and communicate everything to development?
V0 proof of concept - research and design.
The goal of V0 (3 months long) was a final deliverable of a mid-to-high fidelity Figma-only clickable prototype that the client could use to demo to potential investors and customers (in particular, data stewards).
I served as a UX Designer in a 3-designer team that worked with 4 client Product Owners and an Engagement Manager.
V0 established the key concepts that V1 continued a few months later. The actual algorithm run was impossible to depict accurately without real technical resources, so the scope of V0 included only the initial matching and contract signing layer.
As a team of 3 UX designers, we uncovered key background concepts in data science, healthcare regulations, established the 2 main personas, and mapped the initial information architecture.
During primary research we met with 5 data stewards and 6 algorithm owners. We discovered that contracts were a sore point, so V0's scope concentrated on contracting.
We gathered requirements by running workshops with the 4 product owners to translate their expectations for each version of the product.
My team and I began with napkin sketches of the final 2-3 features, which translated to low fidelity and then mid fidelity wireframes. We chose not to show any example data through mid-fidelity (instead of high fidelity) because the priority of the demos was to show a high-level vision.
As we only had 1.5 months to design everything for this prototype, we made educated design decisions based on research but also made quite a few assumptions, including:
The project organization IA - we assumed 1 research initiative with multiple possible matches
1 single status defining each project - disproven by V1 as each project doesn't operate as a simple sequence because of multiple matches and constant re-running
Hierarchy of objects - instead of having to drilling down into each project, users should be able to access the test reports at the top level
Client product owners presented this prototype to great success! Investors and future users were so interested that they demanded the product now.
This led to Engagement 2 a.k.a. V1: building out the core product value.
V1 requirements and research.
The goal of V1 (6 months long) was a Minimally Viable Product (MVP) that turned into final EscrowAI product - main interactions and enclave activity, plus hook to get people in.
This time, I was the Lead UX Designer that also managed 2 other UX Designers a 3-designer team. I worked with the same 4 client Product Owners + 1 more Product Owner, an Engagement Manager, a Scrum Master, an Architect, and 7 total developers (working in Africa and Asia). As there was no product presence, I took on Product Manager duties to fill our team gaps.
Planning was absolutely key this time around, because we were actually building out the promised product.
During demos, the client product owners found that customers already matched and signed contracts outside of the system, so the real need was different from what we designed in V0.
Therefore, V1's scope was to build out the core project interaction features so that customers can perform their first runs.
Our clients lacked roadmap, and Launch's development side also lacked a roadmap because they had so many technicalities to figure out. Therefore, I need to make my own roadmap.
I made a design team timeline based on the length of the project and mapped that onto a calendar in 2 week sprints. The rest of the team followed this calendar as well.
This is the sequence of questions that I asked to drill down again and again in my design process. Every stage was just peeling off layer by layer of information, asking the client product owners questions in order for them to make decisions and then coaxing more information out of them gradually and systematically.
The questions didn't get neatly answered in 1 meeting — I had recurring meetings with all 4-5 client product owners where I continued to coax information gradually out of them that turned into requirements. This was incredibly important as the team did not have clear requirements before this.
I zeroed in on the specific parts of each process that were absolutely essential to this MVP.
Site map and feature prioritization
Once I had enough information, I built a sitemap of all proposed features. At this stage, there were still too many features to address.
I noticed how most of the functionality lay in the bottommost layer and all the pages above were just a way to get there. I ran a workshop with the product owners to nail down which of these bottom features were most important.
I ran a usability study with 3 data stewards and 3 algorithm owners by asking questions as they clicked through the V0 prototype.
Key findings:
Generally, a contracted project includes only 1 data steward and 1 algorithm owner
algorithm-side users may not actually be very technical. they are data scientists, not developers
data stewards are very strict about only allowing certain users to do or see certain things, and they are afraid of algorithm owner seeing too much about their dataset
V1 design and execution.
I had approximately 3 months to translate these requirements into final designs.
I spent this time mapping and confirming workflows (user steps, use cases, etc.) with the product owners and architect, as there was no begin development work until the team had solid requirements.
Development struggled to start because the clean room was theoretically possible but difficult in execution. I worked with the architect responsible for the clean room functionality to manage expectations — I explored an optimistic end goal ('what we can do') while the architect managed expectations ('what we can't do').
Due to the time crunch, I decided to use a pre-existing design system. I worked with my designer and the development team to select MUI, which had excellent developer documentation. We only changed a few colors and kept everything else boilerplate, which allowed development to work off of lower fidelity wireframes.
I also spun up technical documentation, which was developer documentation that I rephrased & reorganized for users doing technical onboarding.
I developed what I called a 'slice' model to link up the sitemap, wireframes, and Jira stories into a paradigm. This allowed the team to all speak the same modular language, and populated the Jira board neatly and efficiently.
Each feature group is a ‘slice’ (outlined vertically in the sitemap), each wireframe page is an ‘epic’, and each block on a page is a ‘story’.
Final site map for wireframing
Based on the top prioritized features, I created a final pared down site map of every page I would design.
Final designs
Designs - Home page (data steward version)
The data steward views their multiple projects — each project contains a description, a status, and last updates. Note: a data steward can change their home page view to algorithm owner mode (the top header color changes in response).
Designs - Project page (algorithm owner version)
The algorithm owner drills into a project to see their current tests, datasets, keys, and documents.
Designs - Algorithm Upload process pages
Each project requires 1 algorithm, which requires a multi-step wizard to add to a project. An algorithm owner must first obtain a token from BeekeeperAI, and only then can they upload and push their algorithm out in a docker container.
Note: due to time constraints, this flow bypassed the high-fidelity design point and was implemented from the mid-fidelity wireframes.
Designs - Initiate a Run pages
An algorithm owner can initiate a run after they import all the pieces to create a test, and then initiating that test.
Note: due to time constraints, this flow bypassed the high-fidelity design point and was implemented from the mid-fidelity wireframes.
results.
BeekeeperAI successfully launched in April 2022 , lists EscrowAI as a SaaS offering Microsoft Marketplace and as a partner solution in Azure confidential computing, and continues to gain more and more collaborators (most recently Massive Bio, datosX Digital Health Labs, cStructure).
I'm extremely proud to work on such an extraordinary project while at Launch Consulting!
Post-launch, I gathered all the initiatives that were discussed but ultimately deprioritized for the V1 MVP effort. These included:
Platform-wide notifications
Context-switching from a user’s Data Steward view to their Algorithm Owner view & vice versa
Admin experience
learnings.
This was my first time owning a product from start to finish, and marks my shift from a mid-level UX Designer into a Lead Designer. Through this experience, I found my niche — B2B & technical products.
I also discovered that I valued design freedom & end-to-end ownership of the product experience, and I loved the process of transforming loose premises and data into insights into a tangible product.
If I'm not given a product roadmap (or requirements, really), I don't have time to wait for somebody else to build one -- I'll build my own. Me taking initiative gives the whole team momentum to move forward.
With our product owners, I played Good Cop ('what can we do?') while my dev manager partner played Bad Cop ('no, we can't do that'). We triangulated using our complimentary superpowers to get what we wanted.
If I don’t understand something, it usually means other people don’t either. It’s alway worth the extra time & resources to ask questions even if it means feeling silly in the moment.
