Data Exchange Grants
Share and manage AWS data entitlements with trusted organizations members without monetization.
Iterative Product Development
Data Enterprise
about.
Product Design
Enterprise Product Development
Multi-Engagement Project
Iterative Product Design
CLIENT:
AWS Data Exchange (in-house)
DURATION:
5 months | V1 - Main effort
3 months | V2 - Additional effort
ROLE:
Senior UX Designer
AWS Data Exchange (ADX) is a service that allows AWS customers to easily share and manage data entitlements from other organizations at scale.
Previously, users had to register as AWS Marketplace sellers/buyers and list their data sets as a AWS Marketplace data product in order to exchange data. As ADX's in-house designer, I designed Data Grants to allow the option to purely exchange data between 2 parties without monetizing anything through AWS Marketplace.
context.
AWS Data Exchange (ADX) is a service that helps AWS customers easily share and manage data entitlements from other organizations at scale. ADX has hierarchy, delivery, hosting, and security within AWS suite of products, so it’s a great place to store and manage data.
Though the underlying structure was designed for pure data sharing, ADX was optimized to create and deliver monetized data products for AWS Marketplace. Therefore, users were required to also register for AWS Marketplace in order to use ADX.
With the new data grants feature, users can now use ADX as a standalone service to send and receive data sets without needing to register, list, or purchase data through AWS Marketplace.
Originally, a user needs to register for both ADX and AWS Marketplace to exchange data (in the form of data products). The registration process for a AWS Marketplace seller/provider requires setting up financial flows and listing at least 1 product (data or otherwise) publicly on Marketplace.
Following registration, a seller would go to ADX create a data set(s) and package it as a data product, which would list to AWS Marketplace. A corresponding buyer would see this product on Marketplace and purchase it, then go to ADX to gain access (a.k.a. entitlement) to that data set(s). This means jumping back and forth between both services.
According to prospective and existing ADX customers, there are 2 major reasons they want to use ADX for delivery without monetizing via AWS Marketplace:
External Billing
Customers are selling/buying data products and want to deliver via ADX but bill outside. Many already have their own billing system and would rather skip ADX entirely than overhaul their setup.
Internal Exchange
Customers simply don't want to charge money because they're exchanging data sets internally, using ADX's resources and hierarchy to organize and transfer.
Introducing ADX data grants — a lightweight method of sharing data that is a parallel path to the original ADX/Marketplace data product path.
challenges.
Data Grants was built in 2 projects:
Version 1 began April 2023 and ended November 2023 (launching at AWS Re:Invent 2023). This was the main effort, consisting of multiple new pages, and the redesign of a key existing page as well as the main service navigation.
Version 2 began April 2024 and ended November 2024 (launching at AWS Re:Invent 2024). This followup effort API-fyed the previously console-only experience and improved some backend processes from Version 1.
I carefully designed around the underlying architecture of ADX, as optimization wasn't our primary goal and we were operating on limited time and resources.
I optimized when I could — ex. I saw an opportunity to address a repeated page in the Entitled data nested pages. The product details page occurred in 'My Subscriptions' and also in 'Entitled Data', where the user had to click through multiple layers to access the data itself. I spoke with the former head of PM for ADX and learned that we were planning for users coming through AWS License Manager who might need additional monetary context.
I successfully convinced the team to take out the product details page for all data grants as there is no monetary context, leaving the product layer of data products alone. This cost no additional engineering time.
AWS's sheer size and hierarchal structure demands a unique and time-consuming system of checkpoints that require full approval from such parties as:
Product managers and product leadership
Related designers and design leadership
Engineering resources and engineering leadership
Launching a feature requires full approval from all parties at multiple points, which may take 1-2 weeks from each initial point. This requires working backwards from the launch date to schedule these reviews as 1-2 hour meeting blocks for 5+ team members. For design, the milestones of UX Sign off (design ends, engineering begins) and Fit and Finish (engineering ends) require special reviewers called bar raisers to sign off.
We launched V1 of this feature in November 2023, so I obtained design bar-raiser approval by September (2 months prior), which I applied for in July (2 months prior to that).
ADX was always synonymous with (monetized) data products, so how do we add an alternate/secondary product offering to the existing information architecture?
V1 requirements and research.
The goal of V1 (6-7 months) was to create the main data grants feature. The scope included creating all the new pages (sent and received, creation pages for a new unit) as well as modifying key existing pages and revamping the ADX's left navigation.
I worked closely with product and development to confirm user needs and finalize requirements.
Data grants started out as the ‘tech-only program' a.k.a. TOP — a lightweight specialized path fast-track existing functionality — 'how to get the other party data access ASAP without all of the formalities of ADX and having to access AWS Marketplace?'
We initially planned for the feature to not have much user interface and to be mostly programmatic (API-only) access, meaning that users can integrate data sharing into customers’ existing business processes via code.
Starting out, the PM and I weren't sure how big this feature would be.
We initially pitched it as 'lightweight' — are data grants simple enough to only live as data sets to sent/deliver within their entitled data sets page, or should they have their own layer of metadata and live on their own page?
On the other hand, this is a reforging of ADX's original product offering. ADX is a whole service, so should data grants exist as its own service? … but the process of registering a new AWS service is going to take forever.
Generally, there were overlaps between 'data owner' to sender and 'data consumer' to receiver, but the current ADX personas of sellers and buyers were not an exact match.
During this time, I also investigated the overlap with other AWS service Datazone which also manages data without monetization. Key differences include:
AWS Datazone organizes data by different business domains inside a single XXL organization, as it's marketed primarily toward internal usage.
Datazone is not self-service - a customer will purchase this service for their assigned AWS admin to use.
Soft requirements
Many requirements weren't confirmed. I designed multiple options for many pages so I could seek clarification by demoing them to users and engineers.
Examples of requirements that I later clarified:
1 share to 1 dataset, or 1 share to many data sets? —> answer: we only need 1 share to 1 dataset.
Does each grant need a human-readable name for each account? —> answer: no, account ID number is sufficient.
Does ADX need to clearly state and link directly to a contract to justify the sharing of data? —> answer: no, both parties contract outside of AWS so they don't need to justify anything.
PM and I initiated 14-15 video calls with existing ADX (seller) customers to discuss pain points, whether TOS will fit their needs, and resolve requirement questions.
Key findings:
The vast majority of customers only need 1 data set per 1 share, with no need for groupings.
Many customers have very specific existing billing configurations they don’t want to change. Most companies have different departments dealing with procurement/billing vs. data access, and don’t want to cross wires.
Each shared dataset needs to include metadata about the sending process for audit purposes.
There are no cases of receiver requesting the share. The sender will always initiate the share.
V1 design.
I designed concurrently alongside research and requirement-gathering. I clarified requirements through quick mockups and user interviews.
The following are examples of wireframe-specific decisions I made as the designer.
Definition: Entitled data = data that was purchased by the buyer and can now be accessed.
The Entitled data page was designed to house the following layers: Product > subscription > data set > revision (a.k.a. version). The original design allowed a user to access all 4 layers through a secondary navigation (in addition to the existing left navigation) without .
Reasons why it needed a redesign:
Data grants do not have a subscription layer.
New nav is not compliant with the AWS Cloudscape style guide, which causes friction with design bar raisers during milestone reviews.
The secondary nav is hardcoded at exactly 280px wide regardless of screen width, which truncates the each layer name and shrinks the available horizontal page space.
The user needs 1 click to open each layer, which adds extra clicks.
We needed to add the additional Data Grants feature to the left navigation, which gave me an opportunity to rehash the entire order.
Specific points that were raised:
Should we separate out Data Grants and Data Products into entirely different sections, or merge them into the same section? Conclusion: key actions for data products (accepting an offer, which transforms into a live subscription) does not translate into an equivalent action for data grants. Therefore, we must have separate, clearly labeled sections.
Should we continue to divide pages into sections based on the 2 Marketplace purchase phases + a separate provider section? Conclusion: defining sections using personas is necessary as many pages are persona-specific, but Marketplace purchase phases do not align exactly to Data Grant phases.
What to name the unit?
We needed a strong verbal identity to differentiate this new feature from the default ADX data products. The initial name was ’tech-only shares’ (TOS) which was simplified to 'data shares'. Unfortunately, 'data share' is far too generic.
Some considerations:
'Simple data share' or 'direct entitlement or share' was considered as this was similar to S3 (simple storage service)
Databricks expresses the ability to share with an outside consumer with 1 click as 'grant share’ or 'delta share' so they don't have a problem with 'share' as a generic action.
An existing customer said that they associate the word 'share' with AWS Redshift datashare.
I brainstormed some suggestions and held a workshop with the product and engineering teams to vote. Interestingly, the final combination of 'data grant' didn't get any votes during this workshop!
V1 wireframes
The scope of V1 was to create the main data grants feature. The scope included: creating/viewing/deleting both sent and received data grants, as well as redesigning entitled data and the left navigation.
Create new data grant
When the seller lands on the sent data grants page, they will see a list of steps (for first-time senders), and a table showing the data grants they've previously created. The 'Activity history' tab shows the log of all actions taken by you.
Before the sender can click 'create data grant', they need to first create a data set as an asset so that they can wrap a data grant around it. This is also the first step in publishing a product to the AWS Marketplace. Once a data set is created, the sender can start the data grant creation wizard by selecting the data set and giving it a unique name and description. They must enter AWS account ID of the account they want to send this data grant to, and specify when they want access to end.
It might take some time for the data grant to finish creating. You can see that the status is ‘in progress’ both here on the specific grant's details page, on the main sent data grants table, and on the Activity history tab.
Receive new data grant
When the receiver lands on the received data grants page, they will see a table that contains all of the data grants that have been shared with them that you need to take action on. If they are accepted or expire (see expiration date), they move to the 2nd tab as these data grants are no longer actionable.
A receiver can accept a data grant by selecting from the table and clicking ‘accept data grant’. They can also click into a single data grant first to see more information before accepting the grant. Upon acceptance, the link to the entitled data set becomes active. This data set is now entitled to the receiver, meaning that they now have to right to access and use the data.
Entitled data
A receiver accesses their allowed data sets in the Entitled data page on the left navigation. For V1 of Data Grants, this page has been redesigned into a table experience that is sorted by data sets first, with the AWS Marketplace product or data grant that the datasets came from shown in a separate column labeled ‘source’.
Note: for the current version, each data grant only has 1 data set, so there is no separate source for data grants and the receiver can access the data grant details on the data set page. This may change in the future if data grants can have more than 1 data set per grant.
New navigation
In order to position Data Grants alongside AWS Marketplace Data Products, I completely redesigned the left navigation.
I spotlighted data access (entitled and owned data — IMO the most important pages) and Data Grants in the top-most section, as data grants can be seen as a purer form of sharing data. This section contains both pages important to both the seller/sender and buyer/receiver personas.
The bottom section contains all Data Product-specific pages, which I have reordered into persona-specific subsections that no longer emphasize the 2 stages of buyer shopping. I've also renamed certain pages and sub-sections to specify how they are relevant only to AWS Marketplace data products.
V2 design.
The goal of V2 (6 months) was to API-fy the Data Grants feature, fix certain things we couldn’t implement in V1, and add the ability to redistribute a received data grant.
In August 2023, the ADX team was dispersed to other service teams with only a few developers left to support key functions — primarily, data grants. I rejoined the team temporarily in 2024 to support the launch of V2. The following are examples of wireframe-specific changes I made as the designer.
For V2, we added a new persona: the secondary receiver.
This type of receiver gains access to the data grant through AWS License Manager via redistribution from a primary receiver (the original receiver that this data grant was sent to). This secondary receiver gets access to the data set through the Entitled data page only. They can't view a received grant through the main received grants page or see any detailed information about the grant, and they definitely can't send the data again.
This meant that alongside all other page updates, I needed to design a settings page where the sender can opt into linking with License Manager.
A sender can now allow/disallow distribution from their (primary) receiver. To address this, I added to the data grant creation process.
A (primary) receiver can now redistribute their received data grant by going to AWS License Manager from their grant's details page or Entitled data details page. To address this, I added the ability to go to License Manager from the received grant's detail pages.
Instant data grants
In V1, it took some time for the data grant to finish creating. In V2, data grant creation is now instantaneous, so a grant will no longer need ‘in progress’ as a status or the Activity history tab to view incomplete jobs.
V2 wireframes
The scope of V2 was to add the ability to redistribute a received data grant. The scope included: updates to the sender's data grant creation process, updates to the received data grants details page, and updates to the Entitled data page.
Redistribute view
For senders, step 2 of the Data Grants creation wizard now requires an additional selection.
For receivers, if they can redistribute they can navigate to AWS License Manager straight from their received data grant (as long as they set it up first). Any secondary receivers who receive the redistributed data grant in Entitled data won't be able to see the 'source' (a.k.a. original grant details).
Settings
ADX now requires a settings page to manage the new integration with AWS License Manager. A (primary) receiver gets notified from their received grant's details page to set this up before they can perform the redistribution action.
result.
Developing data grants as a feature fundamentally changed AWS Data Exchange from a supporting player to AWS Marketplace into its own self-sustaining service.
Nowadays, ADX defines Data Grants as its primary function, and adds monetization through AWS Marketplace as an optional feature.
Although AWS Data Exchange's team is no longer as robust as before, possible additions for v3 may include:
Grouping of grants in main table
Contracting attached to grants
1+ data sets per grant — 2 additional layers of grouping
1+ data sets per grant
1+ grants per group
Ability for a data consumer to request a data set — this would require consumer to know the data exists, and let the producer know they want it. Need some form of discovery (a mini marketplace browsing page)
Possible future integration with snowflake so that data can be shared from snowflake (alongside ADX) from ADX
Ability to save data expiration presets
Ability to limit granting to 1 or more regions
learnings
Large organizations like AWS make big moves with far-reaching consequences — what used to be a case of ‘work with my team to add 1 alternate path to the main product offering’ turned into 'my team is now scattered to the winds and the alternate path is now the main product offering'.
I’m proud to have made such a huge impact on this service, and I'll always cherish working with such an exceptional, responsive team.
Leadership shakeups can cause completely unforeseen results. I couldn’t have predicted from the beginning that this small parallel feature I designed would become the main function of the service.
A technically dense project can be hard to onboard onto, but an incredible team makes it worth it. I had the most amazing product and engineering partners, and I absolutely leapt at the chance to work on V2 with some of the same folks.
Be audacious and take any opportunity to improve the existing experience. The left nav and entitled data pages wouldn’t have changed so profoundly if I hadn't insisted.
