Hybrid Interaction | Scalable systems | Retail AI
AiCap
At Onki.ai, I led UX design for a wine-retailing smart assistant robot AiCap, crafting hybrid voice and screen interactions and developing a scalable design system that enhanced user engagement and streamlined future development. The design framework was built to extend easily to other retail verticals, supporting broader product expansion.
Responsibilities:
Ideation, UI/UX/IxD, Prototyping, Design system
Role:
UX Design Lead (team of 3)
Time length:
2 month
Challenge
AiCap set out to build a scalable, multilingual smart assistant through thoughtful UX and flexible conversational design, addressing the high cost and growing need for effective shopping assistance in large retail environments to help customers confidently choose the right wine.
Solution
We saw the growing need for personalized shopping assistance in retail stores, but instead of creating separate solutions for each product category, we focused on designing a single scalable smart assistant platform. By combining hybrid voice-screen interactions, multilingual support, and a flexible design system, we enabled engaging, tailored experiences for shoppers while reducing operational costs for retailers and paving the way for expansion across diverse retail sectors.
It was a brand-new product space, and we had to determine how a smart assistant could outperform human assistance and identify the key customer experiences to prioritize. I focused on user journey mapping, prototyping hybrid voice-screen interactions, and conducting usability tests to guide our design direction.
Through rapid iteration, we explored different design concepts, interaction models, and visual patterns. Discarding early ideas helped us refine our approach, ultimately building a scalable framework ready to support diverse retail applications.
A Voice-First Wine Assistant MVP
Our initial MVP focused solely on enabling voice interactions to help customers inquire about wines in-store. It delivered basic responses to product questions, including price and shelf location, but operated as a standalone feature without yet being integrated into the broader product experience. This voice-first foundation allowed us to test core conversational flows and identify key areas for future hybrid integration and design refinement.
Validating Conversational Foundations
This MVP allowed us to test core voice interactions, uncover user expectations, and identify challenges in conversational accuracy and response design. The insights gathered shaped our approach for integrating screen-based elements and designing a cohesive hybrid experience in future iterations.
Competitive Environment
In large retail stores, choosing products like wine often requires specialized knowledge that staff can’t always provide consistently. Existing digital kiosks or apps rely heavily on screen-based interactions, often lacking conversational engagement and feeling disconnected from the physical shopping experience.
Concise Product Information Display
Created modular layouts for presenting key product details (e.g. price, location, attributes) in a clear, digestible way, forming a reusable component for different retail verticals.
Opportunity to Stand Out
While voice technology has become more common, it’s frequently treated as secondary to screens, rather than integrated as an equal part of the experience. We saw an opportunity to break through this barrier by designing a smart assistant where voice and screen interactions blend naturally, enhancing—not just duplicating—the in-store experience. Our goal was to build a system that engages customers in seamless dialogue, provides tailored recommendations, and bridges digital convenience with physical retail, all while being scalable across diverse product categories.
Guided Product Discovery
Designed conversational flows to help users articulate preferences (like taste, use cases, or price range) and receive tailored recommendations—a pattern adaptable for any product category beyond wine.
Hybrid Voice-Screen Transitions
Developed rules and visual cues for seamless shifting between voice and screen interactions, ensuring users never lose context—a scalable approach for various retail assistants.
Outcomes
Voice-only MVP launched to validate core conversational flows
Scalable hybrid interaction patterns for future retail verticals
Modular design system to accelerate future development
User confidence boosted in product selection
Over several months, we progressed from a standalone voice prototype to a scalable framework ready to support diverse retail experiences. We didn’t settle for standard voice add-ons but instead explored innovative ways to merge voice and screen into a cohesive user journey. I’m proud of how much we learned and the solid foundation we built for future products.
Dynamic Conversation Templates
Tested multiple interaction styles—from sales-driven to casual dialogue—allowing flexible tone and personality across diverse retail scenarios and future product lines.
Clear User Exit Paths
Established consistent methods for users to pause, exit, or shift conversations, ensuring the assistant remains helpful without being intrusive—a universal pattern for conversational experiences.
Voice-First Assistant MVP
We initially launched a standalone voice assistant capable of answering wine-related questions, providing product details, and suggesting recommendations. This MVP allowed us to validate core conversational flows and identify opportunities for improvement.
Seamless Hybrid Interactions
Building on early learnings, I designed transitions between voice and screen, enabling users to switch modes naturally without losing context. This became a reusable design pattern for future retail products.
Guided Product Recommendations
I developed conversational flows to help users describe their tastes or occasions and receive tailored wine suggestions, moving beyond static browsing experiences.
Behind the Scenes
None of this would have been possible without a strong collaborative environment. We held daily co-working sessions and weekly meetings focused on goal setting and reflection. We practiced pair design, frequent brainstorming, and built flexible processes tailored to our team’s needs.
Concise Product Displays
Essential product info—like price, shelf location, and pairings—was presented clearly and modularly, ensuring it could adapt to other retail verticals.
Teamwork Makes the Dream Work
I’m deeply grateful to my teammates for making all of this possible. Through this project, I was reminded once again how collaboration and diverse perspectives are essential for solving complex challenges and delivering innovative solutions.
Scalable Framework
All design patterns were built to be reusable, setting the foundation for expanding the assistant into other product lines beyond wine.
Collaboration and Transparency
To keep everyone aligned, I regularly recorded my ideas and design iterations using Loom. This approach helped our team stay synchronized, track progress, and share feedback in a way that was less formal yet highly effective.
OVERCOMING OBSTACLES
Creating a seamless hybrid assistant felt effortless for users—but demanded significant problem-solving behind the scenes. One of the biggest obstacles was defining consistent patterns and clear rules for when and how voice and screen should switch over, without confusing users or losing context.
I explored several research papers on hybrid interactions, which offered valuable insights but also highlighted how underdeveloped and evolving this field still is. While it would have been simpler to rely on standard screen-first designs, I insisted on prototyping and testing new flows. Through iteration and close collaboration with engineers and stakeholders, we established transition rules and modular patterns that worked not only for wine retail but could scale to future product lines. There’s still space to further refine these patterns as hybrid interaction research progresses.
Components that Speed Up the Game
We needed to test and scale rapidly, so it was crucial for our tools to support that pace. In building our design system, I created components with flexible structures, enabling dozens of variations and allowing us to iterate quickly and efficiently.
PRODUCT HIGHLIGHTS