Mouse is Dead
The tech for spatial computing exists, but nobody's using it. I researched why (10+ participants, 3 research methods) and built the bridge: a collection of 3 gesture-controlled prototypes across 3 devices, each solving a real hands-free problem with just 3–4 gestures. Research-grounded, accessible (webcam only), and strategically placed in leisure/home contexts where users are most open to adopting new interaction patterns.
Gestural UI
Vibe Coding
Thesis Research
Designing the Transition from Screens to Space
Spatial computing technology already exists, Vision Pro, AR glasses, wrist sensors, but nobody's changing how they interact daily.
This thesis investigates why and designs the missing transition path. Through mixed-methods UX research and a collection of gesture-controlled prototypes, I identified that the core barrier isn't technology, it's the absence of gradual, human-centered transition design that respects cognitive load, context, and user comfort.
Problem
The future exists. Nobody's using It.
User Research
Understanding the Barriers
Five participants tested a gesture-controlled PDF reader prototype, alternating between traditional mouse/trackpad and gesture modes. The results show gesture mode dramatically increased body awareness and often felt intuitive and fun, but also clearly raised cognitive load.
Participants described "training my hands more than reading," struggling with gesture memory, similar gesture confusion (undo vs mark mode), and mis-detection anxiety. Traditional mode felt stable and controlled but left bodies static and disengaged. The radar chart visualization made the trade-off unmistakable.
Co-Design Workshop
Wall of Pain Points & Future Visions
Key Findings
Gestures Work, In the Right Context
Three findings shaped everything. Users actually enjoyed gesture mode — it felt playful and increased body awareness — but in focused tasks like reading, gestures competed for the same mental attention as the task itself. Mental burden was the #1 pain point across all research: mis-detection anxiety, gesture confusion, and overload. When designing their ideal futures, participants consistently chose small gestures, home settings, and always wanted fallback to traditional input. The insight was clear: the problem isn't gestures — it's context.
Prototype Collection
Strategic Design Path from Screens to Space
Instead of one prototype, I designed a strategic collection where each prototype bridges a specific adoption barrier using accessible Phase 1 technology, just a webcam or phone camera. Each targets a different device, cognitive load level, and daily context, forming stepping stones from screen-bound interaction toward spatial computing. The collection demonstrates that transition design should start in leisure/home environments, where people are most open to new interaction patterns, the strategic doorway to broader adoption.
Prototype #1
Gestural PDF Reader
Prototype #2
Cooking Copilot
Prototype #3
Photo Curator
Phone storage is always full, but cleaning up photos on a tiny screen is tedious. Photo Curator moves this to a larger display where users can actually see their photos before deciding.
Four gestures: swipe up to delete, left/right to navigate, heart shape to favorite, both hands up to undo. Low cognitive load, leisure context, home environment, exactly where research showed people are most receptive to new interaction patterns.
Design Rationale
Every Choice Traces Back to Research
Minimal gesture vocabulary (3–4 per prototype) because workshops showed complex sets cause overload. Familiar patterns like swipe-up-to-dismiss to leverage existing muscle memory. Undo gesture to address the #1 pain point, mis-detection anxiety. Hybrid touch/gesture modes because users demanded control over when their body is involved. Home and leisure contexts because relaxed environments are where people most readily adopt new interactions, the strategic doorway to broader change.
Strategic UX
Not Designing the Endpoint. Designing the Path.
The core barrier to spatial computing adoption isn't technology, it's the missing transition path.
This prototype collection demonstrates how to bridge that gap through strategic context selection, minimal gestures, familiar patterns, and hybrid modes.
These design principles can serve as a framework for anyone working on the shift from screen-based to spatial interaction. Strategic UX means designing the path that gets people there, not just imagining where they'll end up.
