Design for Spatial Input
tl;dr:
The post introduces an interactive system that uses hand-eye coordination to deliver an immersive user experience. Standard and custom gestures, precise navigation through eye tracking, and direct touch interaction with ergonomic consideration are key features. The system compensates for lack of physical touch with audio cues, and ensures accessibility via compatibility with assistive technologies, fully utilizing the spatial medium's potential.
Guidelines
- 👋 Hand-Eye Interactions: The system is primarily interacted with through hand gestures, supported by eye targeting, making for a more intuitive and immersive experience.
- 👌 Standard Gestures: Pinching, dragging, zooming, and rotating are standard gestures modeled after familiar multi-touch interactions on touch screen devices, providing a seamless transition for users.
- ✋ Custom Gestures: Developers have the ability to define custom gestures for unique app behaviors, provided they are distinctly different from standard gestures and don’t strain the user or have a high rate of false activations.
- 👁️ Eye Tracking: The system utilizes eye direction as a signal of intent, allowing for precise and satisfying interactions that feel natural and intuitive.
- 🖥️ Interaction Origin: The start point of certain interactions, like zooming or pointer movement, is determined by the user’s eye focus, enhancing the precision of interaction.
- 💻 Direct Interaction: The system supports direct touch, letting users interact with the virtual environment using their fingertips, such as scrolling through a page or typing on a virtual keyboard.
- 💪 Ergonomics Consideration: When designing for direct interaction, it’s essential to consider the ergonomics to avoid user fatigue, especially for apps that require more direct touch.
- 🔊 Audio Feedback: Audio cues are used to supplement visual cues, providing more comprehensive feedback in the absence of physical touch, making interactions feel more satisfying and reliable.
- 🧑🦯 Assistive Technology Compatibility: The design and development of interactions should consider those using assistive technologies, ensuring accessibility for all users.
- 🌐 Exploiting Spatial Medium: The combination of hand gestures and eye input allows for the creation of novel and delightful interactions, fully exploiting the potential of the spatial medium.
Keywords
- Hand-Eye Coordination: The ability to track the movements of the hands and adjust the direction and power of these movements based on the visual information perceived.
- Multi-touch Interactions: Multiple points of contact with the interface, such as pressing two fingers on a screen to zoom in or out, or twisting them to rotate an image.
- Custom Gestures: Specific, unique movements designed and defined by developers for their applications, not part of the standard system gestures.
- Eye Tracking: The process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head, used in the system to allow precise and intuitive interactions.
- Direct Touch Interaction: Interaction that involves physically reaching out and touching the screen or interface to control the system.
- Ergonomics: The practice of designing products, systems, or processes to take proper account of the interaction between them and the people who use them, taken into account here to avoid user fatigue.
- Audio Cues: Sound signals used to provide feedback and guide the user’s interaction with the system.
- Assistive Technologies: Array of devices that allow people with disabilities to interact with technology and data, considered in the system to ensure accessibility for all users.
- Spatial Medium: The use of physical space for interaction in a virtual environment, exploited fully in this system to provide immersive experiences.