An augmented reality (AR) device is interfaced to an Internet of Things (IoT) device by receiving IoT device information in a standardized schema from the IoT device. The IoT device information includes device inputs and available output information for the IoT device. A predetermined AR user interface widget to render for the IoT device is determined from the received IoT device information. The predetermined AR user interface widget converts user inputs to the AR device into the device inputs for the IoT device. Upon selection of the IoT device by the AR device, the predetermined AR user interface widget for the selected IoT device is provided to the AR device as an overlay on a display of the AR device. Device input received in response to user interaction with the AR user interface widget is provided to the IoT device in an input type expected by the IoT device.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
2. The method of claim 1, further comprising receiving location information from the AR device.
3. The method of claim 2, further comprising selecting the IoT device based on a proximity of the AR device to the IoT device.
This invention relates to augmented reality (AR) systems that interact with Internet of Things (IoT) devices. The problem addressed is efficiently selecting and controlling IoT devices in an AR environment, particularly when multiple devices are present. The solution involves an AR device detecting nearby IoT devices and selecting one based on proximity to the AR device. The AR device may display a virtual representation of the selected IoT device, allowing a user to interact with it through the AR interface. The system may also determine the relative position and orientation of the AR device and the IoT device to enhance interaction accuracy. Additionally, the AR device may receive user input to control the selected IoT device, such as adjusting settings or triggering actions. The invention ensures seamless integration between AR interfaces and IoT devices, improving usability in smart environments.
4. The method of claim 2, further comprising selecting the IoT device based on verbal input from the user or based on an orientation of the AR device toward the IoT device.
This invention relates to an augmented reality (AR) system for controlling Internet of Things (IoT) devices. The system addresses the challenge of efficiently selecting and interacting with IoT devices in a cluttered environment, where manual selection via menus or voice commands may be cumbersome or inaccurate. The invention improves upon prior methods by enabling device selection through either verbal input or spatial orientation of the AR device toward the target IoT device. When a user provides a verbal command, the system identifies the intended IoT device based on natural language processing. Alternatively, the system uses spatial tracking to determine which IoT device the AR device is oriented toward, allowing for intuitive, hands-free selection. This method enhances usability by reducing the need for precise input and minimizing errors in device identification. The system may also include features such as displaying device status or control options in the AR interface once the device is selected, providing a seamless interaction experience. The invention is particularly useful in smart home or industrial environments where multiple IoT devices are present, improving efficiency and user satisfaction.
5. The method of claim 1, further comprising automatically presenting the predetermined AR user interface widget upon selection of the IoT device.
Augmented reality (AR) systems enhance user interaction with Internet of Things (IoT) devices by overlaying digital information onto the physical environment. A challenge in these systems is efficiently providing users with relevant controls and data for IoT devices without requiring manual navigation through complex menus. This invention addresses this issue by automatically displaying a predetermined AR user interface widget when a user selects an IoT device. The widget is tailored to the specific device, offering direct access to its functions and status information. The system first identifies the IoT device, such as a smart thermostat or lighting system, and then retrieves the corresponding AR widget from a database. The widget is rendered in the user's AR view, positioned near the device, and includes interactive elements like buttons, sliders, or status indicators. The widget may also adapt dynamically based on the device's current state or user preferences. This approach streamlines user interaction by eliminating the need for manual searches or menu navigation, improving efficiency and usability. The invention may also include features like voice control integration, gesture recognition, or multi-device management within the AR interface. The system ensures seamless interaction by continuously monitoring device selection and updating the displayed widget accordingly.
6. The method of claim 1, further comprising receiving state data from the IoT device and providing the state data to the display of the AR device.
7. The method of claim 1, wherein receiving AR device input in response to user interaction with the predetermined AR user interface widget displayed by the AR device comprises receiving hand gestures or body gestures from the user.
Augmented reality (AR) systems enhance real-world environments by overlaying digital content, but traditional interaction methods often rely on physical controllers or voice commands, which can be cumbersome or impractical in certain scenarios. This invention addresses the need for more intuitive and natural user interaction within AR environments by enabling gesture-based control through AR user interface widgets. The system involves an AR device that displays a predetermined AR user interface widget, which serves as a visual anchor for user interaction. The device captures hand or body gestures performed by the user in proximity to or in response to this widget. These gestures are processed to determine user intent, allowing for seamless control of AR functions without requiring physical controllers or voice input. The widget may include visual elements like buttons, sliders, or menus, and gestures such as swipes, pinches, or specific body movements are recognized to trigger corresponding actions within the AR environment. This approach improves usability by leveraging natural human movements, reducing reliance on external input devices, and enhancing immersion. The system is particularly useful in applications where hands-free or gesture-based interaction is preferred, such as industrial training, gaming, or remote collaboration. The invention ensures that AR interactions remain intuitive and responsive, aligning with the growing demand for more natural user interfaces in digital environments.
8. The method of claim 1, wherein receiving IoT device information comprises registering the IoT device in response to the IoT device broadcasting its presence to a backend server upon power up or the backend server periodically polling for new IoT devices.
This invention relates to Internet of Things (IoT) device management, specifically addressing the challenge of efficiently registering and tracking IoT devices in a network. The method involves automatically registering IoT devices when they connect to a backend server, either by broadcasting their presence upon power-up or by the server periodically polling for new devices. Once registered, the system collects and stores device information, including identifiers, capabilities, and network details. The backend server then uses this data to manage device interactions, such as authentication, communication protocols, and service provisioning. The system ensures seamless integration of new devices into the network while maintaining up-to-date records for monitoring and control. This approach simplifies device onboarding and reduces manual configuration, improving scalability and reliability in IoT deployments. The method supports various IoT applications, including smart home systems, industrial automation, and remote monitoring, by providing a standardized way to register and manage diverse device types. The periodic polling feature ensures continuous discovery of new devices, while the broadcast mechanism allows immediate registration upon device activation. The system enhances operational efficiency by automating device registration and maintaining accurate device inventories.
9. The method of claim 1, wherein providing the predetermined AR user interface widget for the selected IoT device to the AR device as an overlay on the display of the AR device comprises the IoT device, upon selection of the IoT device, pushing out a preferred AR overlay that is communicated to the AR device.
This invention relates to augmented reality (AR) interfaces for interacting with Internet of Things (IoT) devices. The problem addressed is the lack of intuitive, device-specific AR interfaces for controlling and monitoring IoT devices in real-world environments. The solution involves dynamically providing AR user interface widgets tailored to selected IoT devices, enhancing usability and functionality. When a user selects an IoT device, the device itself pushes a preferred AR overlay to the AR device. This overlay is customized to the specific IoT device, ensuring the interface matches the device's capabilities and user needs. The AR overlay is displayed on the AR device's screen, allowing users to interact with the IoT device through the AR interface. This approach eliminates the need for generic or preconfigured AR interfaces, improving efficiency and user experience. The system ensures seamless integration between IoT devices and AR interfaces, enabling real-time control and monitoring. The IoT device determines the appropriate AR overlay based on its configuration and user preferences, dynamically adapting to different devices and scenarios. This method enhances interoperability and usability in smart environments, making IoT device management more intuitive and effective.
10. The method of claim 1, wherein determining the predetermined AR user interface widget to render for the IoT device comprises comparing the received IoT device information to available standardized AR user interface widgets to determine at least one standardized AR user interface widget that can interpret the IoT device inputs and outputs, and selecting from the available standardized AR user interface widgets a standardized AR user interface widget as the predetermined AR user interface widget.
This invention relates to augmented reality (AR) interfaces for Internet of Things (IoT) devices, addressing the challenge of providing standardized, user-friendly AR controls for diverse IoT devices. The method involves dynamically selecting and rendering an appropriate AR user interface widget for an IoT device based on its capabilities. When an IoT device is detected, its input and output specifications are analyzed to identify compatible standardized AR interface widgets. These widgets are pre-designed templates capable of interpreting the device's data and commands. The system compares the device's specifications against the available widgets to find matches, then selects the most suitable widget to display in the AR environment. This ensures users interact with IoT devices through consistent, intuitive interfaces regardless of the device's manufacturer or protocol. The approach simplifies IoT device management by eliminating the need for custom interfaces for each device, reducing development time and improving user experience. The method supports real-time adaptation, allowing the AR interface to adjust as new IoT devices are added or existing ones are updated. This solution is particularly useful in smart home, industrial automation, and healthcare environments where multiple IoT devices require unified control.
12. The AR device of claim 11, further comprising a processor that implements an operating system, wherein the registration module and communication module are implemented by the processor via an application programming interface or system layer of the operating system.
Augmented reality (AR) devices enhance real-world environments by overlaying digital content. A challenge in AR systems is efficiently managing and integrating multiple software components, such as registration and communication modules, to ensure seamless interaction between hardware and software layers. This can lead to inefficiencies, compatibility issues, or performance bottlenecks. The invention addresses this by providing an AR device with a processor that runs an operating system. The device includes a registration module for aligning digital content with the real-world environment and a communication module for exchanging data between the AR device and external systems. The operating system implements these modules through an application programming interface (API) or a system layer, ensuring standardized and optimized integration. This approach streamlines development, improves compatibility, and enhances performance by leveraging the operating system's native capabilities. The system layer or API acts as an intermediary, abstracting hardware-specific details and providing a unified framework for module interactions. This design reduces complexity and ensures consistent functionality across different AR devices.
13. The AR device of claim 11, wherein the AR widget user interface maps multiple AR devices to a corresponding AR widget user interface for the IoT device.
Augmented reality (AR) devices are used to interact with Internet of Things (IoT) devices, but managing multiple AR devices and their interfaces can be complex. This invention addresses the challenge by providing an AR device with a user interface that maps multiple AR devices to a corresponding AR widget user interface for a single IoT device. The AR device includes a display for presenting the AR widget user interface, which allows users to control and monitor the IoT device. The system ensures that interactions from any AR device are synchronized and reflected in the AR widget user interface, providing a unified control experience. The AR device also includes a processor to execute instructions for managing the AR widget user interface and a communication module to transmit and receive data from the IoT device. This solution simplifies multi-device management by centralizing control through a single, consistent AR interface, improving usability and efficiency in IoT device interactions.
14. The AR device of claim 13, wherein the AR widget user interface draws graphics and specifies a gesture data transformation that is needed to control the corresponding AR widget user interface for the IoT device.
Augmented reality (AR) devices are used to interact with Internet of Things (IoT) devices by displaying AR widgets that represent and control these devices. A challenge in this domain is efficiently mapping user gestures to specific AR widget controls, ensuring intuitive and responsive interactions. This invention addresses this by enhancing an AR device with a specialized AR widget user interface that not only renders graphics but also defines the required gesture data transformations needed to manipulate the corresponding IoT device. The interface processes user gestures, translates them into the appropriate control signals, and applies the necessary transformations to ensure seamless interaction. This allows users to interact with IoT devices through AR widgets in a more natural and precise manner, improving usability and control accuracy. The system dynamically adjusts gesture inputs to match the expected behavior of the IoT device, reducing errors and enhancing the overall user experience. By integrating gesture data transformation directly into the AR widget interface, the invention streamlines the interaction process, making it more efficient and user-friendly.
15. The AR device of claim 11, wherein the communication module receives location information of the AR device and selects the IoT device based on a proximity of the AR device to the IoT device.
Augmented reality (AR) devices are used to overlay digital information onto the physical world, enhancing user interaction with their environment. A challenge in AR systems is efficiently connecting AR devices with nearby Internet of Things (IoT) devices to provide contextually relevant information. This invention addresses this by enabling an AR device to automatically identify and interact with nearby IoT devices based on their physical proximity. The AR device includes a communication module that receives location data, such as GPS coordinates or indoor positioning signals, to determine its current position. Using this data, the communication module identifies IoT devices within a defined proximity range. The selection process prioritizes IoT devices closest to the AR device, ensuring relevant interactions. For example, an AR device in a smart home might automatically connect to nearby smart lights or thermostats to display or control their settings. This proximity-based selection reduces unnecessary data exchange and improves user experience by focusing on the most relevant IoT devices. The system may also adjust the proximity threshold dynamically based on user preferences or environmental factors.
16. The AR device of claim 11, wherein the communication module receives verbal input from the user or orientation data regarding orientation of the AR device toward the IoT device and selects the IoT device based on the verbal input from the user or based on the orientation data.
Augmented reality (AR) devices are used to interact with Internet of Things (IoT) devices, but selecting the correct IoT device for interaction can be cumbersome, especially in environments with multiple IoT devices. This invention addresses the problem by providing an AR device with a communication module that simplifies IoT device selection. The communication module receives verbal input from the user, such as spoken commands or queries, and identifies the intended IoT device based on the verbal input. Alternatively, the communication module uses orientation data from the AR device, such as the direction the device is pointing, to determine which IoT device the user is targeting. This allows the AR device to automatically select the IoT device the user wants to interact with, either through voice commands or by pointing the AR device in the device's direction. The system enhances user experience by reducing manual selection steps and improving interaction efficiency in AR environments.
17. The AR device of claim 11, wherein the AR widget user interface automatically presents the predetermined AR user interface widget to the AR device when the AR device is within a predetermined distance from the IoT device.
Augmented reality (AR) devices are increasingly used to interact with Internet of Things (IoT) devices, but existing systems often require manual user input to access relevant AR interfaces. This invention addresses the problem by automatically presenting AR user interface widgets when an AR device is near an IoT device, improving usability and reducing user effort. The AR device includes a positioning system to determine its location relative to nearby IoT devices. When the AR device is within a predetermined distance from an IoT device, the system automatically retrieves and displays a predefined AR user interface widget associated with that IoT device. The widget provides relevant controls, status information, or other interactive elements specific to the IoT device, such as adjusting settings or monitoring performance. The system may also adjust the widget's appearance or functionality based on the user's proximity or orientation relative to the IoT device. This automation eliminates the need for manual searches or selections, streamlining the interaction process. The invention enhances user experience by providing context-aware, hands-free access to IoT device controls through AR interfaces.
18. The AR device of claim 11, wherein the communication module receives state data from the IoT device and provides the state data to the display of the AR device.
Augmented reality (AR) devices are used to overlay digital information onto the physical world, enhancing user interaction with their environment. A challenge in AR systems is integrating real-time data from Internet of Things (IoT) devices to provide contextually relevant information. This invention addresses this by enabling an AR device to receive and display state data from IoT devices, improving situational awareness and user experience. The AR device includes a communication module that establishes a connection with one or more IoT devices. The communication module retrieves state data from these devices, such as sensor readings, operational status, or other relevant information. This data is then processed and transmitted to the AR device's display, where it is overlaid onto the user's view of the physical environment. The display may include a heads-up display (HUD) or other visual output mechanism to present the information in a user-friendly manner. The AR device may also include a processing unit that interprets the state data and determines how to present it. For example, the processing unit could analyze sensor data from an IoT-enabled thermostat and display temperature readings or energy consumption metrics in the user's field of view. Additionally, the AR device may support user interactions, allowing the user to control or configure the IoT devices through the AR interface. By integrating IoT device state data into the AR experience, this invention enhances the utility of AR devices in smart home, industrial, or other IoT-enabled environments, providing users with real-time, context-aware information.
19. The AR device of claim 11, wherein the AR widget user interface selects the predetermined AR user interface widget to render for the IoT device by comparing the received IoT device information to available standardized AR user interface widgets to determine at least one standardized AR user interface widget that can interpret the IoT device inputs and outputs, and selects from the available standardized AR user interface widgets a standardized AR user interface widget as the predetermined AR user interface widget.
An augmented reality (AR) system integrates with Internet of Things (IoT) devices to provide standardized AR user interfaces for controlling and monitoring IoT devices. The system addresses the challenge of creating consistent, user-friendly interfaces for diverse IoT devices by dynamically selecting and rendering appropriate AR widgets based on device capabilities. The AR device receives information about an IoT device, including its inputs and outputs, and compares this data against a library of standardized AR user interface widgets. Each widget is designed to interpret specific types of IoT device inputs and outputs, ensuring compatibility. The system identifies one or more widgets that can handle the IoT device's data and selects the most suitable one to render in the AR environment. This approach eliminates the need for custom interfaces for each IoT device, streamlining user interaction and reducing development complexity. The selected widget provides a unified, intuitive interface for users to interact with the IoT device through the AR system.
Cooperative Patent Classification codes for this invention.
June 15, 2022
June 25, 2024
Discover thousands of AI-analyzed patents with comprehensive breakdowns, multimedia content, and expert insights.