Method to create try-on experience wearing virtual 3D eyeglasses is provided using 2D image data of eyeglasses. Virtual 3D eyeglasses are constructed using set of 2D images for eyeglasses. Virtual 3D eyeglasses is configured onto 3D face or head model and being simulated as being fittingly worn by the wearer. Each set of 2D images for eyeglasses includes a pair of 2D lens images, a frontal frame image, and at least one side frame image. Upon detection of a movement of the face and head of wearer in real-time, the 3D face or head model and the configuration and alignment of virtual 3D eyeglasses are modified or adjusted accordingly. Features such as trimming off of portion of the glasses frame, shadow creating and environment mapping are provided to the virtual 3D eyeglasses in response to translation, scaling, and posture changes made to the head and face of the wearer in real-time.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method to create real-time try-on experience of wearing virtual 3D eyeglasses by a wearer, comprising: obtaining a plurality of 2D images for a plurality of, pairs of eyeglasses, the pairs of eyeglasses are organized into a group of 2D images, each pair of eyeglasses has a set of 2D images; when one designed pair of eyeglasses from the group of 2D images is selected by the wearer, constructing a pair of virtual 3D eyeglasses using the set of 2D images for the designed pair of eyeglasses; constructing a 3D face or head model of the wearer based upon one or more facial or head images of the wearer; fitting the pair of virtual 3D eyeglasses onto the 3D face or head model of the wearer, with the pair of virtual 3D eyeglasses being simulated as being worn by the wearer; and rotating the virtual 3D eyeglasses together with the 3D face or head model according to a change in real-time posture of the face of the wearer; wherein each set of the 2D images for each pair of eyeglasses comprises a frontal frame image and at least one side frame image.
The system creates a real-time virtual try-on experience for 3D eyeglasses. It uses 2D images of various eyeglass pairs, organized into a group where each pair has its own set of images including a frontal frame and at least one side frame image. When a user selects a pair of glasses, the system constructs a virtual 3D model of those glasses using the corresponding 2D images. Simultaneously, it creates a 3D model of the user's face or head based on real-time facial or head images. The virtual 3D glasses are then fitted onto the 3D face/head model, simulating the glasses being worn. The virtual glasses and face model rotate together in response to the user's head movements.
2. The method as claimed in claim 1 , wherein the step of fitting the pair of virtual 3D eyeglasses onto the 3D face or head model of the wearer comprises: moving the virtual 3D eyeglasses together with the 3D face or head model according to the real-time posture of the face of the wearer and one reference point of the face.
Building on the real-time virtual try-on experience described previously, when fitting the virtual 3D eyeglasses onto the 3D face or head model, the virtual 3D eyeglasses are moved together with the 3D face or head model according to the real-time posture of the face of the wearer, using a reference point on the user's face to ensure proper alignment and movement synchronization.
3. The method as claimed in claim 1 , wherein the step of fitting the pair of virtual 3D eyeglasses onto the 3D face or head model of the wearer comprises: zooming in and out the virtual 3D eyeglasses together with the 3D face or head model according to a change in real-time scaling size of the face of the wearer.
Expanding on the real-time virtual try-on experience, the system adjusts the size of the virtual glasses based on the scaling of the user's face in real time. When fitting the virtual 3D eyeglasses onto the 3D face or head model, the virtual 3D eyeglasses zoom in and out together with the 3D face or head model according to a change in real-time scaling size of the face of the wearer, simulating the effect of moving closer to or further from a mirror.
4. The method as claimed in claim 1 , wherein while the pair of virtual 3D eyeglasses is being simulated as being fittingly worn by the wearer, a right-side glasses frame portion inside the face is trimmed off upon a rotation angle of the 3D face or head model of the wearer is at least 5 degrees to the right.
During the real-time virtual try-on simulation, if the user rotates their head more than 5 degrees to the right, the portion of the right-side glasses frame that would be inside the user's face is automatically trimmed or hidden from view. This prevents the glasses from appearing to clip through the face model.
5. The method as claimed in claim 1 , wherein while the pair of virtual 3D eyeglasses is being simulated as being fittingly worn by the wearer, a left-side glasses frame portion inside the face is trimmed off upon a rotation angle of the 3D face or head model of the wearer is at least 5 degrees to the left.
During the real-time virtual try-on simulation, if the user rotates their head more than 5 degrees to the left, the portion of the left-side glasses frame that would be inside the user's face is automatically trimmed or hidden from view. This prevents the glasses from appearing to clip through the face model.
6. The method as claimed in claim 1 , wherein each set of the 2D images for each pair of eyeglasses further comprises a pair of 2D lens images; and the pair of 2D lens images of the set of 2D images are based on 2D image that are set at a designated alpha channel value to change the transparency of the lenses of the pair of 3D virtual eyeglasses.
In addition to the frontal and side frame images, each set of 2D images for a pair of eyeglasses also includes a pair of 2D lens images. These lens images have an alpha channel value set to control their transparency, allowing the user to see through the virtual lenses to varying degrees, like tinted or clear lenses.
7. The method as claimed in claim 1 , wherein each set of the 2D images for each pair of eyeglasses further comprises a pair of 2D lens images; and an environmental mapping effect is applied to the pair of lens images to configure and show a transient modified lens image in place of the 2D lens image in real-time upon satisfying a set criteria according to the real-time posture of the face of the wearer.
In addition to the frontal and side frame images, each set of 2D images for a pair of eyeglasses includes a pair of 2D lens images. The system applies an environment mapping effect to these lens images. This means that, based on the user's head position and other criteria, a dynamically modified lens image is shown, reflecting the surrounding environment and creating realistic lens reflections in real-time.
8. The method as claimed in claim 1 , wherein a shadow effect is applied to the face image of the wearer in real-time using the 3D face or head model.
During the real-time virtual try-on experience, the system adds a shadow effect to the user's face image. This shadow is dynamically generated based on the 3D face or head model and the simulated lighting, enhancing the realism of the virtual try-on by making the virtual glasses appear more integrated with the user's face.
9. An augmented reality system for real-time try-on of 3D virtual eyeglasses by a wearer, comprising: a server, the server having a first memory, a plurality of 2D images for a plurality of pairs of eyeglasses are organized into a group of 2D images, the group of 2D images are stored in the first memory; and a host, the host comprising a 3D virtual eyeglasses try-on simulation program, a processor, a camera, a display device and a second memory, wherein each pair of eyeglasses has a set of 2D images in the group of 2D images, the set of 2D images is stored in the second memory; the processor is configured to run the 3D virtual eyeglasses try-on simulation program for constructing a pair of virtual 3D eyeglasses using the set of 2D images for one designed pair of eyeglasses retrieved from the second memory when the designed pair of eyeglasses from the group of 2D images is selected by the wearer, constructing a 3D face or head model of the wearer based upon one or more facial and head images of the wearer captured by the camera in real-time, configuring the pair of virtual 3D eyeglasses onto the 3D face or head model of the wearer, with the pair of virtual 3D eyeglasses being simulated as being fittingly worn by the wearer; the display device is configured to display the wearer trying-on the virtual 3D eyeglasses in real-time; and each set of the 2D images for each pair of eyeglasses stored in the first memory and in the second memory, respectively, comprising a frontal frame image and at least one side frame image; wherein the processor is configured to run the 3D virtual eyeglasses try-on simulation program for rotating the virtual 3D eyeglasses together with the 3D face or head model according to a change in real-time posture of the face of the wearer.
This augmented reality system enables real-time virtual try-on of 3D eyeglasses. A server stores 2D images of various eyeglass pairs, including frontal and side frame views. A host device (e.g. computer, phone) has a camera, display, processor, and memory. It runs a program that constructs a 3D model of selected glasses from their 2D images. It also creates a 3D model of the user's face using camera input. The program fits the virtual glasses onto the 3D face model for display. As the user moves their head, the virtual glasses rotate accordingly, simulating a realistic try-on experience.
10. The augmented reality system as claimed in claim 9 , wherein the processor is configured to run the 3D virtual eyeglasses try-on simulation program for moving the virtual 3D eyeglasses together with the 3D face or head model according to the real-time posture of the face of the wearer and one reference point of the face.
The augmented reality system, described previously, moves the virtual 3D eyeglasses together with the 3D face or head model according to the real-time posture of the face of the wearer, using a reference point on the user's face to ensure proper alignment and movement synchronization.
11. The augmented reality system as claimed in claim 9 , wherein the processor is configured to run the 3D virtual eyeglasses try-on simulation program for zooming in and out the virtual 3D eyeglasses together with the 3D face or head model according to a change in real-time scaling size of the face of the wearer.
The augmented reality system, described previously, adjusts the size of the virtual glasses based on the scaling of the user's face in real time. The system zooms in and out the virtual 3D eyeglasses together with the 3D face or head model according to a change in real-time scaling size of the face of the wearer, simulating the effect of moving closer to or further from a mirror.
12. The augmented reality system as claimed in claim 9 , wherein while the pair of virtual 3D eyeglasses is being simulated as being fittingly worn by the wearer, a right-side glasses frame portion inside the face is trimmed off upon a rotation angle of the 3D face or head model of the wearer is at least 5 degrees to the right.
In the augmented reality system, described previously, while the virtual glasses are simulated on the user, if the head rotates more than 5 degrees to the right, the portion of the right glasses frame inside the face is automatically trimmed off to avoid clipping.
13. The augmented reality system as claimed in claim 9 , wherein while the pair of virtual 3D eyeglasses is being simulated as being fittingly worn by the wearer, a left-side glasses frame portion inside the face is trimmed off upon a rotation angle of the 3D face or head model of the wearer is at least 5 degrees to the left.
In the augmented reality system, described previously, while the virtual glasses are simulated on the user, if the head rotates more than 5 degrees to the left, the portion of the left glasses frame inside the face is automatically trimmed off to avoid clipping.
14. The augmented reality system as claimed in claim 9 , wherein each set of the 2D images for each pair of eyeglasses further comprises a pair of 2D lens images; and the pair of 2D lens images of the set of 2D images are based on 2D image that are set at a designated alpha channel value to change the transparency of the lenses of the pair of 3D virtual eyeglasses.
In the augmented reality system, described previously, each eyeglass set includes 2D lens images with a defined alpha channel. This alpha channel controls the transparency of the virtual lenses, allowing simulation of different lens tints and opacities.
15. The augmented reality system as claimed in claim 9 , wherein each set of the 2D images for each pair of eyeglasses further comprises a pair of 2D lens images; and an environmental mapping effect is applied to the pair of lens images to configure and show a transient modified lens image in place of the 2D lens image in real-time upon satisfying a set criteria according to the real-time posture of the face of the wearer.
In the augmented reality system, described previously, each eyeglass set includes 2D lens images that use environmental mapping. Based on the user's head posture, the lens image dynamically reflects the surrounding environment, providing a realistic reflection effect.
16. The augmented reality system as claimed in claim 9 , wherein a shadow effect is applied to the face image of the wearer in real-time using the 3D face or head model.
The augmented reality system, described previously, applies a shadow effect to the face image using the 3D face or head model. This real-time shadowing enhances the realism of the virtual try-on experience.
17. The augmented reality system as claimed in claim 9 , wherein the 3D virtual eyeglasses try-on simulation program is an app, and the host is a mobile phone or an electronic wireless device.
In the augmented reality system, the 3D virtual eyeglasses try-on simulation program is implemented as a mobile app running on a smartphone or similar wireless device.
18. The augmented reality system as claimed in claim 9 , wherein the 3D virtual eyeglasses try-on simulation program is a webpage, the webpage comprising a virtual mirror for trying-on 3D virtual eyeglasses by the wearer, the virtual mirror is configured under a webcam mode or a virtual model mode, upon operating under the webcam mode, real-time facial tracking is performed and a 3D face or head model of the wearer based upon one or more facial or head images of the wearer is generated based upon the real-time facial tracking results.
In the augmented reality system, the 3D virtual eyeglasses try-on simulation program is a webpage, featuring a virtual mirror for trying on glasses. The mirror operates in webcam mode (using real-time face tracking) or virtual model mode. In webcam mode, it generates a 3D face or head model based on the real-time facial tracking results.
19. A method to create real-time try-on experience of wearing virtual 3D eyeglasses by a wearer, comprising: obtaining a plurality of 2D images for a plurality of pairs of eyeglasses, the pairs of eyeglasses are organized into a group of 2D images, each pair of eyeglasses has a set of 2D images; when one designed pair of eyeglasses from the group of 2D images is selected by the wearer, constructing a pair of virtual 3D eyeglasses using the set of 2D images for the designed pair of eyeglasses; constructing a 3D face or head model of the wearer based upon one or more facial or head images of the wearer; fitting the pair of virtual 3D eyeglasses onto the 3D face or head model of the wearer, with the pair of virtual 3D eyeglasses being simulated as being worn by the wearer; and moving the virtual 3D eyeglasses together with the 3D face or head model according to the real-time posture of the face of the wearer and one reference point of the face; wherein each set of the 2D images for each pair of eyeglasses comprises a frontal frame image and at least one side frame image.
The system creates a real-time virtual try-on experience for 3D eyeglasses. It uses 2D images of various eyeglass pairs, organized into a group where each pair has its own set of images including a frontal frame and at least one side frame image. When a user selects a pair of glasses, the system constructs a virtual 3D model of those glasses using the corresponding 2D images. Simultaneously, it creates a 3D model of the user's face or head based on real-time facial or head images. The virtual 3D glasses are then fitted onto the 3D face/head model, simulating the glasses being worn. The virtual glasses and face model move together according to the real-time posture of the face of the wearer, using a reference point on the user's face to ensure proper alignment and movement synchronization.
20. The method as claimed in claim 19 , wherein the step of fitting the pair of virtual 3D eyeglasses onto the 3D face or head model of the wearer comprises: zooming in and out the virtual 3D eyeglasses together with the 3D face or head model according to a change in real-time scaling size of the face of the wearer.
Building on the real-time virtual try-on experience described previously, the system adjusts the size of the virtual glasses based on the scaling of the user's face in real time. When fitting the virtual 3D eyeglasses onto the 3D face or head model, the virtual 3D eyeglasses zoom in and out together with the 3D face or head model according to a change in real-time scaling size of the face of the wearer, simulating the effect of moving closer to or further from a mirror.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
July 31, 2014
May 30, 2017
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.