Disclosed herein are methods and systems for generating a user-hair-color model. One embodiment takes the form of a process that includes obtaining video data depicting a head of a user. The process also includes determining a set of line segments of pixels of the video data, wherein each line segment in the determined set of line segments intersects an upper contour of the depicted head of the user. The process also includes grouping at least some of the pixels of at least one of the line segments in the determined set of line segments into three sets of pixels based at least in part on respective color data of the pixels. The three sets of pixels include a skin-pixel set, a hair-pixel set, and a background-pixel set. The process also includes updating a user hair-color model based at least in part on the skin-pixel set.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method comprising: obtaining video data depicting a head of a user; determining a set of line segments of pixels of the video data, wherein each line segment in the determined set of line segments intersects an upper contour of the depicted head of the user; grouping at least some of the pixels of at least one of the line segments in the determined set of line segments into three sets of pixels based at least in part on respective color data of the pixels, wherein the three sets of pixels include a skin-pixel set, a hair-pixel set, and a background-pixel set; and updating a user hair-color model based at least in part on the hair-pixel set.
A method for creating a user hair-color model from video data. The method includes: capturing video of a user's head; finding line segments of pixels that intersect the top of the user's head in the video; classifying pixels from at least one of these line segments into three groups (skin, hair, and background) based on their color; and then refining a hair-color model based on the pixel colors identified as hair.
2. The method of claim 1 , further comprising generating the upper contour of the depicted head of the user.
The method from the previous description also includes generating the outline of the top of the user's head from the video data, which is used to determine the intersecting line segments of pixels. This upper contour is used to find the lines needed for pixel classification.
3. The method of claim 2 , wherein generating the upper contour of the depicted head of the user comprises: obtaining depth data that is associated with the video data; identifying a region of the video data that corresponds with depth values that are within a threshold depth range; and identifying an upper boundary of the identified region as being the upper contour of the depicted head of the user.
In the method including generating the outline of the top of the user's head, generating this upper contour involves: using depth data associated with the video; identifying a region where the depth values are within a certain range (indicating the head's presence); and defining the upper edge of that region as the upper contour of the head.
4. The method of claim 2 , wherein generating the upper contour of the depicted head of the user comprises: obtaining a head contour that estimates an outline of the depicted head of the user; and identifying an upper boundary of the obtained head contour as being the upper contour of the depicted head of the user.
In the method including generating the outline of the top of the user's head, generating the upper contour involves: getting an initial estimate of the head's outline; and using the upper edge of this estimated outline as the upper contour of the head.
5. The method of claim 1 , wherein determining the set of line segments of pixels of the video data comprises: identifying a center of the depicted head of the user; identifying at least one line segment of pixels of the video data that originates from the identified center of the depicted head of the user and intersects the upper contour; and determining the identified at least one line segment of pixels of the video data to be in the set of line segments.
In the method for creating a user hair-color model, determining the set of line segments that intersect the top of the user's head involves: locating the center of the user's head in the video; finding lines that start at the head's center and cross the upper contour; and including these lines in the set of line segments used for pixel classification.
6. The method of claim 1 , wherein the grouping assigns each of the pixels to one and only one of the three sets of pixels.
In the method for creating a user hair-color model, the pixel grouping ensures that each pixel is assigned to only one of the three categories: skin, hair, or background. Each pixel gets exclusively labeled.
7. The method of claim 1 , wherein the grouping assigns at least one of the pixels to more than one of the three sets of pixels.
In the method for creating a user hair-color model, the pixel grouping allows pixels to be assigned to more than one of the three categories: skin, hair, and background. This allows for uncertainty in classification, where a pixel might partially belong to multiple categories.
8. The method of claim 1 , wherein grouping the pixels comprises employing a k-means clustering algorithm to group the pixels.
In the method for creating a user hair-color model, grouping pixels into skin, hair, and background uses a k-means clustering algorithm. This algorithm automatically separates pixels into the three groups based on color similarity.
9. The method of claim 1 , wherein grouping the pixels comprises employing a Viterbi algorithm to group the pixels.
In the method for creating a user hair-color model, grouping pixels into skin, hair, and background uses a Viterbi algorithm. This algorithm likely leverages sequential information along the line segments to improve classification accuracy.
10. The method of claim 1 , wherein grouping the pixels comprises employing a support vector machine algorithm to group the pixels.
In the method for creating a user hair-color model, grouping pixels into skin, hair, and background uses a support vector machine (SVM) algorithm. The SVM is trained to classify pixels based on their color features.
11. The method of claim 1 , wherein grouping the pixels comprises employing an expectation maximization algorithm to group the pixels.
In the method for creating a user hair-color model, grouping pixels into skin, hair, and background uses an expectation maximization (EM) algorithm. The EM algorithm is used to estimate the parameters of a mixture model that describes the color distribution of each category.
12. The method of claim 1 , wherein: grouping the pixels comprises performing multiple independent groupings of at least some of the pixels of at least one of the line segments in the determined set of line segments; each performed grouping respectively outputs three sets of pixels based at least in part on respective color data of the pixels; each respective three sets of pixels includes a respective skin-pixel set, a respective hair-pixel set, and a respective background-pixel set; the method further comprises selecting one of the respective hair-pixel sets based on one or more hair-pixel-set quality metrics; and updating the user hair-color model based at least in part on the hair-pixel set comprises updating the user hair-color model based at least in part on the selected hair-pixel set.
In the method for creating a user hair-color model: the pixel grouping involves multiple independent classifications of pixels from the line segments; each classification produces its own sets of skin, hair, and background pixels based on color data; then, one of the hair-pixel sets is chosen based on quality metrics; and finally, the hair-color model is updated using the chosen, high-quality hair-pixel set.
13. The method of claim 12 , wherein each independent grouping of at least some of the pixels of at least one of the line segments in the determined set of line segments employs a different type of grouping algorithm.
In the method performing multiple groupings of pixels, each independent grouping uses a different type of algorithm to classify the pixels into skin, hair and background, so that different algorithms are used and compared.
14. The method of claim 12 , wherein each independent grouping of at least some of the pixels of at least one of the line segments in the determined set of line segments employs a same type of grouping algorithm and a different set of grouping algorithm parameters.
In the method performing multiple groupings of pixels, each independent grouping uses the same algorithm to classify the pixels into skin, hair and background, but with different parameter settings for the algorithm.
15. The method of claim 1 , further comprising updating a background-color model based at least in part on the background-pixel set.
In the method for creating a user hair-color model, a background-color model is also updated based on the pixels identified as background. This improves the accuracy of background segmentation.
16. The method of claim 1 , wherein the user hair-color model is at least one of a histogram, a machine learning classifier, a Gaussian mixture, and an array of color values with respective color counts.
In the method for creating a user hair-color model, the hair-color model can be a histogram of colors, a machine learning classifier, a Gaussian mixture model representing color distributions, or an array storing color values and their corresponding frequencies.
17. The method of claim 1 , wherein updating the user hair-color model based at least in part on the hair-pixel set comprises, for each pixel in the hair-pixel set: adding a plurality of user-hair-color-model entries to the user hair-color model, the entries corresponding to a color of the pixel as well as colors that are within a threshold tolerance of the color of the pixel.
In the method for creating a user hair-color model, updating the hair-color model involves, for each pixel classified as hair: adding entries to the model not only for the pixel's exact color, but also for similar colors within a specified tolerance. This broadens the model's representation of the user's hair color.
18. A system comprising: a communication interface; a processor; and non-transitory computer-readable data storage containing instructions executable by the processor for causing the system to carry out a set of functions, the set of functions including: obtaining video data depicting a head of a user; determining a set of line segments of pixels of the video data, wherein each line segment in the determined set of line segments intersects an upper contour of the depicted head of the user; grouping at least some of the pixels of at least one of the line segments in the determined set of line segments into three sets of pixels based at least in part on respective color data of the pixels, wherein the three sets of pixels include a skin-pixel set, a hair-pixel set, and a background-pixel set; and updating a user hair-color model based at least in part on the hair-pixel set.
A system for creating a user hair-color model includes a processor, a communication interface, and memory storing instructions that, when executed, cause the system to: capture video of a user's head; find line segments of pixels that intersect the top of the user's head; classify pixels from at least one of these line segments into three groups (skin, hair, and background) based on their color; and then refine a hair-color model based on the pixel colors identified as hair.
19. A method comprising: obtaining video data depicting a head of a user; obtaining an upper contour of the head of the user; determining a set of line segments of pixels, each line segment in the set of line segments crossing the upper contour; selecting a first number of k-means initial configurations, each selected k-means initial configuration including, for each line segment in the set of line segments, two initial mean colors that are configuration-independent and one initial mean color that is configuration-dependent; performing the first number of independent k-means analyses on the set of line segments, each independent k-means analysis (i) using a different one of the selected k-means initial configurations and (ii) identifying, for each line segment in the set of line segments, three final mean colors respectively derived from the three initial mean colors; for each of the performed k-means analyses, using the final mean colors derived from the configuration-dependent initial mean colors to identify a respective set of hair-color pixels from the pixels in the line segments; selecting one of the identified sets of hair-color pixels based on one or more hair-color-pixel-set criteria; and using the selected set of hair-color pixels to generate a user-hair-color model of the head of the user.
A method for creating a user hair-color model from video data: capture video of a user's head; determine the upper contour of the user's head; find line segments of pixels crossing that contour; run multiple k-means clustering analyses, each starting with a different initial configuration of color means; for each analysis, identify the "hair" pixels based on the configuration-dependent color mean; select the best set of "hair" pixels according to specified criteria; and use those pixels to generate the user's hair-color model. Initial K-means configuation include two fixed color guesses and a configurable hair color guess.
20. The method of claim 19 , wherein one of the two configuration-independent initial mean colors is representative of a color of a face of the user, the other of the two configuration-independent initial mean colors is representative of a color of background of the video data, and the configuration-dependent initial mean color is representative of a color of hair of the user.
In the k-means clustering hair-color model method, one of the two fixed color guesses is representative of the face color, and the other fixed color is representative of the background color, while the configurable color guess represents the hair color of the user.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
September 1, 2015
March 28, 2017
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.