Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. An information processing apparatus comprising: an input unit configured to sequentially acquire a first plurality of frame images from an image capturing apparatus for capturing a target object; a storage unit configured to store the first plurality of frame images acquired by the input unit; a measurement unit configured to select a first frame image from the first plurality of frame images, and configured to measure the position and orientation of the image capturing apparatus, when the image capturing apparatus has captured frame images which follow the first frame image, based on the first frame image; a first determination unit configured to determine whether a value of the position and orientation of the image capturing apparatus measured by the measurement unit is within a prescribed range; a second determination unit configured to determine whether a second plurality of frame images from the first plurality of frame images are stored in the storage unit between the first frame image and a frame image for which the value of the position and orientation of the image capturing apparatus is determined not to be within the prescribed range; and a control unit configured to perform control such that the measurement unit measures the position and orientation of the image capturing apparatus using a second frame image that is separated by a preset number of frames from the first frame image, in a case where the second determination unit determines that the second plurality of frame images are stored in the storage unit.
An information processing apparatus tracks the position and orientation of an image capturing apparatus (e.g., on a head-mounted display) relative to a target object. The apparatus sequentially acquires a series of frame images and stores them. It selects a first frame, measures the device's position/orientation based on that frame, and continues measuring in subsequent frames. If the measured position/orientation changes beyond a threshold, the apparatus checks if a certain number of frames have been stored since the first frame. If so, it restarts the measurement using a frame separated by a preset number of frames from the first frame to avoid accumulating errors over long sequences without re-initialization.
2. The information processing apparatus according to claim 1 , wherein the measurement unit selects frame images that follow the designated frame image from the first plurality of stored frame images, uses a frame image precedent to the selected frame image and the currently selected frame image to calculate a change in optical flow, and matches the feature of the target object in the preceding frame image to the feature of the target image in the currently selected frame image if a calculated value of the optical flow is greater than or equal to a prescribed threshold value, and measures the position and orientation of the image capturing apparatus.
This information processing apparatus, described in the previous claim, refines its position/orientation measurement by calculating optical flow between consecutive frame images. Specifically, after selecting a frame image and considering subsequent frames, it calculates the optical flow between the current frame and the previous frame. If the calculated optical flow exceeds a threshold, it matches features of the target object between the two frames to measure the change in the camera's position and orientation. This helps to accurately track the target object when there is significant movement in the images.
3. The information processing apparatus according to claim 1 , wherein the measurement unit measures the position and orientation of the image capturing apparatus by matching the feature of the target object in a frame image precedent to the selected frame image to the feature of the target object in the first frame image.
This information processing apparatus, described in the first claim, measures the position and orientation of the image capturing apparatus by matching features of the target object in the current frame image to the features of the target object in the initial (first) frame image. This establishes direct correspondences between the current view and the initial view to estimate the device's movement relative to the initial starting point.
4. The apparatus according to claim 3 , wherein the feature of the target object is a line segment.
In the apparatus that measures position and orientation by matching features as described in the previous claim, the "feature of the target object" that is matched between frames is specifically a line segment. The apparatus detects and matches line segments in the image to track the target object's pose.
5. The apparatus according to claim 3 , wherein the feature of the target object is a feature point.
In the apparatus that measures position and orientation by matching features as described two claims ago, the "feature of the target object" that is matched between frames is specifically a feature point (e.g., a corner or a distinctive blob). The apparatus detects and matches these points in the image to track the target object's pose.
6. The information processing apparatus according to claim 1 , wherein the second frame image is selected every preset number of frames, from the first plurality of frame images stored in the storage unit.
This information processing apparatus, described in the first claim, selects a second frame image for re-initialization (when position/orientation drifts too far) by choosing a frame every preset number of frames from the first plurality of frame images stored. For example, if the preset number is 10, it might choose frame 1, then frame 11, then frame 21, etc., for re-initialization attempts.
7. The information processing apparatus according to claim 6 , wherein the preset number of frames is determined based on a value of a frame rate of the frame images.
In the apparatus that selects a second frame every preset number of frames, as described in the previous claim, the value of the "preset number of frames" is determined based on the frame rate of the captured images. Higher frame rates might allow for a smaller preset number (more frequent re-initialization attempts), while lower frame rates might necessitate a larger preset number to maintain tracking accuracy and efficiency.
8. The apparatus according to claim 1 , wherein the image capturing apparatus is attached on a head mount display.
In the apparatus which performs tracking and re-initialization of position/orientation as described in the first claim, the "image capturing apparatus" is specifically attached to a head-mounted display. This implies the apparatus is used for tracking the head movement and orientation of the user wearing the display.
9. A control method of an information processing apparatus, comprising: an input step of acquiring sequentially a first plurality of frame images from an image capturing apparatus for capturing a target object; a storage step of storing the first plurality of frame images acquired in the input step; a measurement step of (i) selecting a first frame image from the first plurality of frame images, and (ii) measuring the position and orientation of the image capturing apparatus, when the image capturing apparatus has captured frame images which follow the first frame image, based on the first frame image; a first determination step of determining whether a value of the position and orientation of the image capturing apparatus measured in the measurement step is within a prescribed range; a second determination step of determining whether a second plurality of frame images from the first plurality of frame images are stored in the storage unit between the first frame image and a frame image for which the value of the position and orientation of the image capturing apparatus is determined not to be within the prescribed range; a control step of performing control to measure the position and orientation of the image capturing apparatus using a second frame image that is separated by a preset number of frames from the first frame image, in a case where it is determined in the second determination step that the second plurality of frame images are stored in the storage unit.
A method for tracking the position and orientation of an image capturing apparatus relative to a target object. The method involves sequentially acquiring a series of frame images and storing them. A first frame is selected, and the device's position/orientation is measured based on that frame, continuing measurement in subsequent frames. If the measured position/orientation changes beyond a threshold, the method checks if a certain number of frames have been stored since the first frame. If so, it restarts the measurement using a frame separated by a preset number of frames from the first frame to avoid accumulating errors over long sequences without re-initialization.
10. A non-transitory computer readable storage medium which stores a computer program for causing a computer to execute the control method of an information processing apparatus according to claim 9 .
A non-transitory computer-readable storage medium (e.g., a hard drive, SSD, or flash drive) stores a computer program that, when executed by a computer, performs the tracking and re-initialization method for an image capturing apparatus as described in the previous claim. This includes acquiring and storing frame images, measuring position/orientation, detecting drift beyond a threshold, and re-initializing the measurement using frames separated by a pre-determined number, to ensure accurate and robust tracking of a target object.
Unknown
September 2, 2014
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.