Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method for image registration, the method comprising: establishing a three-dimensional reference model of a scene; acquiring a target image of the scene, the target image being captured with a sensor; determining the viewpoint of the sensor that captured the target image using one or more three-dimensional geoarcs, wherein the viewpoint of the sensor is determined relative to the three-dimensional reference model; and generating a composite three-dimensional representation of the scene by associating data from the target image with data from the three-dimensional reference model, wherein the viewpoint of the sensor is used to perform the association; wherein the step of determining the viewpoint of the sensor further comprises: identifying one or more feature pairs in the three-dimensional reference model; identifying one or more feature pairs in a target image; for each feature pair in the target image, associating the feature pair in the target image with one of the feature pairs in the three-dimensional reference model, estimating an angle associated with the feature pair in the target image, generating a three-dimensional geoarc surface associated with the three-dimensional reference model, wherein the geoarc surface represents relationships between the feature pair in the target image and the estimated angle; and identifying locations in three-dimensional space relative to three-dimensional reference model where two or more three-dimensional geoarc surfaces overlap.
Image registration method: First, create a 3D model of a scene. Then, capture an image of that scene using a sensor. Next, determine the sensor's location and orientation (viewpoint) using 3D-GeoArcs relative to the 3D model. This involves identifying matching feature pairs (e.g., corners of buildings) in both the 3D model and the image. For each pair, estimate the angle between them in the image. Create a 3D-GeoArc surface representing the possible sensor viewpoints that would result in that angle, based on the 3D model. The viewpoint is identified where multiple GeoArc surfaces overlap. Finally, create a combined 3D representation of the scene by merging the image data with the 3D model data, using the calculated sensor viewpoint to align them.
2. The method for image registration of claim 1 wherein the step of generating a three-dimensional geoarc surface includes representing uncertainty in the estimated angle by varying the thickness of the geoarc surface.
The image registration method (create 3D model; capture image; determine sensor viewpoint using 3D-GeoArcs by identifying feature pairs, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles, and finding overlapping GeoArc surfaces; generate composite 3D representation) refines the 3D-GeoArc surface generation by representing uncertainty in the estimated angle. This is done by making the GeoArc surface thicker or thinner, indicating a range of possible viewpoints instead of a single precise location.
3. The method for image registration of claim 2 wherein the generated geoarc surface may overlap a previously generated geoarc surface, creating a three-dimensional volume.
The image registration method (create 3D model; capture image; determine sensor viewpoint using 3D-GeoArcs by identifying feature pairs, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles with thickness representing uncertainty, and finding overlapping GeoArc surfaces; generate composite 3D representation) allows generated 3D-GeoArc surfaces to overlap, which forms a 3D volume where multiple viewpoints are possible given the image and 3D model feature correspondences. The overlap indicates a higher probability of the correct sensor viewpoint.
4. The method for image registration of claim 1 further comprising: selecting as the determined viewpoint of the sensor a location where the most geoarc surfaces overlap.
The image registration method (create 3D model; capture image; determine sensor viewpoint using 3D-GeoArcs by identifying feature pairs, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles, and finding overlapping GeoArc surfaces; generate composite 3D representation) selects the sensor viewpoint as the location where the greatest number of 3D-GeoArc surfaces overlap. This location is considered the most likely sensor position because it satisfies the constraints imposed by the most feature pairs identified between the target image and the 3D reference model.
5. The method for image registration of claim 1 further comprising: for each feature pair in the target image, refining the generated three-dimensional geoarc surface by ignoring or removing portions of the three-dimensional geoarc surface which relate to viewpoints that are incorrect based on checks with reference data.
The image registration method (create 3D model; capture image; determine sensor viewpoint using 3D-GeoArcs by identifying feature pairs, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles, and finding overlapping GeoArc surfaces; generate composite 3D representation) further improves the 3D-GeoArc surface by refining it for each feature pair in the target image. It does this by removing or ignoring portions of the 3D-GeoArc surface that correspond to incorrect viewpoints, based on comparisons with known reference data. This reduces the search space for the correct viewpoint and improves accuracy.
6. The method for image registration of claim 1 further comprising: validating the determined viewpoint of the sensor by referring to reference data to predict additional features that should be visible in the target image if the determined viewpoint of the sensor is correct.
The image registration method (create 3D model; capture image; determine sensor viewpoint using 3D-GeoArcs by identifying feature pairs, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles, and finding overlapping GeoArc surfaces; generate composite 3D representation) includes validating the determined sensor viewpoint. It does this by using reference data to predict what additional features should be visible in the target image if the determined viewpoint is correct. This acts as a consistency check to confirm the accuracy of the viewpoint estimation.
7. The method for image registration of claim 1 wherein the step of generating a composite three-dimensional representation further comprises: determining the location of the sensor and the angle of the sensor relative to the composite three-dimensional representation; and determining the location of one or more objects visible in the target image relative to the composite three-dimensional representation by, for each object, adding a translational offset to the location of the sensor.
The image registration method (create 3D model; capture image; determine sensor viewpoint using 3D-GeoArcs by identifying feature pairs, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles, and finding overlapping GeoArc surfaces; generate composite 3D representation) generates a composite 3D representation by first determining the sensor's location and orientation (angle) relative to the combined 3D representation. Then, for each object visible in the target image, it determines the object's location relative to the composite 3D representation by adding a translational offset to the sensor's location.
8. The method for image registration of claim 1 wherein the step of generating a composite three-dimensional representation of the scene includes projecting in real time one or more objects associated with the target image into the 3D composite scene.
The image registration method (create 3D model; capture image; determine sensor viewpoint using 3D-GeoArcs by identifying feature pairs, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles, and finding overlapping GeoArc surfaces; generate composite 3D representation) generates a composite 3D representation of the scene by projecting objects from the target image into the 3D scene in real-time. This enables dynamic updates to the 3D scene based on new image data.
9. The method for image registration of claim 1 wherein establishing a three-dimensional reference model comprises establishing a three-dimensional reference model using information from a geospatial intelligence system database.
In the image registration method (capture image; determine sensor viewpoint using 3D-GeoArcs by identifying feature pairs, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles, and finding overlapping GeoArc surfaces; generate composite 3D representation), the 3D reference model is created using information from a geospatial intelligence system database. This allows for the use of existing, high-quality 3D models for image registration.
10. The method for image registration of claim 1 wherein the features of the feature pairs identified in the three-dimensional reference model and in the target image are characterized such that they are invariant with the rotation and scale of the reference model and the target image.
In the image registration method (capture image; determine sensor viewpoint using 3D-GeoArcs by identifying feature pairs, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles, and finding overlapping GeoArc surfaces; generate composite 3D representation), the features (e.g., corners, edges) identified in the 3D reference model and the target image are chosen to be rotation and scale-invariant. This means the algorithm can still match corresponding features even if the reference model and the target image are rotated or scaled differently.
11. The method for image registration of claim 1 wherein the step of determining the viewpoint of the sensor includes partitioning the three-dimensional reference model into a number of regions and determining a potential viewpoint of the sensor within one or more of the regions.
In the image registration method (capture image; determine sensor viewpoint using 3D-GeoArcs by identifying feature pairs, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles, and finding overlapping GeoArc surfaces; generate composite 3D representation), the process of determining the sensor viewpoint includes partitioning the 3D reference model into regions and determining a potential sensor viewpoint within one or more of these regions. This reduces computational complexity by limiting the search for the viewpoint to specific areas of the 3D model.
12. A method for image registration, the method comprising: identifying one or more feature pairs in a three-dimensional reference model; identifying one or more feature pairs in a target image; for each feature pair in the target image, associating the feature pair in the target image with one of the feature pairs in the three-dimensional reference model, estimating an angle associated with the feature pair in the target image, generating a three-dimensional geoarc surface associated with the three-dimensional reference model, wherein the geoarc surface represents relationships between the feature pair in the target image and the estimated angle; and identifying locations in three-dimensional space relative to three-dimensional reference model where two or more three-dimensional geoarc surfaces overlap.
Image registration method: Identify matching feature pairs (e.g., corners of buildings) in both a 3D reference model of a scene and a target image of that scene. For each pair, estimate the angle between them in the image. Create a 3D-GeoArc surface representing the possible sensor viewpoints that would result in that angle, based on the 3D model. The viewpoint is identified where multiple GeoArc surfaces overlap.
13. The method for image registration of claim 12 further comprising varying the thickness of the geoarc surface to represent uncertainty in the estimated angle.
The image registration method (identifying feature pairs in a 3D model and target image, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles, and finding overlapping GeoArc surfaces) refines the 3D-GeoArc surface generation by representing uncertainty in the estimated angle. This is done by making the GeoArc surface thicker or thinner, indicating a range of possible viewpoints instead of a single precise location.
14. The method for image registration of claim 13 wherein the generated geoarc surface may overlap a previously generated geoarc surface, creating a three-dimensional volume.
The image registration method (identifying feature pairs in a 3D model and target image, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles with thickness representing uncertainty, and finding overlapping GeoArc surfaces) allows generated 3D-GeoArc surfaces to overlap, which forms a 3D volume where multiple viewpoints are possible given the image and 3D model feature correspondences. The overlap indicates a higher probability of the correct sensor viewpoint.
15. An air vehicle comprising: a sensor adapted to capture images; a data processing system communicatively coupled to the sensor, the data processing system programmed to: establish a three-dimensional reference model of a scene; acquire a target image of the scene from the sensor; determine the viewpoint of the sensor that captured the target image using one or more three-dimensional geoarcs, wherein the viewpoint of the sensor is determined relative to the three-dimensional reference model; and generate a composite three-dimensional representation of the scene by associating data from the target image with data from the three-dimensional reference model, wherein the viewpoint of the sensor is used to perform the association; wherein to determine the viewpoint of the sensor, said data processing system is further programmed to: identify one or more feature pairs in the three-dimensional reference model; identify one or more feature pairs in a target image; for each feature pair in the target image, associate the feature pair in the target image with one of the feature pairs in the three-dimensional reference model, estimate an angle associated with the feature pair in the target image, generate a three-dimensional geoarc surface associated with the three-dimensional reference model, wherein the geoarc surface represents relationships between the feature pair in the target image and the estimated angle; and identify locations in three-dimensional space relative to three-dimensional reference model where two or more three-dimensional geoarc surfaces overlap.
An air vehicle (e.g., drone) has a camera and a computer. The computer runs image registration: First, it creates a 3D model of a scene. Then, it gets an image of that scene from the camera. Next, it determines the camera's location and orientation (viewpoint) using 3D-GeoArcs relative to the 3D model. This involves identifying matching feature pairs (e.g., corners of buildings) in both the 3D model and the image. For each pair, estimate the angle between them in the image. Create a 3D-GeoArc surface representing the possible camera viewpoints that would result in that angle, based on the 3D model. The viewpoint is identified where multiple GeoArc surfaces overlap. Finally, create a combined 3D representation of the scene by merging the image data with the 3D model data, using the calculated camera viewpoint to align them.
16. The air vehicle of claim 15 wherein to generate a composite three-dimensional representation of the scene, said data processing system is further programmed to project, in real time, one or more objects associated with the target image into the 3D composite scene.
The air vehicle (camera, computer with image registration: create 3D model; capture image; determine sensor viewpoint using 3D-GeoArcs by identifying feature pairs, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles, and finding overlapping GeoArc surfaces; generate composite 3D representation) generates a composite 3D representation of the scene by projecting objects from the target image into the 3D scene in real-time. This enables dynamic updates to the 3D scene based on new image data from the camera.
17. The air vehicle of claim 15 wherein the sensor is located on the air vehicle such that landscapes and scenes may fall within the sensor's field of view.
The air vehicle (camera, computer with image registration: create 3D model; capture image; determine sensor viewpoint using 3D-GeoArcs by identifying feature pairs, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles, and finding overlapping GeoArc surfaces; generate composite 3D representation) has its camera positioned so that it can capture images of landscapes and scenes below.
18. The air vehicle of claim 15 wherein the data processing system includes a memory, wherein the memory is operable to store reference data, including the three-dimensional reference model, and wherein the memory is operable to store images captured by the sensor.
The air vehicle (camera, computer with image registration: create 3D model; capture image; determine sensor viewpoint using 3D-GeoArcs by identifying feature pairs, estimating angles, creating 3D-GeoArc surfaces representing relationships between image feature pairs and estimated angles, and finding overlapping GeoArc surfaces; generate composite 3D representation) includes memory to store reference data, including the 3D reference model, as well as the images captured by the camera.
Unknown
October 7, 2014
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.