The invention is related to methods and apparatus that use a visual sensor and dead reckoning sensors to process Simultaneous Localization and Mapping (SLAM). These techniques can be used in robot navigation. Advantageously, such visual techniques can be used to autonomously generate and update a map. Unlike with laser rangefinders, the visual techniques are economically practical in a wide range of applications and can be used in relatively dynamic environments, such as environments in which people move. One embodiment further advantageously uses multiple particles to maintain multiple hypotheses with respect to localization and mapping. Further advantageously, one embodiment maintains the particles in a relatively computationally-efficient manner, thereby permitting the SLAM processes to be performed in software using relatively inexpensive microprocessor-based computer systems.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method of autonomous localization and mapping, the method comprising: visually observing an environment via a visual sensor; maintaining a map of landmarks in a data store, where the map of landmarks is based at least in part on visual observations of the environment; receiving data from a dead reckoning sensor, where data from the dead reckoning sensor relates to movement of the visual sensor within the environment; using data from the dead reckoning sensor and a prior pose estimate to predict a new device pose in a global reference frame at least partly in response to a determination that a known landmark has not been encountered; and using data from the visual sensor to predict a new device pose in the global reference frame at least partly in response to a determination that a known landmark has been recognized, where the new device pose estimate is based at least in part on a previous pose estimate associated with the known landmark and using the visual sensor data to update the map; wherein the data from the visual sensor comprises 2D image coordinates of features associated with the recognized landmark.
A method for a robot to navigate autonomously and create a map of its surroundings. The robot uses a camera to see the environment and builds a map of landmarks in a computer's memory based on what it sees. The robot also uses sensors like wheel encoders or a pedometer to measure its own movement. When the robot moves and doesn't see a known landmark, it uses its movement sensors and its best guess of its location to predict where it is now. If the robot recognizes a landmark in its camera view, it refines its location estimate based on where it remembers the landmark being and updates the map based on the new observation, using 2D image coordinates of the landmark features.
2. The method as defined in claim 1 , further comprising using the autonomous localization and mapping in a mobile robot.
The autonomous localization and mapping method, which involves visually observing an environment via a camera, maintaining a landmark map in memory based on visual observations, receiving movement data from sensors like wheel encoders, predicting a new location when a known landmark is not seen using movement data and previous location estimates, and predicting and updating the location and map using camera data when a known landmark is recognized, where the new location estimate uses a previous landmark location and the visual sensor data to update the map; all this functionality is used to enable navigation in a mobile robot.
3. The method as defined in claim 1 , wherein the map comprises one or more maps.
The autonomous localization and mapping method, which involves visually observing an environment via a camera, maintaining a landmark map in memory based on visual observations, receiving movement data from sensors like wheel encoders, predicting a new location when a known landmark is not seen using movement data and previous location estimates, and predicting and updating the location and map using camera data when a known landmark is recognized, where the new location estimate uses a previous landmark location and the visual sensor data to update the map; uses one or more maps to represent the environment, potentially allowing for different map layers or representations to improve accuracy or efficiency.
4. The method as defined in claim 1 , wherein the visual sensor corresponds to one or more cameras.
The autonomous localization and mapping method, which involves visually observing an environment via a camera, maintaining a landmark map in memory based on visual observations, receiving movement data from sensors like wheel encoders, predicting a new location when a known landmark is not seen using movement data and previous location estimates, and predicting and updating the location and map using camera data when a known landmark is recognized, where the new location estimate uses a previous landmark location and the visual sensor data to update the map; uses one or more cameras as the visual sensor to observe the environment, potentially providing stereo vision or multiple viewpoints.
5. The method as defined in claim 1 , wherein the dead reckoning sensor corresponds to at least one of an odometer or a pedometer.
The autonomous localization and mapping method, which involves visually observing an environment via a camera, maintaining a landmark map in memory based on visual observations, receiving movement data from sensors like wheel encoders, predicting a new location when a known landmark is not seen using movement data and previous location estimates, and predicting and updating the location and map using camera data when a known landmark is recognized, where the new location estimate uses a previous landmark location and the visual sensor data to update the map; uses at least one of an odometer (wheel encoder) or a pedometer as the dead reckoning sensor to measure movement.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
September 13, 2012
August 13, 2013
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.