Visual Inertial Odometry (VIO) Tracker. More...

Classes
struct	Config
	Configuration parameters of the VIO tracker. More...

struct	ImuState
	IMU state: velocity, gyro bias, accelerometer bias. More...

struct	MultisensorSettings
	Multisensor odometry settings. More...

struct	RGBDSettings
	RGBD odometry settings. More...

struct	State
	State of the odometry tracker. More...

Public Types
enum class	MulticameraMode : uint8_t { Performance , Precision , Moderate }
	Multicamera mode. More...

enum class	OdometryMode : uint8_t { Multicamera , Inertial , RGBD , Mono , Multisensor }
	Odometry mode. More...

using	ImageSet = std::vector< Image >
	Image set.

using	Gravity = Vector3f

Public Member Functions
	Odometry (const Rig &rig, const Config &cfg=GetDefaultConfig())
	Construct a tracker. More...

	Odometry (Odometry &&other) noexcept
	Move constructor. More...

	~Odometry ()
	Destructor.

PoseEstimate	Track (const ImageSet &images, const ImageSet &masks={}, const ImageSet &depths={}, const cuvslam::internal::Internals *internals=nullptr)
	Track a rig pose using current frame. More...

void	RegisterImuMeasurement (uint32_t sensor_index, const ImuMeasurement &imu)
	Register IMU measurement. More...

std::vector< Observation >	GetLastObservations (uint32_t camera_index) const
	Get Last Observations. More...

std::vector< Landmark >	GetLastLandmarks () const
	Get Last Landmarks. More...

std::optional< Gravity >	GetLastGravity () const
	Get Last Gravity. More...

std::optional< ImuState >	GetImuState () const
	Get current IMU state (velocity, gyro bias, acc bias). More...

void	GetState (State &state) const
	Get tracker state. More...

std::unordered_map< uint64_t, Vector3f >	GetFinalLandmarks () const
	Get all final landmarks from all frames. More...

const std::vector< uint8_t > &	GetPrimaryCameras () const
	Get primary camera indices used for tracking. More...

void	ApplyPersistentInternalParameters (const std::vector< cuvslam::internal::InternalParameter > &parameters)
	Apply internal parameters by string key/value pairs. More...

Static Public Member Functions
static Config	GetDefaultConfig ()
	Get default configuration. More...

Detailed Description

Visual Inertial Odometry (VIO) Tracker.

Class Documentation

◆ cuvslam::Odometry::Config

struct cuvslam::Odometry::Config

Configuration parameters of the VIO tracker.

Class Members
MulticameraMode	multicam_mode	Multicamera mode. Default: MulticameraMode::Precision.
OdometryMode	odometry_mode	Odometry mode. Default: OdometryMode::Multicamera.
bool	use_gpu	Enable tracking using GPU. Default: true.
bool	async_sba	Enable SBA asynchronous mode. Default: true.
bool	use_motion_model	Enable internal pose prediction mechanism. Default: true. If frame rate is high enough it improves tracking performance and stability. As a general rule it is better to use a pose prediction mechanism tailored to a specific application. If you have an IMU, consider using it to provide pose predictions to cuVSLAM.
bool	use_denoising	Enable image denoising. Disable if the input images have already passed through a denoising filter. Default: false
bool	rectified_stereo_camera	Enable fast and robust tracking between rectified cameras with principal points on the horizontal line. Default: false
bool	enable_observations_export	Enable GetLastObservations(). Warning: export flags slow down execution and result in additional memory usage. Default: false
bool	enable_landmarks_export	Enable GetLastLandmarks(). Warning: export flags slow down execution and result in additional memory usage. Default: false
bool	enable_final_landmarks_export	Enable GetFinalLandmarks(). Warning: export flags slow down execution and result in additional memory usage. This flag also sets enable_landmarks_export and enable_observations_export. Default: false
float	max_frame_delta_s	Maximum frame delta in seconds. Odometry will warn if time delta between frames is higher than the threshold. Default: 1.f
string_view	debug_dump_directory	Directory where input data will be dumped in edex format.
bool	debug_imu_mode	Enable IMU debug mode. Default: false.
RGBDSettings	rgbd_settings	RGBD odometry settings.
MultisensorSettings	multisensor_settings	Multisensor odometry settings. Used only when odometry_mode == OdometryMode::Multisensor.

◆ cuvslam::Odometry::ImuState

struct cuvslam::Odometry::ImuState

IMU state: velocity, gyro bias, accelerometer bias.

Class Members
Vector3f	velocity
Vector3f	gyro_bias
Vector3f	acc_bias

◆ cuvslam::Odometry::MultisensorSettings

struct cuvslam::Odometry::MultisensorSettings

Multisensor odometry settings.

Warning: Experimental: tracking may be inaccurate or fail for some sensor configurations and scenes.

Used only when Config::odometry_mode == OdometryMode::Multisensor.

Multisensor mode supports any mix of:

one or more plain RGB cameras
one or more RGB-D cameras (any subset of the rig)
a single optional IMU (configured through Rig::imus)

The rig's IMU presence drives IMU fusion automatically — there is no separate with_imu flag. The only fields here describe depth handling.

Class Members
vector< int32_t >	depth_camera_ids	Camera ids (matching Rig::cameras indices) that supply depth images at Track() time. Empty means: no depth (pure multi-camera-RGB). Cameras not listed here are treated as RGB-only. Mixed rigs (some RGB, some RGB-D) are supported by listing only the depth-capable cameras.
float	depth_scale_factor	Scale factor for depth measurements (denominator: raw depth divided by this is meters). Applied uniformly across all depth cameras. Default: 1.f.
bool	enable_depth_stereo_tracking	Allow stereo 2D tracking between depth-aligned cameras and other cameras. Default: true. Multisensor mode benefits from cross-camera 2D tracks (depth-aligned cameras are typically informative anchors), so we enable this by default. Set to false to opt out.

◆ cuvslam::Odometry::RGBDSettings

struct cuvslam::Odometry::RGBDSettings

RGBD odometry settings.

Class Members
float	depth_scale_factor	Scale of provided depth measurements. Default: 1.f. The scale factor is the denominator used to convert raw depth values from the input depth image to actual distance measurements in meters. For example, in TUM RGB-D a factor of 5000 is used for 16-bit PNG images, meaning each pixel value should be divided by 5000 to get the depth in meters, while a factor of 1 is used for 32-bit float images, where the depth values are already in meters.
int32_t	depth_camera_id	Depth camera id. Depth image is supposed to be pixel-to-pixel aligned with some RGB camera image. This field specifies camera id, that depth is aligned with. Default: -1
bool	enable_depth_stereo_tracking	Allows stereo 2D tracking between depth-aligned camera and any other camera. Default: false.

◆ cuvslam::Odometry::State

struct cuvslam::Odometry::State

State of the odometry tracker.

Only available if data export is enabled in Config.

Class Members
typedef unordered_map< uint8_t, shared_ptr< Context > >	ContextMap

Class Members
uint64_t	frame_id	Internal frame id.
int64_t	timestamp_ns	Timestamp in nanoseconds.
Pose	delta	Pose change since last keyframe.
bool	keyframe	Is this frame a keyframe?
bool	warming_up	Is the tracker in warming up phase?
optional< Gravity >	gravity	Optional gravity. Available in Inertial or Multisensor mode with an IMU.
vector< Observation >	observations	Observations for this frame.
vector< Landmark >	landmarks	Landmarks for this frame.
ContextMap	context	Opaque context information for this frame (used internally by Slam)

Member Typedef Documentation

◆ Gravity

using cuvslam::Odometry::Gravity = Vector3f

Gravity acceleration vector (magnitude ~9.81 m/s²) in the rig / VO frame; +Y is down (OpenCV), so it is approximately (0, +g, 0) when the rig is upright.

Member Enumeration Documentation

◆ MulticameraMode

enum cuvslam::Odometry::MulticameraMode : uint8_t

strong

Multicamera mode.

Multicamera mode defines which cameras will be used for mono SOF (primary cameras)

Enumerator
Performance	primary cameras auto selection, each secondary camera must be connected to only one primary camera
Precision	all cameras are primary cameras
Moderate	primary cameras auto selection, secondary cameras may be connected to more than one primary camera

◆ OdometryMode

enum cuvslam::Odometry::OdometryMode : uint8_t

strong

Odometry mode.

Odometry mode defines what data is expected and how it will be used by odometry tracker

Enumerator
Multicamera	Uses multiple synchronized stereo cameras, all cameras need to have frustum overlap with at least one another camera.
Inertial	Uses stereo camera and IMU measurements. A single stereo-camera with a single IMU sensor is supported.
RGBD	Uses RGB-D camera for tracking. A single RGB-D camera is supported. RGB & Depth images must be aligned.
Mono	Uses a single camera, tracking is accurate up to scale.
Multisensor	Warning Experimental: tracking may be inaccurate or fail for some sensor configurations and scenes. Unified multi-sensor mode (cuNLS-based). Supports any mix of plain RGB cameras, RGB-D cameras (any subset of the rig), with or without a single IMU. IMU fusion is enabled automatically when the rig contains an IMU; sba_mode is forced to the inertial bundler in that case. Per-camera depth presence is configured through MultisensorSettings::depth_camera_ids. @note Requirements (construction throws `std::invalid_argument` otherwise): - Build must have cuNLS enabled (`-DUSE_CUNLS=ON`). - The rig must provide at least one RGB-D camera through MultisensorSettings::depth_camera_ids, or at least one camera pair with overlapping frustums. A single RGB-D camera is valid, with or without an IMU. - The current cuNLS solver supports pinhole cameras. Other camera models emit a warning and are not supported in Multisensor mode. - Depth images passed to `Track()` must use `Encoding::MONO` with `DataType::UINT16` or `DataType::FLOAT32`; see `Track()` for the per-frame matching rules against `MultisensorSettings::depth_camera_ids`.

Constructor & Destructor Documentation

◆ Odometry() [1/2]

cuvslam::Odometry::Odometry	(	const Rig &	rig,
		const Config &	cfg = `GetDefaultConfig()`
	)

explicit

Construct a tracker.

Parameters

[in]	rig	rig setup
[in]	cfg	tracker configuration

Exceptions

std::runtime_error	if tracker fails to initialize
std::invalid_argument	if rig or config is invalid

◆ Odometry() [2/2]

cuvslam::Odometry::Odometry ( Odometry && other )

noexcept

Move constructor.

Parameters

[in] other other tracker

Member Function Documentation

◆ GetDefaultConfig()

static Config cuvslam::Odometry::GetDefaultConfig ( )

inlinestatic

Get default configuration.

See also: Config for default values.

Returns: default configuration

◆ Track()

PoseEstimate cuvslam::Odometry::Track	(	const ImageSet &	images,
		const ImageSet &	masks = `{}`,
		const ImageSet &	depths = `{}`,
		const cuvslam::internal::Internals *	internals = `nullptr`
	)

Track a rig pose using current frame.

Track current frame synchronously: the function blocks until the tracker has computed a pose. By default, this function uses visual odometry to compute a pose. If visual odometry tracker fails to compute a pose, in Inertial mode or Multisensor mode with an IMU the function returns the position calculated from user-provided IMU data. If after several calls of Track() visual odometry is not able to recover, then invalid pose will be returned.

The track will output poses in the same coordinate system until a loss of tracking.

Image timestamps have to match. cuVSLAM will use timestamp from the camera 0 image. If a camera rig provides "almost synchronized" frames, the timestamps should be within 1 millisecond. The number of images for tracker should not exceed rig->num_cameras.

Parameters

[in]	images	an array of synchronized images no more than rig->num_cameras. Must use ImageData::DataType::UINT8. Partial ImageSet is supported, for example due to frame drops. Corresponding cameras are identified by Image::camera_index.
[in]	masks	(Optional) an array of corresponding masks no more than rig->num_cameras. Must use ImageData::DataType::UINT8. Partial ImageSet is supported, for example if mask is calculated on for some cameras. Corresponding cameras are identified by Image::camera_index.
[in]	depths	(Optional) an array of depth images. In OdometryMode::RGBD exactly one depth image must be provided. In OdometryMode::Multisensor pass one depth image per depth-providing camera; each entry is matched to its rig camera by Image::camera_index and every camera_index must appear in MultisensorSettings::depth_camera_ids. Other modes must pass an empty array. Must use ImageData::Encoding::MONO and ImageData::DataType::UINT16 or ImageData::DataType::FLOAT32.
[in]	internals	(Optional) pointer to internal per-frame development parameters; pass nullptr (default) to use built-in defaults. Not intended for production use.

Returns: On success PoseEstimate contains estimated rig pose, on failure PoseEstimate::world_from_rig will be nullopt.

Exceptions

std::invalid_argument	if image parameters are invalid
std::runtime_error	in case of unexpected errors

◆ RegisterImuMeasurement()

void cuvslam::Odometry::RegisterImuMeasurement	(	uint32_t	sensor_index,
		const ImuMeasurement &	imu
	)

Register IMU measurement.

If visual odometry loses camera position, it briefly continues execution using user-provided IMU measurements while trying to recover the position. You should call these functions in monotonic timestamp order however many IMU measurements you have between image acquisitions:

tracker.Track
tracker.RegisterImuMeasurement
...
tracker.RegisterImuMeasurement
tracker.Track

IMU measurements and frame images both have timestamps, so calls to Track() and RegisterImuMeasurement() on the same Odometry instance must be made in non-decreasing timestamp order and externally serialized. Do not call them concurrently from different threads. If IMU samples are captured on a separate thread, buffer them and submit them in timestamp order from the same sequence that calls Track(), or protect all calls with caller-owned synchronization.

Parameters

[in]	sensor_index	Sensor index; must be 0, as only one sensor is supported now
[in]	imu	IMU measurements

Exceptions

std::invalid_argument if IMU fusion is disabled or if called out of the order of timestamps

See also: Track

◆ GetLastObservations()

std::vector<Observation> cuvslam::Odometry::GetLastObservations ( uint32_t camera_index ) const

Get Last Observations.

Get an array of observations from the last VO frame for a specific camera

Parameters

[in] camera_index Index of the camera to get observations for

Returns: Array of observations

Exceptions

std::invalid_argument if stats export is disabled

See also: Observation

◆ GetLastLandmarks()

std::vector<Landmark> cuvslam::Odometry::GetLastLandmarks ( ) const

Get Last Landmarks.

Get an array of landmarks from the last VO frame; Landmarks are 3D points in the last camera frame.

Returns: Array of landmarks

Exceptions

std::invalid_argument if stats export is disabled

See also: Landmark

◆ GetLastGravity()

std::optional<Gravity> cuvslam::Odometry::GetLastGravity ( ) const

Get Last Gravity.

Get gravity acceleration vector in the last VO / rig frame (+Y down, OpenCV convention).

Returns: Optional gravity vector. Empty if gravity is not yet available.

Exceptions

std::invalid_argument if IMU fusion is disabled

See also: Gravity

◆ GetImuState()

std::optional<ImuState> cuvslam::Odometry::GetImuState ( ) const

Get current IMU state (velocity, gyro bias, acc bias).

Returns: IMU state or nullopt before the first successful track.

◆ GetState()

void cuvslam::Odometry::GetState ( State & state ) const

Get tracker state.

Only available if data export is enabled in Config.

Parameters

[out] state Odometry state to be filled

Exceptions

std::invalid_argument if stats export is disabled

See also: State

◆ GetFinalLandmarks()

std::unordered_map<uint64_t, Vector3f> cuvslam::Odometry::GetFinalLandmarks ( ) const

Get all final landmarks from all frames.

Landmarks are 3D points in the odometry start frame.

Returns: std::unordered_map<uint64_t, Vector3f>

Exceptions

std::invalid_argument if stats export is disabled

See also: Landmark

◆ GetPrimaryCameras()

const std::vector<uint8_t>& cuvslam::Odometry::GetPrimaryCameras ( ) const

Get primary camera indices used for tracking.

Returns: Vector of primary camera indices

◆ ApplyPersistentInternalParameters()

void cuvslam::Odometry::ApplyPersistentInternalParameters ( const std::vector< cuvslam::internal::InternalParameter > & parameters )

Apply internal parameters by string key/value pairs.

Allows setting internal runtime settings by name. Unknown keys log a warning and are ignored. Invalid values or keys not applicable to the current odometry mode throw std::invalid_argument.

For internal use only.

Supported keys (grouped by prefix):

SBA (all modes): sba.num_sba_frames, sba.num_inertial_sba_frames, sba.num_fixed_sba_frames, sba.num_sba_iterations, sba.robustifier_scale, sba.use_sba_winsorizer

Note: sba.async and sba.mode are construction-time settings that determine whether the SBA background thread is spawned and which bundler is used. They cannot be changed after the tracker is created. Set Odometry::Config::async_sba and Odometry::Config::odometry_mode before constructing the Odometry object instead.

StateMachine / IMU gravity estimation (Inertial mode only): sm.gravity_update_period_ns, sm.max_integration_time_ns, sm.min_num_kf_for_gravity, sm.min_time_period_ns, sm.max_time_period_ns

Parameters

[in] parameters Key/value pairs to apply.

Classes

Public Types

Public Member Functions

Static Public Member Functions

Detailed Description

Class Documentation

◆ cuvslam::Odometry::Config

◆ cuvslam::Odometry::ImuState

◆ cuvslam::Odometry::MultisensorSettings

◆ cuvslam::Odometry::RGBDSettings

◆ cuvslam::Odometry::State

Member Typedef Documentation

◆ Gravity

Member Enumeration Documentation

◆ MulticameraMode

◆ OdometryMode

Constructor & Destructor Documentation

◆ Odometry() [1/2]

◆ Odometry() [2/2]

Member Function Documentation

◆ GetDefaultConfig()

◆ Track()

◆ RegisterImuMeasurement()

◆ GetLastObservations()

◆ GetLastLandmarks()

◆ GetLastGravity()

◆ GetImuState()

◆ GetState()

◆ GetFinalLandmarks()

◆ GetPrimaryCameras()

◆ ApplyPersistentInternalParameters()