How to use evo to compute ATE and RPE for SLAM evaluation

To use the evo library for evaluating SLAM algorithms, you need to follow a series of steps to compute the Absolute Trajectory Error (ATE) and Relative Pose Error (RPE). Below is a guide on how to use evo for this purpose, including the required file formats for the estimated and ground truth trajectories, as well as example Python code.

1. Installingevo

First, you need to install evo. You can install it via pip:

    pip install evo --upgrade --no-binary evo
    
    
      
    

This will install the latest version of the evo package. My version is v1.30.3.

2. Required Input File Format

Both the ground truth trajectory and the estimated trajectory must be provided in the TUM dataset format. This format is simple and consists of a text file where each line contains a timestamp followed by the pose (position and orientation):

    timestamp tx ty tz qx qy qz qw
    
    
      
    

• timestamp : The time at which the pose was recorded.
  • tx, ty, tz : The position (translation) of the camera in the world coordinate frame.
  • qx, qy, qz, qw : The orientation of the camera as a quaternion.

Example of trajectory file in TUM format:

0 0.0 0.0 0.0 0.0 0.0 0.0 1.0
    2.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0
    3.0 2.0 0.0 0.0 0.0 0.0 0.0 1.0
    ...
    
    
      
      
      
      
    

You need two files:

1. Ground truth trajectory file (e.g., groundtruth.txt).
2. Estimated trajectory file (e.g., estimated.txt).

3. Aligning Trajectories and Computing Error Metrics

Once you have the estimated and ground truth trajectories in the correct format, you can use evo to compute the ATE and RPE.

3.1 Compute Absolute Trajectory Error (ATE)

To compute the ATE (Absolute Trajectory Error) , use the following command:

    evo_ape tum groundtruth.txt estimated.txt --align
    
    
      
    

This will compute the ATE between your estimated trajectory (estimated.txt) and the ground truth trajectory (groundtruth.txt). The --align flag performs SE(3) alignment (rotation and translation alignment).

You can add the --plot flag to visualize the trajectories and the error:

    evo_ape tum groundtruth.txt estimated.txt --align --plot
    
    
      
    

3.2 Compute Relative Pose Error (RPE)

To compute the RPE (Relative Pose Error) , use the following command:

    evo_rpe tum groundtruth.txt estimated.txt --align --delta 1
    
    
      
    

The --align flag performs SE(3) alignment. The --delta option specifies the time interval between consecutive poses for which the relative error is computed (in this case, delta = 1 step).

You can also visualize the RPE with the --plot flag:

    evo_rpe tum groundtruth.txt estimated.txt --align --delta 1 --plot
    
    
      
    

4. Python Example Code for ATE and RPE

If you want to compute the ATE and RPE programmatically using Python, you can use the evo API directly as shown below:

    import evo.tools.file_interface as file_io
    import evo.core.trajectory as traj
    from evo.core.metrics import PoseRelation, APE, RPE
    from evo.core.sync import associate_trajectories
    import numpy as np
    
    # Load ground truth and estimated trajectories
    gt_traj = file_io.read_tum_trajectory_file("groundtruth.txt")
    est_traj = file_io.read_tum_trajectory_file("estimated.txt")
    
    # Synchronize trajectories based on timestamps
    gt_traj, est_traj = associate_trajectories(gt_traj, est_traj)
    
    # Align estimated trajectory to ground truth using SE(3) alignment
    est_traj_aligned = traj.align_trajectory(est_traj, gt_traj)
    
    # 1. Compute ATE (Absolute Trajectory Error)
    ape_metric = APE(PoseRelation.translation_part)
    ape_metric.process_data((gt_traj, est_traj_aligned))
    ape_stats = ape_metric.get_all_statistics()
    
    print("ATE RMSE: ", ape_stats['rmse'])
    print("ATE Mean: ", ape_stats['mean'])
    print("ATE Median: ", ape_stats['median'])
    
    # 2. Compute RPE (Relative Pose Error)
    rpe_metric = RPE(PoseRelation.translation_part, delta=1)
    rpe_metric.process_data((gt_traj, est_traj_aligned))
    rpe_stats = rpe_metric.get_all_statistics()
    
    print("RPE RMSE: ", rpe_stats['rmse'])
    print("RPE Mean: ", rpe_stats['mean'])
    print("RPE Median: ", rpe_stats['median'])
    
    
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
    

5. Explanation of Results

ATE (Absolute Trajectory Error) :

* `rmse`: Root Mean Square Error of the translation part of the error.
* `mean`: Mean translation error.
* `median`: Median translation error.

RPE (Relative Pose Error) :

* `rmse`: Root Mean Square Error for the relative pose errors.
* `mean`: Mean relative error.
* `median`: Median relative error.

These statistics provide a quantitative assessment of the performance of the SLAM system.

6. Visualizing Results

To visualize the trajectories and errors, you can use evo’s plotting functionality. For example, after computing ATE or RPE, you can generate a plot using:

    evo_ape tum groundtruth.txt estimated.txt --align --plot
    
    
      
    

This will show a graphical representation of the ground truth and estimated trajectories, along with the translational errors.

7. Additional Options

Scaling : If your SLAM algorithm doesn’t recover scale (e.g., monocular visual SLAM), you can perform Sim(3) alignment (scaling, rotation, and translation) by adding the --correct_scale flag:

    evo_ape tum groundtruth.txt estimated.txt --align --correct_scale

    
    
        

Saving Outputs : You can save the output statistics to a file by adding the --save_results and --save_plot options:

    evo_ape tum groundtruth.txt estimated.txt --align --save_results results.zip --save_plot plot.pdf