Advertisement

论文学习--Learning High-Speed Flight in the Wild

阅读量：

文章目录

可选

复制代码 
* 运行

  • 代码梳理

    • test

      • 仿真数据采集
      • 轨迹测试

    • 代码逻辑

  • 论文阅读

    • 摘要
    • 方法
      • Privileged Expert
      • Sendorimotor Agent
      • Actions

方法详述
* 该领域权威专家
* 该学生政策
* 训练环境

Git

该系统基于GitHub平台（https://github.com/uzh-rpg/agile_autonomy）实现了一种先进的自主导航算法

子文链接

随后又撰写了另外三篇博客

代码运行

编译环境

Ubuntu18.04 + gcc6 + cuda11.3 + ROS melodic + anaconda3(conda 4.10.1) + python3.8 + open3d v0.10.0

编译步骤

参考git源码的Readme.md

【可选】

可以选择采用HTTPS方式进行依赖包的下载。即可以通过以下方式对src/agile_autonomy/dependencies.yaml文件进行修改。

复制代码 
    repositories:
      catkin_boost_python_buildtool:
    type: git
    url: https://github.com/ethz-asl/catkin_boost_python_buildtool.git
    version: master
      catkin_simple:
    type: git
    url: https://github.com/catkin/catkin_simple.git
    version: master
      eigen_catkin:
    type: git
    url: https://github.com/ethz-asl/eigen_catkin.git
    version: master
      eigen_checks:
    type: git
    url: https://github.com/ethz-asl/eigen_checks.git
    version: master
      gflags_catkin:
    type: git
    url: https://github.com/ethz-asl/gflags_catkin.git
    version: master
      glog_catkin:
    type: git
    url: https://github.com/ethz-asl/glog_catkin.git
    version: master
      mav_comm:
    type: git
    url: https://github.com/ethz-asl/mav_comm.git
    version: master
      minimum_jerk_trajectories:
    type: git
    url: https://github.com/uzh-rpg/minimum_jerk_trajectories.git
    version: master
      minkindr:
    type: git
    url: https://github.com/ethz-asl/minkindr.git
    version: master
      minkindr_ros:
    type: git
    url: https://github.com/ethz-asl/minkindr_ros.git
    version: master
      numpy_eigen:
    type: git
    url: https://github.com/ethz-asl/numpy_eigen.git
    version: master
      rpg_common:
    type: git
    url: https://github.com/kelia/rpg_common.git
    version: main
      rotors_simulator:
    type: git
    url: https://github.com/ethz-asl/rotors_simulator.git
    version: master
      rpg_mpc:
    type: git
    url: https://github.com/uzh-rpg/rpg_mpc.git
    version: feature/return_full_horizon
      rpg_quadrotor_common:
    type: git
    url: https://github.com/uzh-rpg/rpg_quadrotor_common.git
    version: master
      rpg_quadrotor_control:
    type: git
    url: https://github.com/uzh-rpg/rpg_quadrotor_control.git
    version: devel/elia
      rpg_single_board_io:
    type: git
    url: https://github.com/uzh-rpg/rpg_single_board_io.git
    version: master
      rpg_flightmare:
    type: git
    url: https://github.com/antonilo/flightmare_agile_autonomy.git
    version: main
      rpg_mpl_ros:
    type: git
    url: https://github.com/uzh-rpg/rpg_mpl_ros.git
    version: master
      assimp_catkin:
    type: git
    url: https://github.com/uzh-rpg/assimp_catkin.git
    version: master
    
    
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
    
    代码解读

[1] 下载源码

复制代码 
    mkdir agile_autonomy_ws
    cd agile_autonomy_ws
    mkdir -p agile_autonomy_ws/src
    cd agile_autonomy_ws
    catkin init
    catkin config --extend /opt/ros/melodic
    catkin config --merge-devel
    catkin config --cmake-args -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-fdiagnostics-color
    cd src
    git clone https://github.com/uzh-rpg/agile_autonomy.git
    vcs-import < agile_autonomy/dependencies.yaml
    cd rpg_mpl_ros
    git submodule update --init --recursive
    
    #install extra dependencies (might need more depending on your OS)
    sudo apt-get install libqglviewer-dev-qt5
    
    # Install external libraries for rpg_flightmare
    sudo apt install -y libzmqpp-dev libeigen3-dev libglfw3-dev libglm-dev
    
    # Install dependencies for rpg_flightmare renderer
    sudo apt install -y libvulkan1 vulkan-utils gdb
    
    
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
    
    代码解读

[2] 先安装Open3D

源码安装C++版本的open3D， v0.10.0. 由于采用gcc6编译，因此不用最新版本v0.13.0的open3D。

复制代码 
    git clone --recursive https://github.com/intel-isl/Open3D
    
    # You can also update the submodule manually
    git submodule update --init --recursive
    
    # 切换到指定版本
    git checkout  v0.10.0
    
    # On Ubuntu
    util/install_deps_ubuntu.sh
    
    mkdir build
    cd build
    cmake -DCMAKE_INSTALL_PREFIX=<open3d_install_directory> ..
    # On Ubuntu
    make -j$(nproc)
    make install
    
    
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
    
    代码解读

[3] 修改Open3D的相关路径

因为Open3D被配置为位于指定路径；因此，在源代码中相应位置调整Open3D的搜索路径。请在以下三个文件中进行调整：

  • mpl_test_node/VerificationNode.Cmake
    • open3d_conversion/Open3DConversion.Cmake
    • agile_autonomy/DataGeneration/TrajectorySamplers.Cmake

对应行 find_package(Open3D HINTS *：

复制代码 
    find_package(Open3D HINTS /home/yourname/open3d_install/lib/cmake/)
    
    
      
    
    代码解读

[4] 开始编译

复制代码 
    catkin build
    
    
      
    
    代码解读

这里还是会报找不到Open3D， 如下：

复制代码 
    CMake Error at /home/wxm/Documents/codes/agile_automomy_wxm/devel/share/open3d_conversions/cmake/open3d_conversionsConfig.cmake:173 (message):
      Project 'mpl_test_node' tried to find library 'Open3D'.  The library is
      neither a target nor built/installed properly.  Did you compile project
      'open3d_conversions'? Did you find_package() it before the subdirectory
      containing its code is included?
    
    
    
      
      
      
      
      
      
    
    代码解读

解决方案：在指定CMake文件中的第157行位置处添加完整的Open3D安装路径设置项

复制代码 
    foreach(path  *;/home/yourname/open3d_install/lib)
    
    
      
    
    代码解读

然后再继续编译，

复制代码 
    # Build and re-source the workspace
    catkin build
    
    
      
      
    
    代码解读

[5] 报错2

复制代码 
    error: missing template arguments before ‘timing_spline’
    
    
      
    
    代码解读

解决方案：修复文件 agile_automomy_wxm/src/agile_autonomy/data_generation/traj_sampler/src/traj_sampler.cpp 中的第266行内容，请具体如下。

复制代码 
    Timing<double> timing_spline;
    
    
      
    
    代码解读

[6] 报错3

复制代码 
    error: missing template arguments before ‘(’ token
       Eigen::Quaternion(std::cos(-cam_pitch_angle_ / 2.0), 0.0,
    
    
      
      
    
    代码解读

优化 agile_automomy_wxm/src/agile_autonomy/data_generation/agile_autonomy.cpp文件中的第461号行的操作

复制代码 
    Eigen::Quaterniond(std::cos(-cam_pitch_angle_ / 2.0), 0.0,
                        std::sin(-cam_pitch_angle_ / 2.0), 0.0);
    
    
      
      
    
    代码解读

[7] 运行中报错

运行时，可能会报找不到一些python库，根据提示，直pip安装即可。

运行时如果报cv_bridge报如下错误，这是需要源码编译一个cv_bridge.

复制代码 
    from cv_bridge.boost.cv_bridge_boost import getCvType
    ImportError: dynamic module does not define module export function (PyInit_cv_bridge_boost)
    
    
      
      
    
    代码解读

源码安装cv_bridge

复制代码 
    conda deactivate
    mkdir  cv_bridge_ws
    cd cv_bridge_ws
    mkdir src
    cd src
    git clone https://github.com/ros-perception/vision_opencv.git
    # Find version of cv_bridge in your repository
    apt-cache show ros-melodic-cv-bridge | grep Version
    Version: 1.13.0-0bionic.20210921.205941
    git checkout 1.13.0 
    catkin config  --install
    catkin build
    source  install/setup.bash  --extend
    
    
      
      
      
      
      
      
      
      
      
      
      
      
      
    
    代码解读

[8] 配置学习环境

复制代码 
    # Create your learning environment
    roscd planner_learning
    conda create --name tf_24 python=3.8
    conda activate tf_24
    conda install tensorflow-gpu
    pip install rospkg==1.2.3,pyquaternion,open3d,opencv-python
    
    
      
      
      
      
      
      
    
    代码解读

[9] 下载flighemare渲染环境

Download the flightmare standalone available from the provided link: https://zenodo.org/record/5517791/files/standalone.tar?download=1, decompress and extract it, then copy and place it into the flightrender directory located at https://github.com/antonilo/flightmare_agile_autonomy/tree/main/flightrender.

运行

1.打开一个终端，输入

复制代码 
    cd agile_autonomy_ws
    source devel/setup.bash
    roslaunch agile_autonomy simulation.launch
    
    
      
      
      
    
    代码解读
  1. 再打开另一个终端，输入
复制代码 
    cd agile_autonomy_ws
    source devel/setup.bash
    source  cv_bridge_ws/install/setup.bash  --extend
    conda activate tf_24
    python test_trajectories.py --settings_file=config/test_settings.yaml
    
    
      
      
      
      
      
    
    代码解读
  1. 运行界面如下，一共包含三个窗口。
请添加图片描述

代码梳理

test

程序运行时主要分为两个阶段：第一项操作负责初始化仿真环境并生成相应的数据；第二步则将来自仿真传感器的数据导入到训练好的模型中以生成无人机的最佳飞行轨迹，并实时监控无人机的实际运动状态。

复制代码 
    roslaunch agile_autonomy simulation.launch
    
    
      
    
    代码解读
复制代码 
    python test_trajectories.py --settings_file=config/test_settings.yaml
    
    
      
    
    代码解读

仿真数据采集

在仿真环境中创建一个无人机后，在Unity中基于给定的无人机姿态和图像尺寸等参数设置下，能够生成仿真RGB双目摄像头，并输出相应的深度信息图。

轨迹测试

默认一共测试50次。
其中一次的测试log如下：

复制代码 
    ==========================
     RESET SIMULATION     
    ==========================
    It worked well. (Arrived at 285 / 324)
    Giving a stop from python
    
    Unpausing Physics...
    
    Placing quadrotor...
    success: True
    status_message: "SetModelState: set model state done"
    Received call to Clear Buffer and Restart Experiment
    
    Resetting experiment
    
    Done Reset
    Doing experiment 8
    Reading pointcloud from ../data_generation/data/rollout_21-11-16_14-36-51/pointcloud-unity.ply
    min max pointcloud
    [29.90007973 29.90000534  6.71293974]
    [-29.90000534  10.09995842  -1.48705816]
    Reading Trajectory from ../data_generation/data/rollout_21-11-16_14-36-51/reference_trajectory.csv
    Loaded traj ../data_generation/data/rollout_21-11-16_14-36-51/reference_trajectory.csv with 324 elems
    Starting up!
    Starting up!
    Starting up!
    Starting up!
    Starting up!
    Starting up!
    Starting up!
    Starting up!
    Starting up!
    Starting up!
    Starting up!
    Starting up!
    Starting up!
    Starting up!
    Starting up!
    Starting up!
    Starting up!
    Crashing into something!
    Crashing into something!
    It worked well. (Arrived at 263 / 324)
    Giving a stop from python
    experiment done
    ------- 9 Rollout ------------
    expert_usage is 19.101
    number_crashes is 2.000
    travelled_dist is 16.563
    closest_distance is 0.119
    -------------------------------
    Rollout dir is ../data_generation/data/rollout_21-11-16_14-36-51
    
    
    
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
    
    代码解读
复制代码 
    [ INFO] [1637053941.578871631, 1024.850000000]: [/hummingbird/autopilot] Switched to BREAKING state
    [ INFO] [1637053942.599715262, 1025.360000000]: [/hummingbird/autopilot] Switched to HOVER state
    [ INFO] [1637053944.262924978, 1025.628000000]: [/hummingbird/autopilot] OFF command received
    [ INFO] [1637053944.263037491, 1025.628000000]: [/hummingbird/autopilot] Switched to OFF state
    [ INFO] [1637053944.263474245, 1025.628000000]: Received off command, stopping maneuver execution!
    [ INFO] [1637053944.263595630, 1025.628000000]: Switching to kOff
    [ INFO] [1637053946.541124498, 1026.765000000]: Received off command, stopping maneuver execution!
    [ INFO] [1637053946.541246332, 1026.766000000]: Switching to kOff
    [ INFO] [1637053947.542290912, 1027.019000000]: [/hummingbird/rpg_rotors_interface] Interface armed
    [ INFO] [1637053947.542323540, 1027.019000000]: [/hummingbird/autopilot] START command received
    [ INFO] [1637053947.542396557, 1027.019000000]: [/hummingbird/autopilot] Absolute state estimate available, taking off based on it
    [ INFO] [1637053947.542428793, 1027.019000000]: [/hummingbird/autopilot] Switched to START state
    [ INFO] [1637053952.204707651, 1029.023000000]: Solving MPC with hover as initial guess.
    [ INFO] [1637053960.008035347, 1032.918000000]: [/hummingbird/autopilot] Switched to BREAKING state
    [ INFO] [1637053960.148160505, 1032.988000000]: [/hummingbird/autopilot] Switched to HOVER state
    [ INFO] [1637053972.565306967, 1039.187000000]: Initiated Logging, computing reference trajectory and generating point cloud!
    [ INFO] [1637053972.565415887, 1039.187000000]: Starting maneuver computation
    Initiated acrobatic sequence
    Creating directories in [/home/wxm/Documents/codes/agile_autonomy_ws/src/agile_autonomy/data_generation/agile_autonomy/../data]
    [ INFO] [1637053974.602435105, 1040.204000000]: Find elevation
    [ INFO] [1637053974.602561021, 1040.205000000]: Place trees
    [ INFO] [1637053976.604247155, 1041.204000000]: Spawning [2255] trees, poisson mode is [2209].
    [ INFO] [1637053976.604346633, 1041.205000000]: Incrementing seed to 70.
    [ INFO] [1637053977.906371979, 1041.855000000]: Start creating pointcloud
    [ INFO] [1637053977.906433051, 1041.855000000]: Scale pointcloud: [60.00, 20.00, 10.00]
    [ INFO] [1637053977.906451500, 1041.855000000]: Origin pointcloud: [-0.00, 20.00, 1.89]
    [ INFO] [1637054004.909340006, 1055.338000000]: Pointcloud saved
    Opened new odometry file: /home/wxm/Documents/codes/agile_autonomy_ws/src/agile_autonomy/data_generation/agile_autonomy/../data/rollout_21-11-16_17-12-52/odometry.csv
    Saving trajectory to CSV. 
    Trajectory filename: /home/wxm/Documents/codes/agile_autonomy_ws/src/agile_autonomy/data_generation/agile_autonomy/../data/rollout_21-11-16_17-12-52/reference_trajectory.csv
    Saved trajectory to file.
    [ INFO] [1637054004.926331444, 1055.347000000]: Gogogo!
    [ INFO] [1637054004.926386841, 1055.347000000]: Switching to kAutopilot
    [ INFO] [1637054005.026160529, 1055.397000000]: Maneuver computation successful!
    [ INFO] [1637054005.026284428, 1055.397000000]: Maneuver computation took 16.2100 seconds.
    [ INFO] [1637054005.152199453, 1055.460000000]: Selected trajectory #0
    [ INFO] [1637054005.162462001, 1055.465000000]: Solving MPC with hover as initial guess.
    [ INFO] [1637054005.163936578, 1055.466000000]: [/hummingbird/autopilot] Switched to COMMAND_FEEDTHROUGH state
    [ INFO] [1637054005.283365602, 1055.525000000]: Selected trajectory #0
    [ INFO] [1637054005.418361186, 1055.592000000]: Selected trajectory #0
    [ INFO] [1637054005.546865663, 1055.656000000]: Selected trajectory #0
    [ INFO] [1637054005.682420173, 1055.724000000]: Selected trajectory #0
    [ INFO] [1637054005.814040633, 1055.790000000]: Selected trajectory #0
    [ INFO] [1637054005.948414538, 1055.857000000]: Selected trajectory #1
    [ INFO] [1637054006.080307769, 1055.923000000]: Selected trajectory #0
    [ INFO] [1637054006.212933667, 1055.989000000]: Selected trajectory #0
    [ INFO] [1637054006.349705622, 1056.057000000]: Selected trajectory #0
    [ INFO] [1637054006.484988932, 1056.125000000]: Selected trajectory #0
    [ INFO] [1637054006.615398050, 1056.190000000]: Selected trajectory #0
    [ INFO] [1637054006.749158400, 1056.257000000]: Selected trajectory #0
    [ INFO] [1637054006.884676191, 1056.325000000]: Selected trajectory #0
    [ INFO] [1637054007.014193683, 1056.389000000]: Selected trajectory #0
    [ INFO] [1637054007.150219775, 1056.457000000]: Selected trajectory #0
    [ INFO] [1637054007.285815896, 1056.525000000]: Selected trajectory #0
    [ INFO] [1637054007.416023906, 1056.590000000]: Selected trajectory #0
    [ INFO] [1637054007.549148150, 1056.657000000]: Selected trajectory #0
    [ INFO] [1637054007.683953972, 1056.724000000]: Selected trajectory #0
    [ INFO] [1637054007.819368611, 1056.791000000]: Selected trajectory #0
    [ INFO] [1637054007.949866909, 1056.857000000]: Selected trajectory #0
    [ INFO] [1637054008.085218053, 1056.924000000]: Selected trajectory #0
    [ INFO] [1637054008.216562474, 1056.990000000]: Selected trajectory #0
    [ INFO] [1637054008.350844266, 1057.057000000]: Selected trajectory #0
    [ INFO] [1637054008.485598554, 1057.123000000]: Selected trajectory #0
    [ INFO] [1637054008.618634733, 1057.190000000]: Selected trajectory #0
    [ INFO] [1637054008.751611357, 1057.257000000]: Selected trajectory #0
    [ INFO] [1637054008.885042986, 1057.324000000]: Selected trajectory #0
    [ INFO] [1637054009.016160422, 1057.389000000]: Selected trajectory #0
    [ INFO] [1637054009.153358343, 1057.458000000]: Selected trajectory #0
    [ INFO] [1637054009.284585798, 1057.523000000]: Selected trajectory #0
    [ INFO] [1637054009.417833454, 1057.590000000]: Selected trajectory #0
    [ INFO] [1637054009.554909065, 1057.658000000]: Selected trajectory #0
    [ INFO] [1637054009.684010018, 1057.723000000]: Selected trajectory #0
    [ INFO] [1637054009.822174455, 1057.792000000]: Selected trajectory #0
    [ INFO] [1637054009.954099619, 1057.857000000]: Selected trajectory #0
    [ INFO] [1637054010.087778279, 1057.924000000]: Selected trajectory #0
    [ INFO] [1637054010.216160307, 1057.989000000]: Selected trajectory #0
    [ INFO] [1637054010.355049595, 1058.058000000]: Selected trajectory #1
    [ INFO] [1637054010.493161090, 1058.127000000]: Selected trajectory #0
    [ INFO] [1637054010.617860860, 1058.189000000]: Selected trajectory #0
    [ INFO] [1637054010.752438215, 1058.257000000]: Selected trajectory #0
    [ INFO] [1637054010.888457315, 1058.324000000]: Selected trajectory #0
    [ INFO] [1637054011.022363300, 1058.391000000]: Selected trajectory #0
    [ INFO] [1637054011.152986033, 1058.457000000]: Selected trajectory #0
    [ INFO] [1637054011.287158368, 1058.524000000]: Selected trajectory #0
    [ INFO] [1637054011.419963912, 1058.590000000]: Selected trajectory #1
    [ INFO] [1637054011.551984385, 1058.656000000]: Selected trajectory #0
    [ INFO] [1637054011.689502372, 1058.725000000]: Selected trajectory #1
    [ INFO] [1637054011.820987537, 1058.791000000]: Selected trajectory #1
    [ INFO] [1637054011.955258672, 1058.858000000]: Selected trajectory #0
    [ INFO] [1637054012.086914492, 1058.923000000]: Selected trajectory #0
    [ INFO] [1637054012.222633486, 1058.991000000]: Selected trajectory #0
    [ INFO] [1637054012.354479495, 1059.057000000]: Selected trajectory #0
    [ INFO] [1637054012.488525237, 1059.124000000]: Selected trajectory #0
    [ INFO] [1637054012.621122964, 1059.190000000]: Selected trajectory #0
    [ INFO] [1637054012.753466361, 1059.256000000]: Selected trajectory #0
    [ INFO] [1637054012.892174870, 1059.325000000]: Selected trajectory #0
    [ INFO] [1637054013.024166737, 1059.391000000]: Selected trajectory #0
    [ INFO] [1637054013.155947755, 1059.457000000]: Selected trajectory #0
    [ INFO] [1637054013.290224788, 1059.524000000]: Selected trajectory #0
    [ INFO] [1637054013.421112235, 1059.590000000]: Selected trajectory #0
    [ INFO] [1637054013.559066533, 1059.658000000]: Selected trajectory #0
    [ INFO] [1637054013.692403028, 1059.725000000]: Selected trajectory #0
    [ INFO] [1637054013.822663259, 1059.790000000]: Selected trajectory #0
    [ INFO] [1637054013.958871348, 1059.858000000]: Selected trajectory #0
    [ INFO] [1637054014.090127192, 1059.923000000]: Selected trajectory #0
    [ INFO] [1637054014.219357813, 1059.988000000]: Selected trajectory #0
    [ INFO] [1637054014.354345334, 1060.055000000]: Selected trajectory #0
    [ INFO] [1637054014.491573089, 1060.124000000]: Selected trajectory #0
    [ INFO] [1637054014.625786710, 1060.191000000]: Selected trajectory #0
    [ INFO] [1637054014.759140338, 1060.257000000]: Selected trajectory #1
    [ INFO] [1637054014.892043398, 1060.324000000]: Selected trajectory #0
    [ INFO] [1637054015.023581521, 1060.390000000]: Selected trajectory #0
    [ INFO] [1637054015.158324608, 1060.457000000]: Selected trajectory #0
    [ INFO] [1637054015.293084738, 1060.524000000]: Selected trajectory #0
    [ INFO] [1637054015.423784131, 1060.589000000]: Selected trajectory #0
    [ INFO] [1637054015.557652264, 1060.656000000]: Selected trajectory #0
    [ INFO] [1637054015.692585591, 1060.724000000]: Selected trajectory #0
    [ INFO] [1637054015.823870935, 1060.789000000]: Selected trajectory #0
    [ INFO] [1637054015.959402968, 1060.857000000]: Selected trajectory #0
    [ INFO] [1637054016.096347502, 1060.925000000]: Selected trajectory #0
    [ INFO] [1637054016.231029848, 1060.992000000]: Selected trajectory #1
    [ INFO] [1637054016.365640987, 1061.059000000]: Selected trajectory #0
    [ INFO] [1637054016.493614867, 1061.123000000]: Selected trajectory #0
    [ INFO] [1637054016.627199198, 1061.190000000]: Selected trajectory #0
    [ INFO] [1637054016.761573804, 1061.257000000]: Selected trajectory #0
    [ INFO] [1637054016.895532532, 1061.324000000]: Selected trajectory #0
    [ INFO] [1637054017.029675794, 1061.391000000]: Selected trajectory #0
    [ INFO] [1637054017.160851167, 1061.457000000]: Selected trajectory #0
    Close Odometry File.
    [ INFO] [1637054017.259214657, 1061.506000000]: Switching to kComputeLabels
    [ WARN] [1637054017.467031923, 1061.610000000]: [/hummingbird/autopilot] Did not receive control command inputs anymore but last thrust command was high, will switch to hover
    
    
    
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
    
    代码解读

代码逻辑

  • PlaNet充当网络模型角色，在仿真环境中接收无人机实时采集的RGB视觉数据、IMU速度计信息（odometry）、深度感知数据以及预设参考运动轨迹信息作为输入参数。
    系统通过PlaNet预测出无人机未来的运动轨迹路径，并将该预测结果发布至 /hummingbird/trajectory_predicted 专题Topic上供后续模块使用。
  • AgileAutonomy平台订阅PlaNet发布的预测轨迹信息，并将其存储于 reference_trajectory变量中以便后续状态更新使用。
    随后系统调用MpcController模块生成并发布 quadrotor 带姿态指令，并将该指令推送给状态估计器 state_predictor_ 进行实时位姿更新。
  • 在仿真环境中基于当前无人机定位 odometry 参数以及设定的有效场宽FOV参数值配置下，
    系统能够自动完成对当前场景下无人机所见 RGB 视觉数据与深度感知数据的采集与融合处理工作。
    其中系统会利用getImageFromUnity()函数获取RGB视觉图，并利用SGB算法（sgm_gpu::SgmGpu）从双目视觉数据中提取出精确的深度感知信息作为真实深度基准数据源。

论文阅读

摘要

传统的自主导航系统通常包括感知、地图构建以及路径规划这三个主要环节。尽管串行处理在低速场景中表现良好，在高速环境下将各个步骤分开处理会导致计算延迟和累积误差。为此，本文提出了一种"端到端"的直接映射方法。通过神经网络接收带噪声传感器数据，并生成一条规划路径。这种方法能够有效减少计算延迟，并增强对噪声数据的鲁棒性。训练过程中采用的是带有噪声干扰的真实环境仿真数据。

方法

MPC：基于优先学习的方法用于模型训练过程。
仿真环境采用的是Flightmare、RotorS Gazebo与Unity的结合使用。

在这里插入图片描述

Privileged Expert

通过传统的离线规划方法计算这样的一组无障碍轨迹。

应用Metropolis-Hasting Sampling方法选择前三个轨迹作为后续网络训练的基础约束。

Sendorimotor Agent

输入信息：深度视觉数据 + 飞行动态参数（速度与姿态） + 目标导航指令
系统输出信息：运动轨迹集合（其中包含潜在冲突轨迹的集合）

Actions

通过将网络输入的数据映射到多边形轨迹空间，并按照预估的碰撞成本对这些轨迹进行排序。随后将具有最低预估碰撞成本的轨迹传输给MPC用于无人机路径规划。

方法详述

The privileged expert

无人机运动规划系统是一种基于采样的路径规划方案。给定状态信息和环境点云数据后，在保证不与障碍物发生碰撞的前提下生成一条避障成功的目标路径；该路径的存在意味着无人机能够到达预设航线附近并保持稳定飞行状态；当无人机逐渐靠近预设航线并远离障碍物时（即处于避障与趋近两者结合的过程中），其所在位置的安全性概率也随之提高。通过Metropolis-Hastings算法进行随机采样，在满足收敛条件的前提下实现了对复杂环境下的路径优化；结合三次B-样条曲线与均匀节点采样方法实现路径插值，在保证平滑性的同时降低了计算复杂度；该系统不具备实时响应能力，并且由于采用固定间隔的采样策略导致了显著的计算开销

The student policy

不同于专家轨迹，the student policy产生一个实时的无碰撞轨迹，且仅需要板载传感器测量。
测量包含SGM生成的深度图像，平台的速度和姿态，期望的飞行方向。没有输入点云的情况下仍然能够获取得到轨迹。主要有两个挑战：一是仅可以观测到部分环境，且传感器有噪声；二是分布是多模态的。高概率的可能有多个运动假设，但是进行平均之后，概率又很低。
我们设计了一个神经网络来解决这两个问题。网络有两个分支，能够产生一个视觉，惯导，和参考信息的潜在编码，输出3条轨迹和对应的碰撞代价。我们采用MobileNet-V3架构来有效从深度图像上提取特征，然后进行1维卷积来生成32的特征向量。当前平台的速度和姿态被串联到期望的参考方向上，且经过一个四层感知隐藏节点和LeakyReLU处理。我们再次使用1维卷积来生成32维的特征向量。视觉和姿态信息被串联通过另一个四层感知隐藏节点和LeakyReLU激活。最后预测每个节点，一条轨迹和其碰撞代价。总之，我们的架构接收一个深度图像，平台速度，飞机姿态（表示为旋转矩阵），和参考的方向。从这些输入，来预测一个含有碰撞成本的轨迹。不同于专家学习，神经网络预测的轨迹不能描述全状态，仅仅描述当前位置。这个表示比B样条更通用，且不需要大量的插值计算。
我们利用从专家轨迹中生成的三条轨迹进行有监督学习。考虑到多假设预测，我们采用R-WTA来处理。

Training environment

Flightmare仿真实验环境中搭建场景用于数据采集。所有环境均布设于平坦区域，并在此处放置物体。生成的目标分为两类：仿真中的树形结构与凸体集合。这些物体呈随机排列分布。 创建了850个不同的实验场景。对每个场景进行全局路径规划计算，在起始点延伸至离起始点40米远的方向上完成路径规划计算结果输出后发现 该全局路径仅能被专家系统识别而不受学生政策的影响 选择性地利用教师反馈机制以提升模型性能从而实现了对复杂动态系统的有效建模过程 在仿真实验环境下 通过SGM技术生成深度图并结合真实位姿信息实现了对目标状态的真实描述 最终系统部署于配备有Inter RealSense 435设备的产品上该设备内置深度成像功能可同时获取深度图像与姿态信息从而保证了系统的完整性和可靠性

全部评论 (0)

还没有任何评论哟~

相关文章推荐

论文学习--Learning High-Speed Flight in the Wild

文章目录 Git 子文链接 代码运行 编译环境 编译步骤 【可选】 [1]下载源码 [2]先安装Open3D [3]修改Open3D的相关路径 [4]开始编译 [5]报错2 [6]报错3 [7]运行中...

学习笔记（二）Learning High-Speed Flight in the Wild 修改策略

一、README.md该部分内容 ————————————————————————————————————————————————— 一、改变执行速度或环境 Youcanchangetheaverag...

学习笔记（三）Learning High-Speed Flight in the Wild 训练模型

一、利用收集的数据进行训练测试 在学习笔记（二）中，利用dagger收集了五个轨迹数据，在这里进行之前的训练测试： 在train文件夹内就是之前采集到的数据，而在val文件夹内，是在之前下载的数据集中...

论文Agile Autonomy: Learning High-Speed Flight in the Wild开源代码运行记录

AgileAutonomy:LearningHighSpeedFlightintheWild LearningHighSpeedFlightintheWild 论文链接：<https://www.sc...

《Learning High Speed Flight in the Wild》生成专家轨迹部分--数据记录

暂时主要是记录下数据。simulation.launch运行中数据如下（第1次训练数据： [INFO][1642060400.137616415,3.221000000]:Unityisready! ...

FOCUS : Shedding Light on the High Search Response Time in the Wild (学习记录)

问题：解决在运维中，发现高搜索响应时间之后，使用机器学习算法发现异常 AIops看的第一篇论文学习记录。 对web应用中highsearchresponsetime请求时间长影响因素的分析 然后使用决...

Deep Learning Face Attributes in the Wild

文章要解决的问题 Predictingfaceattributesfromwebimages 方法的主要想法 ItcascadestwoCNNsLNetandANetforfacelocalizati...

深度学习论文（十）---Multiple-Human Parsing in the Wild

TowardsRealWorldHumanParsing:MultipleHumanParsingintheWild JLi，JZhao，YWei，CLang，YLi May232017 <https...

论文笔记：Learning From Documents in the Wild to Improve Document Unwarping

1广告 论文2022年发表在SIGGRAPH顶会上。 预训练出来的模型有139M。 对文档的去扭曲变形效果在我们调研的深度学习模型里面算是最好的。 2摘要 文档图像去扭曲是文档数字化和分析的重要内容。...

【论文笔记】Learning from Synthetic Data for Crowd Counting in the Wild

BacktoPaperNotes BasicIntro Authors:QiWang导师,JunyuGao（主力）,WeiLin,YuanYuanfrom西北工业大学（西安） Keywords:Cro...