Skip to content

Latest commit

 

History

History
167 lines (150 loc) · 6.42 KB

README.md

File metadata and controls

167 lines (150 loc) · 6.42 KB
Raw

Racecar

Here is a set up guide for onboard development using Jetson Orin Nano. Should also work for most of other jetson boards, but no guarentees.

[Setting up Git]

  • Configure local client

    git config --global user.name "your_github_username"
git config --global user.email "your_github_email"
git config -l

  • Then clone Racecar repo, put in your personal access token in password

    git clone https://github.com/armlabstanford/Racecar.git
> Cloning into `YOUR-REPOSITORY`...
Username: <type your username>
Password: <type your password or personal access token (GitHub)

  • Go to repo and save your credential

    git pull -v
git config --global credential.helper cache

[Setting up ssh keyless remote access]

  • See if you already have a ssh key generated: 
    ls -al ~/.ssh/id_*.pub
  • If not, generate a key pair: 
    ssh-keygen -t rsa -b 4096 -C ""
  • Throw your id_rsa.pub content into racecar home/.ssh/authorized_keys

[Setting up ROS Noetic]

  • Refer to this ROS official guide
  • Also need ddynamic reconfig 
    sudo apt install ros-noetic-ddynamic-reconfigure
  • for realsense-ros, add 
    1. find_package( OpenCV REQUIRED )
2. include_directories(
include
${realsense2_INCLUDE_DIR}
${catkin_INCLUDE_DIRS}
${OpenCV_INCLUDE_DIRS}
)
3. target_link_libraries(${PROJECT_NAME}
${realsense2_LIBRARY}
${catkin_LIBRARIES}
${CMAKE_THREAD_LIBS_INIT}
${OpenCV_LIBRARIES}
)
  • You might need to add this line if catkin doesn't find the realsense package 
    set(realsense2_DIR /usr/lib/x86_64-linux-gnu/cmake/realsense2)

[Setting up telemetry]

  • install pip3 and jtop 
    sudo apt install python3-pip ros-noetic-rosbridge-server
sudo pip3 install -U jetson-top

[Setting up mamba env]

  • Install mamba 
    cd ~/Downloads
wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-pypy3-Linux-aarch64.sh .
bash Mambaforge-pypy3-Linux-aarch64.sh
  • Create environment and install packages 
    mamba create -n racecar python=3.9
mamba activate racecar
mamba install scipy
pip install pythoncrc numpy evdev pyOpenSSL twisted autobahn service-identity tornado pymongo empy catkin_pkg pyyaml rospkg Pillow defusedxml pycryptodomex gnupg tqdm
pip install git+https://github.com/LiamBindle/PyVESC
pip uninstall PyCRC
pip install pythoncrc
  • Don't forget to go to pyvesc to comment the get firmware line and unindent the next line.
  • Also need to add your user to the dialout group 
    sudo usermod -a -G dialout $USER

[Setting up OpenCV]

  • Download opencv-3.16 opencv-contrib-3.16 from github
  • Create a folder named opencv/ in home directory, unzip both to folder
  • Install necessary packages 
    sudo apt install build-essential cmake git pkg-config libgtk-3-dev \
      libavcodec-dev libavformat-dev libswscale-dev libv4l-dev \
      libxvidcore-dev libx264-dev libjpeg-dev libpng-dev libtiff-dev \
      gfortran openexr libatlas-base-dev python3-dev python3-numpy \
      libtbb2 libtbb-dev libdc1394-22-dev
  • Go to opencv/ execute commmands below: 
    mkdir -p build && cd build
cmake -DWITH_CUDA=ON -DENABLE_FAST_MATH=1 -DCUDA_FAST_MATH=1 -DWITH_CUBLAS=1 -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib-3.4.16/modules ../opencv-3.4.16/
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local  -D INSTALL_C_EXAMPLES=ON  -D INSTALL_PYTHON_EXAMPLES=ON  -D OPENCV_GENERATE_PKGCONFIG=ON  -D OPENCV_EXTRA_MODULES_PATH=../opencv_contrib-3.4.16/modules -D BUILD_EXAMPLES=ON ../opencv-3.4.16
make -j6
  • Now go to the cmakelist.txt of racecar/ change OpenCV_DIR to your build path. E.g: 
    set(OpenCV_DIR /home/racecar/Documents/opencv/build)

[Setting up librealsense]

  • Intel has no official release for arm64 cpu, so we have to build it ourself
  • We are referring to this guide to build the librealsense ourself. 
    sudo apt-get update && sudo apt-get -y upgrade
sudo apt-get install -y git libssl-dev libusb-1.0-0-dev pkg-config libgtk-3-dev
cd ~/Documents
git clone https://github.com/IntelRealSense/librealsense.git
cd ./librealsense
# Remove all realsense cameras before run
./scripts/setup_udev_rules.sh
# Start build
mkdir build && cd build
cmake ../ -DBUILD_PYTHON_BINDINGS:bool=true
sudo make uninstall && sudo make clean && sudo make -j6 && sudo make install

[TODOs]

  • Add launch script
  • Add telemetry publishers
  • Add additional driving camera
  • Add current sensing

[Setting up video stream]

  • On source pc (jetson) You may replace host= with whatever target ip you want to send to

    gst-launch-1.0 v4l2src device=/dev/video4 ! 'video/x-raw,width=960,height=540,framerate=60/1' ! videoconvert ! 'video/x-raw,format=I420' ! x264enc speed-preset="ultrafast" tune=zerolatency option-string="sps-id=0" ! rtph264pay ! udpsink host=KWA20.local port=5003 sync=false -e

    gst-launch-1.0 v4l2src device=/dev/video0 ! 'image/jpeg,width=1280,height=720,framerate=30/1' ! jpegdec ! videoconvert ! 'video/x-raw,format=I420' ! x264enc speed-preset="ultrafast" tune="zerolatency" option-string="sps-id=0" ! rtph264pay ! udpsink host=Kens-MacBook-Pro-2.local port=5003 sync=false -e

    gst-launch-1.0 v4l2src device=/dev/video0 ! 'image/jpeg,width=1280,height=720,framerate=30/1' ! jpegdec ! videoconvert ! 'video/x-raw,format=I420' ! x264enc speed-preset="ultrafast" tune="zerolatency" option-string="sps-id=0" ! rtph264pay ! udpsink host=KWA20.local port=5003 sync=false -e

  • Wait 30s, and go to target pc (your laptop)

    gst-launch-1.0 -v udpsrc port=5003 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink

  • performance:

    • rosbridge, uncompressed: 600ms
    • rosbridge, compressed: 160ms
    • gst: 70ms (only 1 viewer restriction applies, although you can relay it to ros)
    • local 30fps: 33ms

  • useful tools:

ffplay /dev/video0
v4l2-ctl --device=/dev/video0 --all

On macos the gst-launch is at: /Library/Frameworks/GStreamer.framework/Commands/gst-launch-1.0