Merge pull request #3 from peterdavidfagan/pr-migrate_documentation

Migrate setup documentation to justthedocs

.github/workflows/pages.yaml

@@ -0,0 +1,59 @@
+name: Deploy Documentation
+
+on:
+  push:
+    branches: 
+      - "main"
+    paths:
+      - "docs/**"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+# Allow one concurrent deployment
+concurrency:
+  group: "pages"
+  cancel-in-progress: true
+
+jobs:
+  # Build job
+  build:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: docs
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+      - name: Setup Ruby
+        uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: '3.1' # Not needed with a .ruby-version file
+          bundler-cache: true # runs 'bundle install' and caches installed gems automatically
+          cache-version: 0 # Increment this number if you need to re-download cached gems
+          working-directory: '${{ github.workspace }}/docs'
+      - name: Setup Pages
+        id: pages
+        uses: actions/configure-pages@v3
+      - name: Build with Jekyll
+        run: bundle exec jekyll build --baseurl "${{ steps.pages.outputs.base_path }}"
+        env:
+          JEKYLL_ENV: production
+      - name: Upload artifact
+        uses: actions/upload-pages-artifact@v1
+        with:
+          path: "docs/_site/"
+
+  deploy:
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    runs-on: ubuntu-latest
+    needs: build
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v2

docs/Gemfile

@@ -0,0 +1,7 @@
+source 'https://rubygems.org'
+
+gem "jekyll", "~> 4.3.3" # installed by `gem jekyll`
+# gem "webrick"        # required when using Ruby >= 3 and Jekyll <= 4.2.2
+
+gem "just-the-docs", "0.7.0" # pinned to the current release
+# gem "just-the-docs"        # always download the latest release

docs/_config.yml

@@ -0,0 +1,9 @@
+title: DROID Docs
+description: Developer documentation for recreating the DROID platform setup.
+theme: just-the-docs
+
+url: https://peterdavidfagan.github.io/DROID # this url not being used right now
+color_scheme: r2d2 # TODO: update name to DROID
+
+aux_links:
+  Official GitHub Repository: https://github.com/AlexanderKhazatsky/R2D2

docs/_sass/color_schemes/r2d2.scss

@@ -0,0 +1,7 @@
+@import "./color_schemes/light";
+
+// add custom colours here
+$cardinal-red: #8C1515; // can be updated assumed primary author's university color scheme
+
+$link-color: $cardinal-red;
+$btn-primary-color: $cardinal-red;

docs/contribution-guidelines.md

@@ -0,0 +1,7 @@
+---
+layout: default
+title: Contribution Guidelines
+nav_order: 6
+---
+
+# Contribution Guidelines

docs/dataset-schema.md

@@ -0,0 +1,38 @@
+---
+layout: default
+title: Dataset Schema
+nav_order: 5
+---
+
+# Dataset Schema
+
+| Field | Datatype |
+| --------- | -------- |
+| steps | `Dataset` |
+| steps/action | `Tensor(shape=(6,), dtype=float64)` |
+| steps/action_dict | `FeaturesDict` |
+|steps/action_dict/cartesian_position| `Tensor(shape=(6,), dtype=float64)` |
+|steps/action_dict/cartesian_velocity| `Tensor(shape=(6,), dtype=float64)` |
+|steps/action_dict/gripper_position| `Tensor(shape=(1,), dtype=float64)` |
+|steps/action_dict/gripper_velocity| `Tensor(shape=(1,), dtype=float64)` |
+|steps/action_dict/joint_position| `Tensor(shape=(7,), dtype=float64)` |
+|steps/action_dict/joint_velocity| `Tensor(shape=(7,), dtype=float64)` |
+|steps/discount| `Scalar(shape=(), dtype=float32)` |
+|steps/is_first| `Scalar(shape=(), dtype=bool)` |
+|steps/is_last| `Scalar(shape=(), dtype=bool)` |
+|steps/is_terminal| `Scalar(shape=(), dtype=bool)` |
+|steps/language_embedding| `Tensor(shape=(512,), dtype=float32)` |
+|steps/language_embedding_2| `Tensor(shape=(512,), dtype=float32)` |
+|steps/language_embedding_3| `Tensor(shape=(512,), dtype=float32)` |
+|steps/language_instruction| `Text(shape=(), dtype=string)` |
+|steps/language_instruction_2| `Text(shape=(), dtype=string)` |
+|steps/language_instruction_3| `Text(shape=(), dtype=string)` |
+|steps/observation| `FeaturesDict` |
+|steps/observation/cartesian_position| `Tensor(shape=(6,), dtype=float64)` |
+|steps/observation/exterior_image_1_left| `Image(shape=(180, 320, 3), dtype=uint8)` |
+|steps/observation/exterior_image_2_left| `Image(shape=(180, 320, 3), dtype=uint8)` |
+|steps/observation/gripper_position| `Tensor(shape=(1,), dtype=float64)` |
+|steps/observation/joint_position| `Tensor(shape=(7,), dtype=float64)` |
+|steps/observation/wrist_image_left| `Image(shape=(180, 320, 3), dtype=uint8)` |
+|reward| `Scalar(shape=(), dtype=float32)` |
+

docs/example-workflows/calibrating-cameras.md

@@ -0,0 +1,32 @@
+---
+layout: default
+title: Calibrating Cameras
+parent: Example Workflows
+nav_order: 2
+---
+
+# Prerequisites
+
+This guide assumes you have already setup DROID application software to run on your host machine or through Docker. Proceed with this guide having launched the control server on the NUC and the GUI application on the laptop.
+
+# Calibrating Cameras
+
+The GUI will let you know if any of your camera’s have yet to be calibrated, or if any of your camera’s haven’t been calibrated in a while (this is to make sure you don’t accidentally move them and forget). Since the hand camera is relatively fixed, it is okay to calibrate significantly less frequently than the other cameras, but it is still good practice to calibrate it every now and then.
+
+Useful Information:
+* If calibration is successful, you will be brought back to the calibration hub. If it is unsuccessful, the GUI will inform you as such.
+* During calibration, you will see a visualization of the pose estimation of the charuco board. Stable, green boxes along with stable axes lines are good! When these are not present, a calibration is likely to fail. If things start to significantly jumble during the automated calibration procedure, feel free to press B and try again.
+* Because the board is heavy, the robot may move differently when the board is attached to the gripper.
+* **WARNING:** If A button click is taking a really long time. The camera might have failed. You can confirm this by looking in the terminal for a line that resembles “can’t claim interface…”. The solution is to simply close the GUI, and load it up again. Your previous calibration info will not be lost.
+
+## Mounting Calibration Board
+
+Please follow the instructions in the assembly guide for mounting the calibration board.
+
+## Calibrating a 3rd-Person Camera
+
+<iframe src="https://drive.google.com/file/d/1kcE4YzeJYJLbKanY2N8R66tiCqWjk6ij/preview" width="640" height="480" allow="autoplay"></iframe>
+
+## Calibration a Hand-Mounted Camera
+
+<iframe src="https://drive.google.com/file/d/1Raqb35VDrr4YvSBaola5kSR-QFNKZ32w/preview" width="640" height="480" allow="autoplay"></iframe>

docs/example-workflows/data-collection.md

@@ -0,0 +1,44 @@
+---
+layout: default
+title: Collecting Data
+parent: Example Workflows
+nav_order: 3
+---
+
+# Prerequisites
+
+**Important:** Before proceeding please ensure you have followed the camera calibration guide and calibrated your cameras. 
+
+This guide assumes you have already setup DROID application software to run on your host machine or through Docker. Proceed with this guide having launched the control server on the NUC and the GUI application on the laptop.
+
+# Collecting Data
+
+## Using the GUI
+
+* The GUI will start by prompting you to enter your name.
+	* Note 1: Your trajectories will be associated with the name you type in. So, make sure to be consistent, and type in your full name! The login page will not let you pass until you have done so.
+	* Note 2: Press shift to see the camera feed. Anytime you see a camera feed, confirm that there are exactly 6 images on screen. If there is anything different, halt data collection, unplug and replug the 3 camera wires, and restart the GUI.
+
+* After this, you will be presented with the task configuration page. This is where you enter all the tasks that are currently doable from the scene you have created for your robot. You may select from the predefined tasks using the checkboxes, or enter your own tasks in the text box. For your convenience, use the shift button to toggle between this page and the camera feed. This can be useful for checking what is in view of your cameras.
+	* In the upper left corner are 3 tabs:
+	* Task Ideas: A google doc of task ideas for inspiration split by room type (ex: bedroom, kitchen, etc)
+	* Preferred Tasks: We will be collecting demonstrations for 20 abstract tasks, and 20 specific tasks as a backup. These latter 20 tasks are specific instances of the 20 abstract tasks. We ask that you make sure that around once every couple xdays, you collect some trajectories for as many of these tasks as possible. Clicking this tab will bring you to a page that lists all of the specific tasks and allows you to keep track of which ones you have collected data for.
+	* Franka Website: This will bring you to your Franka’s website, where you can lock / unlock robot joints, as well as activate FCI mode.
+
+* In the lower right corner are another 3 tabs:
+	* Collect Trajectory: This will bring you to the requested task page, where you will be prompted with your task for the next trajectory. You may press A to begin the trajectory, or B to sample a new task if it’s necessary.
+	* Calibrate: This will bring you to the camera calibration page. Click the camera that you would like to calibrate. Note that because we need to turn the cameras on in high resolution, button clicks may take a while here. See the section below for more information on camera calibration.
+	* Practice: This will allow you to collect a trajectory without the data being saved. You can use this tool however you see fit.
+
+* Periodically, the GUI will prompt you with desired scene changes. Proceed with them as instructed. When the scene changes involve altering a camera position, make sure to recalibrate the associated camera!
+
+* Miscellaneous Notes:
+	* Finish trajectories in such a way that the robot can be reset (ex: nothing in gripper, as it will be dropped.
+	* Try to create scenes with as many available tasks as possible.
+	* Although we want you to stick to the requested tasks, use your best judgment.
+	* At any time, hold 'Return' for 5 seconds to reset the robot
+
+# Uploading Data
+
+Instructions for uploading data can be found in [this](https://github.com/AlexanderKhazatsky/R2D2/tree/main/scripts) readme.
+

docs/example-workflows/evaluating-policies.md

@@ -0,0 +1,15 @@
+---
+layout: default
+title: Evaluating Policies
+parent: Example Workflows
+nav_order: 5
+---
+
+# Evaluating Policies
+
+* Run a variant of [this](https://github.com/AlexanderKhazatsky/R2D2/blob/main/scripts/evaluation/evaluate_policy.py) file with the desired properties.
+	* Make sure to update policy_logdir and model_id with the desired values
+	* Any properties that are added to the variant will be loaded from the training folder values.
+* It is recommended to start off by evaluating policies in the “Practice” tab for simplicity.
+* Note: The robot will only move when you are holding down the side button of the controller. However, instead of the actions coming from your handle as during data collection, the actions will come from the policy.
+

docs/example-workflows/example-workflows.md

@@ -0,0 +1,7 @@
+---
+layout: default
+title: Example Workflows
+nav_order: 4
+has_children: true
+permalink: /docs/example-workflows
+---

docs/example-workflows/teleoperation.md

@@ -0,0 +1,21 @@
+---
+layout: default
+title: Teleoperating the Robot
+parent: Example Workflows
+nav_order: 1
+---
+
+# Prerequisites
+
+This guide assumes you have already setup DROID application software to run on your host machine or through Docker. Proceed with this guide having launched the control server on the NUC and the GUI application on the laptop.
+
+# Teleoperating the Robot
+
+* To teleoperate the robot, your oculus controller should be plugged into your laptop, and the permissions prompt should be accepted (done by putting the headset on and clicking “Accept”. Also, the controller has to be in view of the headset cameras. It detects the pose of the handle via infrared stickers on the handle.
+* To control the robot, we will use only the right controller. If you would like to use the left controller for teleoperation instead, change [this](https://github.com/AlexanderKhazatsky/R2D2/blob/5f2f96b5cf9d95dde67fda21a8ab776683aeeae7/r2d2/controllers/oculus_controller.py#L16) parameter.
+* Teleoperation works by applying the changes to the oculus handle’s pose to the robot gripper’s pose. The trigger on the front of the controller is used to control the gripper. Actions are only applied when the trigger on the side of the controller is being held.
+* It is important for intuitive control that the controller’s definition of forward is aligned with the direction of the robot. The controller defines “forward” on the first step where the side button of the controller is held. At any point, you can redefine the forward direction by pressing down on the joystick until you hear a click. At some point, try changing the definition of forward to get a better feel for its purpose.
+* To practice, select `Practice` from the GUI application.
+
+<iframe src="https://drive.google.com/file/d/1Rg10T5rVaK9m_0BYXq2EkVdNgLpZnj3P/preview" width="640" height="480" allow="autoplay"></iframe>
+

docs/example-workflows/training-policies.md

@@ -0,0 +1,13 @@
+---
+layout: default
+title: Training Policies
+parent: Example Workflows
+nav_order: 4
+---
+
+# Training Policies
+
+* Run a variant of [this](https://github.com/AlexanderKhazatsky/R2D2/blob/main/scripts/training/train_policy.py) file with the desired properties.
+	* If you are doing a simple BC from images sanity check task, run [this](https://github.com/AlexanderKhazatsky/R2D2/blob/main/scripts/training/sanity_check/image_obs.py) file
+* If the function of any hyperparameters are unclear, message Sasha to inquire about their utility.
+