diff --git a/dev_requirements.txt b/dev_requirements.txt index da83d52c..11ef90be 100644 --- a/dev_requirements.txt +++ b/dev_requirements.txt @@ -22,6 +22,7 @@ docutils<0.18 slidingwindow opencv-python sphinx_rtd_theme +furo sphinx_markdown_tables tqdm twine diff --git a/docs/conf.py b/docs/conf.py index 3d08d82e..7e453423 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -4,6 +4,7 @@ import sys from typing import Any import urllib.request +import pydata_sphinx_theme import recommonmark from recommonmark.parser import CommonMarkParser @@ -59,11 +60,11 @@ 'sphinx.ext.napoleon', 'sphinx.ext.todo', 'sphinx.ext.viewcode', - 'sphinx_markdown_tables', -] + 'sphinx_markdown_tables'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] + # The master toctree document. master_doc = 'index' @@ -78,14 +79,8 @@ todo_include_todos = False # -- Options for HTML output ------------------------------------------- - on_rtd = os.environ.get('READTHEDOCS', None) == 'True' -if not on_rtd: # only import and set the theme if we're building docs locally - import sphinx_rtd_theme - - html_theme = 'sphinx_rtd_theme' - html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] + ["../.."] - +html_theme = 'furo' html_static_path = [] # -- Options for HTMLHelp output --------------------------------------- diff --git a/docs/deepforestr.md b/docs/deepforestr.md index 405a744c..89a85129 100644 --- a/docs/deepforestr.md +++ b/docs/deepforestr.md @@ -1,7 +1,194 @@ # Using DeepForest from R -The file can't be edited on GitHub because it is created in runtime. +An R wrapper for DeepForest is available in the [deepforestr package](https://github.com/weecology/deepforestr). +Commands are very similar with some minor differences due to how the wrapping process +using [reticulate](https://rstudio.github.io/reticulate/) works. -Look at the python `conf`_. module +## Installation -.. _conf: https://github.com/weecology/DeepForest/blob/main/docs/conf.py \ No newline at end of file +`deepforestr` is an R wrapper for the Python package, [DeepForest](https://deepforest.readthedocs.io/en/latest/). +This means that *Python* and the `DeepForest` Python package need to be installed first. + +### Basic Installation + +If you just want to use DeepForest from within R run the following commands in R. +This will create a local Python installation that will only be used by R and install the needed Python package for you. +If installing on Windows you need to [install RTools](https://cran.r-project.org/bin/windows/Rtools/) before installing the R package. + +```R +install.packages('reticulate') # Install R package for interacting with Python +reticulate::install_miniconda() # Install Python +reticulate::py_install(c('gdal', 'rasterio', 'fiona')) # Install spatial dependencies via conda +reticulate::conda_remove('r-reticulate', packages = c('mkl')) # Remove package that causes conflicts on Windows (and maybe macOS) +reticulate::py_install('DeepForest', pip=TRUE) # Install the Python retriever package +devtools::install_github('weecology/deepforestr') # Install the R package for running the retriever +install.packages('raster') # For visualizing output for rasters +``` + +**After running these commands restart R.** + +### Advanced Installation for Python Users + +If you are using Python for other tasks you can use `deepforestr` with your existing Python installation +(though the [basic installation](#basic-installation) above will still work by creating a separate miniconda install and Python environment). + +#### Install the `DeepForest` Python package + +Install the `DeepForest` Python package into your prefered Python environment +using `pip`: + +```bash +pip install DeepForest +``` + +#### Select the Python environment to use in R + +`deepforestr` will try to find Python environments with `DeepForest` +(see the `reticulate` documentation on [order of discovery](https://rstudio.github.io/reticulate/articles/versions.html#order-of-discovery-1) for more details) installed. +Alternatively you can select a Python environment to use when working with `deepforestr` (and other packages using `reticulate`). + +The most robust way to do this is to set the `RETICULATE_PYTHON` environment +variable to point to the preferred Python executable: + +```R +Sys.setenv(RETICULATE_PYTHON = "/path/to/python") +``` + +This command can be run interactively or placed in `.Renviron` in your home directory. + +Alternatively you can select the Python environment through the `reticulate` package for either `conda`: + +```R +library(reticulate) +use_conda('name_of_conda_environment') +``` + +or `virtualenv`: + +```R +library(reticulate) +use_virtualenv("path_to_virtualenv_environment") +``` + +You can check to see which Python environment is being used with: + +```R +py_config() +``` + +#### Install the `deepforestr` R package + +```R +remotes::install_github("weecology/deepforestr") # development version from GitHub +``` + +## Getting Started + +### Load the current release model + +```R +library(deepforestr) + +model = df_model() +model$use_release() +``` + +### Predict a single image + +#### Return the bounding boxes in a data frame + +```R +image_path = get_data("OSBS_029.png") # Gets a path to an example image +bounding_boxes = model$predict_image(path=image_path, return_plot=FALSE) +head(bounding_boxes) +``` + +#### Return an image for visualization + +```R +image_path = get_data("OSBS_029.png") # Gets a path to an example image +predicted_image = model$predict_image(path=image_path, return_plot=TRUE) +plot(raster::as.raster(predicted_image[,,3:1]/255)) +``` + +### Predict a tile + +#### Return the bounding boxes in a data frame + +```R +raster_path = get_data("OSBS_029.tif") # Gets a path to an example raster tile +bounding_boxes = model$predict_tile(raster_path, return_plot=FALSE) +head(bounding_boxes) +``` + +#### Return an image for visualization + +```R +raster_path = get_data("OSBS_029.tif") # Gets a path to an example raster tile +predicted_raster = model$predict_tile(raster_path, return_plot=TRUE, patch_size=300L, patch_overlap=0.25) +plot(raster::as.raster(predicted_raster[,,3:1]/255)) +``` + +Note at `patch_size` is an integer value in Python and therefore needs to have the `L` at the end of the number in R to make it an integer. + +### Predict a set of annotations + +```R +csv_file = get_data("testfile_deepforest.csv") +root_dir = get_data(".") +boxes = model$predict_file(csv_file=csv_file, root_dir = root_dir, savedir=".") +``` + +### Training + +#### Set the training configuration + +```R +annotations_file = get_data("testfile_deepforest.csv") + +model$config$epochs = 1 +model$config["save-snapshot"] = FALSE +model$config$train$csv_file = annotations_file +model$config$train$root_dir = get_data(".") +``` + +Optionally turn on `fast_dev_run` for testing and debugging: + +```R +model$config$train$fast_dev_run = TRUE +``` + +#### Train the model + +```R +model$create_trainer() +model$trainer$fit(model) +``` + +### Evaluation + +```R +csv_file = get_data("OSBS_029.csv") +root_dir = get_data(".") +results = model$evaluate(csv_file, root_dir, iou_threshold = 0.4) +``` + +### Saving & Loading Models + +#### Saving a model after training + +```R +model$trainer$save_checkpoint("checkpoint.pl") +``` + +#### Loading a saved model + +```R +new_model = df_model() +after = new_model$load_from_checkpoint("checkpoint.pl") +pred_after_reload = after$predict_image(path = img_path) +``` + +*Note that when reloading models, you should carefully inspect the model parameters, such as the score_thresh and nms_thresh. +These parameters are updated during model creation and the config file is not read when loading from checkpoint! +It is best to be direct to specify after loading checkpoint.* \ No newline at end of file diff --git a/docs/index.rst b/docs/index.rst index 13ca2cf8..79379bef 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,7 +1,34 @@ -Welcome to DeepForest documentation! +Welcome to DeepForest! ============================================== -DeepForest is a python package for predicting individual tree crowns from RGB imagery. +DeepForest is a python package for airborne object detection and classification. + +In DeepForest you can use pretrained models for tree crown prediction + +.. image:: ../www/OSBS_sample.png + +For bird detection + +.. image:: ../www/Bird_panel.jpg + +or train your own object detection and classification model. + +*Why DeepForest?* + +Observing the abundance and distribution of individual organisms is one of the foundations of ecology. Connecting broad-scale changes in organismal ecology, such as those associated with climate change, shifts in land use, +and invasive species requires fine-grained data on individuals at wide spatial and temporal extents. To capture these data, ecologists are turning to airborne data collection from uncrewed aerial vehicles, piloted aircraft, +and earth-facing satellites. Computer vision, a type of image-based artificial intelligence, has become a reliable tool for converting images into ecological information by detecting and classifying ecological objects within airborne imagery. +There have been many studies demonstrating that with sufficient labeling, computer vision can yield near-human level performance for ecological analysis. +However, almost all of these studies rely on isolated computer vision models that require extensive technical expertise and human data labeling. +In addition, the speed of innovation in computer vision makes it difficult for even experts to keep up with new innovations. +To address these challenges, the next phase for ecological computer vision needs to reduce the technical barriers and move towards general models that can be applied across space, time and taxa. + +The goal of DeepForest is to be *simple*, *customizable*, and *modular*. DeepForest tries to create simple functions, like 'predict_tile', and keep unneeded details away from average user. The truth is that most machine learning projects fail not because of fancy models, but because of data and project organization. DeepForest tries to create simple defaults, use existing labeling tools and interfaces, and tries to be minimally impactful in learning new code and API. + +*Feedback? How are you using DeepForest?* + +The most helpful thing you can do is leave feedback on DeepForest git `repo https://github.com/weecology/DeepForest/issues`_. No feature or issue, or positive affirmation is too small. Please do it now! + Source code is available here: (https://github.com/weecology/DeepForest.git). .. toctree:: diff --git a/docs/landing.md b/docs/landing.md index f8e82979..680dc8fd 100644 --- a/docs/landing.md +++ b/docs/landing.md @@ -1,22 +1,20 @@ # What is DeepForest? -DeepForest is a python package for training and predicting individual tree crowns from airborne RGB imagery. DeepForest comes with a prebuilt model trained on data from the National Ecological Observatory Network. Users can extend this model by annotating and training custom models. +DeepForest is a python package for training and predicting ecological objects in airborne imagery. DeepForest currently comes with a tree crown object detection model and a bird detection model. Both are single class modules that can be extended to species classification based on new data. Users can extend these models by annotating and training custom models. ![](../www/image.png) -# What's new in DeepForest 1.0.0? - -The original project was written in tensorflow based on keras-retinanet. When tensorflow updated to 2.0 there were breaking changes and the authors of keras-retinanet decided to not attempt to recover the project. -The rapid development of open machine learning community resources means that tensorflow 1.14.0, which is required for deepforest, will rapidly become out of date. Fear not, starting in 1.0.0, the model now depends on the pytorch and torchvision and will be stable for the forseeable future. - ## How does deepforest work? DeepForest uses deep learning object detection networks to predict bounding boxes corresponding to individual trees in RGB imagery. -DeepForest is built on the retinanet model from the [torchvision package](http://pytorch.org/vision/stable/index.html) and designed to make training models for tree detection simpler. +DeepForest is built on the object detection module from the [torchvision package](http://pytorch.org/vision/stable/index.html) and designed to make training models for tree detection simpler. -For more about the motivation behind DeepForest, see a recent talk I gave at the Florida Musuem on Natural History +For more about the motivation behind DeepForest, see some recent talks I've done on computer vision for ecology and practical applications to machine learning in environmental monitoring. + + + ## License Free software: [MIT license](https://github.com/weecology/DeepForest/blob/master/LICENSE) diff --git a/environment.yml b/environment.yml index 75ebcaf4..049dc072 100644 --- a/environment.yml +++ b/environment.yml @@ -9,6 +9,7 @@ dependencies: - pyyaml>=5.1.0 - pytest - pytest-profiling + - furo - numpydoc - geopandas - h5py