Skip to content

Commit

Permalink
Add guide to engines (deepjavalibrary#660)
Browse files Browse the repository at this point in the history
Explains what an engine is, what engines we support, and default engines.

This includes a few miscellaneous doc fixes.

Change-Id: I24ca0260ecdec8d7ab3bddfa3fad9b3beaa31c4f
  • Loading branch information
zachgk authored Feb 18, 2021
1 parent f59197d commit eb0ae86
Show file tree
Hide file tree
Showing 5 changed files with 37 additions and 2 deletions.
1 change: 1 addition & 0 deletions api/src/main/java/ai/djl/engine/Engine.java
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
* contain methods to detect information about the usable machine hardware and to create a new
* {@link NDManager} and {@link Model}.
*
* @see <a href="http://docs.djl.ai/docs/engine.html">Engine Guide</a>
* @see EngineProvider
*/
public abstract class Engine {
Expand Down
31 changes: 31 additions & 0 deletions docs/engine.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Engines

The [Engine](https://javadoc.io/doc/ai.djl/api/latest/ai/djl/engine/Engine.html) is one of the most fundamental classes in DJL. Most of the core functionality in DJL including NDArrays, NDManager, and Models are only interfaces. They form a tree of interfaces with the root as the Engine class.

The implementations of these interfaces are provided by the various engines using a Java service loader. This means that DJL is able to take advantage of much of the performance optimization and hardware support work which has gone on in these engines. As they are updated, DJL can take advantage of the updates. And, DJL is able to freely switch between engines to keep up with performance advancements.

In addition, the engines are very useful for production systems. Models trained in those engines in python can often be imported and run in Java through DJL. This makes it much easier to integrate into existing Java servers or any of the powerful Java production ecosystem. Because they are run with the same engine they are trained in, there shouldn't be any loss in performance or accuracy either.

For training in DJL, the choice of engine is less important. Any engine that fully implements the DJL specification would have similar results and the performance does not differ too much. As you are encouraged to write engine agnostic code, you can even switch between the engines as easily as switching dependencies. In general, you should use the recommended engine (below) unless you have a good reason to use a different one.

## Supported Engines

Currently, the engines that are supported by DJL are:

- [MXNet](../mxnet/README.md) - **Recommended Engine** with full support
- [PyTorch](../pytorch/README.md) - full support
- [TensorFlow](../tensorflow/README.md) - supports inference and some NDArray operations
- [ONNX Runtime](../onnxruntime/README.md) - supports basic inference
- [DLR](../dlr/README.md) - supports basic inference
- [TFLite](../tflite/README.md) - supports basic inference
- [PaddlePaddle](../paddlepaddle/README.md) - supports basic inference

## Setup

In order to choose an engine, it must be added into the Java classpath. Usually this means additional Maven or Gradle dependencies. Many engines require multiple dependencies be added, so look at the engine README for your desired engine to learn what dependencies are necessary.

It is also possible to load multiple engines simultaneously. When DJL starts up, it chooses a default engine from the available engines. Most of the API that requires an engine such as `NDManager.newBaseManager()` or `Model.newInstance()` will internally use the default engine. As of DJL 10.0, it chooses the default engine based on a ranking of how much we recommend the engine, but before that it was chosen at random. For those calls, you can also choose the engine manually by getting an engine with `Engine.getEngine(engineName)` and then calling the equivalent method on the Engine class such as `engine.newBaseManager()`.

Some calls will also take advantage of all possible engines. For example, model loading will try all of the engines available to see if any work for the model you are trying to load.

You can also choose the default engine manually. Each engine has a name which can be found in the engine's javadoc or README. You can set the default engine by setting either the "DJL_DEFAULT_ENGINE" environment variable or the "ai.djl.default_engine" Java property. Either one should be set to the name of the desired default engine.
1 change: 1 addition & 0 deletions docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ nav:
- 'jupyter/rank_classification_using_BERT_on_Amazon_Review.ipynb'
- 'jupyter/transfer_learning_on_cifar10.ipynb'
- Guides:
- 'docs/engine.md'
- Models:
- 'docs/load_model.md'
- 'docs/model-zoo.md'
Expand Down
4 changes: 3 additions & 1 deletion docs/quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ Deep Java Library (DJL) is designed to be easy to get started with and simple to
The easiest way to learn DJL is to read the [beginner tutorial](../jupyter/tutorial/README.md) or
our [examples](../examples/README.md).

<iframe width="560" height="315" src="https://www.youtube.com/embed/EyB4gBECaNU" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
You can also view our 1.5 hour long (in 8 x ~10 minute segments) DJL 101 tutorial video series:

<iframe width="560" height="315" src="https://www.youtube.com/embed/?list=PLC1JzXeHJitDmZBHupMGZE2zHzU8fEvfD&listType=playlist" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

## Prerequisites

Expand Down
2 changes: 1 addition & 1 deletion website/demo.html
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ <h5 class="header col s12 light">Experience Java to solve your DL problem in a s
</div>
<div class="row center">
<a href="https://github.com/aws-samples/djl-demo"
class="btn-large waves-effect waves-light light-blue accent-2">Checkout our repository</a>
class="btn-large waves-effect waves-light light-blue accent-2">Checkout our Demos</a>
<a href="https://demodocs.djl.ai/"
class="btn-large waves-effect waves-light light-blue accent-2">Documentation</a>
</div>
Expand Down

0 comments on commit eb0ae86

Please sign in to comment.