jupyter/load_mxnet_model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Load MXNet model\n",
    "\n",
    "In this tutorial, you learn how to load an existing MXNet model and use it to run a prediction task.\n",
    "\n",
    "\n",
    "## Preparation\n",
    "\n",
    "This tutorial requires the installation of Java Kernel. For more information on installing the Java Kernel, see the [README](https://github.com/awslabs/djl/blob/master/jupyter/README.md) to install Java Kernel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n",
    "\n",
    "%maven ai.djl:api:0.3.0-SNAPSHOT\n",
    "%maven ai.djl:repository:0.3.0-SNAPSHOT\n",
    "%maven ai.djl:model-zoo:0.3.0-SNAPSHOT\n",
    "%maven ai.djl.mxnet:mxnet-engine:0.3.0-SNAPSHOT\n",
    "%maven ai.djl.mxnet:mxnet-model-zoo:0.3.0-SNAPSHOT\n",
    "%maven org.slf4j:slf4j-api:1.7.26\n",
    "%maven org.slf4j:slf4j-simple:1.7.26\n",
    "%maven net.java.dev.jna:jna:5.3.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Include MXNet engine dependency\n",
    "\n",
    "This tutorial uses MXNet engine as its backend. MXNet has different [build flavor](https://mxnet.apache.org/get_started?version=v1.5.1&platform=linux&language=python&environ=pip&processor=cpu) and it is platform specific.\n",
    "Please read [here](https://github.com/awslabs/djl/blob/master/examples/README.md#engine-selection) for how to select MXNet engine flavor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "String osName = System.getProperty(\"os.name\");\n",
    "String classifier = osName.startsWith(\"Mac\") ? \"osx-x86_64\" : osName.startsWith(\"Win\") ? \"win-x86_64\" : \"linux-x86_64\";\n",
    "\n",
    "%maven ai.djl.mxnet:mxnet-native-mkl:jar:${classifier}:1.6.0-c-SNAPSHOT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import java.awt.image.*;\n",
    "import java.nio.file.*;\n",
    "import java.util.*;\n",
    "import java.util.stream.*;\n",
    "import ai.djl.*;\n",
    "import ai.djl.inference.*;\n",
    "import ai.djl.ndarray.*;\n",
    "import ai.djl.ndarray.index.*;\n",
    "import ai.djl.modality.*;\n",
    "import ai.djl.modality.cv.*;\n",
    "import ai.djl.modality.cv.util.*;\n",
    "import ai.djl.modality.cv.transform.*;\n",
    "import ai.djl.mxnet.zoo.*;\n",
    "import ai.djl.translate.*;\n",
    "import ai.djl.training.util.*;\n",
    "import ai.djl.util.*;\n",
    "import ai.djl.basicmodelzoo.cv.classification.*;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: Prepare your MXNet model\n",
    "\n",
    "This tutorial assumes that you have a MXNet model trained using Python. A MXNet symbolic model usually contains the following files:\n",
    "* Symbol file: {MODEL_NAME}-symbol.json - a json file that contains network information about the model\n",
    "* Parameters file: {MODEL_NAME}-{EPOCH}.params - a binary file that stores the parameter weight and bias\n",
    "* Synset file: synset.txt - an optional text file that stores classification classes labels\n",
    "\n",
    "This tutorial uses a pre-trained MXNet `resnet18_v1` model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use [DownloadUtils.java] for downloading files from internet."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load DownloadUtils.java"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DownloadUtils.download(\"https://mlrepo.djl.ai/model/cv/image_classification/ai/djl/mxnet/resnet/0.0.1/resnet18_v1-symbol.json\", \"build/resnet/resnet18_v1-symbol.json\", new ProgressBar());\n",
    "DownloadUtils.download(\"https://mlrepo.djl.ai/model/cv/image_classification/ai/djl/mxnet/resnet/0.0.1/resnet18_v1-0000.params.gz\", \"build/resnet/resnet18_v1-0000.params\", new ProgressBar());\n",
    "DownloadUtils.download(\"https://mlrepo.djl.ai/model/cv/image_classification/ai/djl/mxnet/synset.txt\", \"build/resnet/synset.txt\", new ProgressBar());\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Load your model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Path modelDir = Paths.get(\"build/resnet\");\n",
    "Model model = Model.newInstance();\n",
    "model.load(modelDir, \"resnet18_v1\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Create a `Translator`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Pipeline pipeline = new Pipeline();\n",
    "pipeline.add(new CenterCrop()).add(new Resize(224, 224)).add(new ToTensor());\n",
    "Translator<BufferedImage, Classifications> translator = new ImageClassificationTranslator.Builder()\n",
    "                .setPipeline(pipeline)\n",
    "                .setSynsetArtifactName(\"synset.txt\")\n",
    "                .build();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Load image for classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "var img = BufferedImageUtils.fromUrl(\"https://djl-ai.s3.amazonaws.com/resources/images/kitten.jpg\");\n",
    "img"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Run inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Predictor<BufferedImage, Classifications> predictor = model.newPredictor(translator);\n",
    "Classifications classifications = predictor.predict(img);\n",
    "\n",
    "classifications"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "Now, you can load any MXNet symbolic model and run inference."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Java",
   "language": "java",
   "name": "java"
  },
  "language_info": {
   "codemirror_mode": "java",
   "file_extension": ".jshell",
   "mimetype": "text/x-java-source",
   "name": "Java",
   "pygments_lexer": "java",
   "version": "12.0.2+10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}