benchmark-openCV-JuliaImage/JuliaImage-bench.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Benchmarking JuliaImages.jl\n",
    "### Using:\n",
    "[BenchmarkTools.jl](https://github.com/JuliaCI/BenchmarkTools.jl) | `jl`\n",
    "\n",
    "[timeit](https://docs.python.org/3/library/timeit.html) | `py`\n",
    "### Against :\n",
    "1. OpenCV\n",
    "\n",
    "Version used : **4.1.0**\n",
    "2. scikit-image\n",
    "\n",
    "Version used : **0.16.2**\n",
    "3. PIL using Pillow\n",
    "\n",
    "Version used : **6.1.0**\n",
    "\n",
    "## Machine on which Benchmarks were carried out : \n",
    "64Bit Windows 10 with 16 GB of RAM and i5-7200U@2.5Ghz\n",
    "\n",
    "Sample image from https://testimages.juliaimages.org/\n",
    "\n",
    "<hr>\n",
    "\n",
    "**Note :** Compared mean times by timing only one sample in timeit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Getting the Required Packages\n",
    "using Images, ImageTransformations, FileIO, BenchmarkTools, TestImages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test 1: Loading Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  3.39 MiB\n",
       "  allocs estimate:  322\n",
       "  --------------\n",
       "  minimum time:     76.631 ms (0.00% GC)\n",
       "  median time:      105.579 ms (0.00% GC)\n",
       "  mean time:        110.319 ms (0.41% GC)\n",
       "  maximum time:     209.470 ms (0.00% GC)\n",
       "  --------------\n",
       "  samples:          46\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@benchmark img = load(\"img/mountainstream.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result:\n",
    "\n",
    "| Framework \t| Time Taken (ms)|\n",
    "|-----------\t|--------------\t|\n",
    "| `Images.jl` \t|  110.319    \t|\n",
    "| `OpenCV`   \t|25.222    \t|\n",
    "| `PIL`         |0.248      |\n",
    "    \n",
    "$PIL < OpenCV < Images.jl$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test 2: Saving Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  3.39 MiB\n",
       "  allocs estimate:  305\n",
       "  --------------\n",
       "  minimum time:     257.997 ms (0.00% GC)\n",
       "  median time:      336.976 ms (0.00% GC)\n",
       "  mean time:        316.100 ms (0.10% GC)\n",
       "  maximum time:     368.297 ms (0.00% GC)\n",
       "  --------------\n",
       "  samples:          16\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = load(\"img/mountainstream.png\")\n",
    "@benchmark save(\"copy_julia.png\", img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result:\n",
    "\n",
    "| Framework \t| Time Taken (ms)|\n",
    "|-----------\t|--------------\t|\n",
    "| `Images.jl` \t|  316.1    \t|\n",
    "| `OpenCV`   \t|40.224   \t|\n",
    "| `PIL`         |174.674      |\n",
    "    \n",
    "$OpenCV <  PIL < Images.jl$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test 3: Resizing Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  29.25 MiB\n",
       "  allocs estimate:  6\n",
       "  --------------\n",
       "  minimum time:     324.998 ms (0.00% GC)\n",
       "  median time:      344.478 ms (0.08% GC)\n",
       "  mean time:        376.705 ms (4.31% GC)\n",
       "  maximum time:     583.193 ms (0.00% GC)\n",
       "  --------------\n",
       "  samples:          14\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = load(\"img/mountainstream.png\")\n",
    "@benchmark big_img = imresize(img, ratio=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result:\n",
    "\n",
    "| Framework \t| Time Taken (ms)|\n",
    "|-----------\t|--------------\t|\n",
    "| `Images.jl` \t|376.705    \t|\n",
    "| `OpenCV`   \t|33.105    \t|\n",
    "| `PIL`         |57.582      |\n",
    "    \n",
    "$ OpenCV < PIL < Images.jl$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test 4: Greyscaling Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  384.34 KiB\n",
       "  allocs estimate:  8\n",
       "  --------------\n",
       "  minimum time:     1.790 ms (0.00% GC)\n",
       "  median time:      2.021 ms (0.00% GC)\n",
       "  mean time:        2.198 ms (1.84% GC)\n",
       "  maximum time:     18.771 ms (0.00% GC)\n",
       "  --------------\n",
       "  samples:          2264\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = load(\"img/mountainstream.png\")\n",
    "@benchmark gray_img = Gray.(img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result:\n",
    "\n",
    "| Framework \t| Time Taken (ms)|\n",
    "|-----------\t|--------------\t|\n",
    "| `Images.jl` \t|2.198    \t|\n",
    "| `OpenCV`   \t|0.269    \t|\n",
    "| `PIL`         |0.985     |\n",
    "    \n",
    "$ OpenCV < PIL < Images.jl$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test 5: Applying Gaussian Blur"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  18.70 MiB\n",
       "  allocs estimate:  603\n",
       "  --------------\n",
       "  minimum time:     39.857 ms (0.00% GC)\n",
       "  median time:      42.566 ms (0.00% GC)\n",
       "  mean time:        44.348 ms (4.84% GC)\n",
       "  maximum time:     93.842 ms (0.00% GC)\n",
       "  --------------\n",
       "  samples:          113\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = load(\"img/mountainstream.png\")\n",
    "@benchmark gauss = imfilter(img, Kernel.gaussian(5))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result:\n",
    "\n",
    "| Framework \t| Time Taken (ms)|\n",
    "|-----------\t|--------------\t|\n",
    "| `Images.jl` \t|   44.348\t|\n",
    "| `OpenCV`   \t|4.007 \t|\n",
    "| `PIL`         |33.232     |\n",
    "    \n",
    "$ OpenCV < PIL < Images.jl$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test 6: Generating (Greyscale) Histogram "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  386.06 KiB\n",
       "  allocs estimate:  5\n",
       "  --------------\n",
       "  minimum time:     19.395 ms (0.00% GC)\n",
       "  median time:      20.976 ms (0.00% GC)\n",
       "  mean time:        22.056 ms (0.10% GC)\n",
       "  maximum time:     55.325 ms (0.00% GC)\n",
       "  --------------\n",
       "  samples:          227\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = load(\"img/mountainstream.png\")\n",
    "@benchmark edges, counts  = imhist(img,256)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result:\n",
    "\n",
    "| Framework \t| Time Taken (ms)|\n",
    "|-----------\t|--------------\t|\n",
    "| `Images.jl` \t|22.056   \t|\n",
    "| `OpenCV`   \t|1.29 \t|\n",
    "| `PIL`         |1.773     |\n",
    "    \n",
    "$ OpenCV < PIL < Images.jl$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test 7: Changing colorscale to HSV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  4.50 MiB\n",
       "  allocs estimate:  8\n",
       "  --------------\n",
       "  minimum time:     6.093 ms (0.00% GC)\n",
       "  median time:      6.461 ms (0.00% GC)\n",
       "  mean time:        7.373 ms (7.22% GC)\n",
       "  maximum time:     64.638 ms (0.00% GC)\n",
       "  --------------\n",
       "  samples:          677\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = load(\"img/mountainstream.png\")\n",
    "@benchmark imghsv = HSV.(img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result:\n",
    "\n",
    "| Framework \t| Time Taken (ms)|\n",
    "|-----------\t|--------------\t|\n",
    "| `Images.jl` \t|7.373   \t|\n",
    "| `OpenCV`   \t|3.675\t|\n",
    "| `PIL`         |22.607     |\n",
    "    \n",
    "$ OpenCV < Images.jl < PIL$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test 8: Calculating Integral Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  9.00 MiB\n",
       "  allocs estimate:  7\n",
       "  --------------\n",
       "  minimum time:     4.437 ms (0.00% GC)\n",
       "  median time:      5.785 ms (0.00% GC)\n",
       "  mean time:        7.329 ms (16.98% GC)\n",
       "  maximum time:     73.310 ms (13.81% GC)\n",
       "  --------------\n",
       "  samples:          681\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = load(\"img/mountainstream.png\")\n",
    "@benchmark integral_img = integral_image(img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result:\n",
    "\n",
    "| Framework \t| Time Taken (ms)|\n",
    "|-----------\t|--------------\t|\n",
    "| `Images.jl` \t|7.329   \t|\n",
    "| `OpenCV`   \t|0.226\t|\n",
    "| `scikit-image`         |37.098     |\n",
    "\n",
    "Note: PIL didn't have a method to calculate integral image. Benchmarked against scikit-image instead.\n",
    "\n",
    "    \n",
    "$ OpenCV < Images.jl < scikit-image$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test 9: Rotating Image (90 Degrees)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  1.13 MiB\n",
       "  allocs estimate:  5\n",
       "  --------------\n",
       "  minimum time:     16.047 ms (0.00% GC)\n",
       "  median time:      16.682 ms (0.00% GC)\n",
       "  mean time:        17.877 ms (0.82% GC)\n",
       "  maximum time:     59.864 ms (0.00% GC)\n",
       "  --------------\n",
       "  samples:          280\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = load(\"img/mountainstream.png\")\n",
    "@benchmark rotated = imrotate(img, pi/2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result:\n",
    "\n",
    "| Framework \t| Time Taken (ms)|\n",
    "|-----------\t|--------------\t|\n",
    "| `Images.jl` \t|17.877   \t|\n",
    "| `OpenCV`   \t|4.317 \t|\n",
    "| `PIL`         |2.808     |\n",
    "\n",
    "    \n",
    "$ PIL < OpenCV < Images.jl$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test 10: Corner Detection using Harris Method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  70.88 MiB\n",
       "  allocs estimate:  101873\n",
       "  --------------\n",
       "  minimum time:     75.884 ms (8.44% GC)\n",
       "  median time:      83.419 ms (9.94% GC)\n",
       "  mean time:        86.072 ms (10.41% GC)\n",
       "  maximum time:     136.603 ms (24.07% GC)\n",
       "  --------------\n",
       "  samples:          59\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = load(\"img/mountainstream.png\")\n",
    "@benchmark corners = imcorner(img; method = harris)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result:\n",
    "\n",
    "| Framework \t| Time Taken (ms)|\n",
    "|-----------\t|--------------\t|\n",
    "| `Images.jl` \t|86.072   \t|\n",
    "| `OpenCV`   \t|22.802\t|\n",
    "| `scikit-image`         |112.88     |\n",
    "\n",
    "    \n",
    "$  OpenCV < Images.jl < scikit-image$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 11: Morphological Operation - Erode"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  1.13 MiB\n",
       "  allocs estimate:  3\n",
       "  --------------\n",
       "  minimum time:     405.601 μs (0.00% GC)\n",
       "  median time:      512.500 μs (0.00% GC)\n",
       "  mean time:        706.785 μs (23.49% GC)\n",
       "  maximum time:     71.665 ms (99.14% GC)\n",
       "  --------------\n",
       "  samples:          7020\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = load(\"img/mountainstream.png\")\n",
    "@benchmark imge = erode(img, [5,5]) #over 5x5 1's Kernel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result:\n",
    "\n",
    "| Framework \t| Time Taken (μs)|\n",
    "|-----------\t|--------------\t|\n",
    "| `Images.jl` \t|706.785  \t|\n",
    "| `OpenCV`   \t|1190.893 \t|\n",
    "| `scikit-image`         |40561.481     |\n",
    "\n",
    "    \n",
    "$ Images.jl <  OpenCV < scikit-image$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 12: Morphological Operation - Opening "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  1.13 MiB\n",
       "  allocs estimate:  3\n",
       "  --------------\n",
       "  minimum time:     408.499 μs (0.00% GC)\n",
       "  median time:      469.999 μs (0.00% GC)\n",
       "  mean time:        652.171 μs (22.81% GC)\n",
       "  maximum time:     33.034 ms (0.00% GC)\n",
       "  --------------\n",
       "  samples:          7595\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = load(\"img/mountainstream.png\")\n",
    "@benchmark imgc = closing(img, [5,5]) #over 5x5 1's Kernel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result:\n",
    "\n",
    "| Framework \t| Time Taken (μs)|\n",
    "|-----------\t|--------------\t|\n",
    "| `Images.jl` \t|652.171  \t|\n",
    "| `OpenCV`   \t|1587.633 \t|\n",
    "| `scikit-image`         |68730.478     |\n",
    "\n",
    "    \n",
    "$ Images.jl <  OpenCV < scikit-image$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test 13: Morphological Operation - TopHat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  2.25 MiB\n",
       "  allocs estimate:  5\n",
       "  --------------\n",
       "  minimum time:     960.600 μs (0.00% GC)\n",
       "  median time:      1.263 ms (0.00% GC)\n",
       "  mean time:        1.687 ms (19.92% GC)\n",
       "  maximum time:     28.826 ms (95.24% GC)\n",
       "  --------------\n",
       "  samples:          2937\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = load(\"img/mountainstream.png\")\n",
    "@benchmark imgth = tophat(img, [9,9])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result:\n",
    "\n",
    "| Framework \t| Time Taken (ms)|\n",
    "|-----------\t|--------------\t|\n",
    "| `Images.jl` \t|1.687 \t|\n",
    "| `OpenCV`   \t|1.82 \t|\n",
    "| `scikit-image`         |65.544     |\n",
    "\n",
    "    \n",
    "$ Images.jl <  OpenCV < scikit-image$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test 14: Morphological Operation - BottomHat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  2.25 MiB\n",
       "  allocs estimate:  5\n",
       "  --------------\n",
       "  minimum time:     950.299 μs (0.00% GC)\n",
       "  median time:      1.079 ms (0.00% GC)\n",
       "  mean time:        1.482 ms (20.42% GC)\n",
       "  maximum time:     23.362 ms (92.39% GC)\n",
       "  --------------\n",
       "  samples:          3346\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = load(\"img/mountainstream.png\")\n",
    "@benchmark imgth = bothat(img, [9,9])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result:\n",
    "\n",
    "| Framework \t| Time Taken (ms)|\n",
    "|-----------\t|--------------\t|\n",
    "| `Images.jl` \t|1.482\t|\n",
    "| `OpenCV`   \t|1.913 \t|\n",
    "| `scikit-image`         |81.389     |\n",
    "\n",
    "    \n",
    "$ Images.jl <  OpenCV < scikit-image$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test 15: Segmentation- Connected Components"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: \n",
       "  memory estimate:  10.85 MiB\n",
       "  allocs estimate:  70\n",
       "  --------------\n",
       "  minimum time:     9.715 ms (0.00% GC)\n",
       "  median time:      10.430 ms (0.00% GC)\n",
       "  mean time:        12.003 ms (9.84% GC)\n",
       "  maximum time:     36.741 ms (58.12% GC)\n",
       "  --------------\n",
       "  samples:          416\n",
       "  evals/sample:     1"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = load(\"img/mountainstream.png\")\n",
    "@benchmark markers = label_components(img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## End of report."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please accept now sir."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 1.3.0",
   "language": "julia",
   "name": "julia-1.3"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.3.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}