Road Damage Detection YOLOV4 tflite

1. Setting Up Environment
   • Using Conda
   • Using Pip
2. Convert YOLOv4 to TensorFlow
   • Run Object Detection
3. Convert YOLOv4 to tflite
   • Run Objection Detection using TFLite Model
4. FPS Comparison

Setting Up Environment

Using Conda

CPU

# CPU
conda env create -f conda-cpu.yml


# activate environment on Windows or Linux
conda activate tf-cpu

# activate environment on Mac
source activate tf-cpu

GPU

# GPU
conda env create -f conda-gpu.yml

# activate environment on Windows or Linux
conda activate tf-gpu

# activate environment on Mac
source activate tf-gpu

Using Pip

# CPU
pip install -r requirements.txt

# GPU
pip install -r requirements-gpu.txt

Note: If installing GPU version with Pip, you need to install CUDA and cuDNN in your system. You can find the tutorial for Windows here.

Download Weights File

Download yolov4.weights file 245 MB: yolov4.weights (Google-drive mirror yolov4.weights )

If using tiny version, download yolov4-tiny.weights file instead. I am using original yolov4 because its more accurate (although slower).

Convert YOLOv4 to TensorFlow

# Convert darknet weights to tensorflow
## yolov4
python save_model.py --weights ./data/yolov4.weights --output ./checkpoints/yolov4-416 --input_size 416 --model yolov4 

## yolov4-tiny
python save_model.py --weights ./data/yolov4-tiny.weights --output ./checkpoints/yolov4-tiny-416 --input_size 416 --model yolov4 --tiny

Run Object Detection

# Run yolov4 on image
python detect.py --weights ./checkpoints/yolov4-416 --size 416 --model yolov4 --image ./data/kite.jpg

python detect.py --weights ./checkpoints/yolov4-tiny-416 --size 416 --model yolov4 --image ./data/kite.jpg --tiny

# Run on multiple images
python detect.py --weights ./checkpoints/yolov4-tiny-416 --size 416 --model yolov4 --images "./data/kite.jpg, ./data/girl.jpg"

# Run yolov4 on video
python detect_video.py --weights ./checkpoints/yolov4-416 --size 416 --model yolov4 --video ./data/japan.mp4 --output ./detections/video_output.avi

# Run yolov4 on webcam
python detect_video.py --weights ./checkpoints/yolov4-416 --size 416 --model yolov4 --video 0 --output ./detections/webcam_output.avi

Output

Convert to tflite

# Save tf model for tflite converting
python save_model.py --weights ./data/yolov4.weights --output ./checkpoints/yolov4-416 --input_size 416 --model yolov4 --framework tflite

# yolov4
python convert_tflite.py --weights ./checkpoints/yolov4-416 --output ./checkpoints/yolov4-416.tflite

# yolov4 quantize float16
python convert_tflite.py --weights ./checkpoints/yolov4-416 --output ./checkpoints/yolov4-416-fp16.tflite --quantize_mode float16

# yolov4 quantize int8
python convert_tflite.py --weights ./checkpoints/yolov4-416 --output ./checkpoints/yolov4-416-int8.tflite --quantize_mode int8

Run Objection Detection using TFLite Model

# Run demo tflite model
python detect.py --weights ./checkpoints/yolov4-416.tflite --size 416 --model yolov4 --image ./data/kite.jpg --framework tflite

# Run demo tflite on videos
python detect_video.py --weights ./checkpoints/yolov4-416.tflite --size 416 --model yolov4 --video ./data/japan.mp4 --output ./detections/video_output.avi --framework tflite

Yolov4 and Yolov4-tiny int8 quantization have some issues. I will try to fix that. You can try Yolov3 and Yolov3-tiny int8 quantization

Convert to TensorRT

Similar to above method, you can convert darknet yolov3 or yolov4 models to tensorflow and then to TensorRT.

# yolov4
python save_model.py --weights ./data/yolov4.weights --output ./checkpoints/yolov4.tf --input_size 416 --model yolov4
python convert_trt.py --weights ./checkpoints/yolov4.tf --quantize_mode float16 --output ./checkpoints/yolov4-trt-fp16-416

FPS Comparison

We use japan.mp4 for all the experiments and report following results.

OpenCV GPU: 9.96 FPS

Darknet GPU: 13.5 FPS

TensorFlow GPU: 11.5 FPS

TFLite CPU: 2 FPS

TFLite GPU (On Android): To be tested

Evaluate on COCO 2017 Dataset

# run script in /script/get_coco_dataset_2017.sh to download COCO 2017 Dataset
# preprocess coco dataset
cd data
mkdir dataset
cd ..
cd scripts
python coco_convert.py --input ./coco/annotations/instances_val2017.json --output val2017.pkl
python coco_annotation.py --coco_path ./coco 
cd ..

# evaluate yolov4 model
python evaluate.py --weights ./data/yolov4.weights
cd mAP/extra
python remove_space.py
cd ..
python main.py --output results_yolov4_tf

mAP50 on COCO 2017 Dataset

Detection	512x512	416x416	320x320
YoloV4	61.96	57.33

Traning your own model

# Prepare your dataset
# If you want to train from scratch:
In config.py set FISRT_STAGE_EPOCHS=0 
# Run script:
python train.py

# Transfer learning: 
python train.py --weights ./data/yolov4.weights

The training performance is not fully reproduced yet, so I recommended to use Alex's Darknet to train your own data, then convert the .weights to tensorflow or tflite.

TODO

• Convert YOLOv4 to TensorRT
• YOLOv4 tflite on android
• YOLOv4 tflite on ios
• Training code
• Update scale xy
• ciou
• Mosaic data augmentation
• Mish activation
• yolov4 tflite version
• yolov4 in8 tflite version for mobile

References

• YOLOv4: Optimal Speed and Accuracy of Object Detection YOLOv4.
• darknet