Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add oriented bounding box head for object detection tasks #73

Open
wants to merge 80 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
7c16e47
Resnet Variants (#9)
DrejcPesjak Feb 20, 2024
8e35f25
MLFlow Upload Fix (#10)
klemen1999 Feb 20, 2024
15bd923
CLI Source Option (#11)
kozlov721 Feb 21, 2024
2797278
Fix Removed Tensor Metadata (#12)
kozlov721 Feb 21, 2024
2c62a08
Forbid Extra Fields (#13)
kozlov721 Feb 22, 2024
0b51fa0
Automatic Inference of attach_index (#14)
kozlov721 Feb 24, 2024
bd67595
Backbone Fix (#15)
kozlov721 Feb 27, 2024
f42192c
Uploading logs to MLFlow (#16)
kozlov721 Feb 28, 2024
e1ab39b
Generate NN archive from training configs (#17)
jkbmrz Mar 20, 2024
b3b4e32
Extend NN Archive Generation Test Coverage (#18)
jkbmrz Mar 25, 2024
351e0c5
Upload All Checkpoints (#19)
kozlov721 Apr 11, 2024
9c4cadb
LuxonisML v0.1.0 (#20)
kozlov721 Apr 11, 2024
f425fdb
SIGTERM Handling (#21)
kozlov721 Apr 15, 2024
ca57063
Task Label Groups Support (#22)
kozlov721 Apr 24, 2024
d1d71f0
Tensor Core Float16 Precision (#24)
kozlov721 May 8, 2024
0830043
Metrics - Fixed Missing Reset (#25)
kozlov721 May 14, 2024
5a31f72
Deterministic Training Support (#23)
klemen1999 May 15, 2024
99b1857
Custom Loaders Support (#27)
kozlov721 May 21, 2024
b6b4688
enums handling (#31)
kozlov721 May 21, 2024
72afb72
GPUStatsMonitor (#29)
kozlov721 May 24, 2024
5893c3e
More Efficient Keypoint Export (#28)
JSabadin May 24, 2024
4110f78
Added active param to augmentations (#32)
klemen1999 May 24, 2024
36a92a6
Fix Archiver Pre-Processing (#34)
kozlov721 May 30, 2024
1d9998b
EfficientRep Variants (#33)
JSabadin May 31, 2024
c2e98b7
Support for LuxonisML - Annotation Refactor (#37)
kozlov721 Jun 6, 2024
abe7d3d
Changed Imports in Config (#38)
kozlov721 Jun 7, 2024
669a023
Multi Input Support (#36)
CaptainTrojan Jun 13, 2024
bb9b01d
Updated Tuner (#26)
klemen1999 Jun 13, 2024
bf69480
CLI Inspect Fix (#41)
kozlov721 Jun 16, 2024
88e8ff5
Tuning - Augmentation Subsets Support (#35)
klemen1999 Jun 19, 2024
248fa48
Support For Export Without Weights (#43)
kozlov721 Jun 21, 2024
4dda255
Openvino version in blobconverter config (#44)
klemen1999 Jun 27, 2024
7c31555
Fix onnx export (#45)
klemen1999 Jul 9, 2024
db24760
Multi-GPU Training Support (#42)
MatejRojec Jul 11, 2024
c70b2a6
Ensure checkpoint is always generated (#46)
klemen1999 Jul 11, 2024
57258ee
New Keypoint Heads and Losses (#40)
JSabadin Jul 11, 2024
ef606b7
Fix Views (#47)
kozlov721 Jul 17, 2024
5e4dd76
Remove `dataset_id` (#48)
kozlov721 Jul 26, 2024
c0305d6
Fix Archive Tests (#50)
kozlov721 Jul 30, 2024
3d5307d
Fixed New `mlflow` Run (#49)
kozlov721 Jul 30, 2024
fc4bf11
Suppressing `mlflow` Exceptions (#52)
kozlov721 Aug 2, 2024
a20de5f
Hotfix for Freezing `polars` (#54)
kozlov721 Aug 3, 2024
611b93c
Full Custom Tasks Support (#53)
kozlov721 Aug 7, 2024
c116982
Miscellaneous Fixes and Features (#55)
kozlov721 Aug 12, 2024
04d008b
Fix: Update NN Archive Generation (#56)
conorsim Aug 15, 2024
4fc6111
Adaptive Detection Loss Speed-Up (#51)
JSabadin Aug 15, 2024
6c7045f
Remove `num_workers` Hotfix (#58)
JSabadin Aug 15, 2024
4af2fab
Fixed Retrieving Labels in AdaptiveDetectionLoss (#59)
kozlov721 Aug 16, 2024
9b17a70
Unified Cores and Cleanup (#57)
kozlov721 Aug 19, 2024
55df6ef
Stricter Config Types (#60)
kozlov721 Aug 19, 2024
0d1a8a1
Semantic Nodes Categorization (#61)
kozlov721 Aug 19, 2024
356e5eb
Support for Multi-Split Views (#62)
kozlov721 Aug 20, 2024
9c8683b
Option to Parse Data in `LuxonisLoaderTorch` (#63)
kozlov721 Aug 21, 2024
3c2b2ae
Fix Archiver (#64)
kozlov721 Aug 22, 2024
54fc144
Generalized Progress Bar (#66)
kozlov721 Aug 27, 2024
19a5d2b
Fail-Proof Checkpoint Usage in Callbacks (#65)
kozlov721 Aug 27, 2024
8870968
LuxonisML 0.3.0 Updates (#68)
kozlov721 Aug 28, 2024
723dff1
[Feat]: add obb head
mak-en Sep 1, 2024
ae8aa1a
Merge branch 'dev' into obb-head
mak-en Sep 1, 2024
5acabfc
Config `pin_memory` Option (#67)
JSabadin Sep 3, 2024
8a99628
[Fix]: change target dimensions in _preprocess_target
mak-en Sep 3, 2024
9bf8f8a
Merge branch 'dev' into obb-head
mak-en Sep 3, 2024
8026126
[Fix]: change obb decoupled block dimensions for DFL
mak-en Sep 4, 2024
7b5adf5
[WiP]: add nms for obb
mak-en Sep 5, 2024
25cb8ef
[WiP]: add a metrics module for obb
mak-en Sep 8, 2024
c911a91
[WiP]: implement an obb visualizer
mak-en Sep 10, 2024
0df07a5
Increased default validation interval (#71)
klemen1999 Sep 13, 2024
b3f9f5d
[Fix] Corrected config valid sequence for predefined models (#72)
klemen1999 Sep 17, 2024
ae9c26d
[WiP]: add obb_loader and minor fixes
mak-en Sep 17, 2024
3318205
Merge branch 'dev' into obb-head
mak-en Sep 17, 2024
4265845
[Fix]: remove debug files
mak-en Sep 17, 2024
7daabdc
Code Cleanup and Improved Tests (#69)
kozlov721 Sep 19, 2024
1297a04
Update CI Badge (#75)
kozlov721 Sep 19, 2024
d2a95e5
Merge branch 'dev' into obb-head
kozlov721 Sep 19, 2024
af77626
formatting
kozlov721 Sep 19, 2024
a037c50
leftover merge discrepancies
kozlov721 Sep 19, 2024
e7ab0c0
[Fix] Updated `EfficientBBoxHead` parser to `YOLO` (#74)
klemen1999 Sep 20, 2024
2449850
DDRNet for Semantic Segmentation (#70)
sokovninn Sep 20, 2024
b918547
Merge branch 'dev' into obb-head
mak-en Sep 22, 2024
6ab3204
[Fix]: fix docstrings
mak-en Sep 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Custom Loaders Support (#27)
* support for custom loaders and datasets

* updated configs

* custom loaders in inspect command

* updated inspect for multi-task labels

* removed custom loader from test config

* deleted comment

* deleted comment

* removed custom dataset

* removed comment

* skipping archiver test untill fixed in luxonis-ml

* [Automated] Updated coverage badge

---------

Co-authored-by: GitHub Actions <actions@github.com>
kozlov721 and actions-user authored May 21, 2024
commit 99b18575784ea9a86125884cfb4203d60cff9b86
5 changes: 3 additions & 2 deletions configs/classification_model.yaml
Original file line number Diff line number Diff line change
@@ -15,8 +15,9 @@ model:
thickness: 2
include_plot: True

dataset:
name: cifar10_test
loader:
params:
dataset_name: cifar10_test

trainer:
preprocessing:
6 changes: 4 additions & 2 deletions configs/coco_model.yaml
Original file line number Diff line number Diff line change
@@ -95,12 +95,14 @@ tracker:
wandb_entity: luxonis
is_mlflow: False

dataset:
name: coco_test
loader:
train_view: train
val_view: val
test_view: test

params:
dataset_name: coco_test

trainer:
accelerator: auto
devices: auto
5 changes: 3 additions & 2 deletions configs/detection_model.yaml
Original file line number Diff line number Diff line change
@@ -10,8 +10,9 @@ model:
params:
use_neck: True

dataset:
name: coco_test
loader:
params:
dataset_name: coco_test

trainer:
preprocessing:
5 changes: 3 additions & 2 deletions configs/example_export.yaml
Original file line number Diff line number Diff line change
@@ -12,8 +12,9 @@ model:
backbone: MicroNet
task: binary

dataset:
name: coco_test
loader:
params:
dataset_name: coco_test

trainer:
preprocessing:
5 changes: 3 additions & 2 deletions configs/example_tuning.yaml
Original file line number Diff line number Diff line change
@@ -11,8 +11,9 @@ model:
backbone: MicroNet
task: binary

dataset:
name: coco_test
loader:
params:
dataset_name: coco_test

trainer:
preprocessing:
5 changes: 3 additions & 2 deletions configs/keypoint_bbox_model.yaml
Original file line number Diff line number Diff line change
@@ -8,8 +8,9 @@ model:
predefined_model:
name: KeypointDetectionModel

dataset:
name: coco_test
loader:
params:
dataset_name: coco_test

trainer:
preprocessing:
5 changes: 3 additions & 2 deletions configs/resnet_model.yaml
Original file line number Diff line number Diff line change
@@ -29,8 +29,9 @@ model:
thickness: 2
include_plot: True

dataset:
name: cifar10_test
loader:
params:
dataset_name: cifar10_test

trainer:
batch_size: 4
5 changes: 3 additions & 2 deletions configs/segmentation_model.yaml
Original file line number Diff line number Diff line change
@@ -11,8 +11,9 @@ model:
backbone: MicroNet
task: binary

dataset:
name: coco_test
loader:
params:
dataset_name: coco_test

trainer:
preprocessing:
1 change: 1 addition & 0 deletions luxonis_train/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from .attached_modules import *
from .core import *
from .models import *
from .utils import *

113 changes: 49 additions & 64 deletions luxonis_train/__main__.py
Original file line number Diff line number Diff line change
@@ -5,8 +5,10 @@
from typing import Annotated, Optional

import cv2
import torch
import typer
from torch.utils.data import DataLoader

from luxonis_train.utils.registry import LOADERS

app = typer.Typer(help="Luxonis Train CLI", add_completion=False)

@@ -105,7 +107,6 @@ def inspect(
"""Inspect dataset."""
from lightning.pytorch import seed_everything
from luxonis_ml.data import (
LuxonisDataset,
TrainAugmentations,
ValAugmentations,
)
@@ -117,7 +118,7 @@ def inspect(
get_unnormalized_images,
)
from luxonis_train.utils.config import Config
from luxonis_train.utils.loaders import LuxonisLoaderTorch, collate_fn
from luxonis_train.utils.loaders import collate_fn
from luxonis_train.utils.types import LabelType

overrides = {}
@@ -134,79 +135,63 @@ def inspect(

image_size = cfg.trainer.preprocessing.train_image_size

dataset = LuxonisDataset(
dataset_name=cfg.dataset.name,
team_id=cfg.dataset.team_id,
dataset_id=cfg.dataset.id,
bucket_type=cfg.dataset.bucket_type,
bucket_storage=cfg.dataset.bucket_storage,
)
augmentations = (
TrainAugmentations(
image_size=image_size,
augmentations=[
i.model_dump() for i in cfg.trainer.preprocessing.augmentations
],
train_rgb=cfg.trainer.preprocessing.train_rgb,
keep_aspect_ratio=cfg.trainer.preprocessing.keep_aspect_ratio,
)
if view == "train"
else ValAugmentations(
image_size=image_size,
augmentations=[
i.model_dump() for i in cfg.trainer.preprocessing.augmentations
],
train_rgb=cfg.trainer.preprocessing.train_rgb,
keep_aspect_ratio=cfg.trainer.preprocessing.keep_aspect_ratio,
)
augmentations = (TrainAugmentations if view == "train" else ValAugmentations)(
image_size=image_size,
augmentations=[i.model_dump() for i in cfg.trainer.preprocessing.augmentations],
train_rgb=cfg.trainer.preprocessing.train_rgb,
keep_aspect_ratio=cfg.trainer.preprocessing.keep_aspect_ratio,
)

loader_train = LuxonisLoaderTorch(
dataset,
view=view,
augmentations=augmentations,
loader = LOADERS.get(cfg.loader.name)(
view=view, augmentations=augmentations, **cfg.loader.params
)

pytorch_loader_train = torch.utils.data.DataLoader(
loader_train,
batch_size=4,
num_workers=1,
pytorch_loader = DataLoader(
loader,
batch_size=1,
num_workers=0,
collate_fn=collate_fn,
)

if save_dir is not None:
os.makedirs(save_dir, exist_ok=True)

counter = 0
for data in pytorch_loader_train:
imgs, label_dict = data
images = get_unnormalized_images(cfg, imgs)
for i, img in enumerate(images):
for label_type, labels in label_dict.items():
if label_type == LabelType.CLASSIFICATION:
continue
elif label_type == LabelType.BOUNDINGBOX:
img = draw_bounding_box_labels(
img, labels[labels[:, 0] == i][:, 2:], colors="yellow", width=1
)
elif label_type == LabelType.KEYPOINT:
img = draw_keypoint_labels(
img, labels[labels[:, 0] == i][:, 1:], colors="red"
for data in pytorch_loader:
imgs, task_dict = data
for task, label_dict in task_dict.items():
images = get_unnormalized_images(cfg, imgs)
for i, img in enumerate(images):
for label_type, labels in label_dict.items():
if label_type == LabelType.CLASSIFICATION:
continue
elif label_type == LabelType.BOUNDINGBOX:
img = draw_bounding_box_labels(
img,
labels[labels[:, 0] == i][:, 2:],
colors="yellow",
width=1,
)
elif label_type == LabelType.KEYPOINT:
img = draw_keypoint_labels(
img, labels[labels[:, 0] == i][:, 1:], colors="red"
)
elif label_type == LabelType.SEGMENTATION:
img = draw_segmentation_labels(
img, labels[i], alpha=0.8, colors="#5050FF"
)

img_arr = img.permute(1, 2, 0).numpy()
img_arr = cv2.cvtColor(img_arr, cv2.COLOR_RGB2BGR)
if save_dir is not None:
counter += 1
cv2.imwrite(
os.path.join(save_dir, f"{counter}_{task}.png"), img_arr
)
elif label_type == LabelType.SEGMENTATION:
img = draw_segmentation_labels(
img, labels[i], alpha=0.8, colors="#5050FF"
)

img_arr = img.permute(1, 2, 0).numpy()
img_arr = cv2.cvtColor(img_arr, cv2.COLOR_RGB2BGR)
if save_dir is not None:
counter += 1
cv2.imwrite(os.path.join(save_dir, f"{counter}.png"), img_arr)
else:
cv2.imshow("img", img_arr)
if cv2.waitKey() == ord("q"):
exit()
else:
cv2.imshow(task, img_arr)
if save_dir is None and cv2.waitKey() == ord("q"):
exit()


@app.command()
39 changes: 5 additions & 34 deletions luxonis_train/callbacks/test_on_train_end.py
Original file line number Diff line number Diff line change
@@ -1,43 +1,14 @@
import lightning.pytorch as pl
from luxonis_ml.data import LuxonisDataset, ValAugmentations
from torch.utils.data import DataLoader

from luxonis_train.utils.config import Config
from luxonis_train.utils.loaders import LuxonisLoaderTorch, collate_fn
import luxonis_train
from luxonis_train.utils.registry import CALLBACKS


@CALLBACKS.register_module()
class TestOnTrainEnd(pl.Callback):
"""Callback to perform a test run at the end of the training."""

def on_train_end(self, trainer: pl.Trainer, pl_module: pl.LightningModule) -> None:
cfg: Config = pl_module.cfg

dataset = LuxonisDataset(
dataset_name=cfg.dataset.name,
team_id=cfg.dataset.team_id,
dataset_id=cfg.dataset.id,
bucket_type=cfg.dataset.bucket_type,
bucket_storage=cfg.dataset.bucket_storage,
)

loader_test = LuxonisLoaderTorch(
dataset,
view=cfg.dataset.test_view,
augmentations=ValAugmentations(
image_size=cfg.trainer.preprocessing.train_image_size,
augmentations=[
i.model_dump() for i in cfg.trainer.preprocessing.augmentations
],
train_rgb=cfg.trainer.preprocessing.train_rgb,
keep_aspect_ratio=cfg.trainer.preprocessing.keep_aspect_ratio,
),
)
pytorch_loader_test = DataLoader(
loader_test,
batch_size=cfg.trainer.batch_size,
num_workers=cfg.trainer.num_workers,
collate_fn=collate_fn,
)
trainer.test(pl_module, pytorch_loader_test)
def on_train_end(
self, trainer: pl.Trainer, pl_module: "luxonis_train.models.LuxonisModel"
) -> None:
trainer.test(pl_module, pl_module._core.pytorch_loaders["test"])
3 changes: 2 additions & 1 deletion luxonis_train/core/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
from .archiver import Archiver
from .core import Core
from .exporter import Exporter
from .inferer import Inferer
from .trainer import Trainer
from .tuner import Tuner

__all__ = ["Exporter", "Trainer", "Tuner", "Inferer", "Archiver"]
__all__ = ["Exporter", "Trainer", "Tuner", "Inferer", "Archiver", "Core"]
2 changes: 1 addition & 1 deletion luxonis_train/core/archiver.py
Original file line number Diff line number Diff line change
@@ -45,7 +45,7 @@ def __init__(
cfg=self.cfg,
dataset_metadata=self.dataset_metadata,
save_dir=self.run_save_dir,
input_shape=self.loader_train.input_shape,
input_shape=self.loaders["train"].input_shape,
)

self.model_name = self.cfg.model.name
Loading