Skip to content

blob tracker

Jump to bottom
Alexandre Marcireau edited this page Jun 19, 2018 · 10 revisions

The Blob Tracker example implements an object tracker based on Gaussian blobs. It illustrates a technique for drawing polygons on top of an other display.

It focuses on the following Tarsier handlers:

It also uses the BlobDisplay from the Chameleon library.

We aim at creating a change detection display with ellipses drawn on top to show the blob trackers. We are going to use two blob displays: one for the trackers on the visible layer, and one for the trackers on the hidden layer. The blob display must be shaped after the change detection display. We use the onPaintAreaChanged callback to change the blob display position and size whenever the change detection display is modified. The Timer must trigger both the change detection display and the blob display. The latter is triggered with the update function.

The main.qml file contains:

import QtQuick 2.3
import QtQuick.Layouts 1.1
import BackgroundCleaner 1.0
import ChangeDetectionDisplay 1.0
import BlobDisplay 1.0

Item {
    id: item
    visible: true
    width: 304
    height: 240

    Timer {
        interval: 20
        running: true
        repeat: true
        onTriggered: {
            backgroundCleaner.triggerDraw();
            changeDetectionDisplay.triggerDraw();
            hiddenBlobDisplay.update();
            blobDisplay.update();
        }
    }

    BackgroundCleaner {
        id: backgroundCleaner
        width: item.width
        height: item.height
    }

    ChangeDetectionDisplay {
        objectName: "changeDetectionDisplay"
        id: changeDetectionDisplay
        width: item.width
        height: item.height
        canvasSize: "304x240"
        onPaintAreaChanged: {
            blobDisplay.width = paintArea.width
            blobDisplay.height = paintArea.height
            blobDisplay.x = paintArea.x
            blobDisplay.y = paintArea.y
            hiddenBlobDisplay.width = paintArea.width
            hiddenBlobDisplay.height = paintArea.height
            hiddenBlobDisplay.x = paintArea.x
            hiddenBlobDisplay.y = paintArea.y
        }
    }

    BlobDisplay {
        objectName: "hiddenBlobDisplay"
        id: hiddenBlobDisplay
        canvasSize: "304x240"
        strokeColor: "#ff5b5a"
        strokeThickness: 1
        fillColor: "#33ff5b5a"
    }

    BlobDisplay {
        objectName: "blobDisplay"
        id: blobDisplay
        canvasSize: "304x240"
        strokeColor: "#00cb56"
        strokeThickness: 1
        fillColor: "#3300cb56"
    }
}

We want to mask isolated events to prevent noise from disturbing the gaussian trackers. The Tarsier library provides the tarsier::MaskIsolated handler to do so. Moreover, we want to duplicate each event to use them to trigger both the change detection display and the blob tracker. A lambda function could be used to trigger both the handlers. It is also possible to use the tarsier::Replicate handler, which can trigger any number of handlers whenever it is sent an event.

To track the blobs, we use the tarsier::TrackBlobs handler. We use twelve initial trackers and decent parameters. These settings need to be adapted to each task. The blob tracker expects seven callback: three callbacks (promote - update - demote) for both the visible and hidden layers, and one common delete callback.

The main.cpp file now contains:

#include <sepia.hpp>
#include <tarsier/maskIsolated.hpp>
#include <tarsier/replicate.hpp>
#include <tarsier/trackBlobs.hpp>
#include <chameleon/backgroundCleaner.hpp>
#include <chameleon/changeDetectionDisplay.hpp>
#include <chameleon/blobDisplay.hpp>
#include <QtGui/QGuiApplication>
#include <QtQuick/QQuickView>
#include <QtCore/QFileInfo>

int main(int argc, char* argv[]) {
    QGuiApplication app(argc, argv);

    qmlRegisterType<chameleon::BackgroundCleaner>("BackgroundCleaner", 1, 0, "BackgroundCleaner");
    qmlRegisterType<chameleon::ChangeDetectionDisplay>("ChangeDetectionDisplay", 1, 0, "ChangeDetectionDisplay");
    qmlRegisterType<chameleon::BlobDisplay>("BlobDisplay", 1, 0, "BlobDisplay");

    QSurfaceFormat format;
    format.setDepthBufferSize(24);
    format.setStencilBufferSize(8);
    format.setVersion(3, 3);
    format.setProfile(QSurfaceFormat::CoreProfile);
    QQuickView view(QUrl::fromLocalFile(QFileInfo("../../source/main.qml").absoluteFilePath()));
    view.setFormat(format);
    view.setResizeMode(QQuickView::SizeRootObjectToView);
    view.show();

    auto changeDetectionDisplay = view.rootObject()->findChild<chameleon::ChangeDetectionDisplay*>("changeDetectionDisplay");
    auto blobDisplay = view.rootObject()->findChild<chameleon::BlobDisplay*>("blobDisplay");
    auto hiddenBlobDisplay = view.rootObject()->findChild<chameleon::BlobDisplay*>("hiddenBlobDisplay");

    auto atisEventStreamObservable = sepia::make_atisEventStreamObservable(
        "/Users/Bob/Desktop/recording.es",
        sepia::make_split(
            tarsier::make_maskIsolated<sepia::DvsEvent, 304, 240, 10000>(
                tarsier::make_replicate<sepia::DvsEvent>(
                    tarsier::make_trackBlobs<sepia::DvsEvent>(
                        {
                            tarsier::Blob{38 * 1, 40 * 1, 250, 0, 250},
                            tarsier::Blob{38 * 3, 40 * 1, 250, 0, 250},
                            tarsier::Blob{38 * 5, 40 * 1, 250, 0, 250},
                            tarsier::Blob{38 * 7, 40 * 1, 250, 0, 250},
                            tarsier::Blob{38 * 1, 40 * 3, 250, 0, 250},
                            tarsier::Blob{38 * 3, 40 * 3, 250, 0, 250},
                            tarsier::Blob{38 * 5, 40 * 3, 250, 0, 250},
                            tarsier::Blob{38 * 7, 40 * 3, 250, 0, 250},
                            tarsier::Blob{38 * 1, 40 * 5, 250, 0, 250},
                            tarsier::Blob{38 * 3, 40 * 5, 250, 0, 250},
                            tarsier::Blob{38 * 5, 40 * 5, 250, 0, 250},
                            tarsier::Blob{38 * 7, 40 * 5, 250, 0, 250},
                        },
                          1e5, // activityDecay
                            0, // minimumProbability
                            1, // promotionActivity
                          0.1, // deletionActivity
                         0.99, // meanInertia
                         0.99, // covarianceInertia
                            1, // repulsionStrength
                           20, // repulsionLength
                         0.05, // attractionStrength
                          100, // attractionResetDistance
                        10000, // pairwiseCalculationsToSkip
                        [&](std::size_t id, const tarsier::Blob& blob) {
                            blobDisplay->promoteBlob(id, blob);
                        },
                        [&](std::size_t id, const tarsier::Blob& blob) {
                            blobDisplay->updateBlob(id, blob);
                        },
                        [&](std::size_t id, const tarsier::Blob& blob) {
                            blobDisplay->demoteBlob(id, blob);
                        },
                        [&](std::size_t id, const tarsier::Blob& blob) {
                            hiddenBlobDisplay->promoteBlob(id, blob);
                        },
                        [&](std::size_t id, const tarsier::Blob& blob) {
                            hiddenBlobDisplay->updateBlob(id, blob);
                        },
                        [&](std::size_t id, const tarsier::Blob& blob) {
                            hiddenBlobDisplay->demoteBlob(id, blob);
                        },
                        [&](std::size_t id, const tarsier::Blob& blob) {
                            blobDisplay->deleteBlob(id, blob);
                            hiddenBlobDisplay->deleteBlob(id, blob);
                        }
                    ),
                    [&](sepia::DvsEvent dvsEvent) -> void {
                        changeDetectionDisplay->push(dvsEvent);
                    }
                )
            ),
            [](sepia::ThresholdCrossing) -> void {}
        ),
        [](std::exception_ptr) {}
    );

    return app.exec();
}

The premake4.lua file must import the following Chameleon components:

local mocFiles = { -- must contain Chameleon components paths
    '/usr/local/include/chameleon/backgroundCleaner.hpp',
    '/usr/local/include/chameleon/changeDetectionDisplay.hpp',
    '/usr/local/include/chameleon/blobDisplay.hpp',
}
Clone this wiki locally