Traffic Light Negotiation and Perception-Based Detection (Project 223)

[robertogl]

Contribute to the discussion by asking and/or answering questions, commenting, or sharing your ideas for solutions to project 223

[sai303030]

i need some idea about project and abstract of the project

[robertogl]

Hi @sai303030,

What do you mean precisely?

[sai303030]

I was take up a project under the domain of machine learning and tittle of my project is traffic light negotiation and perception based detection , I need some tips and abstract of the project..

[nikhiltkumar]

Where can I find hints for the solution?

[robertogl]

Hi @nikhiltkumar,

In the background material section we have links to several examples that can be very useful for starting the project and work towards its completion. Please let us know if there is anything in particular you need help with.

[Vikas-ABD]

can I get any reading papers for this project

[dikokob]

Hi @robertogl,

I hope you are well.

I am from the University of Cape Town, and one of my students is doing the project.

However, it seems we need to purchase Road Runner Scenario. Please let me know if this is included in our licence.

[robertogl]

Hello @dikokob,

University of Cape Town should have a full-suite Wide-Campus license, which includes all the products. However, let me just confirm with the account manager.

[robertogl]

I got the confirmation that you have Road Runner and Road Runner Scenario as you can see in this list. However, Road Runner installation can be a bit tedious. Here the instructions.

I hope this helps.

[dikokob]

@robertogl thanks. I'll let you know how it goes.

I have asked my student to put a bit of a twist on the project by making a South African (i.e. load shedding) scenario. Would this be eligible for them to submit it for MathWorks Excellence in Innovation?

Reach out on [email protected]

[robertogl]

Of course! That would even introduce extra complexity to the project.

[megwilson8]

Hi @robertogl,
I am currently doing my thesis at UCT and the topic is Traffic light negotiation with perception based detection. I think you were communicating with my supervisors above. I am trying to integrate a road runner scenario into the prebuilt traffic light negotiation with unreal engine to create a South African scenario and adapt the Simulink for these conditions.

I have created a road runner scene and exported it to unreal engine. When I export the RoadRunner scene there are some errors which are effecting the driving scenario. Is there a way to fix these errors so I can implement the Roadrunner scene in the prebuilt traffic light negotiation with unreal engine? Is there also a way to find the errors before exporting the file, debugging?

I have attached the error messages to this message.

[pfryscak]

Thank you. Quick question: what version of MATLAB are you using?

[megwilson8]

I am using the 2023a for both RoadRunner and Matlab.

[pfryscak]

Engen Intersection Corner Fixed.zip
Hi! Import into ADT still has issues; ADT doesn't support lanes that taper, for example. I sent an internal message to that team asking what they suggest.

In the meanwhile, attached is a corrected version of your file, that will export from RoadRunner without error. There was an odd issue at one corner near the petrol station that I edited.

[megwilson8]

@pfryscak really appreciate the help.
Many thanks!

[megwilson8]

@pfryscak I have updated the Road Runner file but still receiving errors when I upload the file into MATLAB Driving Scenario. The main error is the lane width which I'm not sure how to check and edit? I have uploaded both the RoadRunner and Driving Scenario file to github (https://github.com/megwilson8/Traffic-Light-Negotiation-.git ) and attached screenshots of the errors below.

[pfryscak]

Hi. I have not yet received detailed instructions from the Automated Driving Toolbox (ADT) team. In the meanwhile, please take a look at https://www.mathworks.com/help/roadrunner/ref/lanewidthtool.html. Make a copy of your RRSCENE and adjust the lanes to have uniform width along their length and no tapers (forming lanes). Try importing that into ADT and see if it reduces the errors.

[megwilson8]

I have put this on hold for the moment and have move to perception based detection of the traffic light states. Using the simulation environment of Traffic Light Negotiation using Unreal Engine I have saved the simulation video and have attempted to train the detector to detect the traffic light states. I used Video Labeler in MATLAB but the detection is very inaccurate sometimes never detecting the traffic light or detecting everything but the traffic light.
I went with the approach of labeling each state of the traffic light and using this to train the detector but this was very inaccurate. I then tried to train the detection from only one labeled file for instance the video only had labels of green. This had a slightly better response but still very far from anything I can work with.
Is there a better way to use perception based detection to detect the traffic lights as I am trying to modify the model to be able to detect the traffic light state without using the state flow input.

[megwilson8]

@pfryscak

[pfryscak]

Hi @megwilson8. Once things are outside of the RoadRunner realm, I'm afraid I have less information. I will see if a colleague can help.

[robertogl]

Hi @megwilson8,

There are several factors to consider here. For example, the data that you use to train the detector. You need to have enough data and from different perspective, distance, and background with a good balance for all three lights. Also, what kind of detector are you using and how are you training it? Training a Yolov2 with 3 classes (one per color) should work. Otherwise there are other techniques that do not adopt NN that still work fine.

[megwilson8]

Hi @robertogl,
I will try the Yolov2 training.
I'm not sure how familiar you are with the Traffic Light Negotiation with Unreal Engine Visualization model but I am trying to train a detector to use perception based detection to determine the state of the traffic light. I also need to train the light to detect a broken traffic light (no colour) but I cant find where I could implement this into the model, displaying no colour.
Do you know how I can implement a scenario where there is no state showing on the traffic light?

[zy18879589365]

Is there a publicly available dataset for training?