Commuting to and from work can be a nightmare. Cars advance slowly in stop and go traffic, crawling from one traffic jam at stoplights to the next. At peak rush hour especially, there is no chance of sailing through a series of green lights. The research teams at the institute branch for industrial automation INA, at the Fraunhofer Institute for Optronics, System Technologies and Image Exploitation IOSB, want to change this with their “KI4LSA” project, which uses artificial intelligence to enable smart, predictive light switching. The project partners are Stührenberg GmbH, Cichon Automatisierungstechnik GmbH, Stadtwerke Lemgo GmbH, the city of Lemgo (associated) and Straßen.NRW (associated). The German Federal Ministry of Transport and Digital Infrastructure (BMVI) is funding the project, which ends in summer 2022.
Conventional traffic lights use rule-based controls, but this rigid approach does not work for all traffic situations. In addition, the sensors currently in use—induction loop technology embedded in the road surface— provide only a rough impression of the actual traffic situation. The researchers at Fraunhofer IOSB-INA are working to address these problems. Instead of conventional sensors, they are using high-resolution cameras and radar sensors to more precisely capture the actual traffic situation. This allows the number of vehicles waiting at a junction to be determined accurately in real time. The technology also detects the average speed of the cars and the waiting times. The real-time sensors are combined with artificial intelligence, which replaces the usual rigid control rules. The AI uses deep reinforcement learning (DRL) algorithms, a method of machine learning that focuses on finding intelligent solutions to complex control problems. “We used a junction in Lemgo, where our testing is carried out, to build a realistic simulation and trained the AI on countless iterations within this model. Prior to running the simulation, we added the traffic volume measured during rush hour into the model, enabling the AI to work with real data. This resulted in an agent trained using deep reinforcement learning: a neural network that represents the lights control,” Arthur Müller, project manager and scientist at the Fraunhofer IOSB-INA, explains the DRL approach. The algorithms trained in this way calculate the optimum switching behavior for the traffic lights and the best phase sequence to shorten waiting times at the junction, reduce journey times and thus lower the noise and CO2 pollution caused by queuing traffic. The AI algorithms run in an edge computer in the control box at the junction. One advantage of the algorithms is that they can be tested, used and scaled up to include neighboring lights that form a wider network.
Big impact when scaled up
The simulation phases carried out on the congested Lemgo junction fitted with intelligent lights demonstrated that the use of artificial intelligence could improve traffic flow by 10–15%. Over the coming months, the trained agent will now take to the streets for further evaluation in a real-life laboratory. This testing will also consider the influence of the traffic metrics on parameters like noise pollution and emissions. However, the unavoidable “simulation to reality gap” presents a challenge. “The assumptions about traffic behavior that were used in the simulation are not a 1:1 representation of reality. So, the agent will need to be adjusted accordingly,” Müller says. “If this is successful, the effects of scaling up will be huge. Just think of the large number of traffic lights even in a small town like Lemgo.”
The EU estimates that traffic jams cause economic damage totaling 100 billion euros per year for its member states. According to Müller, AI traffic lights provide an opportunity to use our existing infrastructure more efficiently. “We are the first team in the world to test deep reinforcement learning for traffic light control under real-world conditions. And we hope that our project will inspire others to similar endeavors.”
Intelligent traffic signal systems for pedestrians
The “KI4PED” project focuses on pedestrians rather than vehicles. In a project scheduled to run until the end of July 2022, Fraunhofer IOSB-INA is working together with Stührenberg GmbH and associated partners Straßen.NRW, the city of Lemgo and the city of Bielefeld to develop an innovative approach for the needs-based control of pedestrian signals. This should be particularly beneficial for vulnerable people, such as older people or those with disabilities. The aim is to reduce waiting times and improve safety at pedestrian crosswalks by enabling longer crossing times. According to current studies, the “walk” times are too short for these groups of people. The buttons currently in use, generally in small yellow boxes, do not deliver any information about the number or age of crossers, or indeed their other needs. The project partners want to use AI in combination with high-resolution LiDAR sensors to automate the process and automatically adjust and increment the crossing times according to the needs of the pedestrians. The AI performs person detection and tracking based on data from LiDAR sensors and applies it in an embedded system in real time.
“For data-protection purposes, we are using LiDAR sensors rather than camera-based systems. These present the pedestrians as 3D point-clouds, meaning that they cannot be individually identified,” explains Dr. Dennis Sprute, project manager and scientist at Fraunhofer IOSB-INA. LiDAR sensors (light detection and ranging) emit pulsed light waves into the surrounding environment, which bounce off nearby objects and return to the sensor. The sensor measures the time it takes for the light to return to calculate the distance it traveled to the object, in this case, the person. These sensors are also resistant to the influences of light, reflections and weather. A feasibility study will be carried out to determine the optimum positions and alignment at the crossing. The AI algorithms will initially be trained for a week at two stoplight crossings in Lemgo and Bielefeld. Sensors tests are also planned on the Fraunhofer IOSB-INA site using various simulated light conditions to determine the detection capabilities.
By using a needs-based control concept adapted to the individual situation, the research partners hope to reduce the waiting times when there are lots of people waiting by 30%. They also aim to reduce the number of incidents of jaywalking by around 25%.
