This year, Women’s Day presented an exciting opportunity to participate in the Amazon Web Services (AWS) DeepRacer Women’s League- India 2021. The problem statement was to model a virtual car to navigate through a miniature digital race track. Multiple technical sessions were conducted over two weeks, to help me build a computerised version of a car with camera and Light Detection and Ranging (LiDAR) sensor inputs.
Helen Thomas won the AWS DeepRacer Women’s League - India 2021
Fun exposure
The competition was set up as a race, where the goal was for participants to ride their car without going off the track and to complete the lap as fast as possible. It is a complex task, and the intelligence to drive like human racers was developed using Artificial Neural Networks (ANN) and Reinforcement Learning (RL). ANN loosely imitates the operation of neurons in the brain to process input information such as camera images of the road ahead, and take decisions on whether to accelerate or brake. RL is a type of machine learning, which is used in many everyday applications including autonomous vehicles, smart speakers, stock trading, medical diagnosis, online search recommendation, not to mention phenomenally successful games such as GO and Atari. Just like humans gain expertise in performing a task over repeated trials and improving each time, RL rewards the car with points whenever the correct action is chosen.This opportunity was, in a way, the first step towards developing a self-driving car. As a Ph.D. transportation engineering student, I was determined to leverage this hands-on experience.
It was clear that efficient training by setting up meaningful reward functions and action space, along with suitably tweaking the hyper-parameters of the algorithm were the keys to winning. After some initial experimentation, I deduced that simple things like positive rewards for keeping close to the centre-line of track or reducing speed at curves worked the best. Equally important was deciding what sensor data should be used as input, and the range of steering angles and speeds for actions.
I started training on a simple oval-shaped virtual track to try out different approaches and gain confidence. However, as the track became more complex with tight turns while progressing through the rounds, training strategies had to be improvised to keep score in the leader board. Suggestions from the community members and my mentor, such as creating an ‘optimal’ racing line, similar to those used in motorsports, were helpful at this stage.
Overall, this league was successful in generating interest and equipping women students with ML tools. I look forward to participating in more challenging ML-based virtual races, and competing with racers around the world. This will also help as it is closely related to research at IISc Bengaluru, where a more realistic game engine-based driving simulator is being developed for Indian road conditions.
The writer is a Ph.D. student from the Indian Institute of Science (IISc), Bengaluru
