by Alina Vereshchaka and Nicole Capozziello
Published September 12, 2019
During the month of June, the Department of Computer Science and Engineering hosted the first-ever UB Reinforcement Learning Challenge. The event invited undergraduate and graduate students to apply their skills and knowledge in the field of reinforcement learning, a prominent area of machine learning.
“Reinforcement learning is a cutting-edge topic,” says Alina Vereshchaka, PhD student in computer science and organizer of the event. “I wanted to give students an opportunity to explore this area by defining their own environment and applying deep reinforcement learning algorithms to solve it.”
As a team or as individuals, students submitted an abstract for a project. Five teams were chosen to advance to the final round, developing their projects over a two-week period. During their final presentations on June 27, students had the opportunity to demonstrate their work in front of an audience of SEAS faculty and staff, students, and members of the general public. The top three teams received a certificate, gift cards and CSE merchandise.
“Despite the challenging task, the students in the CSE4/510 Reinforcement Learning course were able to provide great results within the time limits,” says Vereshchaka. “The finalists’ projects covered a variety of domains, including resource allocation optimization, financial trading, robotics, the stock market, image tracking and image filling.”
The judging committee was comprised of CSE faculty Sargur Shrihari, Wen Dong, and Kenny Joseph, who specialize in artificial intelligence.
“Reinforcement learning forces students to think in new ways about how we might teach machines (and humans) to learn new things,” says Joseph, assistant professor in the Department of Computer Science and Engineering. “The students did a wonderful job in applying this new way of thinking in creative and novel ways, so much so that it forced me reflect on ways that I could better approach similar problems. It was therefore gratifying simply to watch and learn from the students' hard work.”
Reinforcement learning has been demonstrated to be successful in solving challenging problems in areas ranging from games to robotics. However, most of this success has been in single-agent environments, such as robotics, power systems, trading strategies, or personalized recommendations, while real-world problems are mostly based on interactions between multiple agents.
This challenge sought to motivate students to find approaches for defining environments based on multi-agent interactions and teaching these agents an optimal behavior. The multi-agent system is a more realistic way of representing task allocation, team planning and user preferences. It has a large number of applications, including transportation, logistics, robotics, military, graphics, manufacturing and management.
"Common to all the projects was that they strive to solve our society's most important and challenging problems, relevant to the students' everyday life and involving the complex interactions of human and machines. The topics ranged from best scheduling critical resources (snowplows) in preparation for uncertain future situations (weather), to tapping into the wisdom of a crowd to identify truths from complex situations (stock market) and avoid group polarization at the same time," added Dong, assistant professor in the Department of Computer Science and Engineering. "As a result, the students had to creatively apply the knowledge learned from the course in order to solve open-ended questions."
Recognizing self-managing time could be a challenge, the organizers set up events to guide participants through the process of developing their projects. “The RL challenge provided enough structure so that it was not painfully open-ended while still allowing students to work on projects that felt important to them,” says Nathan Margaglio, the first place winner.
“Not only did it provide some incentive to start and, more importantly, finish a project that is challenging and technical in nature but it also helped guide students towards concrete and meaningful goals,” says Margaglio, one of many participants who look forward to increased future opportunities to advance their skills in a practical setting.
One of the finalists was Nazerke Sandibay, a summer exchange student from Nazarbayev University in Kazakhstan. Sandibay, one of the only undergraduate participants, received positive feedback on her work focusing on identifying the optimal trajectory of a “2 degree of freedom robot.”
"The UB Reinforcement Learning Challenge is a valuable experience for students, who are empowered to engage with and solve problems that incorporate multi-agents,” says Vereshchaka. “Best of all, these kinds of focused efforts enable students to show their achievements in reinforcement learning. Hopefully this event becomes a regular occurrence, and will motivate students to challenge themselves in simulation modeling and reinforcement learning.”
The winning teams were:
This event was supported by the Department of Computer Science and Engineering and the School of Engineering and Applied Science. More details can be found here.
Nathan Margaglio earned first place in the Reinforcement Learning Challenge. From left is Wen Dong (CSE faculty member and judge), Nathan Margaglio, Alina Vereshchaka (organizer) and Kenny Joseph (CSE faculty member and judge).
Judges, organizer and participants of the UB Reinforcement Learning Challenge 2019 and CSE4/510 Reinforcement Learning course.
Second place winner Anurag Saykar. From left is Wen Dong (CSE faculty member and judge), Anurag Saykar, Alina Vereshchaka (organizer), Kenny Joseph (CSE faculty member and judge) and Sargur Srihari (CSE faculty member and judge).
Third place winners Nitin Nataraj and Priyanka Pal. From left is Wen Dong (CSE faculty member and judge), Natarai, Pal, Alina Vereshchaka (organizer), Kenny Joseph (CSE faculty member and judge) and Sargur Srihari (CSE faculty member and judge).
Weijin Zhu, finalist of the UB Reinforcement Learning Challenge, with Alina Vereshchaka (organizer).
Zhi Wen Huang, finalist of the UB Reinforcement Learning Challenge, with Alina Vereshchaka (organizer).
Nazerke Sandibay, finalist of the UB Reinforcement Learning Challenge and summer exchange student from Nazarbayev University, Kazakhstan, with Alina Vereshchaka (organizer).