Gokul Puthumanaillam
PhD Student, Roboticist and C++ enthusiast

This work introduced a novel approach to verification of neural network controlled systems, combining the capabilities of the nnenum framework with the Verse toolkit. Addressing a critical gap in the traditional verification process which often does not include the system dynamics analysis while computing the neural network outputs, our integrated methodology enhances the precision and safety of decision-making in complex dynamical systems. By iteratively verifying neural network decisions and propagating system states, we maintain an accurate representation of the system’s behavior over time, a vital aspect in ensuring operational safety. Our approach is exemplified through the verification of the neural network controlled Airborne Collision Avoidance System for Unmanned Aircraft (ACAS Xu). We demonstrate that the integration of nnenum and Verse not only accurately computes reachable sets for the UAS but also effectively handles the inherent complexity and nonlinearity of the system. The resulting analysis provides a nuanced understanding of the system’s behavior under varying operational conditions and interactions with other agents, such as intruder aircraft. The comprehensive simulations conducted as part of this study reveal the robustness of our approach, validating its effectiveness in verifying the safety and reliability of learned controllers. Furthermore, the scalability and adaptability of our methodology suggest its broader applicability in various autonomous systems requiring rigorous safety verification.

Paper: ArXiv

Video coming soon! (I am an amateur in video editing, but I try :) )

This project provides ROS2 support for u-blox ZED F9P GPS receivers. Most of this driver has been written based off the original ROS(1) driver that can be found here.

Features:

• Fast and robust
• Tested only for serial connections
• Can play back PCAP capture logs to test behavior
• Easy to add support for more log types
• Supports ASCII and binary-format logs
• Can synchronize BESTPOS, BESTVEL, and PSRDOP2 logs together in order to produce gps_common/GPSFix messages
• Can produce IMU data from receives with SPAN support

Code repository: Feel free to browse and access the software stack: https://github.com/gokulp01/ros2-ublox-zedf9p

Implemented Meta-Q-Learning for optimizing humanoid walking patterns. The project also demonstrate its effectiveness in improving stability, efficiency, and adaptability. Additionally, this work also explores the transferability of Meta-Q-Learning to new tasks with minimal tuning.

Conducted experiments: Test how adaptable the humanoid is by performing:

1. Side stepping
2. Ascending and Descending

Code repository: Feel free to browse and access the software stack: https://github.com/gokulp01/meta-qlearning-humanoid

RiP is a reachability based behavioural planner that prioritizes certain behaviour over the other. In this case, the priority order being:

1. No interaction with the "Danger" zone
2. Total length of the trajectory<constant (100)
3. Minimize the time spent in "Risk" zone
4. Minimize the total path length

(This order of the can be rearranged to get different results.)

The risk is minimized by assigining risk boundaries around the obstacles and then optimizing the path for length as well as risk. This algorithm is most suited for cases where the obstacles are adverserial and missions where safety is of utmost priority.

All possible explored paths (left) and final chosen path (right)

Code repository: Feel free to browse and access our software stack: http://github.com/gokulp01/RiP-Planner

More on the project and its documentation is in works. Coming soon!

Eve is our entry into the Mahindra $1 million challenge.

It is the culmination of years of hard work and is packed with cutting edge technologies and techniques – made even more impressive as we were the youngest team to qualify for the finals.

I had been heading the Sensing division of the team working towards building Eve. We got the car to level 2-3 autonomy. Extensively contributed in the areas of firmware, sensor fusion, planning and control, amongst other things for Indian road conditions.

*Code repository: * Feel free to browse and access our software stack:

• https://gitlab.com/project-manas/ai
• https://github.com/Project-MANAS (archived)

*Recognition: * We have been invited by tech giants like SAP, Continental, Vector and reputed colleges like NIE Mysore to present and give talks on our work.

Sponsors

• Microsoft
• Nvidia
• Gigabyte
• Continental
• Bossard
• Mahindra Rise
• Sick
• Vector
• Pepperl+Fuchs
• Ximea
• Cooler Master
• National Instruments and many more :)

Robots and other autonomous systems are increasingly being adopted in diverse fields – from health care and security to transport and manufacturing – as they become cheaper and smarter.

However, a significant gap exists between the current reality of robotic capabilities and real-world requirements. The MBZIRC Maritime Grand Challenge, was aimed on finding a practical solution to global maritime security challenges such as illegal fishing, piracy, smuggling and human trafficking.

It involved the developemnt of a heterogeneous collaboration among unmanned aerial vehicles and unmanned surface vehicles to perform complex navigation and manipulation tasks in a GNSS- denied marine environment.

Modified descriptor extractor and high level architecture(right)

Code repository: Software stack(request access): https://github.com/MOONLABIISERB/MBZIRC2023

Never Heard of Solo? It’s the UGV that made the headlines at IGVC in less than twelve months after its predecessor Adam’s entry.

Grand Prize winners and won the Lescoe Cup at IGVC, an international level robotics competition held at Oakland University, Michigan, USA.

Code repository and Report:

• Software stack: https://gitlab.com/project-manas/ai/igvc_2019
• Publications, Report: https://bit.ly/3pOx2Vf
• Feel free to mess around with the simulations: https://projectmanas.in/solo

Awards won:

• Interoperability Challenge-1st
• Design Challenge-2nd
• Cybersecurity challenge-3rd
• Grand prize winners- Lescoe cup

SERAPHIM is a system consisting of a fully autonomous drone and an autonomous Unmanned Ground Vehicle which was our entry to the Student Unmanned Aerial Systems Competition (SUAS) organized by the Association for Unmanned Vehicle Systems International (AUVSI).

The UAV was made keeping in mind a series of tasks that should be completed by the UAS system built:

• Interoperability: The UAS could download mission details, upload aircraft telemetry in real time, and upload mission deliverables to an external networked system.
• Autonomous Flight: The UAS autonomously takes off, flies within boundaries, navigates a series of waypoints, and lands.
• Obstacle Avoidance: The UAS autonomously avoids obstacles, which can be stationary or moving.
• Object Detection, Classification, Localization: The UAS takes pictures of a search area, detects objects of interest, classifies its characteristics, and provides a GPS position.
• Mapping: The UAS takes pictures of an area of interest and stitches the imagery into a map.

Air Delivery. The UAS autonomously drops a payload object so that it lands undamaged at a provided GPS position.

Multi-robot teams have been used for a variety of navigation and mapping applications. They have proven to provide better time efficacy and output than using a singular robot in doing the same. A place where such an implementation may have a lot of potential is in the exploration of planets and other similar terrestrial bodies, which is what this study aims to focus on. With space research thriving more than ever, a decentralized multi-robot system has the potential to deliver results many more times than a single ground robot would.

This paper introduces an optimized space-filling technique and tries to achieve autonomous multi-robot navigation without any path overlap.

Code repository and Report:

• Software stack: https://github.com/raghavthakar/kobu
• Publications, Report: https://github.com/raghavthakar/kobu/blob/main/kobu.pdf
• Watch it in action here: https://www.youtube.com/watch?v=NxmvM7g0AX4

Advisor: Dr. Sujit PB, Multi Robot Autonomy Lab Undergraduate researcher at the Multi-Robot Autonomy(M00N) Lab working on the topic of Object SLAM and sensor fusion based indoor localisation in cluttered environment.

Worked with Prof. Sujit PB(IISER-B) and Prof. Madhava Krishna(IIIT-H) to develop a novel monocular 2D Object Aware SLAM for navigation in indoor environments. Used meta-learning based approach alongside an EKF to fuse the values from the GNSS and the IMU, we were able to show that we can better the results of classical localization methods without significant computational resources.

Currently, I am also leading the navigation stack for the team competing at the MBZIRC. The main problem statement is to localize and search for a target using a heterogeneous swarm comprising of a USV, multiple UAVs and fixed wings in a GNSS denied environment. We are extrapolating the results obtained in the previously developed navigation pipeline for the swarm. To know more about the MBZIRC project, please check out the MBZIRC section.

Paper under preparation.

Bubo is a UAV made and tested for warehouses. Bubo took part in the Flipkart Grid 2.0 and reached the penultimate round of the competition

Named after Bubo, from Clash of the Titans our UAV is a tested and perfected warehouse assistant UAV with the following capabilities:

• Pick up/drop loads to a specific location
• Video surveillance
• Cycle Counting (Barcode & RFID)
• RTH(at low battery)
• Intralogistics (online mapping)

All possible explored paths (left) and final chosen path (right)

Rescue missions, military applications, firefighting missions, industrial automation, and other current concerns are all increasingly benefiting from multi-robot systems. One such problem statement where the practicality of using multiple robots far exceeds using one, is the mapping of large unknown environments. To avoid a negative impact on the benefits, multiple robots must not go over the same area, and hence must be able to communicate effectively. As a fail-safe measure in case of failure of the central node, this communication must be decentralized, and we have presented a blockchain based approach for the same.

All possible explored paths (left) and final chosen path (right)

Paper accepted for presentation at Blockchain Applications in Robotics and Automation, IROS 2021

This work presents a novel approach to effectively search for a target using a multi-robot system. The proposed approach augments the conventional swarm-based stochastic search algorithms by dynamically refining the search space to locate the source. Compared to other search algorithms, including PSO, Cuckoo Search Algorithm, Bat Algorithm, Glowworm Swarm Optimization and Random Walk, our algorithm reduces the time taken, and the amount of exploration done is much more succinct. Similarly, the algorithm makes no concessions in terms of success rate. In extreme scenarios, when the number of particles is fewer than five, the search space is enormous, or the search space is unbounded or noise in the sensor readings, our method stands out and performs far better than its competitors.

All possible explored paths (left) and final chosen path (right)

Paper under review

Baymax is a robot that won RosCore, a national level robotics competition.

BayMax is a really cool mobile robot that was made in 5 hours (4 hours and 37 minutes to be precise). It went on to win RosCore, a national level competition which involved designing the robot with capabilities of navigating, localising and planning its way through a maze while also parallely performing vision based tasks(scanning AR codes and solving them).

Feel free to browse through the source code: https://bitbucket.org/gokulp01/baymax/src/master/

Advisors: Prof. Keyvan Hashtrudi-Zaad, Prof. Joshua Marshall

• Worked with the research team on the SAE AutoDrive II Challenge
• Implemented a full-stack pipeline for self-driving car test-bed, which includes:
• A complete simulation of the map and environment in Carla.
• Perception and planning stack comprised of route planning, prediction, behavioural planning and path generation
• NMPC based controller for the ego vehicle, a Reciprocal Velocity Obstacles based controller for the traffic fleet, an adaptive behavioural planner with a dynamic velocity profiler.

Unscented Kalman filter based sensor fusion for sensors including Lidar, GPS, IMU and Encoders. The work is still in progress and is expected to be completed by September.

Detecting hypokalemic paralysis and partial paralysis using computer vision. Status: Completed

Mentor: Dr. Bhargav Bhatkalkar Team Members: Gokul, Dhruva, Riyaadh

More than 10 Million People around the globe are affected by Parkinson’s Disease. 1 in every 15 people suffering from Parkinson’s Disease experience partial paralysis which if not treated at the right time can be very dangerous.

Worked on NeuroVision, a tool to detect hypokalemic paralysis and partial paralysis using state of the art computer vision techniques. NeuroVision uses Computer Vision and Deep Learning algorithms to detect if a patient is in a state of paralysis. NeuroVision revolves around making the solution as cheap and feasible for anyone to use.

Report: Feel free to go through our product pitch: https://drive.google.com/file/d/1O-7xuAecCF_Ev9lRXNjB5UbMr4e1wMJ6/view?usp=sharing

Awards and recognition:

• 2nd Place in the MIT Health Hackathon.
• Placed 2nd in the country at the Coursera Innovation challenge 2021
• Mentored by Adam McGowan, a consultant for early-stage ventures to help the team explore the possibilities of commercialising the product.

H4rdWeared was presented at the Smart India Hackathon under DRDO’s(Defence Research and Development Organisation) problem statement. H4rdWeared was designed a wearable strap for soldiers with the following features:

• GPS location tracking of the soldier
• Monitor and record maximum number of vital body parameters
• SoS facility.
• Supports location timeline
• Transfers information by using a MANET-type communication system
• BrainFlex integration to monitor the brainwaves of different frequencies and log the soldier’s mental health

Awards and Recognition: One of the top 5 teams to make it to the national level finals Finals was called off because of the pandemic :(

Developed Cosmorepophilia, an operational sleep shift scheduling tool for the NASA Space Apps Challenge

Sleep loss and fatigue may lead to reduced performance and an increased risk to safety during many activities, including spaceflight. Our challenge was to develop an operational sleep shift scheduling tool that provides autonomous customization of a schedule for sleep, exercise, and nutrition to manage fatigue.

Cosmorepophilia provides autonomous customization of a schedule for sleep, exercise and nutrition to manage fatigue especially in astronauts. Cosmorepophilia also includes Cosmorepose, an innovative yet simplistic design of sleep mask, incorporated with some of the consummate technologies to ensure absolute and austere sleep for the cosmic travellers. The design houses vital sleep inducing red light therapy techniques put forth by some of the leading prolific researchers and sleep specialists. Due to the incorporated flexi – PCB, actively measuring significant human sensory organs, users can monitor and get customized Regenerative Learning solutions, aiding them in getting a titillating sleep experience.

Awards and Recognition:

• Global nominees
• People’s Choice Award
• Presented our paper titled Cosmorepophilae- an interstellar circadian rhythm synchronizer and sleep scheduling mechanism for deep space travellers at the Space Habitat Conference 2020.
• Software Stack: The github repo will soon be made publicly available post paper acceptance.

Zeoco revolves around the idea of quantifying the implicit effect of consuming a product or utility concerning the associated carbon footprint index.

Mentor: Ali De Jong, Apple Inc [USA]

Climate change, which is now considered one of the biggest threats to humanity, is also the reason behind various other environmental concerns. Continued negligence might lead us to an irreparably damaged environment. After the partial failure of the Paris Agreement, it is quite evident that we as individuals need to come together to bring about a change on a large scale to have a significant impact. This project discusses our approach towards obtaining a realistic measure of the carbon footprint index being consumed by a user through day-to-day activities performed via a smartphone app and offering incentives in weekly and monthly leader board rankings along with a reward system.

The app helps ease out decision makings on tasks like travel, shopping, electricity consumption, and gain a different and rather numerical perspective over the daily choices.

Code repository and report: Feel free to access our project using the given links:

Report: ArXiv

Source Code: https://github.com/L3thal14/Zeoco

Built Crack, a completely functional modern computer based on Hack.

Crack is a modern computer built from scratch using HDL from first principles which can handle assembly-level codes or machine level codes and execute them accordingly. The results were significantly good and the computer could handle games like pong which were written in a low level language.