My research centers on understanding and improving the control of complex systems to develop robust and generalizable solutions. I am passionate about interdisciplinary research and have explored several distinct domains from reinforcement learning (RL) to visual neuroprosthetics. Despite their differences, these research endeavors share a common thread of developing machine learning models capable of effectively navigating uncertainty in training data to achieve reliable performance in unpredictable situations.

Reinforcement Learning and Generalization

RL has shown remarkable promise in solving complex tasks, but RL agents often struggle to adapt to new and unseen environments. My PhD research began with this generalization challenge in RL. This limitation stems from agents overfitting to the particulars of their training environments, and therefore failing to learn broadly applicable strategies. Existing regularization techniques, such as noise injection and parameter regularization, often fall short in mitigating this issue.

To address these limitations, I developed a Bayesian Neural Network (BNN) regularization technique. This novel approach leverages the parameter uncertainty of a BNN in RL to improve an agent’s ability to generalize. By constraining each parameter distribution, our method encourages the network to learn solutions that are both effective for the training data and robust to variations in their environment’s space.

Neural Control and Neuroprosthetics

I am applying generalization techniques to real-world complex systems, particularly neuroprosthetics. The ability to artificially control neural activity is a fundamental challenge for sensory neuroprostheses. Current approaches face scaling challenges with next-generation implants containing over 1,000 electrode sites.

To overcome these hurdles, I applied deep neural networks (DNN) to predict and control complex neural responses, leading to significantly improved control over visual neuroprosthesis. Our results demonstrated that DNN-optimized stimulation patterns significantly outperformed traditional methods in controlling neural activity.

Future Research Directions

My future research focuses on improving online adaptation in machine learning systems. Some areas I’m particularly interested in:

• Leveraging self-supervised learning to enhance adaptability
• Developing uncertainty-guided adaptive fine-tuning methods
• Building systems that can serve specific individuals rather than just average use cases
• Expanding into new interdisciplinary domains from neuroscience to finance

As AI becomes increasingly integrated into daily life, I aim to develop machine learning solutions that actively adapt to diverse needs not only resolve the “average use case.”