preprints

2023

1. 

   ICE: Information Context Exploration for Sparse MDP’s

   J. Chmura, H. Burhani, and X. Shi

   2023

   Abs arXiv

   Sparse reward environments are known to be challenging for reinforcement learning agents. In such environments, efficient and scalable exploration is crucial. Exploration is a means by which an agent gains information about the environment. We expand on this topic and propose a new intrinsic reward that systemically quantifies exploratory behavior and promotes state coverage by maximizing the information content of a trajectory taken by an agent. We compare our method to alternative exploration based intrinsic reward techniques, namely Curiosity Driven Learning and Random Network Distillation. We show that our information theoretic reward induces efficient exploration and outperforms in various games, including Montezuma Revenge, a known difficult task for reinforcement learning. Finally, we propose an extension that maximizes information content in a discretely compressed latent space which boosts sample efficiency and generalizes to continuous state spaces.

conference & journal articles

2020

1. 

   Genome Gerrymandering: optimal division of the genom einto regions with cancer type specific differences in mutation rates

   A. Young, J. Chmura, Y. Park, Q. Morris, and G. Atwal

   PubMed, 2020

   Abs HTML PDF

   The activity of mutational processes differs across the genome, and is influenced by chromatin state and spatial genome organization. At the scale of one megabase-pair (Mb), regional mutation density correlate strongly with chromatin features and mutation density at this scale can be used to accurately identify cancer type. Here, we explore the relationship between genomic region and mutation rate by developing an information theory driven, dynamic programming algorithm for dividing the genome into regions with differing relative mutation rates between cancer types. Our algorithm improves mutual information when compared to the naive approach, effectively reducing the average number of mutations required to identify cancer type. Our approach provides an efficient method for associating regional mutation density with mutation labels, and has future applications in exploring the role of somatic mutations in a number of diseases.

talks

2019

1. 

   Learning Feature Importance for a Deep Learning Classifier

   J. Chmura

   2019

   PDF

patents

2023

1. 

   System and Method: Multi-Objective Reinforcement Learning for Personalized Client Execution

   M. Azam, J. Chmura, H. Huang, and Z. Yu

   2023

   Abs HTML PDF

   Systems are methods are provided for processing multiple input objectives by a reinforcement learning agent. The method may include: instantiating a reinforcement learning agent that maintains a reinforcement learning neural network and generates, according to outputs of the reinforcement learning neural network, signals for communicating task requests; receiving a plurality of input data representing a plurality of user objectives associated with a task request; generating, based on the reinforcement learning neural network and the plurality of input data, an action output for generating a signal for communicating the task request; computing a reward based on the action output and the plurality of input data; and updating the reinforcement learning neural network based on the reward.

2. 

   System and Method: Multi-Objective Reinforcement Learning with gradient modulation

   M. Azam, J. Chmura, H. Huang, and Z. Yu

   2023

   Abs HTML PDF

   Systems are methods are provided for processing multiple input objectives by a reinforcement learning agent. The method may include: instantiating a reinforcement learning agent that maintains a reinforcement learning neural network and generates, according to outputs of the reinforcement learning neural network, signals for communicating task requests; receiving a plurality of input data representing a plurality of user objectives associated with a task request and a plurality of weights; generating a plurality of preferences based on the plurality of user objectives and the plurality of weights; computing a plurality of loss values; computing a plurality of first gradients based on the plurality of loss values; for a plurality of pairs of references, computing a plurality of similarity metrics; computing an updated gradient based on the first gradients and the plurality of similarity metrics; and updating the reinforcement learning neural network based on the updated gradient.