Research

Statistical Inference for Data Collected with Bandit/RL Algorithms

Statistical Inference for Adaptive Experimentation
Kelly W. Zhang
Ph.D. Thesis, 2023
[pdf]

Statistical Inference After Adaptive Sampling for Longitudinal Data
Kelly W. Zhang, Lucas Janson, Susan A. Murphy
Working paper
[arXiv]

Statistical Inference with M-Estimators on Adaptively Collected Data
Kelly W. Zhang, Lucas Janson, Susan A. Murphy
NeurIPS 2021
Preliminary version at ICML 2021 Workshop on Reinforcement Learning Theory
[arXiv] [proceedings] [PubMed] [video] [slides] [poster] [code]

Inference for Batched Bandits
Kelly W. Zhang, Lucas Janson, Susan A. Murphy
NeurIPS 2020
[arXiv] [proceedings] [PubMed] [video] [slides] [poster] [code]


Designing and Evaluating Reinforcement Learning Algorithms

A mobile health intervention for emerging adults with regular cannabis use: A micro-randomized pilot trial design protocol.
Lara N. Coughlin, Maya Campbell, Tiffany Wheeler, Chavez Rodriguez, Autumn R. Florimbio, Susobhan Ghosh, Yongyi Guo, Pei-Yao Hung, Kelly W. Zhang, Lauren Zimmerman, Erin E. Bonar, Maureen A. Walton, Susan A. Murphy, Inbal Nahum-Shani.
Under submission

Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical Trials
Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Iris Yan, Finale Doshi-Velez, Susan A. Murphy
Under submission
[arXiv]

Did we personalize? Assessing personalization by an online reinforcement learning algorithm using resampling
Susobhan Ghosh*, Raphael Kim*, Prasidh Chhabria, Raaz Dwivedi, Predrag Klasnja, Peng Liao, Kelly W. Zhang, Susan A. Murphy
Machine Learning Journal: Special Issue on Reinforcement Learning for Real Life (to appear)
[arXiv]

Optimizing an adaptive Digital Oral Health Intervention for promoting Oral Self Care Behaviors: Micro-Randomized Trial Protocol
Inbal Nahum-Shani, Zara M. Greer, Anna L. Trella, Kelly W. Zhang, Stephanie Carpenter, David Elashoff, Susan A. Murphy, Vivek Shetty
Contemporary Clinical Trials, 2024
[PubMed] [ClinicalTrials.gov]

Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care
Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, Susan A. Murphy
Thirty-Fifth Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-23)
[arXiv] [code]

Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-implementation Guidelines
Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, Susan A. Murphy
Algorithms 2022 (Special Issue "Algorithms in Decision Support Systems Vol. 2")
Preliminary version at RLDM 2022 (Multi-disciplinary Conference on RL and Decision Making); selected for an oral
[arXiv] [proceedings] [code]

A Bayesian Approach to Learning Bandit Structure in Markov Decision Processes
Kelly W. Zhang, Omer Gottesman, Finale Doshi-Velez
Challenges of Real-World Reinforcement Learning 2020 (NeurIPS Workshop)
[arXiv] [proceedings] [video]


Natural Language Processing

Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis
Kelly Zhang and Samuel Bowman
BlackboxNLP 2018 (EMNLP Workshop)
[arXiv] [proceedings]

Adversarially Regularized Autoencoders
Junbo (Jake) Zhao, Yoon Kim, Kelly Zhang, Alexander Rush, Yann LeCun.
ICML 2018
[arXiv] [proceedings] [code]


Technical Notes