Research
Statistical Inference for Data Collected with Bandit/RL Algorithms
Statistical Inference for Adaptive Experimentation
Kelly W. Zhang
Ph.D. Thesis, 2023
[pdf]
Statistical Inference After Adaptive Sampling for Longitudinal Data
Kelly W. Zhang, Lucas Janson, Susan A. Murphy
Working paper
[arXiv]
Statistical Inference with M-Estimators on Adaptively Collected Data
Kelly W. Zhang, Lucas Janson, Susan A. Murphy
NeurIPS 2021
Preliminary version at ICML 2021 Workshop on Reinforcement Learning Theory
[arXiv]
[proceedings]
[PubMed]
[video]
[slides]
[poster]
[code]
Inference for Batched Bandits
Kelly W. Zhang, Lucas Janson, Susan A. Murphy
NeurIPS 2020
[arXiv]
[proceedings]
[PubMed]
[video]
[slides]
[poster]
[code]
Designing and Evaluating Reinforcement Learning Algorithms
A mobile health intervention for emerging adults with regular cannabis use: A micro-randomized pilot trial design protocol.
Lara N. Coughlin, Maya Campbell, Tiffany Wheeler, Chavez Rodriguez, Autumn R. Florimbio, Susobhan Ghosh, Yongyi Guo, Pei-Yao Hung, Kelly W. Zhang, Lauren Zimmerman, Erin E. Bonar, Maureen A. Walton, Susan A. Murphy, Inbal Nahum-Shani.
Under submission
Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical Trials
Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Iris Yan, Finale Doshi-Velez, Susan A. Murphy
Under submission
[arXiv]
Did we personalize? Assessing personalization by an online reinforcement learning algorithm using resampling
Susobhan Ghosh*, Raphael Kim*, Prasidh Chhabria, Raaz Dwivedi, Predrag Klasnja, Peng Liao, Kelly W. Zhang, Susan A. Murphy
Machine Learning Journal: Special Issue on Reinforcement Learning for Real Life (to appear)
[arXiv]
Optimizing an adaptive Digital Oral Health Intervention for promoting Oral Self Care Behaviors: Micro-Randomized Trial Protocol
Inbal Nahum-Shani, Zara M. Greer, Anna L. Trella, Kelly W. Zhang, Stephanie Carpenter, David Elashoff, Susan A. Murphy, Vivek Shetty
Contemporary Clinical Trials, 2024
[PubMed]
[ClinicalTrials.gov]
Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care
Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, Susan A. Murphy
Thirty-Fifth Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-23)
[arXiv]
[code]
Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-implementation Guidelines
Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, Susan A. Murphy
Algorithms 2022 (Special Issue "Algorithms in Decision Support Systems Vol. 2")
Preliminary version at RLDM 2022 (Multi-disciplinary Conference on RL and Decision Making); selected for an oral
[arXiv]
[proceedings]
[code]
A Bayesian Approach to Learning Bandit Structure in Markov Decision Processes
Kelly W. Zhang, Omer Gottesman, Finale Doshi-Velez
Challenges of Real-World Reinforcement Learning 2020 (NeurIPS Workshop)
[arXiv]
[proceedings]
[video]
Natural Language Processing
Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis
Kelly Zhang and Samuel Bowman
BlackboxNLP 2018 (EMNLP Workshop)
[arXiv]
[proceedings]
Adversarially Regularized Autoencoders
Junbo (Jake) Zhao, Yoon Kim, Kelly Zhang, Alexander Rush, Yann LeCun.
ICML 2018
[arXiv]
[proceedings]
[code]
Technical Notes
- Why the sample mean can be biased on adaptively collected data (three sample illustration)
- Finite horizon RL and Least-Squares Value Iteration
- Least-Squares Methods in Batch RL