avatar

Zhili Feng

Researcher
OpenAI
fzl.007@gmail.com


About Me

I am a researcher at OpenAI.

Research Interests

I look at ML models as information processors – they take in some bits and produce some other bits. Their input and output patterns tell us many aspects of the model, including privacy, memorization, robustness, etc. I am also generally interested in information theory, compression, and many TCS fields like cryptography and online algorithms.

I am lucky to be advised by Prof. Zico Kolter.

Selected Publications

  1. Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws
    Yiding Jiang*, Allan Zhou*, Zhili Feng, Sadhika Malladi, J. Zico Kolter
    Under Review
  2. Adapting Language Models via Token Translation
    Zhili Feng, Tanya Marwah, Nicolo Fusi, David Alvarez-Melis, Lester Mackey
    Under Review
  3. Improving Zero-Shot Reasoning using Dynamic Non-Verbal Tokens
    Tanya Marwah, Zhili Feng, Lester Mackey, Nicolo Fusi, David Alvarez-Melis
    Under Review
  4. Rethinking LLM Memorization through the Lens of Adversarial Compression
    Avi Schwarzschild*, Zhili Feng*, Pratyush Maini, Zachary C Lipton, J. Zico Kolter
    Advances in Neural Information Processing Systems (NeurIPS), 2024
    • Best Paper Award, Workshop on Data Contamination (CONDA) @ ACL 2024
    • Oral Presentation, Red Teaming GenAI Workshop @ NeurIPS 2024
  5. TOFU: A Task of Fictitious Unlearning for LLMs
    Pratyush Maini*, Zhili Feng*, Avi Schwarzschild*, Zachary C Lipton, J. Zico Kolter
    Conference on Language Modeling, 2024
    • Spotlight presentation, Workshop on Secure and Trustworthy Large Language Models @ ICLR 2024
  6. Text Descriptions are Compressive and Invariant Representations for Visual Learning
    Zhili Feng, Anna Bair, J. Zico Kolter
    Transactions on Machine Learning Research (TMLR), 2024
  7. Monotone Deep Boltzmann Machines
    Zhili Feng, Ezra Winston, J. Zico Kolter
    Transactions on Machine Learning Research (TMLR), 2023
  8. On the Neural Tangent Kernel of Equilibrium Models
    Zhili Feng, J. Zico Kolter
    https://arxiv.org/abs/2310.14062
  9. Provable Adaptation across Multiway Domains via Representation Learning
    Zhili Feng, Shaobo Han, Simon S Du
    International Conference on Learning Representations (ICLR), 2022
  10. Non-PSD Matrix Sketching with Applications to Regression and Optimization
    Zhili Feng, Fred Roosta, David P Woodruff
    Conference on Uncertainty in Artificial Intelligence (UAI), 2021
  11. Learning-Augmented k-means Clustering
    Jon Ergun, Zhili Feng, Sandeep Silwal, David P Woodruff, Samson Zhou
    International Conference on Learning Representations (ICLR), 2022
    • Spotlight Presentation
  12. Dimensionality Reduction for the Sum-of-Distances Metric
    Zhili Feng, Praneeth Kacham, David P Woodruff
    International Conference on Machine Learning (ICML), 2021
    • Oral Presentation
  13. Does Data Augmentation Lead to Positive Margin?
    Shashank Rajput*, Zhili Feng*, Zachary B Charles, Po-Ling Loh, Dimitris S Papailiopoulos
    International Conference on Machine Learning (ICML), 2019
    • Oral Presentation
  14. Online Learning with Graph-Structured Feedback against Adaptive Adversaries
    Zhili Feng, Po-Ling Loh
    IEEE International Symposium on Information Theory (ISIT), 2018
  15. Joint Reasoning for Temporal and Causal Relations
    Qiang Ning, Zhili Feng, Hao Wu, Dan Roth
    Annual Meeting of the Association for Computational Linguistics (ACL), 2018
  16. CogCompTime: A Tool for Understanding Time in Natural Language
    Qiang Ning, Ben Zhou, Zhili Feng, Haoruo Peng, Dan Roth
    Empirical Methods in Natural Language Processing (EMNLP), 2018
  17. A Structured Learning Approach to Temporal Relation Extraction
    Qiang Ning, Zhili Feng, Dan Roth
    Empirical Methods in Natural Language Processing (EMNLP), 2017