I’m a Computer Science Ph.D. student at UC Santa Cruz, advised by Liting Hu. I’m also an engineer at Tensormesh and a core committer on LMCache and a maintainer of the vLLM production stack.

My research is about building the next-generation agent infrastructure where agent execution semantics and state are first-class systems abstractions. By exposing and learning these semantics, systems can make better decisions about resource scheduling, state management, computational reuse, and eventually large-scale agent learning.

I’ve worked on:

• learning agent execution to manage state (KV-cache eviction and prefetching, joint compression and eviction, lossless compressed decoding)
• runtimes that expose agent semantics (policy-driven serving layer, LMCache)
• resource management across agents (fairness for multi-modal and multi-agent applications, vLLM production stack)

Previously, I was a software engineer at Huawei and a research intern at Bosch. I received my B.S. from Huazhong University of Science and Technology in 2023, and spent a summer at the National University of Singapore.

• The LMCache Coordinator: a control plane for fleet-wide KV cache

  9 October 2026

  LMCache's MP mode gives every node its own KV cache server, but each server sees only itself. The coordinator is the control plane that makes them act as one cache: it builds a shared, fleet-wide view from cache events and lets pluggable policies act on it.

All posts →

Full list on Google Scholar.

2026

Learning Agent Execution for KV-Cache Management in Agentic Serving

Rui Zhang, Chaeeun Kim, Shaoting Feng, Kuntai Du, Yuhan Liu, Yi Zhong, Cheng-Wei Ching, Junchen Jiang, Liting Hu

PreprintarXiv:2608.14624

abs pdf

A Policy-Driven Runtime Layer for Agentic LLM Serving

Rui Zhang, Chaeeun Kim, Liting Hu

APSysACM SIGOPS Asia-Pacific Workshop on Systems

abs pdf

VeriCache: Turning Lossy KV Cache into Lossless LLM Inference

Jiayi Yao, Samuel Shen, Kuntai Du, Shaoting Feng, Dongjoo Seo, Rui Zhang, Yuyang Huang, Yuhan Liu, Shan Lu, Junchen Jiang

PreprintarXiv:2605.17613

abs pdf

2025

EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Serving

Shaoting Feng, Yuhan Liu, Hanchen Li, Xiaokun Chen, Samuel Shen, Kuntai Du, Zhuohan Gu, Rui Zhang, Yuyang Huang, Yihua Cheng, Jiayi Yao, Qizheng Zhang, Ganesh Ananthanarayanan, Junchen Jiang

PreprintarXiv:2512.14946

abs pdf

LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference

Yuhan Liu, Yihua Cheng, Jiayi Yao, Yuwei An, Xiaokun Chen, Shaoting Feng, Yuyang Huang, Samuel Shen, Rui Zhang, Kuntai Du, Junchen Jiang

PreprintarXiv:2510.09665

abs pdf code

Enabling Fairness Across Multi-modal and Multi-agent Applications

Rui Zhang, Liting Hu

IEEE EDGEInternational Conference on Edge Computing and Communications