🧰 Projects / Software

TokenPowerBench

Open-source benchmark suite

Measures LLM inference power consumption across single-node and multi-node GPU settings, with phase-aware metrics for prefill and decode.

LLM Inference Power Benchmarking Multi-Node GPU

Code| Paper| DOI

LLM Inference Engine Benchmark

Open-source benchmarking tool

Benchmarks energy consumption and efficiency trade-offs across LLM inference engines including vLLM, DeepSpeed, TensorRT-LLM, and Transformers.

LLM Systems Inference Engines Energy Efficiency

Code| Paper| DOI

Awesome Disaggregated LLM Serving

Curated research map

Organizes papers, systems, and open problems around prefill-decode disaggregation, KV-cache movement, MoE serving, and re-aggregation baselines.

LLM Serving Disaggregation KV Cache

Code