Book details
LLM Inference Handbook
Categories Artificial intelligence , LLMs
A technical handbook on running large language models in production, from initial model and hardware choices through deployment and serving. It covers inference fundamentals, performance metrics, GPU memory, quantization and fine-tuning, then optimization techniques such as batching, KV-cache management, speculative decoding, routing, and parallelism