17KB
Inference Engineering — Chapter-11: Appendix B: Recommended Reading
- Source: /library/Inference Engineering/source-file.pdf
- PDF pages: 233–258
- Pages without text: 234, 258
APPENDIX B Recommended Reading
Recommended Reading 233 This book is an introduction to the field of inference engineering. There is endless depth to explore in every one of the technologies and techniques behind performant inference at scale. If you’re in the market for another book to continue learning, I have three recommendations: • AI Engineering: Building Applications with Foundation Models by Chip Huyen (O’Reilly Media, 2025): This incredibly popular book introduces the full breadth of AI engineering topics. • Build a Large Language Model (From Scratch) by Sebastian Raschka (Manning, 2024): This hands-on book provides a detailed look at LLM architecture. • AI Systems Performance Engineering: Optimizing Model Training and Inference Workloads with GPUs, CUDA, and PyTorch by Chris Fregly (O’Reilly Media, 2025): This brand-new book focuses on building for performance. The AI industry moves fast, and new models, research, and imple- mentations are constantly being released. My colleagues and I pub- lish our latest work on the Baseten blog, which you can access at https://www.baseten.com/blog. This appendix provides a list of papers, documentation, books, and blogs to further support your next steps as an inference engineer. Resources are organized by topic and alphabetized by title within each section.
234 Recommended Reading Architecture “Attention is All You Need,” by Ashish Vaswani et al. (Neural Information Processing Systems, 2017), https:// arxiv.org/abs/1706.03762 “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” by Jacob Devlin et al. (North American Chapter of the Association for Computational Linguistics, 2019), https://arxiv.org/abs/1810.04805 “BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models,” by Junnan Li et al. (International Conference on Machine Learning, 2023), https://arxiv.org/abs/2301.12597 Deep Learning, by Ian Goodfellow, Yoshua Bengio, and Aaron Courville (The MIT Press, 2016), https://www. deeplearningbook.org/ Deep Learning with Python (2nd Edition), by François Chollet (Manning, 2021), https://www.manning.com/books/ deep-learning-with-python-second-edition “Denoising Diffusion Probabilistic Models,” by Jonathan Ho et al. (ArXiv abs/2006.11239, 2020), https://arxiv.org/ abs/2006.11239
Recommended Reading 235 “DiT: Scalable Diffusion Models with Transformers,” by William Peebles and Saining Xie (International Conference on Computer Vision (ICCV), 2022), https://arxiv.org/ abs/2212.09748 “FlashAttention: Fast and Memory-Efficient Exact Attention with IO Awareness,” by Tri Dao et al. (ArXiv abs/2205.14135, 2022), https://arxiv.org/abs/2205.14135 “FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning,” by Tri Dao (ArXiv abs/2307.08691, 2023), https://arxiv.org/abs/2307.08691 “FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision,” by Jay Shah et al. (ArXiv abs/2407.08608, 2024), https://arxiv.org/abs/2407.08608 Flash-Attention-4, by Tri Dao (Dao AI Research Lab, 2025), https://github.com/Dao-AILab/flash-attention “Imagen Video: High Definition Video Generation with Diffusion Models,” by Jonathan Ho et al. (ArXiv abs/2210.02303, 2022), https://arxiv.org/abs/2210.02303
236 Recommended Reading “Language Models Are Few-Shot Learners,” by Tom Brown et al. (ArXiv abs/2005.14165), https://arxiv.org/ abs/2005.14165 “Learning Transferable Visual Models from Natural Language Supervision,” by Alec Radford et al. (International Conference on Machine Learning, 2021), https://arxiv.org/abs/2103.00020 “Longformer: The Long-Document Transformer,” by Iz Beltagy et al. (ArXiv abs/2004.05150, 2020), https://arxiv. org/abs/2004.05150 “Mamba: Linear-Time Sequence Modeling with Selective State Spaces,” by Albert Gu and Tri Dao (ArXiv abs/2312.00752, 2023), https://arxiv.org/abs/2312.00752 “Matryoshka Representation Learning,” by Aditya Kusupati et al. (Neural Information Processing Systems, 2022), https://arxiv.org/abs/2205.13147 “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer,” by Noam Shazeer et al. (ArXiv abs/1701.06538, 2017), https://arxiv.org/abs/1701.06538
Recommended Reading 237 “Reformer: The Efficient Transformer,” by Nikita Kitaev et al. (ArXiv abs/2001.04451, 2020), https://arxiv.org/ abs/2001.04451 “Robust Speech Recognition via Large-Scale Weak Supervision,” by Alec Radford et al. (International Conference on Machine Learning, 2022), https://arxiv.org/ abs/2212.04356 “RoFormer: Enhanced Transformer with Rotary Position Embedding,” by Jianlin Su et al. (ArXiv abs/2104.09864, 2021), https://arxiv.org/abs/2104.09864 “SDXL: Improving Latent Diffusion Models for High- Resolution Image Synthesis,” by Dustin Podell et al. (ArXiv abs/2307.01952, 2023), https://arxiv.org/abs/2307.01952 “Segment Anything,” by Alexander Kirillov et al. (2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023), https://arxiv.org/abs/2304.02643 “Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks,” by Nils Reimers and Iryna Gurevych (ArXiv abs/1908.10084, 2019), https://arxiv.org/ abs/1908.10084
238 Recommended Reading “The Llama 3 Herd of Models,” by Aaron Grattafiori et al. (ArXiv 2407.21783, 2024), https://arxiv.org/abs/2407.21783 “Video Diffusion Models,” by Jonathan Ho et al. (ArXiv abs/2204.03458, 2022), https://arxiv.org/abs/2204.03458 “Visual Instruction Tuning,” by Haotian Liu et al. (ArXiv abs/2304.08485, 2023), https://arxiv.org/abs/2304.08485
Recommended Reading 239 Developer Tools BitsAndBytes by bitsandbytes-foundation, https://github. com/TimDettmers/bitsandbytes ComfyUI by comfyanonymous, https://github.com/ comfyanonymous/ComfyUI “CUDA by Example: An Introduction to General Purpose GPU Programming,” by Jason Sanders and Edward Kandrot (NVIDIA developer, 2025), https://developer. nvidia.com/cuda-example “CUDA C++ Programming Guide Release 13.0,” by NVIDIA (2025), https://docs.nvidia.com/cuda/cuda-c-programming- guide/ “CUDA cuBLAS Release 13.0,” by NVIDIA (2025), https:// docs.nvidia.com/cuda/cublas/ CUTLASS by NVIDIA, https://github.com/NVIDIA/cutlass
240 Recommended Reading DeepGEMM by DeepSeek-ai, https://github.com/ deepseek-ai/DeepGEMM Hugging Face Diffusers by Hugging Face, https:// huggingface.co/docs/diffusers/index LMCache by LMCache Project, https://github.com/ LMCache/LMCache “NVIDIA Dynamo Documentation,” by NVIDIA (2025), https://docs.nvidia.com/dynamo/latest/index.html NVIDIA Nsight Systems by NVIDIA, https://developer. nvidia.com/nsight-systems NVIDIA Triton Inference Server by NVIDIA, https://github. com/triton-inference-server/server
Recommended Reading 241 ONNX Runtime by Microsoft, https://onnxruntime.ai/ “PyTorch Performance Tuning Guide,” by Szymon Migacz (PyTorch Foundation, 2020), https://pytorch.org/tutorials/ recipes/recipes/tuning_guide.html “PyTorch Profiler,” by Shivam Raikundalia (PyTorch Foundation, 2021), https://pytorch.org/tutorials/recipes/ recipes/profiler_recipe.html SGLang Project by LMSYS Org, https://github.com/sgl- project/sglang “TensorRT Documentation,” by NVIDIA (2025), https:// docs.nvidia.com/deeplearning/tensorrt/ TensorRT-LLM by NVIDIA, https://github.com/NVIDIA/ TensorRT-LLM
242 Recommended Reading Transformers by Hugging Face, https://huggingface.co/ docs/transformers/index vLLM Project by The Linux Foundation, https://github.com/ vllm-project/vllm
Recommended Reading 243 Frontier Open Models DeepSeek, by DeepSeek AI, https://huggingface.co/ deepseek-ai FLUX, by Black Forest Labs, https://huggingface.co/black- forest-labs Gemma, by Google, https://huggingface.co/google GLM, by Z.ai, https://huggingface.co/zai-org GPT OSS, by OpenAI, https://huggingface.co/openai Kimi, by Moonshot AI, https://huggingface.co/moonshotai
244 Recommended Reading Llama, by Meta Llama, https://huggingface.co/meta-llama MiniMax, by MiniMax AI, https://huggingface.co/MiniMaxAI Mistral, by Mistral AI, https://huggingface.co/mistralai Nemotron, by NVIDIA, https://huggingface.co/nvidia Orpheus, by Canopy Labs, https://huggingface.co/ canopylabs Qwen, by Alibaba Qwen, https://huggingface.co/Qwen
Recommended Reading 245 Wan, by Wan-AI, https://huggingface.co/Wan-AI Whisper, by OpenAI, https://huggingface.co/openai
246 Recommended Reading GPU Infrastructure Designing Data-Intensive Applications, by Martin Kleppmann (O’Reilly Media, 2017) https://dataintensive. net/ Grace Hopper / Grace Blackwell Systems by NVIDIA, https://www.nvidia.com/en-us/data-center/grace-cpu/ GPU Glossary, by Frye et al. (Modal, 2025), https://modal. com/gpu-glossary InfiniBand, by NVIDIA, https://www.nvidia.com/en-us/ networking/products/infiniband/ Kubernetes Documentation, by The Kubernetes Authors (The Linux Foundation, 2025), https://kubernetes.io/docs/ home/
Recommended Reading 247 NVIDIA Blackwell Architecture Technical Brief: Built for the Age of AI Reasoning, (NVIDIA, 2025), https://resources. nvidia.com/en-us-blackwell-architecture?ncid=no-ncid NVIDIA H100 Tensor Core GPU Architecture: Exceptional Performance, Scalability and Security for the Data Center, (NVIDIA, 2023), https://resources.nvidia.com/en-us- hopper-architecture/nvidia-h100-tensor-c “NVIDIA Tesla: A Unified Graphics and Computing Architecture,” by E. Lindholm et al. (IEEE Micro, March– April 2008), https://ieeexplore.ieee.org/document/4523358 NVLink / NVSwitch, by NVIDIA, https://www.nvidia.com/ en-us/data-center/nvlink/ Programming Massively Parallel Processors: A Hands-on Approach, by Wen-mei Hwu, David Kirk, Izzat El Hajj (Morgan Kaufmann, 2022), https://www.goodreads.com/ work/editions/10244675-programming-massively-parallel- processors-a-hands-on-approach SemiAnalysis, by Dylan Patel (SemiAnalysis, 2025), https://semianalysis.com/
248 Recommended Reading Site Reliability Engineering: How Google Runs Production Services, edited by Betsy Beyer et al. (O’Reilly Media, 2017), https://sre.google/books/
Recommended Reading 249 Inference Optimization Research “Adversarial Diffusion Distillation,” by Axel Sauer et al. (European Conference on Computer Vision, 2023), https:// arxiv.org/abs/2311.17042 “Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation,” by Ofir Press, Noah Smith, and Mike Lewis (ArXiv abs/2108.12409, 2021), https://arxiv.org/abs/2108.12409 “AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration,” by Song Han (MIT, 2024), https://github.com/mit-han-lab/llm-awq Cache-DIT by Vipshop, https://github.com/vipshop/cache- dit “CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion,” by Jiayi Yao et al. (Proceedings of the Twentieth European Conference on Computer Systems, 2024), https://arxiv.org/ abs/2405.16444
250 Recommended Reading “Adding Conditional Control to Text-to-Image Diffusion Models,” by Lymin Zhang et al. (International Conference on Computer Vision, 2023), https://arxiv.org/ abs/2302.05543 “Beyond the Buzz: A Pragmatic Take on Inference Disaggregation,” by Tiyasa Mitra et al. (ArXiv abs/2506.05508, 2025), https://arxiv.org/abs/2506.05508 “Break the Sequential Dependency of LLM Inference Using Lookahead Decoding,” by Yichao Fu et al. (ArXiv abs/2402.02057, 2024), https://arxiv.org/abs/2402.02057 “EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty,” by Yuhui Li et al. (ArXiv abs/2401.15077, 2024), https://arxiv.org/abs/2401.15077 “EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees,” by Yuhui Li et al. (Conference on Empirical Methods in Natural Language Processing, 2024), https://arxiv.org/abs/2406.16858 “EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test,” by Yuhui Li et al. (ArXiv abs/2503.01840, 2025), https://arxiv.org/ abs/2503.01840
Recommended Reading 251 “FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving,” by Ye et al. (ArXiv abs/2501.01005, 2025), https://arxiv.org/abs/2501.01005 “GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers,” by Elias Frantar (ArXiv abs/2210.17323, 2022), https://arxiv.org/ abs/2210.17323 “High-Resolution Image Synthesis with Latent Diffusion Models,” by Robin Rombach et al. (Conference on Computer Vision and Pattern Recognition (CVPR), 2021), https://arxiv.org/abs/2112.10752 “Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference,” by Simian Luo (ArXiv abs/2310.04378, 2023), https://arxiv.org/abs/2310.04378 “LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale,” by Tim Dettmers et al. (ArXiv abs/2208.07339, 2022), https://arxiv.org/abs/2208.07339 “Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads,” by Tianle Cai et al. (ArXiv abs/2401.10774, 2024), https://arxiv.org/abs/2401.10774
252 Recommended Reading “Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism,” by Mohammad Shoeybi et al. (ArXiv abs/1909.08053, 2019), https://arxiv.org/ abs/1909.08053 “Efficient Memory Management for Large Language Model Serving with PagedAttention,” by Woosuk Kwon et al. (Proceedings of the 29th Symposium on Operating Systems Principles, 2023), https://arxiv.org/ abs/2309.06180 “Fast Inference from Transformers via Speculative Decoding,” by Yaniv Leviathan et al. (International Conference on Machine Learning, 2022), https://arxiv.org/ abs/2211.17192 “Ring Attention with Blockwise Transformers for Near- Infinite Context,” by Hao Lin et al. (ArXiv abs/2310.01889, 2023), https://arxiv.org/abs/2310.01889 “SageAttention: Accurate 8-Bit Attention for Plug-and- play Inference Acceleration,” by Jintao Zhang et al. (ArXiv abs/2410.02367, 2024), https://arxiv.org/abs/2410.02367 Sequence/Context Parallelism, by Megatron-LM for NVIDIA https://github.com/NVIDIA/Megatron-LM
Recommended Reading 253 SmoothQuant by Song Han (MIT), https://github.com/mit- han-lab/smoothquant “SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot,” by Elias Frantar and Dan Alistarh, (ArXiv abs/2301.00774, 2023), https://arxiv.org/ abs/2301.00774 “SpecVLM: Fast Speculative Decoding in Vision-Language Models,” Haiduo Huang et al. (ArXiv abs/2509.11815, 2025), https://arxiv.org/abs/2509.11815 “TeaCache: Efficient KV Cache Compression via Tensor Decomposition,” by Feng Lu et al. (Alibaba TongYi Vision Intelligence Lab, ArXiv abs/2411.19108, 2025), https:// github.com/ali-vilab/TeaCache
254 Recommended Reading Intelligence Evaluation ARC AGI Prize by Greg Kamradt (2025), https://arcprize. org/ Evals for AI Engineers: Systematically Measuring and Improving AI Applications, by Shreya Shankar and Hamel Husain (O’Reilly Media, forthcoming 2026) https://www. oreilly.com/library/view/evals-for-ai/9798341660717/ Grade School Math: Training Verifiers to Solve Math Word Problems, by Karl Cobbe and Vineet Kosaraju (ArXiv abs/2110.14168, 2021), https://github.com/openai/grade- school-math “How to Fine-Tune Qwen3 to GPT-4o Level Performance,” by Greg Schoeninger (Fine-Tune Fridays, Oxen AI, 2025), https://ghost.oxen.ai/how-to-fine-tune-qwen3-to-gpt-4o- level-performance/ “Humanity’s Last Exam,” by Long Phan et al. (Center for AI Safety and Scale AI, ArXiv abs/2501.14249, 2025), https:// www.lastexam.ai/ “HumanEval: Evaluating Large Language Models Trained on Code,” by Michelle Pokrass, Qiming Yuan, and Yichen Xu (OpenAI, 2021), https://github.com/openai/human-eval
Recommended Reading 255 “MMLU: Measuring Massive Multitask Language Understanding,” by Dan Hendrycks et al. (Proceedings of the International Conference on Learning Representations (ICLR), 2021), https://github.com/hendrycks/test “MTEB: Massive Text Embedding Benchmark,” by Niklas Muenninghoff et al. (Conference of the European Chapter of the Association for Computational Linguistics, 2022), https://arxiv.org/abs/2210.07316 “SWE-Bench: Can Language Models Resolve Real-World Github Issues?” by Carlos Jimenez et al. (Proceedings of the International Conference on Learning Representations (ICLR), 2024), https://www.swebench.com/