Forwarded from Frost's Notes
我们上线了一个LLM的wiki站，介绍了LLM的各种概念和生产环境上部署使用的各种优化方法。学不完，学不完啊
https://bentoml.com/llm/
Modular
LLM Inference Handbook
LLM Inference Handbook
A practical handbook for engineers building, optimizing, scaling and operating LLM inference systems in production.