1. 它怎样组织论文，Mainline 怎样接入
上游分成三层：SUMMARY.md 是总导航，Systems-for-ML 页面按问题分类，会议、年份和单篇笔记继续向下展开。题名会链接 Paper、arXiv 或 Code 等一手材料。Mainline 用本页承接目录和路由，01–41 页负责推理、KV、调度、网络、MoE、训练与 RL 等专题解释。
本次记录取自 upstream develop 的 commit ccaaf4af79d5c711d0053553684f79a441106fd6。日期是 2026-07-15；上游更新后，本页不会自动同步。
2. Systems-for-ML 目录快照
显示 226 / 226 条可点击目录记录。
| 序号 | 论文标题与上游分组 | Mainline track | 原始链接 | 上游文件 |
|---|---|---|---|---|
| 1 | Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices (ICDCS 2017) Automatic Graph Partitioning |
ML systems | Paper Code | cloud edge collaboration |
| 2 | Dynamic Adaptive DNN Surgery for Inference Acceleration on the Edge (INFOCOM 2019) Automatic Graph Partitioning |
ML systems | Paper | cloud edge collaboration |
| 3 | Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge Automatic Graph Partitioning |
ML systems | Paper | cloud edge collaboration |
| 4 | SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud (MobiCom 2020) Automatic Graph Partitioning |
ML systems | Paper | cloud edge collaboration |
| 5 | Walle: An End-to-End, General-Purpose, and Large-Scale Production System for Device-Cloud Collaborative Machine Learning Framework |
compiler / runtime | Paper Code | cloud edge collaboration |
| 6 | A case for disaggregation of ML data processing (arXiv 2210.14826) Uncategorized |
ML systems | Paper | data processing |
| 7 | Disaggregating ML Input Data Processing at Scale Uncategorized |
ML systems | 上游条目未列直接链接 | data processing |
| 8 | GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning (SIGMOD 2023) Uncategorized |
ML systems | Paper | data processing |
| 9 | Pecan: Cost-Efficient ML Data Preprocessing with Automatic Transformation Ordering and Hybrid Placement Uncategorized |
ML systems | Paper Code | data processing |
| 10 | Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training (ISCA 2022) Uncategorized |
ML systems | Paper | data processing |
| 11 | DSA: Domain-Specific Architecture Acronyms |
compiler / runtime | 上游条目未列直接链接 | deep learning compiler |
| 12 | MLIR: Scaling Compiler Infrastructure for Domain Specific Computation (CGO 2021) System Architecture |
compiler / runtime | Paper Homepage | deep learning compiler |
| 13 | TVM: An Automated End-to-End Optimizing Compiler for Deep Learning System Architecture |
compiler / runtime | Paper Code Homepage | deep learning compiler |
| 14 | AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures (ASPLOS 2022) Tensor Program Generation / General Tensor Program Generation |
compiler / runtime | Paper | deep learning compiler |
| 15 | Ansor: Generating High-Performance Tensor Programs for Deep Learning Tensor Program Generation / General Tensor Program Generation |
compiler / runtime | Paper | deep learning compiler |
| 16 | Cocktailer: Analyzing and Optimizing Dynamic Control Flow in Deep Learning Tensor Program Generation / General Tensor Program Generation |
compiler / runtime | Paper | deep learning compiler |
| 17 | EINNET: Optimizing Tensor Programs with Derivation-Based Transformations Tensor Program Generation / General Tensor Program Generation |
compiler / runtime | Paper | deep learning compiler |
| 18 | Effectively Scheduling Computational Graphs of Deep Neural Networks toward Their Domain-Specific Accelerators Tensor Program Generation / General Tensor Program Generation |
compiler / runtime | Paper | deep learning compiler |
| 19 | Welder: Scheduling Deep Learning Memory Access via Tile-graph Tensor Program Generation / General Tensor Program Generation |
compiler / runtime | Paper | deep learning compiler |
| 20 | Event Tensor: A Unified Abstraction for Compiling Dynamic Megakernel Tensor Program Generation / Megakernel Compilation |
compiler / runtime | Paper arXiv | deep learning compiler |
| 21 | Mirage Persistent Kernel: A Compiler and Runtime for Mega-Kernelizing Tensor Programs (arXiv:2512.22219) Tensor Program Generation / Megakernel Compilation |
compiler / runtime | arXiv Code Homepage | deep learning compiler |
| 22 | An Introduction to Computational Networks and the Computational Network Toolkit (MSR-TR-2014-112) Uncategorized |
compiler / runtime | Paper Code Homepage | deep learning framework |
| 23 | Caffe: Convolutional Architecture for Fast Feature Embedding (arXiv 1408.5093) Uncategorized |
compiler / runtime | Paper Homepage Code | deep learning framework |
| 24 | Jittor: a novel deep learning framework with meta-operators and unified graph execution (Science China Information Sciences 2020) Uncategorized |
compiler / runtime | Paper Code | deep learning framework |
| 25 | MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems (NIPS 2016 Workshop on MLSys) Uncategorized |
compiler / runtime | Paper Homepage Code | deep learning framework |
| 26 | OneFlow: Redesign the Distributed Deep Learning Framework from Scratch (arXiv 2110.15032) Uncategorized |
compiler / runtime | Paper Code | deep learning framework |
| 27 | Pathways: Asynchronous Distributed Dataflow for ML (MLSys 2022) Uncategorized |
compiler / runtime | Paper | deep learning framework |
| 28 | PyTorch: An Imperative Style, High-Performance Deep Learning Library (NeurIPS 2019) Uncategorized |
compiler / runtime | Paper Code Homepage | deep learning framework |
| 29 | TensorFlow: A System for Large-Scale Machine Learning (OSDI 2016) Uncategorized |
compiler / runtime | Paper Code Homepage | deep learning framework |
| 30 | XDL: An Industrial Deep Learning Framework for High-dimensional Sparse Data (DLP-KDD 2019) Uncategorized |
compiler / runtime | Paper Code | deep learning framework |
| 31 | EasyScale: Elastic Training with Consistent Accuracy and Improved Utilization on GPUs Elastic Training |
training / RL | Paper Code | deep learning training |
| 32 | A Unified Architecture for Accelerating Distributed DNN Training in Heterogeneous GPU/CPU Clusters Optimizing Network Communication |
training / RL | Paper Code | deep learning training |
| 33 | Hanayo: Harnessing Wave-like Pipeline Parallelism for Enhanced Large Model Training Efficiency Parallelism |
training / RL | Paper Code | deep learning training |
| 34 | One weird trick for parallelizing convolutional neural networks (arXiv 1404.599) Parallelism |
training / RL | Paper | deep learning training |
| 35 | Supporting Very Large Models using Automatic Dataflow Graph Partitioning Parallelism |
training / RL | Paper | deep learning training |
| 36 | Echo: Compiler-based GPU Memory Footprint Reduction for LSTM RNN Training (ISCA 2020) Reduce GPU Memory Footprints / Compression |
training / RL | Paper | deep learning training |
| 37 | Gist: Efficient Data Encoding for Deep Neural Network Training (ISCA 2018) Reduce GPU Memory Footprints / Compression |
training / RL | Paper | deep learning training |
| 38 | Gandiva: Introspective Cluster Scheduling for Deep Learning Reduce GPU Memory Footprints / GPU Sharing |
training / RL | Paper | deep learning training |
| 39 | Salus: Fine-Grained GPU Sharing Primitives for Deep Learning Applications Reduce GPU Memory Footprints / GPU Sharing |
training / RL | Paper Code | deep learning training |
| 40 | Zico: Efficient GPU Memory Sharing for Concurrent DNN Training Reduce GPU Memory Footprints / GPU Sharing |
training / RL | Paper | deep learning training |
| 41 | Capuchin: Tensor-based GPU Memory Management for Deep Learning Reduce GPU Memory Footprints / Tensor Swapping / Recomputation |
training / RL | Paper | deep learning training |
| 42 | Checkmate: Breaking the Memory Wall with Optimal Tensor Rematerialization Reduce GPU Memory Footprints / Tensor Swapping / Recomputation |
training / RL | Paper Code | deep learning training |
| 43 | SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networks (PPoPP 2018) Reduce GPU Memory Footprints / Tensor Swapping / Recomputation |
training / RL | Paper | deep learning training |
| 44 | SwapAdvisor: Pushing Deep Learning Beyond the GPU Memory Limit via Smart Swapping Reduce GPU Memory Footprints / Tensor Swapping / Recomputation |
training / RL | Paper | deep learning training |
| 45 | Training Deep Nets with Sublinear Memory Cost (arXiv 1604.06174) Reduce GPU Memory Footprints / Tensor Swapping / Recomputation |
training / RL | Paper Code | deep learning training |
| 46 | vDNN: Virtualized Deep Neural Networks for Scalable, Memory-Efficient Neural Network Design (MICRO 2016) Reduce GPU Memory Footprints / Tensor Swapping / Recomputation |
training / RL | Paper | deep learning training |
| 47 | DiT: Diffusion Transformer Acronyms |
diffusion / video | 上游条目未列直接链接 | diffusion models |
| 48 | CAT-DM: Controllable Accelerated Virtual Try-on with Diffusion Model Diffusion Model Serving / Image Editing |
diffusion / video | Paper Code | diffusion models |
| 49 | FlashPS: Efficient Generative Image Editing with Mask-aware Caching and Scheduling Diffusion Model Serving / Image Editing |
diffusion / video | Paper arXiv Code | diffusion models |
| 50 | Approximate Caching for Efficiently Serving Text-to-Image Diffusion Models Diffusion Model Serving / Image Generation |
diffusion / video | Paper Slides | diffusion models |
| 51 | Cache Me if You Can: Accelerating Diffusion Models through Block Caching Diffusion Model Serving / Image Generation |
diffusion / video | Paper Homepage | diffusion models |
| 52 | DeepCache: Accelerating Diffusion Models for Free Diffusion Model Serving / Image Generation |
diffusion / video | Paper Code | diffusion models |
| 53 | DiFlow: A System for Micro-Serving Text-to-Image Diffusion Workflows Diffusion Model Serving / Image Generation |
diffusion / video | arXiv | diffusion models |
| 54 | DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models Diffusion Model Serving / Image Generation |
diffusion / video | Paper Code | diffusion models |
| 55 | Katz: Efficient Workflow Serving for Diffusion Models with Many Adapters Diffusion Model Serving / Image Generation |
diffusion / video | Paper arXiv Code | diffusion models |
| 56 | MixFusion: A Patch-Level Parallel Serving System for Mixed-Resolution Diffusion Models Diffusion Model Serving / Image Generation |
diffusion / video | Paper arXiv Code | diffusion models |
| 57 | PipeFusion: Displaced Patch Pipeline Parallelism for Inference of Diffusion Transformer Models (arXiv:2405.14430) Diffusion Model Serving / Image Generation |
diffusion / video | arXiv Code | diffusion models |
| 58 | xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism (arXiv:2411.01738) Diffusion Model Serving / Image Generation |
diffusion / video | arXiv Code | diffusion models |
| 59 | Fast Video Generation with Sliding Tile Attention Diffusion Model Serving / Video Generation |
diffusion / video | Paper OpenReview arXiv Code | diffusion models |
| 60 | FlexCache: Flexible Approximate Cache System for Video Diffusion (arXiv:2501.04012) Diffusion Model Serving / Video Generation |
diffusion / video | arXiv | diffusion models |
| 61 | DiffusionPipe: Training Large Diffusion Models with Efficient Pipelines Diffusion Model Training |
diffusion / video | Paper Slides | diffusion models |
| 62 | Cambricon-D: Full-Network Differential Acceleration for Diffusion Models Domain-Specific Accelerator (DSA) |
diffusion / video | Paper | diffusion models |
| 63 | X-Adapter: Adding Universal Compatibility of Plugins for Upgraded Diffusion Model Supporting Add-on Modules |
diffusion / video | Paper Homepage Code | diffusion models |
| 64 | DLRM: Deep Learning Recommendation Model Acronyms |
ML systems | 上游条目未列直接链接 | dlrm |
| 65 | DisaggRec: Architecting Disaggregated Systems for Large-Scale Personalized Recommendation (arXiv 2212.00939) DLRM Inference |
ML systems | Paper | dlrm |
| 66 | Accelerating Neural Recommendation Training with Embedding Scheduling DLRM Training |
training / RL | Paper Slides Code | dlrm |
| 67 | Bagpipe: Accelerating Deep Recommendation Model Training DLRM Training |
training / RL | Paper | dlrm |
| 68 | Heterogeneous Acceleration Pipeline for Recommendation System Training DLRM Training |
training / RL | arXiv | dlrm |
| 69 | EVStore: Storage and Caching Capabilities for Scaling Embedding Tables in Deep Recommendation Systems GPU Cache |
ML systems | Paper Code | dlrm |
| 70 | UGache: A Unified GPU Cache for Embedding-based Deep Learning GPU Cache |
ML systems | Paper | dlrm |
| 71 | Ekko: A Large-Scale Deep Learning Recommender System with Low-Latency Model Update Model Update |
ML systems | Paper | dlrm |
| 72 | AdaEmbed: Adaptive Embedding for Large-Scale Recommendation Models Pruning |
ML systems | Paper | dlrm |
| 73 | HPO: Hyper-Parameter Tuning Acronyms |
ML systems | 上游条目未列直接链接 | hpo |
| 74 | CherryPick: Adaptively Unearthing the Best Cloud Configurations for Big Data Analytics HPO for Systems |
ML systems | Paper | hpo |
| 75 | Google Vizier: A Service for Black-Box Optimization (KDD 2017) HPO for Systems |
ML systems | Paper | hpo |
| 76 | Morphling: Fast, Near-Optimal Auto-Configuration for Cloud-Native Model Serving HPO for Systems |
ML systems | Paper Code | hpo |
| 77 | Selecting the Best VM across Multiple Public Clouds: A Data-Driven Performance Modeling Approach HPO for Systems |
ML systems | Paper | hpo |
| 78 | Elastic Hyperparameter Tuning on the Cloud Optimizing HPO Workloads |
ML systems | Paper | hpo |
| 79 | Hydro: Surrogate-Based Hyperparameter Tuning Service in Datacenters Optimizing HPO Workloads |
ML systems | Paper Code | hpo |
| 80 | RubberBand: Cloud-based Hyperparameter Tuning Optimizing HPO Workloads |
ML systems | Paper | hpo |
| 81 | LLM: Large Language Model Acronyms |
LLM serving / KV / agent | 上游条目未列直接链接 | llm |
| 82 | LoRA: Low-Rank Adaptation Acronyms |
LLM serving / KV / agent | 上游条目未列直接链接 | llm |
| 83 | RL: Reinforcement Learning Acronyms |
LLM serving / KV / agent | 上游条目未列直接链接 | llm |
| 84 | RLHF: Reinforcement Learning from Human Feedback Acronyms |
LLM serving / KV / agent | 上游条目未列直接链接 | llm |
| 85 | SDC: Silent Data Corruption Acronyms |
LLM serving / KV / agent | 上游条目未列直接链接 | llm |
| 86 | PUZZLE: Efficiently Aligning Large Language Models through Light-Weight Context Switch LLM Alignment |
LLM serving / KV / agent | Paper | llm |
| 87 | ThunderAgent: A Fast, Simple, and Program-Aware Agentic Inference System LLM Inference / Agentic Inference |
LLM serving / KV / agent | Paper arXiv Code | llm |
| 88 | LMPrefill: An Inference Engine for Prefill-only Workloads in Large Language Model Applications LLM Inference / Chunked Prefill |
LLM serving / KV / agent | Paper arXiv | llm |
| 89 | Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve LLM Inference / Chunked Prefill |
LLM serving / KV / agent | Paper Code arXiv | llm |
| 90 | Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantization LLM Inference / Compression |
LLM serving / KV / agent | 上游条目未列直接链接 | llm |
| 91 | Fairness in Serving Large Language Models LLM Inference / Fairness |
LLM serving / KV / agent | Paper Code | llm |
| 92 | Locality-aware Fair Scheduling in LLM Serving (arXiv:2501.14312) LLM Inference / Fairness |
LLM serving / KV / agent | arXiv | llm |
| 93 | Cauchy: A Cost-Efficient LLM Serving System through Adaptive Heterogeneous Deployment (SoCC 2025) LLM Inference / Heterogeneous Deployment |
LLM serving / KV / agent | Paper | llm |
| 94 | Coral: Cost-Efficient Multi-LLM Serving over Heterogeneous Cloud GPUs (arXiv:2605.04357) LLM Inference / Heterogeneous Deployment |
LLM serving / KV / agent | arXiv | llm |
| 95 | Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs (arXiv:2502.00722) LLM Inference / Heterogeneous Deployment |
LLM serving / KV / agent | arXiv | llm |
| 96 | HexGen-2: Disaggregated Generative Inference of LLMs in Heterogeneous Environment (ICLR 2025) LLM Inference / Heterogeneous Deployment |
LLM serving / KV / agent | Paper arXiv | llm |
| 97 | HexGen: Generative Inference of Foundation Model over Heterogeneous Decentralized Environment LLM Inference / Heterogeneous Deployment |
LLM serving / KV / agent | arXiv Code | llm |
| 98 | SageServe: Optimizing LLM Serving on Cloud Data Centers with Forecast Aware Auto-Scaling (SIGMETRICS Abstracts 2026) LLM Inference / Heterogeneous Deployment |
LLM serving / KV / agent | Paper arXiv Code | llm |
| 99 | SpotServe: Serving Generative Large Language Models on Preemptible Instances LLM Inference / Heterogeneous Deployment |
LLM serving / KV / agent | arXiv Code | llm |
| 100 | ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching LLM Inference / KV Cache Management |
LLM serving / KV / agent | 上游条目未列直接链接 | llm |
| 101 | CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion LLM Inference / KV Cache Management |
LLM serving / KV / agent | Paper arXiv Code | llm |
| 102 | CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving LLM Inference / KV Cache Management |
LLM serving / KV / agent | arXiv Code | llm |
| 103 | DroidSpeak: KV Cache Sharing Across Fine-tuned Model Variants LLM Inference / KV Cache Management |
LLM serving / KV / agent | Paper arXiv | llm |
| 104 | Efficient Memory Management for Large Language Model Serving with PagedAttention LLM Inference / KV Cache Management |
LLM serving / KV / agent | Paper arXiv Code Homepage | llm |
| 105 | Jenga: Effective Memory Management for Serving LLM with Heterogeneity LLM Inference / KV Cache Management |
LLM serving / KV / agent | Paper arXiv | llm |
| 106 | Prompt Cache: Modular Attention Reuse for Low-Latency Inference LLM Inference / KV Cache Management |
LLM serving / KV / agent | Paper arXiv | llm |
| 107 | Parrot: Efficient Serving of LLM-based Applications with Semantic Variable LLM Inference / LLM-based Applications |
LLM serving / KV / agent | Paper Code | llm |
| 108 | SGLang: Efficient Execution of Structured Language Model Programs LLM Inference / LLM-based Applications |
LLM serving / KV / agent | Paper arXiv Code | llm |
| 109 | Teola: Towards End-to-End Optimization of LLM-based Applications LLM Inference / LLM-based Applications |
LLM serving / KV / agent | arXiv | llm |
| 110 | CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference (arXiv:2401.11240) LLM Inference / LoRA Serving |
LLM serving / KV / agent | arXiv | llm |
| 111 | Punica: Multi-Tenant LoRA Serving LLM Inference / LoRA Serving |
LLM serving / KV / agent | arXiv Code | llm |
| 112 | S-LoRA: Serving Thousands of Concurrent LoRA Adapters LLM Inference / LoRA Serving |
LLM serving / KV / agent | arXiv Code | llm |
| 113 | dLoRA: Dynamically Orchestrating Requests and Adapters for LoRA LLM Serving LLM Inference / LoRA Serving |
LLM serving / KV / agent | Paper | llm |
| 114 | FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU LLM Inference / Offloading |
LLM serving / KV / agent | Paper Code | llm |
| 115 | LLM in a flash: Efficient Large Language Model Inference with Limited Memory (arXiv 2312.11514) LLM Inference / Offloading |
LLM serving / KV / agent | arXiv | llm |
| 116 | AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving LLM Inference / Parallelism and Partitioning |
LLM serving / KV / agent | Paper Code | llm |
| 117 | DeepSpeed-Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale LLM Inference / Parallelism and Partitioning |
LLM serving / KV / agent | Paper Code Homepage | llm |
| 118 | Efficiently Scaling Transformer Inference LLM Inference / Parallelism and Partitioning |
LLM serving / KV / agent | Paper | llm |
| 119 | EPIC: Efficient Position-Independent Context Caching for Serving Large Language Models (ICML 2025) LLM Inference / Position-Independent Caching (PIC) |
LLM serving / KV / agent | arXiv | llm |
| 120 | DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving LLM Inference / Prefill-Decode (PD) Disaggregation |
LLM serving / KV / agent | Paper Code | llm |
| 121 | Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads (arXiv:2401.11181) LLM Inference / Prefill-Decode (PD) Disaggregation |
LLM serving / KV / agent | arXiv | llm |
| 122 | Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving (FAST 2025) LLM Inference / Prefill-Decode (PD) Disaggregation |
LLM serving / KV / agent | Paper arXiv Slides Code | llm |
| 123 | Prefill-as-a-Service: KVCache of Next-Generation Models Could Go Cross-Datacenter (arXiv:2604.15039) LLM Inference / Prefill-Decode (PD) Disaggregation |
LLM serving / KV / agent | arXiv | llm |
| 124 | Splitwise: Efficient Generative LLM Inference Using Phase Splitting LLM Inference / Prefill-Decode (PD) Disaggregation |
LLM serving / KV / agent | Paper arXiv | llm |
| 125 | FastServe: Iteration-Level Preemptive Scheduling for Large Language Model Inference LLM Inference / Request Scheduling |
LLM serving / KV / agent | Paper arXiv Code | llm |
| 126 | Llumnix: Dynamic Scheduling for Large Language Model Serving LLM Inference / Request Scheduling |
LLM serving / KV / agent | Paper Code | llm |
| 127 | Orca: A Distributed Serving System for Transformer-Based Generative Models LLM Inference / Request Scheduling |
LLM serving / KV / agent | Paper | llm |
| 128 | Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation (SIGMOD 2025) LLM Inference / Retrieval-Augmented Generation (RAG) |
LLM serving / KV / agent | arXiv | llm |
| 129 | CacheFocus: Dynamic Cache Re-Positioning for Efficient Retrieval-Augmented Generation (arXiv:2502.11101) LLM Inference / Retrieval-Augmented Generation (RAG) |
LLM serving / KV / agent | arXiv | llm |
| 130 | RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation (arXiv:2404.12457) LLM Inference / Retrieval-Augmented Generation (RAG) |
LLM serving / KV / agent | arXiv | llm |
| 131 | FaaScale: Unlocking Fast LLM Scaling for Serverless Inference LLM Inference / Serverless Inference |
training / RL | Paper arXiv | llm |
| 132 | HydraServe: Minimizing Cold Start Latency for Serverless LLM Serving in Public Clouds LLM Inference / Serverless Inference |
training / RL | Paper arXiv Code | llm |
| 133 | ServerlessLLM: Low-Latency Serverless Inference for Large Language Models LLM Inference / Serverless Inference |
training / RL | Paper Code arXiv | llm |
| 134 | Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time LLM Inference / Sparsity |
LLM serving / KV / agent | Paper Code | llm |
| 135 | InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management LLM Inference / Sparsity |
LLM serving / KV / agent | Paper | llm |
| 136 | PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU LLM Inference / Sparsity |
LLM serving / KV / agent | Paper arXiv Code | llm |
| 137 | Fast Inference from Transformers via Speculative Decoding LLM Inference / Speculative Decoding |
LLM serving / KV / agent | Paper | llm |
| 138 | Online Speculative Decoding LLM Inference / Speculative Decoding |
LLM serving / KV / agent | arXiv | llm |
| 139 | SpecInfer: Accelerating Generative LLM Serving with Speculative Inference and Token Tree Verification LLM Inference / Speculative Decoding |
LLM serving / KV / agent | arXiv Code | llm |
| 140 | Speculative Decoding with Big Little Decoder (NeurIPS 2023) LLM Inference / Speculative Decoding |
LLM serving / KV / agent | Paper | llm |
| 141 | Measuring Agents in Production LLM Inference / Workload Characterization |
LLM serving / KV / agent | Paper arXiv | llm |
| 142 | TraceLab: Characterizing Coding Agent Workloads for LLM Serving (arXiv:2606.30560) LLM Inference / Workload Characterization |
LLM serving / KV / agent | arXiv Code Homepage | llm |
| 143 | Accelerating the Training of Large Language Models using Efficient Activation Rematerialization and Optimal Hybrid Parallelism LLM Training / Hybrid Parallelism |
training / RL | Paper Code | llm |
| 144 | Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning LLM Training / Hybrid Parallelism |
training / RL | Paper Code Docs | llm |
| 145 | Beat the long tail: Distribution-Aware Speculative Decoding for RL Training LLM Training / RL Post-Training |
training / RL | Paper | llm |
| 146 | Disaggregated RL systems**: split rollout, inference, environment, reward, and training stages across best-fit resources. LLM Training / RL Post-Training |
training / RL | 上游条目未列直接链接 | llm |
| 147 | DynaRL: Flexible and Dynamic Scheduling of Large-Scale Reinforcement Learning Training LLM Training / RL Post-Training |
training / RL | Paper | llm |
| 148 | Fault tolerance**: isolate and recover failures across trainer, rollout, and control-plane roles. LLM Training / RL Post-Training |
training / RL | 上游条目未列直接链接 | llm |
| 149 | HetRL: Efficient Reinforcement Learning for LLMs in Heterogeneous Environments LLM Training / RL Post-Training |
training / RL | Paper | llm |
| 150 | Heterogeneous environments**: make RL training efficient across mixed GPU generations and hardware capabilities. LLM Training / RL Post-Training |
training / RL | 上游条目未列直接链接 | llm |
| 151 | History Doesn't Repeat Itself but Rollouts Rhyme: Accelerating Reinforcement Learning with RhymeRL LLM Training / RL Post-Training |
training / RL | Paper | llm |
| 152 | RLinf: Flexible and Efficient Large-Scale Reinforcement Learning via Macro-to-Micro Flow Transformation LLM Training / RL Post-Training |
training / RL | Paper | llm |
| 153 | ReSpec: Towards Optimizing Speculative Decoding in Reinforcement Learning Systems LLM Training / RL Post-Training |
training / RL | Paper | llm |
| 154 | ReaL: Efficient RLHF Training of Large Language Models with Parameter Reallocation LLM Training / RL Post-Training |
training / RL | Paper arXiv Code | llm |
| 155 | RobustRL: Role-Based Fault Tolerance System for RL Post-Training LLM Training / RL Post-Training |
training / RL | Paper | llm |
| 156 | RollArt: Disaggregated Multi-Task Agentic RL Training at Scale LLM Training / RL Post-Training |
training / RL | Paper | llm |
| 157 | RollPacker: Taming Long-Tail Rollouts for RL Post-Training with Tail Batching LLM Training / RL Post-Training |
training / RL | Paper arXiv | llm |
| 158 | Rollout latency and long-tail mitigation**: predict, batch, reuse, or otherwise reduce long-tail rollout work. LLM Training / RL Post-Training |
training / RL | 上游条目未列直接链接 | llm |
| 159 | Seer: Online Context Learning for Fast Synchronous LLM Reinforcement Learning LLM Training / RL Post-Training |
training / RL | Paper | llm |
| 160 | Speculative decoding for RL**: adapt draft-and-verify generation to RL training constraints such as drafter staleness and rollout distribution shift. LLM Training / RL Post-Training |
training / RL | 上游条目未列直接链接 | llm |
| 161 | Taming the Long-Tail: Efficient Reasoning RL Training with Adaptive Drafter LLM Training / RL Post-Training |
training / RL | Paper arXiv Code | llm |
| 162 | Weave: Efficient Co-Scheduling for Disaggregated RL Post-Training LLM Training / RL Post-Training |
training / RL | Paper | llm |
| 163 | Workflow scheduling and resource reallocation**: reshape RL pipelines or dynamically move compute, memory, communication, and parameters across roles. LLM Training / RL Post-Training |
training / RL | 上游条目未列直接链接 | llm |
| 164 | Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs LLM Training / Reliability and Fault Tolerance |
training / RL | Paper Code | llm |
| 165 | Empirical reliability studies**: characterize production failure modes and operational mitigations from large training runs. LLM Training / Reliability and Fault Tolerance |
training / RL | 上游条目未列直接链接 | llm |
| 166 | Gemini: Fast Failure Recovery in Distributed Training with In-Memory Checkpoints LLM Training / Reliability and Fault Tolerance |
training / RL | Paper | llm |
| 167 | Holmes: Localizing Irregularities in LLM Training with Mega-scale GPU Clusters LLM Training / Reliability and Fault Tolerance |
training / RL | Paper | llm |
| 168 | Large-Scale AI Infra Reliability: Challenges, Strategies, and Llama 3 Training Experience (DSN-S 2025) LLM Training / Reliability and Fault Tolerance |
training / RL | Paper | llm |
| 169 | MegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production LLM Training / Reliability and Fault Tolerance |
training / RL | Paper arXiv | llm |
| 170 | MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs LLM Training / Reliability and Fault Tolerance |
training / RL | Paper Slides Code | llm |
| 171 | Oobleck: Resilient Distributed Training of Large Models Using Pipeline Templates LLM Training / Reliability and Fault Tolerance |
training / RL | Paper arXiv Code | llm |
| 172 | Production reliability infrastructure**: make failures observable, diagnosable, and routinely recoverable at 10K+ GPU scale. LLM Training / Reliability and Fault Tolerance |
training / RL | 上游条目未列直接链接 | llm |
| 173 | Recovery mechanisms**: reduce lost work after failures through redundancy, checkpoint placement, or pre-planned reconfiguration. LLM Training / Reliability and Fault Tolerance |
training / RL | 上游条目未列直接链接 | llm |
| 174 | Robust LLM Training Infrastructure at ByteDance LLM Training / Reliability and Fault Tolerance |
training / RL | Paper arXiv | llm |
| 175 | SDCs in the Wild: Characterizing and Diagnosing SDC-Defective GPUs in Production LLM Training LLM Training / Reliability and Fault Tolerance |
training / RL | Paper | llm |
| 176 | Safeguarding LLM Training at Scale: Online SDC Detection and Insights from 35 Million GPU Hours LLM Training / Reliability and Fault Tolerance |
training / RL | Paper | llm |
| 177 | Workload resilience**: absorb dynamic workload variation before it turns into large efficiency loss or training instability. LLM Training / Reliability and Fault Tolerance |
training / RL | 上游条目未列直接链接 | llm |
| 178 | Morphling: Fast, Near-Optimal Auto-Configuration for Cloud-Native Model Serving Auto-Configuration for Model Serving |
LLM serving / KV / agent | Paper Code | model serving |
| 179 | Serving Unseen Deep Learning Models with Near-Optimal Configurations: a Fast Adaptive Search Approach Auto-Configuration for Model Serving |
LLM serving / KV / agent | Paper Code | model serving |
| 180 | Clipper: A Low-Latency Online Prediction Serving System Model Serving Systems |
LLM serving / KV / agent | Paper Code | model serving |
| 181 | INFaaS: Automated Model-less Inference Serving Model Serving Systems |
LLM serving / KV / agent | Paper Code | model serving |
| 182 | Microsecond-scale Preemption for Concurrent GPU-accelerated DNN Inferences Model Serving Systems |
LLM serving / KV / agent | Paper Code Artifact | model serving |
| 183 | Paella: Low-latency Model Serving with Software-defined GPU Scheduling Model Serving Systems |
LLM serving / KV / agent | Paper | model serving |
| 184 | TensorFlow-Serving: Flexible, High-Performance ML Serving (NIPS 2017 Workshop on ML Systems) Model Serving Systems |
LLM serving / KV / agent | Paper | model serving |
| 185 | Usher: Holistic Interference Avoidance for Resource Optimized ML Inference Model Serving Systems |
LLM serving / KV / agent | Paper Code | model serving |
| 186 | A Survey of Large-Scale Deep Learning Serving System Optimization: Challenges and Opportunities (arXiv 2111.14247) Survey |
LLM serving / KV / agent | Paper | model serving |
| 187 | A Survey of Multi-Tenant Deep Learning Inference on GPU (MLSys 2022 Workshop on Cloud Intelligence / AIOps) Survey |
LLM serving / KV / agent | Paper | model serving |
| 188 | Accelerating Distributed MoE Training and Inference with Lina MoE Inference |
MoE / parallelism | Paper | moe |
| 189 | Optimizing Dynamic Neural Networks with Brainstorm MoE Inference |
MoE / parallelism | Paper | moe |
| 190 | Accelerating Distributed MoE Training and Inference with Lina MoE Training |
MoE / parallelism | Paper | moe |
| 191 | Janus: A Unified Distributed Training Framework for Sparse Mixture-of-Experts Models MoE Training |
MoE / parallelism | Paper | moe |
| 192 | SmartMoE: Efficiently Training Sparsely-Activated Models through Combining Offline and Online Parallelization MoE Training |
MoE / parallelism | Paper Code | moe |
| 193 | Mixtral-8x7B Models |
MoE / parallelism | 上游条目未列直接链接 | moe |
| 194 | DL: Deep Learning Acronyms |
scheduling / placement | 上游条目未列直接链接 | resource scheduler |
| 195 | ML: Machine Learning Acronyms |
scheduling / placement | 上游条目未列直接链接 | resource scheduler |
| 196 | AlloX: Compute Allocation in Hybrid Clusters Scheduling for DL Training Workloads |
scheduling / placement | Paper Code | resource scheduler |
| 197 | AntMan: Dynamic Scaling on GPU Clusters for Deep Learning Scheduling for DL Training Workloads |
scheduling / placement | Paper Code | resource scheduler |
| 198 | Astraea: A Fair Deep Learning Scheduler for Multi-Tenant GPU Clusters (TPDS 2021) Scheduling for DL Training Workloads |
scheduling / placement | Paper | resource scheduler |
| 199 | Balancing Efficiency and Fairness in Heterogeneous GPU Clusters for Deep Learning Scheduling for DL Training Workloads |
scheduling / placement | Paper | resource scheduler |
| 200 | Blox: A Modular Toolkit for Deep Learning Schedulers Scheduling for DL Training Workloads |
scheduling / placement | arXiv Code | resource scheduler |
| 201 | CASSINI: Network-Aware Job Scheduling in Machine Learning Clusters Scheduling for DL Training Workloads |
scheduling / placement | Paper | resource scheduler |
| 202 | Gandiva: Introspective Cluster Scheduling for Deep Learning Scheduling for DL Training Workloads |
scheduling / placement | Paper | resource scheduler |
| 203 | Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads Scheduling for DL Training Workloads |
scheduling / placement | Paper Code | resource scheduler |
| 204 | HiveD: Sharing a GPU Cluster for Deep Learning with Guarantees Scheduling for DL Training Workloads |
scheduling / placement | Paper Code | resource scheduler |
| 205 | Interference-aware Multiplexing for Deep Learning in GPU Clusters: A Middleware Approach Scheduling for DL Training Workloads |
scheduling / placement | Paper Code | resource scheduler |
| 206 | Looking Beyond GPUs for DNN Scheduling on Multi-Tenant Clusters Scheduling for DL Training Workloads |
scheduling / placement | Paper Code | resource scheduler |
| 207 | Lucid: A Non-intrusive, Scalable and Interpretable Scheduler for Deep Learning Training Jobs Scheduling for DL Training Workloads |
scheduling / placement | Paper Code | resource scheduler |
| 208 | Lyra: Elastic Scheduling for Deep Learning Clusters Scheduling for DL Training Workloads |
scheduling / placement | Paper arXiv | resource scheduler |
| 209 | MAPA: Multi-Accelerator Pattern Allocation Policy for Multi-Tenant GPU Servers (SC 2021) Scheduling for DL Training Workloads |
scheduling / placement | Paper Code | resource scheduler |
| 210 | Multi-Resource Interleaving for Deep Learning Training Scheduling for DL Training Workloads |
scheduling / placement | Paper Code | resource scheduler |
| 211 | Optimus: An Efficient Dynamic Resource Scheduler for Deep Learning Clusters (EuroSys 2018) Scheduling for DL Training Workloads |
scheduling / placement | Paper Code | resource scheduler |
| 212 | Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning Scheduling for DL Training Workloads |
scheduling / placement | Paper Code | resource scheduler |
| 213 | Shockwave: Fair and Efficient Cluster Scheduling for Dynamic Adaptation in Machine Learning Scheduling for DL Training Workloads |
scheduling / placement | Paper Code | resource scheduler |
| 214 | Sia: Heterogeneity-aware, goodput-optimized ML-cluster scheduling Scheduling for DL Training Workloads |
scheduling / placement | Paper | resource scheduler |
| 215 | Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads (arXiv 2202.07848) Scheduling for DL Training Workloads |
scheduling / placement | Paper | resource scheduler |
| 216 | Themis: Fair and Efficient GPU Cluster Scheduling (NSDI 2020) Scheduling for DL Training Workloads |
scheduling / placement | Paper | resource scheduler |
| 217 | Tiresias: A GPU Cluster Manager for Distributed Deep Learning Scheduling for DL Training Workloads |
scheduling / placement | Paper Code | resource scheduler |
| 218 | Topology-Aware GPU Scheduling for Learning Workloads in Cloud Environments (SC 2017) Scheduling for DL Training Workloads |
scheduling / placement | Paper Code | resource scheduler |
| 219 | SLAQ: Quality-Driven Scheduling for Distributed Machine Learning Scheduling for General ML Training Workloads |
scheduling / placement | Paper | resource scheduler |
| 220 | Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Vision (arXiv 2205.11913) Survey |
scheduling / placement | Paper | resource scheduler |
| 221 | Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads (ATC 2019) Trace Analysis |
scheduling / placement | Paper | resource scheduler |
| 222 | Characterization and Prediction of Deep Learning Workloads in Large-Scale GPU Datacenters (SC 2021) Trace Analysis |
scheduling / placement | Paper | resource scheduler |
| 223 | Characterizing Deep Learning Training Workloads on Alibaba-PAI (IISWC 2019) Trace Analysis |
scheduling / placement | Paper | resource scheduler |
| 224 | MLaaS in the Wild: Workload Analysis and Scheduling in Large-Scale Heterogeneous GPU Clusters Trace Analysis |
scheduling / placement | Paper | resource scheduler |
| 225 | MSRL: Distributed Reinforcement Learning with Dataflow Fragments Uncategorized |
training / RL | Paper Code | rl |
| 226 | Ray: A Distributed Framework for Emerging AI Applications Uncategorized |
training / RL | Paper Code Homepage | rl |
3. 怎样把目录读回 Mainline
| 上游主题 | Mainline 首选入口 | 下一步要问的问题 |
|---|---|---|
| LLM / model serving | 04 KV、05 scheduler、08 P/D | 论文改变的是 KV identity、allocator、admission 还是远端状态迁移？ |
| MoE / LLM training | 01 architecture、06 placement、18 cluster | router、EP traffic、重叠和容错怎样影响真实训练/推理？ |
| RL / agentic work | 17 agent serving、32 agent research | rollout、environment、reward、training 和工具状态怎样被调度？ |
| diffusion | 11 text diffusion、12 media diffusion | 去噪步骤、latent、video chunk 与 adapter 如何改变系统资源模型？ |
| resource scheduler / compiler / framework | 03 operators、05 scheduler、18 cluster | 优化的是 kernel、批处理、资源放置、弹性还是 end-to-end QoS？ |
4. 上游 taxonomy：按系统问题，不按热度或模型品牌
上游把 Systems-for-ML 分为 LLM、model serving、MoE、diffusion、training、RL、resource scheduler、compiler、framework、data processing 等主题，会议页再按年份切分。Mainline 的模型版本页用于核对“这一代改了什么”，系统论文页则查找状态、算子和调度问题的可复现机制。两条索引互相链接，回答的问题不同。
品牌/版本轴
GLM、Kimi、Qwen、Hy3、MiniMax、DeepSeek、Gemma。关注 checkpoint、训练、RL、开放权重与 API。
系统问题轴
KV、P/D、agent runtime、MoE、network、rollout、compiler、reliability。关注状态、约束、算法和实验。
5. 2025–2026 优先扩展的论文路线
| 路线 | 优先论文 | Mainline 连接点 |
|---|---|---|
| KV / P-D / heterogeneity | Mooncake、Jenga、LMPrefill、DroidSpeak、CacheBlend、Coral | 04、05、08、38、40 |
| Agentic serving / trace | ThunderAgent、TraceLab、Teola、Parrot、SGLang | 09、17、32、40 |
| RL systems | RollArt、Weave、RLinf、DynaRL、Seer、RobustRL | 18、32 和各模型的 post-training 段 |
| MoE / network / reliability | Tessera、MegaScale-Omni、AEGIS、SDCHunter、Mycroft | 06、07、18、29 |
| Diffusion / multimodal systems | DiFlow、FlashPS、Katz、Fast Video | 10、11、12 |
这些条目只表示独立详解的编写顺序，不代表质量排名。每篇解释都会核对原论文、官方代码和 artifact，上游个人笔记不作为可复制的结论。
6. 一篇系统论文进入 Mainline 时必须补上的六个问题
- 对象：它管理的是权重、KV、激活、token、任务 DAG、网络包还是训练样本？
- 不变量：正确性依赖哪些 identity、顺序、版本、权限和失败语义？
- 瓶颈：是 HBM、GEMM、all-to-all、KV 迁移、排队、环境等待还是恢复？
- 机制：调度、allocator、parallelism、cache、kernel 或控制面具体怎样改变路径？
- 证据：硬件、模型、输入、并发、baseline、指标和统计范围是否可重放？
- 边界：结论能推广到哪种模型/拓扑，哪一部分尚未公开？
7. 快照更新与链接失效策略
本页固定记录上游 commit 和日期，不随 GitHub 自动刷新。更新目录时要重新核验 revision、生成目录、保存变更记录并检查 Paper/Code 链接。论文撤回、匿名稿正式出版、仓库改名或模型发布新版时，旧条目仍保留其历史指向，独立解释页另行注明时间边界。
8. 归属、许可与原始图的处理
上游仓库的 MIT 许可允许按其条件复用目录结构和文字，但并不将论文 PDF、作者图、benchmark 数据或项目代码一并变成 MIT。Mainline 因此只保留题名/链接/分类和显式归属；要嵌入原始架构图时，一律从论文或官方项目下载、在图注写原始来源，并遵循相应的合理使用与许可边界。
9. 目录来源与再利用说明
- 上游 Systems-for-ML 目录：本页的分类和题名/链接来源。
- SUMMARY.md：上游导航结构。
- conference-papers-skill.md：上游收录方式与分类语义。
- MIT License：上游仓库许可。所有论文和代码仍应按其各自许可使用。