Quartz sync: Jun 10, 2026, 11:16 AM

Author: leoda1

content/Inference/vllm/src_code/4. vllm v1 调度逻辑 和 distributed 分布式原理.md

@@ -22,8 +22,8 @@ vllm 构建了一套双进程多线程异步的在线推理框架，PD 分离场
 > 假设模型为 30B moe，1P1D，80GB 显存，tp8。num_hidden_layers=48，num_key_value_heads=4，head_dim=128，8k 输入 1k 输出，325 条请求，rps=65，最大一个 batch 里面放 64k。
 > 
 > per token / rank = 2(KV) x 1(KV head) x 128(head_dim) x 2(bf16) x 48 (num_hidden_layers) = 24KB
-> one kv cache block = 24KB x 16 = 384KB
-> 9k 输入输出需要的 block 数量 =  24KB x 9216 / 384KB = 216MB / 384KB = 576 个 block
+> one kv cache block = 24KB x 16 =384
+> 9k 输入输出需要的 block 数量 =  24KB x 9216 / 384KB = 576 个 block
 > decode 的一个 gpu 有 54GB 给 kv cache(log日志看到的)，那么最多就可以同时有 54 * 1024 * 1024 / 384 / 576 = 256 个（8k）请求。理论值与实际打印值接近。
 
 传统引擎把"处理 prompt(prefill)"和"逐字生成(decode)"当成两种不同的阶段分开调度,代码很割裂。V1 把它们统一成一件事:不管你是 prefill 还是 decode,都只是"进度条还差 N 个 token没算"。调度器只干一件事——给你分配这一步算几个 token。
@@ -59,6 +59,8 @@ generator = await handler.****create_chat_completion****(request, raw_request)
 ####
 
 ## 问题
-1. decode 侧每次好了多少 kv cache之后怎么告诉 decode 现在好了哪些的，以及满的时候还会收新的 batch 的 kv cache 过来吗？
-2. decode 计算的时候，调度逻辑在干嘛，在准备搬运新的 kv cache 过来？
-3. 
+
+1. 如果 kv cache 不够大，那么很容易被 prefill 以一个很慢的带宽打满，那么 decode 侧每次的 forward 和 空出来新的 block给 prefill 传kv cache 进来 就会完全串行。（新的 block 空出来收到一部分就可以被nixl batch 起来一起计算）
+2. 如果 kv cache 够大，那么每次 prefill 传进来的正好 decode 侧吃满计算的请求 cache 进来之后，每次 decode forward 计算和prefill 传 kv cache 进来就可以完全 overlap，完全并行。
+
+此时问题就是我怎么在各类固定 kv cache 大小后，在不同 workload / 不同的 decode 消费能力下能通过某种建模得出最低的 kv cache 传输速度 以至于 pd 分离性能没有影响。

content/Inference/vllm/src_code/excalidraw/4. vllm v1 调度逻辑 和 distributed 分布式原理 2026-06-04 21.25.13.excalidraw.md

@@ -0,0 +1,31 @@
+1. **Use Headings and Subheadings**: Clearly define sections with headings (e.g.,
+```
+#
+```
+,
+```
+##
+```
+,
+```
+###
+```
+) to organize content hierarchically.
+
+2. **Bullet Points and Lists**: Use bullet points or numbered lists to break down information into digestible parts.
+
+3. **Consistent Spacing**: Ensure consistent spacing between sections and paragraphs for better readability.
+
+4. **Highlight Key Points**: Use bold or italics to emphasize important information or key terms.
+
+5. **Tables for Structured Data**: Use tables to organize data that fits into rows and columns for clarity.
+
+6. **Quotes and References**: Use blockquotes for quotes and reference links for sources.
+
+7. **Code Blocks**: Use code blocks for any code snippets or technical instructions.
+
+8. **Images and Diagrams**: Include images or diagrams where applicable to visually represent information.
+
+9. **Linking and Cross-referencing**: Use internal links to connect related notes or sections within your vault.
+
+10. do not use ``` markdown

content/Inference/vllm/src_code/excalidraw/4. vllm v1 调度逻辑 和 distributed 分布式原理 2026-06-04 21.25.13.excalidraw.svg

@@ -0,0 +1,124 @@
+Please create an Obsidian note with interactive flashcards using native Obsidian HTML features. The note must include:
+
+1. Frontmatter (at the top) with the following properties:
+
+---
+
+total: {{number of flashcards created}}
+
+topics: ["{{topic 1}}", "{{topic 2}}", "{{topic 3}}"]
+
+created: "{{current date in YYYY-MM-DD format}}"
+
+---
+
+2. Interactive flashcards using HTML details/summary tags (below the frontmatter).
+
+**Instructions:**
+
+- Extract key concepts, facts, definitions, relationships, and important information from the content
+- Identify all testable knowledge points: definitions, concepts, facts, formulas, relationships, processes, dates, names, theories, principles, etc.
+- Extract 3-7 main topics or themes from the content for the topics array in frontmatter
+- **CRITICAL: Randomize the order of flashcards - mix different topics, difficulty levels, and question types together. Do NOT follow the original content order or group by topic. Create a varied, shuffled study experience.**
+- Create 10-30 flashcards depending on content length and density
+- Prioritize information that is:
+  - Fundamental to understanding the topic
+  - Frequently referenced or important
+  - Easy to forget (dates, numbers, specific details)
+  - Part of a sequence or process
+  - A definition or key concept
+
+- Maintain the exact markdown syntax for the frontmatter block (`---` at the top and bottom).
+- Each property in frontmatter must be on its own line with proper YAML indentation
+- Topics should be an array: topics: ["Topic 1", "Topic 2", "Topic 3"]
+- Total should be a number without quotes
+- Created should be a string in quotes with format "YYYY-MM-DD"
+
+- Format each flashcard using HTML details/summary tags:
+  - Each flashcard must be a separate <details> block with class="flashcard"
+  - The <summary> tag must have class="flashcard-question" and contain the question
+  - The answer must be wrapped in <div class="flashcard-answer">
+  - Leave a blank line after </summary> and before </details> for proper spacing
+  - Answers can include markdown formatting (bold, italic, lists, links, etc.)
+  - Answers can span multiple paragraphs
+
+- Questions should test understanding, not just recall
+- Answers should be complete but concise (typically 1-4 sentences, but can be longer if needed)
+- Include context and examples when helpful
+- Use clear, specific language
+- Make questions specific and focused (avoid overly broad questions)
+- Include [[internal links]] to related notes or concepts when relevant
+- Use **bold** for key terms in answers
+- Group related information in answers using lists or structured formatting
+
+- Do not use ``` code blocks or markdown code formatting in the output
+- Focus on accuracy and completeness based on the actual content provided
+
+**Example Output Format:**
+
+---
+
+total: 3
+
+topics: ["Cell Biology", "Energy Production", "Molecular Biology"]
+
+created: "2025-01-09"
+
+---
+
+<details class="flashcard">
+<summary class="flashcard-question">What is the primary function of mitochondria?</summary>
+
+<div class="flashcard-answer">
+
+Mitochondria are organelles that produce ATP (adenosine triphosphate) through cellular respiration. They are often called the "powerhouse of the cell" because they generate most of the cell's energy supply.
+
+Related: See also [[Chloroplasts]] for plant cell energy production.
+
+</div>
+
+</details>
+
+<details class="flashcard">
+<summary class="flashcard-question">What is the difference between DNA and RNA?</summary>
+
+<div class="flashcard-answer">
+
+- **DNA**: Double-stranded, contains thymine, stores genetic information
+- **RNA**: Single-stranded, contains uracil, involved in protein synthesis
+
+</div>
+
+</details>
+
+<details class="flashcard">
+<summary class="flashcard-question">How does photosynthesis work?</summary>
+
+<div class="flashcard-answer">
+
+Photosynthesis occurs in two stages:
+
+1. **Light-dependent reactions**: Chlorophyll absorbs light energy, splits water molecules, and produces ATP and NADPH
+2. **Calvin cycle**: Uses ATP and NADPH to convert carbon dioxide into glucose
+
+</div>
+
+</details>
+
+Important Notes:
+- ALWAYS include frontmatter with total, topics, and created date
+- Each property in frontmatter must be on its own line (no nesting, flat structure like the YouTube template)
+- Topics should be an array: topics: ["Topic 1", "Topic 2", "Topic 3"]
+- Created should be a string in quotes: created: "2025-01-09"
+- Total should be a number without quotes: total: 15
+- Use proper HTML syntax - ensure all tags are properly closed
+- Output the actual HTML tags directly (NOT inside markdown code blocks)
+- Each flashcard should be separated by a blank line for readability
+- **RANDOMIZE THE ORDER: Mix topics, difficulty levels, and question types - do not follow source material order**
+- Include [[internal links]] to related concepts when relevant
+- Answers can include markdown formatting like **bold**, *italic*, lists, and [[internal links]]
+- If content is very long, focus on the most important concepts
+- If content is technical, ensure definitions are clear
+- Maintain the original meaning and accuracy of the source material
+- All flashcards will be in one note - users can click to reveal answers
+- This format works natively in Obsidian without requiring any plugins

content/_NoteCompanion/Templates/enhance.md

@@ -0,0 +1,32 @@
+Contextual Extraction of Discussion Points and Action Items
+
+Instruction:
+Analyze the provided content, which includes:
+	•	Transcript 1: The first transcript of the discussion.
+	•	Transcript 2: The second transcript of the discussion.
+	•	Written Notes: Notes taken by a participant summarizing the discussion.
+
+Task:
+Extract the following while prioritizing the notes written by the participant to infer emphasis and key takeaways:
+	1.	Discussion Points: Summarize the key topics, ideas, or issues discussed. Prioritize points that appear in the written notes and cross-reference with the transcripts for completeness.
+	2.	Action Items: Identify specific tasks, responsibilities, or decisions agreed upon. For each action item, include:
+	•	A brief description of the task.
+	•	The person(s) responsible, if mentioned.
+	•	Any deadlines, if stated.
+
+Output Format:
+
+**Discussion Points:**
+1. [Point 1]
+2. [Point 2]
+...
+
+**Action Items:**
+1. [Task description] - [Responsible person(s)] - [Deadline]
+2. [Task description] - [Responsible person(s)] - [Deadline]
+...
+
+**Supporting Context:**
+- Key excerpts from Transcript 1: [Relevant excerpts related to discussion points and action items].
+- Key excerpts from Transcript 2: [Relevant excerpts related to discussion points and action items].
+- Key highlights from Written Notes: [Direct quotes or summaries from notes].

content/_NoteCompanion/Templates/flash_cards.md

@@ -0,0 +1,97 @@
+---
+
+title: "[Full paper title]"
+
+authors: ["Author 1", "Author 2", "etc"]
+
+year: [Publication year]
+
+journal: "[Journal name]"
+
+volume: "[Vol. number]"
+
+issue: "[Issue number]"
+
+pages: "[Page range]"
+
+doi: "[DOI number]"
+
+url: "[Direct URL to access paper]"
+
+tags: ["academic", "paper", "[field]", "[specific subtopic]"]
+
+research_question: "[Exact question/objective as stated by authors]"
+
+significance: "[Why this research matters in 1-2 specific sentences]"
+
+keywords: ["Keyword 1", "Keyword 2", "etc"]
+
+citation: "[Complete citation in APA/MLA format]"
+
+---
+
+## 1. Key Arguments & Evidence
+
+- **Argument 1**: [Specific claim made by authors] ([p.X])
+    - Evidence: [Concrete data, statistics, or examples supporting this claim] ([p.X])
+    - Methodology used: [Specific methods used to gather this evidence] ([p.X])
+    - Theoretical basis: [Named theory or framework] ([Author Year, p.X])
+- **Argument 2**: [Continue with same detailed format]
+- **Argument 3**: [Add as many as needed with same detailed format]
+
+## 2. Methodology Details
+
+- **Research design**: [Exact type - e.g., "longitudinal cohort study" not just "quantitative"] ([p.X])
+- **Data collection methods**: [Specific techniques used - e.g., "semi-structured interviews using protocol X"] ([p.X])
+- **Sample characteristics**: [Exact details - e.g., "128 participants: 64 female, 64 male, ages 18-35, from University X"] ([p.X])
+- **Key variables**: [Named variables with operational definitions] ([p.X])
+- **Analytical techniques**: [Specific statistical tests or qualitative approaches with software used] ([p.X])
+- **Ethical considerations**: [Any ethics committee approvals or considerations mentioned] ([p.X])
+
+## 3. Substantive Findings
+
+- **Primary finding**: [Most significant result with exact statistics/p-values/effect sizes] ([p.X])
+- **Secondary findings**: [Additional results with specific data points] ([p.X])
+- **Unexpected results**: [Any surprising outcomes with authors' explanations] ([p.X])
+- **Null findings**: [What didn't work or wasn't supported] ([p.X])
+- **Limitations acknowledged**: [Specific constraints identified by authors] ([p.X])
+
+## 4. Scholarly Context
+
+- **Builds on**: [[Author Year]] - [Named works this paper extends] ([p.X])
+- **Contradicts**: [[Author Year]] - [Specific research this challenges] ([p.X])
+- **Resolves**: [Specific debates this settles] ([p.X])
+- **Theoretical framework**: [Named theory/framework the paper operates within] ([p.X])
+- **Research gap addressed**: [Explicit gap identified by authors] ([p.X])
+
+## 5. Key Quotes
+
+- **Central argument**: "[Direct quote]" ([p.X])
+- **Methodology**: "[Direct quote]" ([p.X])
+- **Main finding**: "[Direct quote]" ([p.X])
+- **Implications**: "[Direct quote]" ([p.X])
+- **Future research**: "[Direct quote]" ([p.X])
+
+## 6. Explicit Recommendations & Applications
+
+- **Direct recommendations**: [List all specific recommendations made by authors] ([p.X])
+- **Policy implications**: [Any policy changes suggested] ([p.X])
+- **Practice implications**: [How practitioners should change behavior] ([p.X])
+- **Industry applications**: [Business or commercial applications] ([p.X])
+- **Educational implications**: [Teaching or learning applications] ([p.X])
+- **Future research directions**: [Specific suggestions for further study] ([p.X])
+
+## 7. Critical Reference Mapping
+
+- [[Author Year]] - [Title] ([p.X in reviewed paper]) - [Precise role in paper's argument]
+- [[Author Year]] - [Title] ([p.X in reviewed paper]) - [Precise role in paper's argument]
+- [[Author Year]] - [Title] ([p.X in reviewed paper]) - [Precise role in paper's argument]
+
+## 8. Personal Research Notes
+
+- **Relevance to my work**: [Specific ways this connects to your research]
+- **Methods I could adapt**: [Specific techniques that could be applied]
+- **Gaps I could address**: [How your work might extend this research]
+- **Potential citations**: [Where in your own writing you might cite this]
+- **Related papers in vault**: [[Paper 1]], [[Paper 2]]
+- **Related concepts**: [[Concept 1]], [[Concept 2]]