Skip to content
This repository was archived by the owner on Sep 9, 2025. It is now read-only.

Commit a2dbdf7

Browse files
committed
moved md to rag folder (with images). added export from excalidraw
Signed-off-by: Daniele Martinoli <[email protected]>
1 parent 94a2a8c commit a2dbdf7

File tree

7 files changed

+12610
-8
lines changed

7 files changed

+12610
-8
lines changed

docs/cli/ilab-rag-retrieval.md renamed to docs/rag/ilab-rag-retrieval.md

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,10 @@
99

1010
**Version**: 0.1
1111

12+
**Options to Rebuild Excalidraw Diagrams**:
13+
* Using this [shareable link][shareable-excalidraw]
14+
* Importing the scene from the exported [DSL](./images/rag-ingestion-and-chat.excalidraw)
15+
1216
## 1. Introduction
1317
This document proposes enhancements to the `ilab` CLI to support workflows utilizing Retrieval-Augmented Generation
1418
(RAG) artifacts within `InstructLab`. The proposed changes introduce new commands and options for the embedding ingestion
@@ -23,19 +27,19 @@ The commands are tailored to support diverse user experiences, all enabling the
2327

2428
### 1.2 Model Training Path
2529
This flow is designed for users who aim to train their own models and leverage the source documents that support knowledge submissions to enhance the chat context:
26-
![model-training](../images/rag-model-training.png)
30+
![model-training](./images/rag-model-training.png)
2731

2832
**Note**: documents are processed using `instructlab-sdg` package and are defined using the docling v1 schema.
2933

3034
### 1.3 Tanomony Path (no Training)
3135
This flow is for users who have defined taxonomy knowledge but prefer not to train their own models. Instead, they aim to generate RAG artifacts from source documents to enhance the chat context:
32-
![tanomony-no-training](../images/rag-tanomony-no-training.png)
36+
![tanomony-no-training](./images/rag-tanomony-no-training.png)
3337

3438
**Note**: documents are processed using `docling.DocumentConverter` and are defined using the docling v2 schema.
3539

3640
### 1.4 Plug-and-Play RAG Path
3741
This flow is designed for users who want to enhance their chat experience with pre-trained models by simply integrating the RAG functionality:
38-
![plug-and-play](../images/rag-plug-and-play.png)
42+
![plug-and-play](./images/rag-plug-and-play.png)
3943

4044
**Note**: documents are processed using `docling.DocumentConverter` and are defined using the docling v2 schema.
4145

@@ -285,12 +289,11 @@ chat:
285289
286290
287291
### 2.10 Workflow Visualization
288-
(Link to [shared Excalidraw][shared-excalidraw])
289292
290293
Embedding ingestion pipeline:
291-
![ingestion-mvp](../images/ingestion-mvp.png)
294+
![ingestion-mvp](./images/ingestion-mvp.png)
292295
RAG-based Chat pipeline:
293-
![rag-chat](../images/rag-chat.png)
296+
![rag-chat](./images/rag-chat.png)
294297
295298
### 2.11 Proposed Implementation Stack
296299
> **ℹ️ Note:** This stack is still under review. The proposed list represents potential candidates based on the current state of discussions.
@@ -349,11 +352,11 @@ ilab serve --rag-embeddings --image-name=docker.io/user/my_rag_artifacts:1.0 --p
349352
ilab model chat --rag --retriever-type api --retriever-uri http://localhost:8123
350353
```
351354
355+
[shareable-excalidraw]: https://excalidraw.com/#json=p126_RwjtILDahmnIFK9c,11xoIhRQCkUVRm0Wkg-Ysg
352356
[ilab-knowledge]: https://github.com/instructlab/taxonomy?tab=readme-ov-file#getting-started-with-knowledge-contributions
353357
[sdg-diff-strategy]: https://github.com/instructlab/sdg/blob/main/src/instructlab/sdg/utils/taxonomy.py
354358
[chat_template]: https://github.com/instructlab/instructlab/blob/0a773f05f8f57285930df101575241c649f591ce/src/instructlab/configuration.py#L244
355359
[augment_chat_template]: https://github.com/instructlab/instructlab/blob/48e3f7f1574ae50036d6e342b8d78d8eb9546bd5/src/instructlab/model/backends/llama_cpp.py#L281
356360
[ranking]: https://docs.haystack.deepset.ai/v1.21/reference/ranker-api
357361
[expansion]: https://haystack.deepset.ai/blog/query-expansion
358-
[chunkers]: https://github.com/DS4SD/docling/blob/main/docs/concepts/chunking.md
359-
[shared-excalidraw]: https://excalidraw.com/#json=D_sPMvwB0XbCVoBL1hyAi,R_rUo6ljInJPrcWnbOO5pQ
362+
[chunkers]: https://github.com/DS4SD/docling/blob/main/docs/concepts/chunking.md
File renamed without changes.
File renamed without changes.

0 commit comments

Comments
 (0)