Transformer course on LLMs and multimodal models

This is the repository of the course on LLMs based on the Transformer architecture targeted at people with experience in Python, Machine Learning, and Deep Learning but little or no experience with Transformers and LLMS.

The course focuses on modern LLMs and multimodal models built on the Transformer architecture. A separate module covers the use of Transformers for non-text modalities (time series, tabular data, and others) and modern multimodal architectures. The course also introduces the fundamentals of building AI agents based on LLMs. As a result, you will gain comprehensive knowledge of how modern LLMs, multimodal models, RAG systems, and basic LLM agents operate both in theory and in practice.

The course emphasizes not only theoretical understanding but also practical skills for applying these models and building systems around them. Special attention is given to modern LLM architectures, relevant frameworks such as Transformers, LLM, Ollama, LangChain, and others, as well as to the efficiency of LLM-based system design.

By the end of the course, you will have a coherent understanding of the principles and practical use of LLMs, multimodal models, RAG systems, and basic LLM agents — with a strong emphasis on practical application, proper evaluation, and computational efficiency.

The course features $14$ sessions: one lecture per session, one practical session per session.

The course Syllabus

1. Transformer model
2. Transformer-based Encoders
3. Classification with Transformers
4. Transformer-based decoders
5. Towards ChatGPT
6. Efficient Transformers
7. RAG with Transformers
8. Introduction to AI Agents
9. Uncertainty estimation for Transformers
10. Uncertainty quantification generation tasks
11. Multilingual language models
12. Multimodal dialogue models
13. Transformers for tabular data
14. Transformers for event sequences

Editable materials

Raw versions of the editable course materials (e.g., pptx presentations) can be found here.