Skip to content

Commit dfb4333

Browse files
[programming][machine_learning] Add llama.cpp usage
How to install and use.
1 parent a1d8b8e commit dfb4333

1 file changed

Lines changed: 41 additions & 0 deletions

File tree

src/programming/machine_learning.rst

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -759,6 +759,46 @@ Usage:
759759
760760
$ llamafactory-cli export examples/merge_lora/qwen3_lora_sft.yaml
761761
762+
llama.cpp
763+
~~~~~~~~~
764+
765+
Usage
766+
^^^^^
767+
768+
- Download pre-built binaries and libraries from the `llama.cpp GitHub releases page <https://github.com/ggml-org/llama.cpp/releases/>`__. For example, use the ``b10375`` release. It is recommended to use the "cuda" variant for NVIDIA GPUs and the "vulkan" variant for AMD and Intel GPUs.
769+
770+
- Linux
771+
772+
.. code-block:: sh
773+
774+
$ curl -LO https://github.com/ggml-org/llama.cpp/releases/download/b<BUILD_NUMBER>/llama-b<BUILD_NUMBER>-bin-ubuntu-vulkan-x64.tar.gz
775+
776+
.. code-block:: sh
777+
778+
$ curl -LO https://github.com/ggml-org/llama.cpp/releases/download/b10375/llama-b10375-bin-ubuntu-vulkan-x64.tar.gz
779+
780+
- Extract the archive.
781+
782+
.. code-block:: sh
783+
784+
$ tar --extract --file llama-b<BUILD_NUMBER>-bin-ubuntu-vulkan-x64.tar.gz
785+
$ cd llama-b<BUILD_NUMBER>
786+
787+
.. code-block:: sh
788+
789+
$ tar --extract --file llama-b10375-bin-ubuntu-vulkan-x64.tar.gz
790+
$ cd llama-b10375
791+
792+
- Download and run a model from HuggingFace. For example, ``https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF``. [86]
793+
794+
.. code-block:: sh
795+
796+
./llama-cli -hf unsloth/Qwen3.6-35B-A3B-MTP-GGUF
797+
798+
.. code-block:: sh
799+
800+
./llama-cli -hf <USER>/<MODEL>
801+
762802
Prompt Engineering
763803
~~~~~~~~~~~~~~~~~~
764804

@@ -1123,3 +1163,4 @@ Bibliography
11231163
83. "Evan Luthra on X." X. April 20, 2026. Accessed April 30, 2026. https://x.com/evanluthra/status/2046240203480936608
11241164
84. "What context lengths do people actually run their models at?" Reddit r/LocalLLaMA. February 14, 2026. Accessed April 30, 2026. https://www.reddit.com/r/LocalLLaMA/comments/1mfi8ly/what_context_lengths_do_people_actually_run_their/
11251165
85. "Configuration." Nanocoder Documentation. Accessed July 25, 2026. https://docs.nanocollective.org/nanocoder/docs/v1.28.1/configuration/
1166+
86. "Llama.cpp." ArchWiki. July 15, 2026. Accessed August 12, 2026. https://wiki.archlinux.org/title/Llama.cpp

0 commit comments

Comments
 (0)