Open-Source-LLMs-on-Your-Own-Computer

Chatbot with Llama

Get the Llama models

A screenshot from Hugging Face showing access

A screenshot of the downloaded local files using ls -al .

The appropriate download link of Q5_K_M quantitized Phi-3 mini model from Microsoft.

https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/blob/main/Phi-3-mini-4k-instruct-q4.gguf

Inference on the command-line

Ask a different question in the conversation and A screenshot of the output of the answer..

Use the Chat Template to Carry a Conversation

Ask the next question in the conversation. Write it down as a complete command. A screenshot of the output in response to the next question in the conversation.

wasmedge --dir .:.
--nn-preload default:GGML:AUTO:Phi-3-mini-4k-instruct-q4.gguf
llama-chat.wasm
--prompt-template llama-3-chat
--system-prompt "You are a high school social teacher. Explain the concepts of earth geography."

Create a Web Service API

Ask the next question in the conversation. Write it down as a complete command. A screenshot of the output in response to the next question in the conversation.

Create a Chatbot

Change the “system prompt” for the chatbot-ui to “You are a tour guide for Western tourists in Japan.” Then, submit the screenshot. Ask a few questions about Japan in chatbot-ui. Submit the screenshots. Configure LobeChat to use the http://localhost:8080/v1 as an "OpenAI API proxy address". Submit the screenshot. Chat on LobeChat. Submit the screenshot.

Add Knowledge to the Chatbot

Create a Vector Database for External Knowledge

The output of curl 'http://localhost:6333/collections/chemistry' and a zip file containing the snapshot from the chemistry collection.

Improve the Vector Collection for Longer Context Windows

The chemistry.csv file you generated. The output of curl 'http://localhost:6333/collections/chemistry'. A zip file containing the snapshot from the chemistry collection.

Start an RAG API Server

Use the API to ask a chemistry question and get an answer and API server log that shows the user question, vector search results, and the updated prompt

Create a Web-based Chatbot

The publicly accessible chatbot URL and a screenshot of the chatbot UI showing a relevant conversation

Create a Discord Bot

An invite link to the Discord server that is hosting the bot and a screenshot of the Discord bot showing a relevant conversation

Fine-Tune the Llama Model

Create the Training Dataset

The politics.txt file you created with news headlines. The finetune.json dataset you created for the Alpaca template. A link to the public Hugging Face Dataset repo for your finetune.json data file

Public Hugging Face Dataset repo: https://huggingface.co/datasets/bas369/3_finetune

Fine-tune the Model

A shared link to the notebook you used to fine-tune the LLM. The Hugging Face model repo for the final product.

Co lab Notebook link: https://colab.research.google.com/#fileId=https%3A//huggingface.co/juntaoyuan/validate_chemistry_question/blob/main/Llama_3_1_8b_%2B_Unsloth_finetuning.ipynb Hugging Face model Repo: https://huggingface.co/bas369/finetune_model/tree/main

Run an inference API server for the fine-tuned model

The publicly accessible chatbot URL. A screenshot of the chatbot UI showing a relevant conversation

url: https://0xa9f7efaa07879d7e3276e8626852de6758bd0321.gaianet.network