Installing TinyLlama for Raspberry

wget https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf python -m venv llama2 llama2/bin/pip install llama-cpp-python

======================== Then, in a text editor such as vi, create a file llama2.py that contains the following Python code.

import sys from llama_cpp import Llama llm = Llama(model_path="tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf", verbose=False) output = llm("<user>\n" + sys.argv[1] + "\n<assistant>\n", max_tokens=40) print(output['choices'][0]["text"] + "...")

======================== llama2/bin/python llama2.py "What is MQTT?"

======================== Creating Chat UI for reTerminal DM Screen bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered) sudo systemctl enable nodered.service sudo reboot http://localhost:1880 After the user configuration, paste “https://github.com/kazuhitoyokoi/tinyllama.git" into the Git repository URL field.