Fine-tuned LLaMa2 13B model designed for ReAct-style and Tree-Of-Thoughts style prompting.
MIT License
Fine-tuned Llama2 13B model designed for ReAct-style and Tree-Of-Thoughts style prompting. The codebase has the following desirable features:
Technical details:
### Human:
INSTRUCTIONS_GO_HERE
### Assistant:
Note that there is a single newline at the end of the prompt. Example:
### Human:
What color is the sky?
### Assistant:
The sky is blue.
First, download everything that requires an internet connection into the current project folder. It will increase to around 30GiB in size:
make download-datasets-and-models
Next, transfer the current project folder to the training machine, where the rest of the training can be performed fully offline:
make train
To use the model, a simple chat-like interface is included for demo purposes, it's not very fancy but it's good enough for testing purposes:
make chat
First, run the following command to create ./exported-models/ggml-robot-agent-q5_K_M.bin
, an 8.6GiB GGML file compatible with Llama.cpp:
make generate-ggml
Now to load the model using Llama.cpp:
make chat-llama-cpp
To use Llama.cpp manually, navigate to your llama.cpp folder and start using the model with the following command (replace PATH_TO_PROJECT_FOLDER
with the path to the current project folder):
./main --model PATH_TO_PROJECT_FOLDER/exported-models/ggml-robot-agent-q5_K_M.bin --color --interactive --interactive-first --mirostat 2 --ctx-size 2048 --reverse-prompt $'\n\n### Human:\n' --prompt $'\n\n### Human:\n' --in-suffix $'\n### Assistant:\n'