NoLlama is a lightweight terminal-based alternative to Ollama, enabling interaction with large language models like GPT-4o and LLaMA 70B directly from your terminal. It offers a sleek UI, multiple model choices, colorful markdown rendering, and low memory usage, all without needing a browser.
GPL-3.0 License
NoLlama is a terminal-based interface for interacting with large language models (LLMs) that you can't run locally on your laptop. Inspired by Ollama, NoLlama provides a streamlined experience for chatting with models like GPT-4o, GPT-4o-mini, Claude 3 haiku, Mixtral, LLaMA 70B, and more, directly from your terminal.
While Ollama offers a neat interface for running local LLMs, their performance and capabilities often fall short of these massive models. NoLlama bridges this gap by allowing you to interact with these powerful models using a lightweight terminal UI, complete with colorful markdown rendering, multiple model choices (currently 15+ models), and efficient memory usage.
model
in the chat to switch between models.clear
to clear the chat history.q
, quit
, or exit
to leave the chat.nollama
in the terminal to start.--stream
flag (unstable).torsocks
to route all traffic through the Tor network for privacy.Download the Linux binary from the releases page.
Make the binary executable:
chmod +x nollama
Move the binary to /usr/bin/
for global access:
sudo mv nollama /usr/bin/
Run NoLlama:
Start NoLlama from the terminal by simply typing:
nollama
This will start NoLlama in the default mode.
Note: The provided Linux binary is compiled using Python 3.12.4 and requires glibc 2.38+ to run. This may not work on some stable distributions that ship with older packages, such as Debian. In such cases, it is recommended to build from source to ensure compatibility.
If you'd like to build NoLlama from source, follow these steps:
Clone the Repository:
git clone https://github.com/spignelon/nollama.git
cd nollama
Install Dependencies:
You can install the required dependencies using pip
:
Creating a python virtual environment:
virtualenv .venv
source .venv/bin/activate
pip install -r requirements.txt
Compile the Script (Optional):
If you want to compile the script into a standalone executable, you can use PyInstaller:
First set version_check: bool = False
in .venv/lib/python3.12/site-packages/g4f/debug.py
Then:
pyinstaller --onefile --name=nollama --collect-all readchar nollama.py
Move the Executable to /usr/bin/
:
After compilation, move the binary to /usr/bin/
:
sudo mv dist/nollama /usr/bin/nollama
Run NoLlama:
Start NoLlama by typing:
nollama
Switch Models: Type model
in the chat to choose a different LLM.
Clear Chat: Type clear
to clear the chat history.
Exit: Type q
, quit
, or exit
to leave the chat.
Default Mode: Run NoLlama without any flags for standard operation:
nollama
For enhanced privacy and anonymity, you can use torsocks
to route NoLlama's traffic through the Tor network. This ensures that all requests are anonymized and cannot be traced back to you.
sudo apt update
sudo apt install tor
sudo pacman -S tor
sudo dnf install tor
After installation, you need to enable and start the Tor service:
sudo systemctl enable tor
sudo systemctl start tor
Once Tor is running, you can use torsocks
to run NoLlama anonymously:
torsocks nollama
This will ensure that all your interactions with NoLlama are routed through the Tor network, providing a layer of privacy and anonymity.
Streaming Mode:
NoLlama includes an experimental streaming mode that allows you to see responses as they are generated. This mode is currently unstable and may cause issues. To enable streaming, use the --stream
flag:
nollama --stream
Contributions are welcome! If you have suggestions for new features or improvements, feel free to open an issue or submit a pull request.
NoLlama is not affiliated with Ollama. It is an independent project inspired by the concept of providing a neat terminal interface for interacting with language models, particularly those that are too large to run locally on typical consumer hardware or not available for self hosting.
This project is licensed under the GPL-3.0 License.