A template/demo to show how to use the godot-llm addon
You may simply download the Android apk in the release page to try it out for Android devices.
For Windows/Linux:
Godot LLM Template
from the Godot asset library, or clone this project: git clone https://github.com/Adriankhl/godot-llm-template.git
Godot LLM
directly from the godot asset library, or download the addon from godot-llm and place it to the addons
folderManage External Storage
permission)llava-phi-3-mini-mmproj-f16.gguf
from the same pageFolder and file structure:
├── addons
│ └── godot_llm
│ ├── bin
│ │ ├── libgodot_llm.android.debug.aarch64.so
│ │ ├── libgodot_llm.android.release.aarch64.so
│ │ ├── libgodot_llm.linux.debug.x86_64.so
│ │ ├── libgodot_llm.linux.release.x86_64.so
│ │ ├── libgodot_llm.windows.debug.amd64.dll
│ │ └── libgodot_llm.windows.release.amd64.dll
│ └── godot_llm.gdextension
├── db.gd
├── db.tscn
├── icon.svg
├── icon.svg.import
├── image_to_text.gd
├── image_to_text.tscn
├── LICENSE
├── main.gd
├── main.tscn
├── media
├── project.godot
├── README.md
├── similarity.gd
├── similarity.tscn
├── text_generation.gd
└── text_generation.tscn
You can now open the application with Godot
You will see 4 modes: Text Generation
, Similarity
, Image to Text
, and Vector Database
.
In each mode, you need to click Model
to point to the correct model gguf
file. Additionally, in Image to Text
mode, you need to click Mmproj
to point to your *mmproj*gguf
file.
You will see there are 3 generation mode: Simple
, Instruct
, and Interactive
.
Simple
modeJust edit the prompt and click Generate
. Click Stop
to stop the generation.
You choose None
schema or Person
schema, if you choose None
schema, the model will generate the information of a character with Json format.
Instruct
modeClick Start
first with empty prompt, then modify the prompt and click Continue
when it is available to talk to the AI. This is an interactive mode. Whenever you see the Continue
button is available, either input something in the prompt and click Continue
to send the input, or simply click Continue
to let the AI talk itself.
Interactive
modeSimilar to the instruct mode, except that it relies on a good initial prompt and additoinal reverse prompt and input suffix settings to generate a smooth conversation. The preset prompt should work decently well, click the Start
button with the preset prompt and start talkin to the AI.
Input prompts to compute the Embedding
, or input two prompts to calculate their Similarity
Select an Image
from your device. Alternative, click From View
to capture your game screen as the image. You can Compute Base64
encoding for your image, or decode a base64 string back to Compute Image
. Edit the prompt, and generate text from Image
, Base64
, or View
(your game screen).
Click Metadata: id, year
to define the metadata of the tables, then click Create tables
to create a some tables for embedding storage and retrieval. Click Store text
to split the document into chunks and store into the database, you may also click Split text
to see the chunks. Click Retrieve
to retrive three most similar text chunks with the upper left prompt, which satisfy the sql-where-clause in the middle right. You may also turn on "Verbose stdout", modify the upper prompt to a valid sql statement, and click execute to run the sql statement directly, you will see the result in godot log.