ollama-benchmark

Add in default value (unknown) when grabbing cpu, gpu, and os_version in Linux

Fix Debian Linux not using lshw command to grab cpu info

Fix the line break when grabbing GPU info on Windows machines

Fix Grabbing Nvidia GPU info on Linux

add in flag --ollamabin to explicitly give the path to the developer version of ollama while building your own ollama

• Fix Windows version not detecting cmd or powershell mode
• Fix grabbing ollama version issue on Windows

Bump up to v0.3.10

Fix sending data issue

Fix data file path inside package issue

Try to solve package data filepath for reading

Fix PYPI auto publish issue

• Publish code to PYPI
• Auto adjust models for running benchmark vi memory size
• Let user do this, pip install llm-benchmark
• Then do this, llm_benchmark run

Rename test folder to tests to prepare following the PyPI package folder convention

Ollama Windows version need to put in encoding="utf-8" while writing files and running subprocess.run()

fix command typo in README

Fix llava benchmark to 5 sample images

Finish the run_benchmark.py

• (for mistral model) $ python3 ollama-benchmark/run_benchmark.py -m data/benchmark_models.yml --b data/benchmark1.yml -t instruct
• (for llama2 model) $ python3 ollama-benchmark/run_benchmark.py -m data/benchmark_models.yml --b data/benchmark1.yml -t question-answer
• (for llava model) $ python3 ollama-benchmark/run_benchmark.py -m data/benchmark_models.yml --b data/benchmark1.yml -t vision-image

The basic function to run test ollama API with llama2 and mistral 7B models.

The basic parsing log to compute performance metrics (tokens/s) in the output.