GameVisionAid is an accessibility tool designed to assist visually impaired gamers by enhancing visual cues in video games. It uses real-time object detection to create customizable overlays around enemy players, making it easier to identify and engage with them during gameplay.
OTHER License
Readme.md
, License.md
, Code_of_Conduct.md
, Security.md
.GameVisionAid is an accessibility tool designed to assist visually impaired gamers by enhancing visual cues in video games. It uses real-time object detection to create customizable overlays around enemy players, making it easier to identify and engage with them during gameplay.
Click the link to read Instructions ๐.
game-vision-aid
channel if you need helpIf you find this project useful, please give it a star! Your support is appreciated and helps keep the project growing. ๐
game-vision-aid/
โโโ .github/ # For Issues
โโโ AMD-GPU-Only folder # AMD GPU folder
โโโ โโโ IMPORTANT_FOR_AMD_GPU.md # Important Readme.md for AMD GPU USERS
โโโ โโโ amd_gpu_requirements.bat # AMD GPU requirements batchfile
โโโ โโโ amdgpu-launcher.bat # AMD GPU launcher batchfile
โโโ โโโ config-launcher.bat # AMD config-launcher batchfile
โโโ โโโ config.py # AMD Python Configuration file
โโโ โโโ main.py # AMD Main script file
โโโ โโโ readme.md # Main Readme.md for AMD GPU USERS
โโโ โโโ requirements.txt # AMD List of required Python packages
โโโ banner folder # Banner Image PNG
โโโ models/ # Where the models go
โโโ โโโ fn_v5.pt # Custom YOLOv5 model (if available)
โโโ โโโ model_v5s.pt # Custom YOLOv5 model (if available)
โโโ โโโ fn_v5v5480480Half.onnx # Custom ONNX model (if available) nvidia gpu 2060 super
โโโ โโโ model_fn_v5v5320320Half.onnx # Custom ONNX model (if available) nvidia gpu 2060 super
โโโ โโโ model_v5sv5320320Half.onnx # Custom ONNX model (if available) nvidia gpu 2060 super
โโโ โโโ model_v5sv5480480Half.onnx # Custom ONNX model (if available) nvidia gpu 2060 super
โโโ Export-Models # Export Models Folder
โโโ โโโ models # part of the exporting process
โโโ โโโ ultralytics1/utils # ultralytics/utils
โโโ โโโ utils # utils folder
โโโ โโโ export.py # export script
โโโ โโโ update_ultralytics.bat # update ultralytics batchfile
โโโ src/ # Part of RUST Application
โโโ โโโ main.rs # Part of RUST Application
โโโ .gitignore # gitignore
โโโ CODE_OF_CONDUCT.md # CODE_OF_CONDUCT.md
โโโ Cargo.lock # Part of RUST Application
โโโ Cargo.toml # Part of RUST Application
โโโ Contact.md # Contact
โโโ LICENSE.md # LICENSE
โโโ List-of-Colors.md # List of colors for the Bounding Boxes
โโโ README.md # Project documentation
โโโ SECURITY.md # Security documentation
โโโ config.py # Configuration file
โโโ cudnn_instructions.js # Instructions in JavaScript
โโโ cupy_cuda11x.bat # Cupy Cuda 11x Batchfile
โโโ install_python.bat # Python 3.11.6 Batchfile
โโโ install_pytorch.bat # Pytorch Build wheel for the script GPU
โโโ install_pytorch_cpu_only.bat # Pytorch Build for CPU
โโโ main.py # Main script file
โโโ model_v5s.pt # pt file goes in modelS folder
โโโ model_v5sv5480480Half.onnx # onnx file goes in modelS folder
โโโ model_fn_v5v5320320Half.onnx # onnx file goes in modelS folder
โโโ models-fn_v5.zip # Compressed models to in main models folder
โโโ nodejs-instructions.ps1 # Instructions in Powershell
โโโ notes.txt # Read Notes.txt
โโโ requirements.bat # Installs list of required Python packages
โโโ requirements.txt # List of required Python packages
โโโ update_ultralytics.bat # Update Batch Script for Ultralytics
BetterCam
for fast and efficient screen capturing.YOLOv5
for detecting enemies in video games.color of the overlay boxes
around detected enemies.GPU acceleration
for faster processing with CUDA-enabled GPUs
.NVIDIA GPU with CUDA support
ALL
windows computers.
click start menu
in the search bar
type cmd.exe
click enter. :)
to make sure you have installed Python 3.11.6
correctly:
cmd.exe
python --version
install_python.bat
it will auto add to system path
. V3.11.6
Follow these steps to set up and run GameVisionAid:
Clone the repository:
git clone https://github.com/kernferm/game-vision-aid.git
cd game-vision-aid
Set up a virtual environment (optional but recommended):
python -m venv game-vision-aid_env
To deactivate the virtual environment:
deactivate
On Windows, activate the environment:
game-vision-aid_env\Scripts\activate
Install dependencies:
pip install -r requirements.txt
For NVIDIA GPU Support (Optional):
pip3 install torch==2.4.1+cu118 torchvision==0.19.1+cu118 torchaudio==2.4.1+cu118 --index-url https://download.pytorch.org/whl/cu118
For CPU Only (Laptops with no GPU):
install_pytorch_cpu_only.bat
to install CPU-based PyTorch.install_pytorch_cpu_only.bat
doesn't work, you can manually run the following command in CMD.exe
:
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu/torch_stable.html
AMD GPU
UsersIf you have an AMD GPU, follow the steps below to set up DirectML support for faster processing:
Install AMD GPU dependencies:
amd_gpu_requirements.bat
script to install the necessary dependencies for AMD GPU:
amd_gpu_requirements.bat
This will install DirectML and other required packages, enabling the project to run efficiently on AMD hardware.
Launch the application:
amdgpu-launcher.bat
script to run the application with AMD GPU support.Check ## How to Use config.py
If you encounter any issues, such as an Ultralytics error, follow the steps below:
Run the following command in CMD.exe
to upgrade Ultralytics:
pip install --upgrade ultralytics
The Ultralytics
package is already included in the requirements.txt
and requirements.bat
files.
Use the update_ultralytics.bat
script if you continue to experience Ultralytics errors.
CMD.exe
pip install --upgrade ultralytics
Ultralytics
in the requirements.txtand
requirements.bat`.install_pytorch.bat
as it is tied to cu118
. (Recommended)update_ultralytics.bat
if you have ultralytics error
First, download the CUDA Toolkit 11.8 from the official NVIDIA website:
๐ Nvidia CUDA Toolkit 11.8 - DOWNLOAD HERE
.exe
) and follow the instructions provided by the installer.cmd.exe
terminal and run the following command to ensure that CUDA has been installed correctly:
nvcc --version
This will display the installed CUDA version.
Run the following command in your terminal to install Cupy:
pip install cupy-cuda11x
@echo off
echo MAKE SURE TO HAVE THE WHL DOWNLOADED BEFORE YOU CONTINUE!!!
pause
echo Click the link to download the WHL: press ctrl then left click with mouse
echo https://github.com/cupy/cupy/releases/download/v12.0.0b1/cupy_cuda11x-12.0.0b1-cp311-cp311-win_amd64.whl
pause
echo Installing CuPy from WHL...
pip install https://github.com/cupy/cupy/releases/download/v12.0.0b1/cupy_cuda11x-12.0.0b1-cp311-cp311-win_amd64.whl
pause
echo All packages installed successfully!
pause
Download cuDNN (CUDA Deep Neural Network library) from the NVIDIA website:
๐ Download CUDNN. (Requires an NVIDIA account โ it's free).
Open the .zip
cuDNN file and move all the folders/files to the location where the CUDA Toolkit is installed on your machine, typically:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
Download TensorRT 8.6 GA.
Open the .zip
TensorRT file and move all the folders/files to the CUDA Toolkit folder, typically located at:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
Once all the files are copied, run the following command to install TensorRT for Python:
pip install "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\python\tensorrt-8.6.1-cp311-none-win_amd64.whl"
๐จ Note: If this step doesnโt work, double-check that the .whl
file matches your Python version (e.g., cp311
is for Python 3.11). Just locate the correct .whl
file in the python
folder and replace the path accordingly.
Add the following paths to your environment variables:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\lib
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\libnvvp
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin
Once you have CUDA 11.8 installed and cuDNN properly configured, you need to set up your environment via cmd.exe
to ensure that the system uses the correct version of CUDA (especially if multiple CUDA versions are installed).
cmd.exe
cmd.exe
You need to add the CUDA 11.8 binaries to the environment variables in the current cmd.exe
session.
Open cmd.exe
and run the following commands:
set PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin;%PATH%
set PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\libnvvp;%PATH%
set PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\extras\CUPTI\lib64;%PATH%
These commands add the CUDA 11.8 binary, lib, and CUPTI paths to your system's current session. Adjust the paths as necessary depending on your installation directory.
nvcc --version
This should display the details of CUDA 11.8. If it shows a different version, check the paths and ensure the proper version is set.
Set the Environment Variables for a Persistent Session
If you want to ensure CUDA 11.8 is used every time you open cmd.exe
, you can add these paths to your system environment variables permanently:
Open Control Panel
-> System
-> Advanced System Settings
.
Click on Environment Variables
.
Under System variables
, select Path
and click Edit
.
Add the following entries at the top of the list:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\libnvvp
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\extras\CUPTI\lib64
This ensures that CUDA 11.8 is prioritized when running CUDA applications, even on systems with multiple CUDA versions.
cudnn64_8.dll
is also in your system path:set PATH=C:\tools\cuda\bin;%PATH%
This should properly set up CUDA 11.8 to be used for your projects via cmd.exe
.
config.py
SAVE
your config.py
or from the config-launcher.bat
settings then re start ## Usage Section
on the readme.md
config.py
file allows you to easily configure screen capture, object detection, YOLO model settings, and bounding box colors for your specific needs.# Configuration for BetterCam Screen Capture and YOLO model
# Screen Capture Settings
screenWidth = 480 # Updated screen width for better resolution
screenHeight = 480 # Updated screen height for better resolution
# Object Detection Settings
confidenceThreshold = 0.5 # Confidence threshold for object detection
nmsThreshold = 0.4 # Non-max suppression threshold to filter overlapping boxes
# YOLO Model Settings
modelType = 'onnx' # Choose 'torch' or 'onnx' based on the model you want to load
torchModelPath = 'models/fn_v5.pt' # Path to YOLOv5 PyTorch model
onnxModelPath = 'models/fn_v5.onnx' # Path to YOLOv5 ONNX model
# BetterCam Settings
targetFPS = 60 # Frames per second for capturing
maxBufferLen = 512 # Max buffer length for storing frames
region = None # Region for capture (set to None for full screen)
useNvidiaGPU = True # Set to True to enable GPU acceleration if available
# Colors for Bounding Boxes
boundingBoxColor = (0, 255, 0) # Default bounding box color in BGR format
highlightColor = (0, 0, 255) # Color for highlighted objects
SAVE
your config.py
or from the config-launcher.bat
settings then re start ## Usage Section
on the readme.md
Open the config.py
file.
Set the screenWidth
and screenHeight
to match your preferred screen resolution.
Adjust confidenceThreshold
to modify the object detection sensitivity (default is set to 0.5
for moderate confidence).
Configure the YOLO Model Settings
to specify whether you want to use the PyTorch
or ONNX
model.
Set the desired targetFPS
, and if you have an NVIDIA GPU, ensure useNvidiaGPU
is set to True
for optimal performance.
Customize the boundingBoxColor
and highlightColor
to match your preference for object detection overlays.
Save the changes, and the settings will automatically be applied when you run the program
config-launcher.bat
to open the config.py
sure
to run the config-launcher.bat
in the same folder as the config.py
This color dictionary includes a range of colors that are high-contrast, colorblind-friendly, and visually impaired-friendly. These colors have been selected to provide maximum accessibility and usability for a broad range of users.
Color | RGB Values | Notes |
---|---|---|
Red | (0, 0, 255) | Good contrast with green, blue, yellow |
Green | (0, 255, 0) | High contrast with red, magenta |
Blue | (255, 0, 0) | Standard blue |
Yellow | (0, 255, 255) | High contrast, suitable for colorblindness |
Cyan | (255, 255, 0) | Good contrast with red, blue |
Magenta | (255, 0, 255) | High contrast |
Black | (0, 0, 0) | High contrast with all light colors |
White | (255, 255, 255) | High contrast with dark colors |
Color | RGB Values | Notes |
---|---|---|
Dark Red | (139, 0, 0) | Distinguishable from green, easier for colorblind users |
Orange | (0, 165, 255) | Strong contrast with most colors |
Light Blue | (173, 216, 230) | Easily distinguishable from green |
Purple | (128, 0, 128) | High contrast, distinguishable for most users |
Brown | (165, 42, 42) | Distinct and neutral color |
Pink | (255, 182, 193) | Good contrast, especially for colorblindness |
Lime | (0, 255, 128) | Bright color, good contrast with red and blue |
Turquoise | (64, 224, 208) | Strong contrast with most shades |
Navy | (0, 0, 128) | High contrast for visual impairment |
Gold | (255, 215, 0) | Bright and distinguishable |
Silver | (192, 192, 192) | Neutral, high contrast with darker colors |
Dark Orange | (255, 140, 0) | High contrast and distinguishable |
Indigo | (75, 0, 130) | Deep color, good contrast with light shades |
Teal | (0, 128, 128) | Strong contrast, good for visual impairments |
Olive | (128, 128, 0) | Neutral, good for colorblind users |
Maroon | (128, 0, 0) | Distinct and high contrast |
Sky Blue | (135, 206, 235) | Bright and easily visible |
Chartreuse | (127, 255, 0) | High contrast for most users |
config.region
in config.py
): This setting defines the area of the screen that BetterCam will capture for object detection.
None
: Captures the entire screen.(x1, y1, x2, y2)
: Captures a specific portion of the screen defined by the coordinates (x1, y1)
as the top-left corner and (x2, y2)
as the bottom-right corner (e.g., (100, 100, 800, 600)
will capture a section from (100,100) to (800,600)).By adjusting this setting, you can focus the capture on a particular region of the screen, which can help reduce processing load and ignore unnecessary areas. Itโs especially useful for games or applications where you only need to detect objects in a specific portion of the display.
CMD.exe
non admin mode , copy the file locationright click
on file properties
location
copy the entire locationCMD.exe
cd
paste
the location you just copied
to CMD.exe
remove
the part where it says main.py
at the end of the file location
in CMD.exe
press enter
code below
enter
python main.py
main-launcher.bat
to run the main.py
sure
to run the main-launcher.bat
in the same folder of the main.py
A prompt will appear. Enter the color name you want for the overlay boxes around detected enemies. You can type exit
or q
to quit the program.
The overlay will run in parallel with your game, highlighting detected enemies with the chosen color.
Press q
or type exit
at any time "after you've start the program of course" to close the overlay and exit the program.
Place your .pt
or .onnx
file in the models/ directory.
Update the config.py
file to load your custom model by setting:
torchModelPath = 'models/your_model.pt' # or
onnxModelPath = 'models/your_model.onnx'
Performance on CPU: The overlay may run slower on systems without a GPU. Using a CUDA-enabled GPU is recommended for real-time performance.
Compatibility: This tool is intended for games that allow screen capturing and do not violate any terms of service.
Contributions are welcome! Feel free to submit a pull request or open an issue to discuss improvements, bugs, or new features.
## want to contribute
### please read the repo
- use python 3.11.6
- fork the repo if you want to contribute ONLY!!!
- make a pull request
- there are 5 branches - use `patch-1` to upload or `patch-2` upload new code.
- download the `main-branch`
- trying to get the live feed to work properly - (shows up multiple times, i will double check)
- trying to get the visual colors to work properly
#### if you find anything else, make a notes-2.txt.
This project is proprietary, and all rights are reserved by the author. Unauthorized copying, distribution, or modification of this project is strictly prohibited unless you have written permission from the developer or the FNBUBBLES420 ORG.
Thanks to the developers of:
GameVisionAid
is designed to assist visually impaired or color-blind gamers by enhancing visual cues in video games. It uses real-time object detection to create customizable overlays around enemy players, making it easier to identify and engage with them during gameplay.
This tool runs in parallel with any game and does not modify the game files or violate any terms of service. It captures the screen and provides an overlay to help users better perceive in-game elements. The developer is not responsible for any misuse of this tool.