Kudasai

Move to EasyTL 0.1.1
Added two new OpenAI exceptions to standard retry decorator

Moved to EasyTL v0.1.0
Support for new 4-9-24 OpenAI models
readjustment of how cost calculation is performed, fix a few bugs regarding that.
Changed how cost estimate is displayed.
removed gemini ultra

Kudasai v3.4.2 Patch Notes

Enhancements

• Program Size Optimization:
  • Reduced the program's size by offloading most of the translation functionality to the EasyTL package.
• Requirements and Documentation:
  • Updated requirements.txt to reflect the new dependencies and ensure compatibility with the EasyTL package and Kairyou updates
  • The readme has been updated to reflect changes.

Dependency Updates

• Google GenerativeAI Version Bump:
  • Updated the google-generativeai library to version 0.4.1.

Bug Fixes and Improvements

• Token Counter Correction:
  • Made corrections to token_counter.py to accurately calculate the cost associated with translation requests.
• WebGUI Enhancements:
  • Implemented slight changes to the web GUI to clarify functionality.
• Documentation Overhaul:
  • Program has been redocumented.
• Directory Creation Refactor:
  • Refactored the directory creation process to support nested directories.

Fixed improper purge storage for hugging space instance (does not affect main client)

bumped kairyou to 1.4.1

Gemini now supports 1.5 pro and ultra.

For info on ultra's status please see:
https://github.com/Bikatr7/Kudasai/pull/12

What's Changed

• added gemini 1.5 pro and ultra support by @Bikatr7 in https://github.com/Bikatr7/Kudasai/pull/12

Full Changelog: https://github.com/Bikatr7/Kudasai/compare/v3.4.0...v3.4.1

Release Notes for Kudasai: Major Update

We're excited to announce a major update for Kudasai, introducing significant changes and improvements to enhance your experience and streamline the translation process. This update includes breaking changes, new features, and important bug fixes. Please read through these notes carefully to understand the changes.

If you're just now installing, please see this to get started.

Breaking Changes:

• Updated Python Version: We've upgraded the required Python version to 3.10 to leverage new features and improvements. Ensure your environment is updated to continue using Kudasai without issues.
• Settings File Compatibility: The format of the settings file has changed. The old settings file will not be compatible with this new version. However, Kudasai will offer assistance in updating your existing settings file to the new format. If you choose not to use the automatic update feature, you will need to update your settings file manually.

New Features:

• Gemini Support: Kudasai now supports Gemini as a translation option, expanding your choices for translation services.
• Hugging Face WebGUI: We are now hosting an instance of the WebGUI on Hugging Face, making it easier for users who prefer or need to run Kudasai in a web environment.
• Enhanced Readme: The documentation has been made clearer, providing better guidance and instructions for using Kudasai.
• Linux Mint Compatibility: Through extensive testing, we've ensured that the app works seamlessly on Linux Mint. We encourage testing on other operating systems and welcome feedback.
• Improved WebGUI: The WebGUI has received several improvements, offering a better user experience and more responsive controls.

Bug Fixes and Improvements:

• Preprocessing Import Fix: Resolved an error that prevented the kairyou (preprocessing) module from being imported properly.
• Enhanced JSON Handling: Made slight bug fixes and corrections to json_handler settings. Adjusted demo and default kijiku rules files. Thorough testing on settings changing has been done to ensure functionality.
• UI and Functionality Tweaks: Made slight UI adjustments, improved the responsiveness of the "Send To x" button's in the WebGUI, and added prompts for resetting invalid kijiku rules and resetting to default.
• Stability Enhancements: Reworked crash control mechanisms and improved error catching. Introduced a dedicated exit function and corrected dependency versions.
• Translation Process Refinements: Fixed an issue with missing assert in boot. Messages now include a to_dict function for better compatibility with external services.

Documentation and Miscellaneous:

• Various documentation changes have been made to reflect the updates and new features.
• Adjusted the openai service utilization to incorporate the to_dict method, improving interaction with OpenAI's API.
• Moved to a new structure for kijiku rules, enhancing flexibility and support for additional translation options.

Upgrading to the New Version

To upgrade to this new version of Kudasai, please ensure that your environment meets the Python 3.10 requirement. Upon launching Kudasai after the update, you will be prompted to update your settings file. We strongly recommend using the automatic update feature to ensure compatibility.

We appreciate your support and feedback as we continue to improve Kudasai. If you encounter any issues or have suggestions, please reach out to us through the issues tab on our GitHub repository.

Some slight fixes regarding compatibility with Hugging Face, but nothing major from first beta

Full release notes to come with official v3.4.0 launch

Testing for usability on other devices and linux

Webgui/Console/CLI all appear to be functional

Console and CLI appears to work with Gemini and OpenAI

Feel free to test out, I'll make detailed patch notes later but the current README should be all you need

The webgui is still broken

Moved Gradio to 1.19.2 to meet security requirements.
Updated replacements to fit with COTE Trial
Started work on Gemini integration (proof of concept is there and gemini_service is present, just needs to be integrated into the translation module properly.)
Fix several bugs regarding JSON settings, translation modules use max_tokens properly, GUI updates as well.
Re worked crash control
Readme.md update.

Added Indexing feature, some major optimizations on backend side, webgui updates and speed optimizations
Updates to various demo files and replacements JSON
Support for Fukuin tables via Kairyou
Light prep for Gemini addition

Lots of changes that don't really matter, but will be fully detailed upon v3.4.0

Note that while v3.3.3 was heavily tested, it may still be a bit buggy. Please feel free to open any issues if you run into a problem.

(This release is part of an emergency hotfix to address issues that occurred as seen on:
https://trello.com/b/Wsuwr24S/kudasai

Fixed bad gradio version

Details:
Added Indexing feature, some major optimizations on backend side, webgui updates and speed optimizations
Updates to various demo files and replacements jsons
Support for Fukuin tables via Kairyou
Light prep for gemini addition

What's Changed

• Seperation-of-core-translation-logic-from-kijiku by @Bikatr7 in https://github.com/Bikatr7/Kudasai/pull/8

Full Changelog: https://github.com/Bikatr7/Kudasai/compare/v3.2.2...v3.3.0

Kaiseki bug fix

Full Changelog: https://github.com/Bikatr7/Kudasai/compare/v3.2.0...v3.2.1

Removed unneeded code, stripped out old spacy code since no one used it anyways, old setting is kept for backwards compatability but will changed to default j-e paring of 3

Assorted micro optimizations

Removed the dependency on the requests library.
Corrected version comparison logic in the Toolkit class.
Relocated the Kairyou preprocessor to an external library for import.
API keys storage moved to AppData from the user folder. Or under .config for Linux
Improved logging for handling malformed batches in Kijiku.
Fixed a bug in Kijiku where prompt building didn't properly ignore underscores.

Full Changelog: https://github.com/Bikatr7/Kudasai/compare/v3.1.2...v3.1.3

Updated cote replacements json and demo files in prep for v11

Added 0125 model (s) support

Note that 0125 models may or may not be fully functional by openai api until 2 weeks from now (feb 8), this is just so Kudasai supports them when they come out.

Fixed missing favicon in gui lib files that threw a warning when webgui was started.

Updated Japanese and English language models with specific download paths.
Upgraded Gradio to v4.11 and Spacy to >=3.7, <3.8.
Improved web GUI for compatibility with the new updates.

Run pip install -r requirements.txt to update dependencies, pip may warn about lack of compatibility with old JA & EN models, this doesn't matter unless you use them for something outside Kudasai and are okay to delete.

New Features

1. WebGUI Added: Introduced a Web-based Graphical User Interface for all functionality, arguably better than CLI/Console
2. Archival Functionality: Implemented a feature for archiving outputs.

Enhancements

1. Module Organization: Separated modules into common and gui; moved base modules (FileEnsurer, Toolkit, & Logger) into common.
2. JSON Handling Improvements:
   • Massive improvements in kijiku rules JSON validation.
   • Improvements to kijiku rules JSON CLI/console editing.
   • Added model validation for JSON handler.
3. Refactoring and Documentation:
   • Refactored the Kaiseki and Kijiku class and some aspects of Kairyou for better efficiency.
   • Improved code formatting and documentation across various files.
   • Some documentation changes to katakana_handler.py and json_handler.
4. Error Handling and Messages:
   • Enhanced error messages for invalid input files.
   • Updated README.md for webgui.
   • Better error messages for invalid API keys in kaiseki/kijiku and system message validation.

Bug Fixes

1. Boot Process Correction:
   • Fixed a bug where the boot process was not initiated correctly; now ensured that boot is run by the webgui as well.
2. Output Directory Issue: Fixed a bug in boot where the output directory didn't exist upon startup.
3. API Key Exposure: Resolved an issue where the API key was showing up in the debug log with kaiseki.

Changes and Adjustments

1. Requirement Adjustments: Modified requirements to add specific version requests for spacy and gradio imports.
2. Batch Handling: Added clear_batch; modified push_batch to no longer clear the current batch.
3. Kijiku Rules and Settings:
   • Moved default_kijiku_rules to FileEnsurer.
   • Adjusted default kijiku rules and the default value for setup_needed_files JSON.
   • Changed how FileEnsurer handles ensures kijiku_settings.
4. File and JSON Structuring:
   • Fixed some file names and made adjustments to JSON structuring.
   • Streamlined kairyou setup.
5. Miscellaneous:
   • Added countermeasures for successive runs for Kairyou.
   • Kairyou now indicates when preprocessing is skipped.
   • Updated .gitignore.

Note: For detailed information about each feature and bug fix, refer to the specific commit messages in the repository.