NeuralNetworks

• Fixed:
• • https://github.com/GiorgosXou/NeuralNetworks/issues/32 https://github.com/GiorgosXou/NeuralNetworks/issues/39 warnings (+improvement)

• Added:
• • _2_ 0B00000001-optimization, disabling backpropagation when it's not automatically disabled. helps reduce the size of sketch
• Fixed:
• • Missing MULTIPLY_BY_INT_IF_QUANTIZATION in Layer::FeedForward_Individual(...) (not an issue with EEPROM or PROGMEM)
    
• Improved:
• • Ignore * _inputs during compilation when there's NO_BACKPROP

• Fixed:
• • Crucial mistake made in v2.3.6. Forgot to Increment i in destructor (even though I tested it before releasing it)

• Fixed:
• • Destructor, deallocating last-layer's outputs when not defined as REDUCE_RAM_DELETE_OUTPUTS resulting in potential unadentified behaviours
    
• Improved:
• • NN.Feedforward(...) logic, reduced program space
• • Reduced program space when NO_BACKPROP and, SD.h not used (above #include <NeuralNetwork.h>)

• Improved:
• • REDUCE_RAM_DELETE_OUTPUTS-optimization logic
• • Removed redundant nullification of deleted pointers

• Added:
• • _2_OPTIMIZE 0B00000010 aka REDUCE_RAM_STATIC_REFERENCE_FOR_MULTIPLE_NN_OBJECTS. Or in other words: REDUCE_RAM_STATIC_REFERENCE-optimization support for when using multiple NN objects in your sketch interchangeably. Meaning no more need for manual updating of NN.me ...

• Added:
• • int16_t&int8_t quantization support (for all storage-medias)
• • Training example with tensorflow + quantization
• • Pre-trained int8_t-quantized NN example
• Fixed:
• • https://github.com/GiorgosXou/NeuralNetworks/issues/38 ESP issue with SD.h

• Fixed:
• • https://github.com/GiorgosXou/NeuralNetworks/issues/37 issue due to binary constants marked as deprecated in new cores
• • FIXED: error: extended character X is not valid in an identifier

[!WARNING]
Due to #37 being an issue for many versions of new arduino-cores, I highly-suggest to replace binary-constants in your sketches with the prefix 0 like so: #define _1_OPTIMIZE B00000000 --> #define _1_OPTIMIZE 0B00000000.

• Added:
• • Version message during compilation
• • Reminder REDUCE_RAM_WEIGHTS_LVL2 message during compilation
• Fixed:
• • #35 unnecessary inline declaration of array, causing issues with old versions of gcc
• • #34 Misleading message for multiple custom functions
• • Potential ODR violation
• Replaced:
• • ATtiny85 compilation message, with a more important one related to NN.print()

[!NOTE]
Special thanks to Grzegorz Kuduk for using and referencing my library into his master's thesis "Distributed machine learning in a microcontroller network" PDF.

Bumping up version because I accidently forgot to do so in library.properties and Arduino's library-manager won't show\update-to the latest version otherwise. Additionally: updated README.md and resolved another typo, and issue related to previous version should be resolved when this log gets updated.

• Added:
• • Support for NN without biases
• • Support for Multiple Biases
• • ^^^ ✨ Tensorflow example
• • Optional F() optimization
• Fixed:
• • #26 #28 #29 #30

[!NOTE]
I want to really thanks @ripred for his kind and honest words, especially for "... You did a great job on this and should be proud.". But also, I want to thanks Ivo Ljubičić for using my library in his ✨ master thesis named "Artificial skin concept for human-robot physical interaction". And last but not least @pjurczen for giving me the spark I needed when he said "... I'd love to see this feature added!".

(Additionally: Note that all feature are compatible with almost all optimization use cases)

• Added support for:
• • Internal EEPROM, SD
• • 8-byte double precision
• • Custom activation functions
• • Basic ESP32-S3 SIMD instructions
• Fixed:
• • Multiple #define - compiler warnings
• • Many issues related to ESP32 compilation
• • #14 #17 #18 #19 #20 #21 #22 #23

[!NOTE]
Special thanks to the "National Institute of Telecommunications Research and Training- National University of Engineering, Perú" for using and referencing my library into their beautiful article

• Minor general changes.
• Added more loss functions.
• Added one more optimization property.
• Added even more activation functions (total 14).
• Made the examples clearer & more understandable +etc.
• Reduced ROM,RAM & CPU caused by unnecessary compilation of code.

• Crucial fix for back-propagation with "multiple activation functions".
• Support for "multiple activation function" use without defining Functions.
• Fixed const pointer ISSUE .
• Minor fixes.

Support for:

• More Activation Functions.
• Activation Functions Per Layer.
• Optimizations based on user's Preference.

• Fixed critical issue
  new version is coming soon

• Optimazed Algorithm For Less SRAM Usage.
• Only Sigmoid Activation Function.
• Use of PROGMEM.