djl

An Engine-Agnostic Deep Learning Framework in Java

APACHE-2.0 License

Stars
4.1K
View Code on GitHub Visit Website
Ecosystems: PyTorch, Tensorflow, Java

Bot releases are hidden (Show)

djl - DJL v0.30.0 Release Latest Release

Published by xyang16 about 1 month ago

Key Changes

Enhancement

Bug Fixes

Documentation

CI/CD

Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.29.0...v0.30.0

djl - v0.29.0

Published by ydm-amazon 3 months ago

Key Changes

  • Upgrades for engines

    • Upgrades PyTorch engine to 2.3.1
    • Upgrades TensorFlow engine to 2.16.1
    • Introduces Rust engine CUDA support
    • Upgrades OnnxRuntime version to 1.18.0 and added CUDA 12.4 support
    • Upgrades javacpp version to 1.5.10
    • Upgrades HuggingFace tokenizer to 0.19.1
    • Fixes several issues for LightGBM engine
    • Deprecated llamacpp engine

  • Enhancements for engines and API

    • Adds Yolov8 segmentation and pose detection support
    • Adds metric type to Metic class
    • Improves drawJoints and drawMask behavior for CV model
    • Improves HuggingFace model importing and conversion tool
    • Improves HuggingFace NLP model batch inference performance
    • Adds built-in ONNX extension support
    • Adds several NDArray operators in PyTorch engine
    • Adds fp16 and bf16 support for OnnxRuntime engine
    • Adds CrossEncoder support for NLP models

Enhancements

Bug Fixes

Documentation

CI/CD

New Contributors

Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.28.0...v0.29.0

djl - DJL v0.28.0 Release

Published by tosterberg 5 months ago

Key Changes

Enhancement

Bug Fixes

Documentation

CI/CD

New Contributors

Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.27.0...v0.28.0

djl - v0.27.0

Published by xyang16 7 months ago

Key Changes

Enhancement

Bug Fixes

Documentation

CI/CD

New Contributors

Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.26.0...v0.27.0

djl - DJL v0.26.0 Release

Published by siddvenk 9 months ago

Key Changes

  • LlamaCPP Support. You can use DJL to run supported LLMs using the LlamaCPP engine. See the Chatbot example here to learn more.
  • Manual Engine Initialization. You can configure DJL to not load any engines at startup, and query/register engines programmatically at runtime
  • Engine Updates:
    • PyTorch 2.1.1
    • Huggingface Tokenizers 0.15.0
    • OnnxRuntime 1.16.3
    • XGBoost 2.0.3

Enhancement

Bug Fixes

Documentation

CI/CD

New Contributors

Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.25.0...v0.26.0

djl - DJL v0.25.0 Release

Published by siddvenk 10 months ago

Key Changes

Enhancement

Bug fixes

Documentation and Examples

CI

New Contributors

Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.24.0...v0.25.0

djl - DJL v0.24.0 Release

Published by zachgk about 1 year ago

Key Features

Enhancement

Bug fixes

Documentation and Examples

CI

New Contributors

Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.23.0...v0.24.0

djl - DJL v0.23.0 release

Published by sindhuvahinis over 1 year ago

Key Features

  • Upgrades for engines

    • Upgrades PyTorch engine to 2.0.1
    • Upgrades javacpp version to 1.5.9 (#2636)
    • Upgrades HuggingFace tokenizer to 0.13.3 (#2697)
    • Upgrades OnnxRuntime version to 1.15.0 and other dependencies version (#2658)

  • Enhancements for engines and API

    • Adds XGBoost aarch64 support (#2659)
    • Adds fastText macOS M1 supports (#2639)
    • Creates asynchronous predictStreaming (#2615)

  • Introduces text-generation search algorithm

    • Implements text-generation search algorithm (#2637)
    • Enhancement features for LMSearch (#2642)

Enhancement

  • DJL API improvements:

    • Adds uint16, uint32, uint64, int16, bf16 data type (#2570)
    • Adds NDArray topK operator for PyTorch (#2634)
    • Adds support for unsigned datatype (#2574)
    • Allows subclass access member variable of Predictor (#2582)
    • Makes PredictorContext constructor public (#2586)
    • Refactor ChunkedBytesSupplier to avoid unnecessary conversion (#2587)
    • Move compileJava() into ClassLoaderUtils (#2600)
    • Enable boolean input on ort (#2644)
    • Adds more logs for platform detection (#2646)
    • Improves DJL URL error message (#2678)
    • Avoid exception if SecurityManager is applied (#2665)
    • Masks sensitive env vars in debug print out (#2657)
    • open isImage() method to package children for reuse-enabling custom datasets (#2662)
    • Migrate google analytics (#2654)

  • PyTorch engine improvements

    • Load dependencies with specific order (#2599)
    • Improves IValue tuple of tuple support (#2651)
    • Add basic median support (#2701)

  • Spark extension enhancements

    • Support requirements.txt in model tar file (#2528)
    • Upgrade dependency version in Dockerfile (#2569)
    • Use batch predict in spark (#2545)
    • Change implicit conversions to explicit (#2595)

  • Huggingface tokenizer enhancements

    • Allow creating BPE huggingface tokenizers (2550)

  • Tensorflow engine enhancements

    • Reload javacpp properties (2668)

Breaking change

Bug fixes

  • Avoids exception for cuda version lower than 10.x (#2583)
  • Reverts "[bom] Simplify BOM build script (#2438)" 2598
  • CI fails looking for v3 reverting to v2 (#2604)
  • Fixes the dependencies issue (#2609)
  • Fixes the usage of the repeat function for embedding (#2590)
  • Adds missing djl-zero to bom (#2625)
  • Fixes tabnet predictor (#2643)
  • Fixes error message in X.dot(w) (#2688)
  • Fixes liquid parsing issues in pytorch ndarray cheatsheet (#2690)
  • Fixes getIOU bug (#2674)
  • Fixes setup formatting (#2653)
  • Fixes broken link (#2622)
  • Fixes LocalRepository detection (#2593)
  • Fix jupyter notebook links (#2704)

Documentation and Examples

  • Adds docs on JNI compilation (#2510)
  • Updates import tensorflow model README (#2614)
  • Updates pytorch native JNI development document (#2613)
  • Setup - Running on M1 Macs (#2652)
  • Adds pytorch vs djl ndarray cheatsheet (#2661)
  • Updates timeseries README (#2667)
  • Adds PT NDArray cheat sheet to docs (#2670)
  • Cleans stable_diffusion and add missing .md language blocks (#2635)
  • Updates README (#2596)
  • Fixes typos (#2603)
  • Fixes markdown format (#2608)

CI improvements

  • Upgrades github action gradle plugin to v3 (#2576)
  • Avoids upload djl-serving.tar to S3 if already exist (#2578)
  • Upgrades spotbugs to 5.0.14 (#2594)
  • Upgrades gradle to 8.1.1 (#2611)
  • Publishes PyTorch 2.0.1 jni package (#2699)
  • Removes cu102 test (#2689)
  • Fixes nightly publish ci bug (#2691)
  • Fixes djl-serving release publish workflow script (#2568)
  • Minor fix to the instance spinning (#2606)
  • Adds Triton and FasterTransformers source build instruction (#2605)
  • Publishes triton executable (#2617)
  • Adds http endpoint in the build (#2618)

Contributors

Thank you to the following community members for contributing to this release:
@frankfliu
@KexinFeng
@lanking520
@xyang16
@zachgk
@takanori-ugai
@tosterberg
@siddvenk

New Contributors

Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.22.1...v0.23.0

djl - DJL v0.22.1 release

Published by frankfliu over 1 year ago

Key Features

  • Upgrades and enhancements for Engines
    • Upgrades PyTorch to 1.13.1 (#2245)
    • Upgrades TensorFlow engine to 2.10.1 (#2440)
    • Upgrades XGBoost to 1.7.5 (#2522)
    • DJLServing release 0.22.1

Enhancement

  • Introduces several enhancement for HuggingFace tokenizer:

    • Allows tokenizer native library load from different classloader (#2465)
    • Makes Huggingface model zoo lazy load (#2469)
    • Make Huggingface tokenizers translator factory serializable (#2442)

  • Introduces several enhancement for Spark extension:

    • Adds audio predictors (#2466)
    • Adds more image predictors and change some APIs (#2456)
    • Adds more text predictors (#2443)
    • Adds np_util (#2419)
    • Adds pyspark TextEmbedder and update ImageClassifier (#2414)
    • Adds text generation in pyspark (#2477)
    • Adds text2text generation (#2506)
    • Adds whisper python code (#2513)
    • Upgrades spark version to 3.3.2 (#2523)

  • DJL API improvements:

    • Adds support for unique, bmm, xlogy (#2415)
    • Fixes NDArray.toByteArray() bug (#2436)
    • Adds NDArray.copyTo() support for NDArrayAdapter (#2437)
    • Improves Classifications.toString() print out (#2439)
    • Makes Batchifier serializable (#2441)
    • Loads inputShapes in the loadMetadata method of Linear block (#2448)
    • Adds chunked output support (#2453)
    • Makes audio and cv translator factory serializable (#2455)
    • Adds NamedEntity.toString() function (#2468)
    • Streaming Predict and streamable BytesSupplier (#2470)
    • Mitigates ZipInputStream CVE. (#2473)
    • Adds getProperties() to Model interface (#2476)
    • Adds non-blocking poll() for BytesSupplier (#2478)
    • Makes PassthroughNDManager aware of engine and device (#2484)
    • Fixes telemetry opt out (#2490)
    • Uses sha-256 to avoid security warning (#2495)
    • Moves NeuronUtils to api package (#2496)
    • Adds encode and decode to Input and Output (#2502)
    • Fails model loading if specified translator not found (#2515)
    • Adds a way to check if streaming is supported (#2518)
    • Fixed detect platform for different CUDA version (#2527)
    • Fixes neuron core detection in docker container (#2536)

  • PyTorch engine improvements:

    • Upgrades PyTorch engine to 2.0.0 (#2525)
    • Implements unique operator for PyTorch engine (#2417)
    • Adds yolov5s to pytorch model zoo (#2433)
    • Respect PYTORCH_FLAVOR override to download libtorch (#2486)
    • Print log if graph optimizer is enabled (#2501)

  • OnnxRuntime engine improvements:

    • Adds support for OnnxRuntime Profiler (#2472)

  • MXNet engine improvements:

    • Enables boolean index on mxnet (#2427)

Breaking change

Bug fixes

  • Fixes pytorch-native-cu118 package in BOM (#2535)
  • Fixes spark package name in BOM (#2534)
  • Fixes OnnxRuntime version (#2524)
  • Fixes memory leak in get with and to long, double, float, ... (#2428)

Documentation and Examples

  • Adds timeseries examples document (#2411)
  • Fixes link to doc Mask detection with YOLOv5 (#2529)
  • Adds DeferredTranslatorFactory to tokenizers example (#2511)
  • Updates depednency manage for spark extension (#2531)
  • Updates FAQ and troubleshooting documents (#2454)
  • Cleans inference performance optimization doc (#2519)
  • Adds Yolov5 on Face Mask Detection (#2452)

CI improvements:

  • Simplifies BOM build script (#2438)
  • Avoids re-publish serving tarball (#2479)
  • Fixes gradle 8.0 native publish issue (#2457)
  • Fixes gradle 8.0 publish to release issue (#2460)
  • Upgrades gradle to 8.0.2 (#2449)
  • Uses recommended way to create task in build.gradle (#2451)

Contributors

@frankfliu
@KexinFeng
@lanking520
@nezda
@tipame
@xyang16
@zachgk

New Contributors

Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.21.0...v0.22.1

djl - DJL v0.21.0 release

Published by frankfliu over 1 year ago

Key Features

  • Upgrades and enhancements for Engines
    • Upgrades PyTorch to 1.13.1 (#2245)
    • Upgrades ONNXRuntime to 1.14.0 (#2393)
    • Upgrades HuggingFace tokenizer version to 0.13.2 (#2369)
    • Upgrades XGBoost to 1.7.3 (#2371)
    • Removes Neo-DLR engine from DJL #2373
  • Introduces several improvements for extensions:
    • Adds batch support huggingface tokenizer
    • Adds API improvement for Spark extensions
    • Add a few image processing methods in OpenCV extension (#2320)
    • Adds stft and fft forier transform for audio extension (#2259)
  • Implements NDScope to automatically close NDArray in the scope (#2321)
  • Allows MXNet runs on Ampere GPU (#2313)
  • DJLServing release
    • Adds faster transformer support (#424)
    • Adds Deepspeed ahead of time partition script in DLC (#466)
    • Adds SageMaker MME support (#479)
    • Adds support for stable-diffusion-2-1-base model (#484)
    • Adds support for stable diffusion depth model (#488)
    • Adds out of memory protection for modle loading (#496)
    • Makes load_on_devices per model setting (#493)
    • Adds several per model settings
    • Improves management console model loading and inference UI (#431, #432)
    • Updates deepspeed to 0.8.0 (#465)

Enhancement

  • Introduces several enhancements for timeseries extension:
    • Adds probability distribution support for timeseries (#2025)
    • Add time series dataset support for timeseries package (#2026)
    • Add some basic block and deepAR model (#2027)
    • Enable pytorch deepar model inference in time series package (#2149)
  • Introduces several enhancement for HuggingFace tokenizer:
    • Adds batch encoding support(#2342, #2343, #2337, #2338)
    • Adds batchEncode for text pair (#2339)
    • Adds mean_sqrt_len and weightedmean pooling for TextEmbedding (#2272)
    • Adds more pooling mode form TextEmbedding (#2261)
    • Allows Huggingface model zoo list models in offline mode (#2322)
    • Update TextEmbedding pooling model name (#2314)
  • Introduces a few new examples:
    • Adds clip model to examples (#2239)
    • Adds openai whisper model to examples (#2293)
    • Adds stable diffusion examples (#2246)
  • Introduces several enhancement for Spark extension:
    • Add pyspark support (#2301)
    • Adds spark extension docker image (#2243)
    • Adds Numpy binary translator (#2399)
    • Adds huggingface tokenizer support for Spark (#2311)
    • Refactor Spark extension API (#2370)
  • DJL API improvements:
    • Adds limit and callback for Metrics API (#2362)
    • Adds newBaseManager(String engineName) api (#2275)
    • Falls back to PassthroughNDManager if there is no engine (#2354)
    • Improves Criteria.build() error message (#2397)
    • Improves hybrid engine operators (#2279)
    • Improve NDArray encode/decode performance (#2361)
    • Refactors Engine class (#2303)
    • Implements gatherNd, partial flatten and enable BertOnCode training (#2216)
    • Makes TestDataset constructor protected (#2271)
    • Creates SimplePaddingStackBatchifier (#2384)
    • Creates standard for PreTrained behavior (#2360)
    • Creates the TabularTranslator (#2344)
    • Enables tuning distill_bert embedding layer (#2203)
    • Handle RuntimeException on ImageFactory::newInstance (#2241)
    • Implement AdamW on Pytorch and MXNet (#2206)
    • Improves TranslatorExpansions with pre-processing and post-processing (#2213)
    • Opens LayerNorm.Builder for inheritance (#2309)
    • Adds feature to identify NDArray if double closed (#2352)
    • Adds back support for getting managed Arrays (#2386)
    • Opens Conv2d block constructor for inheritance (#2231)
  • PyTorch engine improvements:
    • Adds scatter function to PtNDArray (#2332)
    • Search for model.pt or model.onnx when loading the model (#2364)
    • Better handle String tensor operations (#2380)
    • Fixes typo in error message (#2355)
    • Makes JNI comptible with PyTorch 1.11.0 (#2263)
    • No longer search java.library.path (#2235)
    • Uses runMethod to replace forward function (#2234)
    • Workaround hann_window issue for PyTorch 1.12.1 (#2262)
    • Adds support for torch::cuda::empty_cache(). (#2305)
    • Fixes memory leak in PyTorch indexing fuction (#2300)
  • LightGBM engine improvements:
    • Fixes atomic move issue (#2258)
    • Fixes byte order for fp32 and fp64 array creation (#2278)
  • Updates library dependencies:
    • Reduces aws s3 extension dependencies (#2378)
    • Reduces hadoop extension dependencies (#2377)
    • Removes unnecessary dependency (#2329)
    • Upgrade dependencies versions (#2371)
  • Adds aarch64 and macOS M1 support for SentencePiece (#2325, #2324)
  • Adds centos 7 support for SentencePiece (#2402)
  • Adds aarch64 support for audio extension (#2250)

Breaking change

  • Removes Neo-DLR engine from DJL #2373
  • Moves RawDataset from basicdata to api module (#2375)
  • Changes Spark extension API (#2388)

Bug fixes

  • Fixes nested model directory issue (#2214)
  • Fixes NPE if block is cleard before training (#2365)
  • Fixes toDebugString() IllegalStateException if NDArray is closed (#2347)
  • Fixes typo in javadoc (#2286)
  • Fixes unittest failure on GPU (#2255)
  • Fixes inconsistencies using RANK in engine providers (#2244)
  • Fixes the content format (#2229)
  • Fixes the name passing bug (#2222)
  • Fixes TrainAmazonReviewRanking example for PyTorch (#2173)
  • Fixes performance issues from freezing MXNet (#2394)

Documentation and Examples

  • Fixes ONNXRuntime Android doc (#2194)
  • Updates readme of tranferFreshFruilt (#2236)
  • Updates cache menagement adds ONNX and Huggingface cache directory (#2334)
  • Updates FAQ (#2328)
  • Fix typo in javadoc of Block and AbstractBlock (#2288)
  • Adds extensions to docs web page (#2284)
  • Adds missing DJLServing docs menu items (#2363)
  • Improves documents (#2396)
  • Updates PyTorch graph exector optimization document (#2374)
  • Updates README for engines (#2282)
  • Fixes whisper model translator (#2341)
  • Documents workaround AudioGrabber bug for whisper model (#2330)
  • Adds tensorflow dependency to pom.xml (#2268)
  • Fixes PyTorch nightly test (#2264)
  • Fixes Onnx version in README (#2253)
  • Update onnx version in README file (#2223)
  • Fix djl-zero doc (#2201)

CI improvements:

  • Adds lost package for Codespaces Dockerfile (#2260)
  • Uses ubuntu-latest as runner (#2199)
  • Updates configure-aws-credentials to v1-node16 (#2196)
  • Adds formatCpp exclusion to gradle plugin (#2215)
  • Allows publish djl-serving from a branch (#2225)
  • Build cu102 JNI for PyTorch 1.11.0 and 1.12.1 (#2254)
  • Fixes jdk18 compile error (#2306)
  • Replace jacoc root report with aggregation report plugin (#2280)
  • Updates github action setup-python to v4 (#2202)
  • Upgrade gradle to 7.6 (#2302)
  • Upgrade testng to 7.7.0 (#2230)
  • Fixes PMD 6.21.0 reported issues (#2296)
  • Removes DLR github actions (#2398)
  • Sets seed in TransferFreshFruitTest for consistent results (#2238)
  • fastText should not depends on any engine #2353
  • Avoid unittest polluting cache directory. (#2228)
  • Minor refactor encoding class (#2336)
  • Split TranslatorTest into smaller test classes. (#2335)

Contributors

  • @dayo05
  • @demq
  • @enpasos
  • @frankfliu
  • @KexinFeng
  • @lanking520
  • @Noricks
  • @siddvenk
  • @SuperMaskv
  • @xyang16
  • @zachgk

New Contributors

Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.20.0...v0.21.0

djl - DJL v0.20.0 release

Published by xyang16 almost 2 years ago

Key Features

  • Upgrades and enhancements for Engines
    • Upgrades PyTorch to 1.13.0 (#2157)
    • Add support for Apple's Metal Performance Shaders (MPS) in PyTorch (#2037)
    • Add system property to config GraphExecutorOptimize (#2156)
    • Upgrades ONNXRuntime to 1.13.1 (#2115)
    • Upgrades Paddle to 2.3.2 (#2116)
    • Upgrades TensorFlow to 2.7.4 (#2121)
    • Upgrades HuggingFace tokenizer version to 0.13.1 (#2127)
    • Upgrades XGBoost to 1.7.1 (#2143)
  • DJLServing
    • Adds large model inference support with MPI mode (#291)
    • Adds built-in DeepSpeed handler (#292)
    • Publishes PaddlePaddle docker image (#342)
  • Adds TabNet Training (#2057)
  • Publishes DJL Zero (#2091)
  • Adds Spark extension (#2162)
  • Introduces several improvements for timeseries extension
  • Adds ImageFeatureExtractor example and resnet base model to model zoo

Enhancement

  • Introduces several enhancements for timeseries extension:
    • Adds probability distribution support for timeseries (#2025)
    • Add time series dataset support for timeseries package (#2026)
    • Update M5Forecast dataset and its unittest (#2105)
    • Add some basic block and deepAR model (#2027)
    • Enable pytorch deepar model inference in time series package (#2149)
  • Introduces several enhancement for HuggingFace tokenizer:
    • Enhance huggingface text embedding translator to support max length padding (#2049)
    • Add cli options to only validate jit model on CPU (#2052)
    • Add batch decoding methods for tokenizers (#2154)
  • Adds new models to DJL model zoo:
    • Adds TabNet model for tabular dataset in modelzoo (#2036)
    • Adds yolo5s to OnnxRuntime model zoo (#2046)
    • Object Detection (#1930)
    • Adds image classification resnet18 base model to model zoo (#2079)
  • DJL API improvements:
    • Adds Sparsemax block (#2028)
    • Updates the SemanticSegmentationTranslator (#2032)
    • Creates Ensembleable (#2043)
    • Handle error when forget to initialize a child block (#2045)
    • Adds draw mask for BitMapWrapper (#2071)
    • Allows show NDArray content in debugger (#2078)
    • Rename transparency to opacity in CategoryMask (#2081)
    • Allows to show NDArray content in Debugger 2 (#2080)
    • Transfer learning with pytorch engine on fresh fruit dataset (#2070)
    • Ensure GradientCollector can clear gradients (#2101)
    • Handles conflict JNA package issue (#2118)
    • Adds Multiplication block (#2110)
    • Allows non-ServingTranslatorFactory for DJLServing (#2148)
    • Adds cumprod operator (#2152)
    • Adds Randperm on PyTorch and MxNet (#2084)
    • Creates translator options (#2145)
  • CI improvements:
    • Add Mac M1 build (#2039)
    • Publishes serving tar and zip (#2014)
    • Uploads djl-bench release artifacts to S3 (#2020)
    • Upgrade deprecated github actions (#2119)
    • Upgrade github actions to latest version (#2122)
    • Compile JNI only when file changes (#2161)
    • Speed up continuous build by not uploading jacoco report (#2166)
    • Respect -SNAPSHOT version in jar manifest (#2177)
    • Allows JNI to be compiled on headless jdk (#2098)
    • Move model zoo download test to canary (#2169)
    • Upgrades PyTorch for Android to 1.13.0 (#2171)
    • Add some unit tests (#2063)
    • Test accumulating gradient collector (#2111)
    • Refactor unit test TestRequirements, add missing TestRequirements (#2120)
  • Upgrade protobuf version to 3.20.2 (#2035)
  • Update deeplabv3 model zoo metadata (#2051)
  • Remove String tensor limitation for model output (#2056)
  • Disables mapLocation when using MPS device (#2061)
  • Adds disablePerSessionThreads option to model loading for ONNXRuntime (#2104)
  • LightGBM inference result matches input type (#2129)
  • Apply no_optimizer_guard only for Android (#2153)
  • Update dependency version (#2176)
  • Reduce nested exception level (#2181)

Documentation and Examples

  • Updates Semantic Segmentation app (#263)
  • Adds the object detection app demo, use onnxruntime engine (#266)
  • Adds stable diffusion demo (#269)
  • Adds Spark extension example (#272)
  • Adds DJLServing Postman examples (#276)
  • Adds DJLServing Java client demo (#277)
  • Adds DJLServing Python client demo (#278)
  • Update README.md (#2010)
  • Update dependency docs for timeseries package (#2004)
  • Update javadoc links (#2017)
  • Use latest javadoc links (#2021)
  • Info added (#2022)
  • Add Mac M1 info in docs (#2040)
  • Some doc fixes (#2042)
  • Improve memory management and batchify docs (#2076)
  • Move serving docs to top level and reorganize (#2100)
  • Update docs top level memu (#2102)
  • Change timeseries dataset source example and add test (#2109)
  • Fix a document issue (#2114)
  • Update dependency document (#2134)
  • Upgrade pytorch 1.13.0 documents (#2158)
  • Add readme for TransferFreshFruit (#2160)
  • Made the sites copyright year dyanamic (#2188)

Breaking change

  • NDArray.toDebugString() signature has been changed (#2078)

Bug Fixes

  • Fixes flooded warning message in PyTorch (#2136)
  • Fixes youtube link in quick start (#2012)
  • Fixes dlr-native build script (#2015)
  • Adds missing condition for benchmark release (#2023)
  • Fixes folder not exist bug when use external pytorch native library (#2033)
  • Fixes bug in OD TF saved model example (#2050)
  • Fixes MXNet engine cu112 document (#2066)
  • Fixes CategoryMask (#2073)
  • Fixes ames and various tabular improvements (#2054)
  • Fixes randomColor (#2075)
  • Fixes remove warning spam during training (#2097)
  • Adds missing test requirements to opencv test (#2108)
  • Fixes android crash issue (#2113)
  • Avoid static setup/tearDown in testng (#2126)
  • Add missing dependency to timeseries (#2128)
  • Fixes mac os build failure (#2131)
  • Fixes char offset (#2137)
  • Adds repo_url in order to fix broken edit links (#2140) (#2141)
  • Fixes format bug and change the name of retrain (#2147)
  • Fixes ndarry operator warnings show up in Jupyter notebook (#2150)
  • Fixes HfModelZoo NPE bug with shadowjar (#2163)
  • Fixes Temporary File Information Disclosure Vulnerability (#2164)
  • Fixes movielens download issue (#2167)
  • Fixes Yolov3 javadoc warning (#2168)
  • CVE-2007-4559 Patch (#2189)

Contributors

  • @asbachb
  • @Carkham
  • @demq
  • @dependabot
  • @frankfliu
  • @JLLeitschuh
  • @KexinFeng
  • @lanking520
  • @patins1
  • @siddvenk
  • @tosterberg
  • @warthecatalyst
  • @wxm2018
  • @xyang16
  • @ylwu-amzn
  • @zachgk

New Contributors

  • @dependabot made their first contribution in (#2035)
  • @ylwu-amzn made their first contribution in (#2049)
  • @tosterberg made their first contribution in (#2097)
  • @asbachb made their first contribution in (#2128)
  • @JLLeitschuh made their first contribution in (#2164)

Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.19.0...v0.20.0

djl - DJL v0.19.0 release

Published by xyang16 about 2 years ago

Key Features

  • Creates new LightGBM engine (#1895)
  • Upgrades and enhancements for Engines
    • Upgrades PyTorch to 1.12.1 (#1894)
    • Upgrades ONNXRuntime to 1.12.1 (#1879)
    • Upgrades Apache MXNet to 1.9.1 (#1898)
    • Publishes new xgboost-gpu package to maven (#1918)
    • Adds ARM support for ONNXRuntime (#1856)
    • Disable autograd by default when PyTorch engine start (#1872)
  • Introduces several enhancement for HuggingFace tokenizer
    • Introduces HuggingFace model zoo (#1984)
    • Adds a few built-in Translators for HuggingFace NLP models
    • Adds macOS M1 support for HuggingFace tokenizer
    • Adds ARM support for HuggingFace tokenizer
    • Adds centos 7 support for HuggingFace tokenizer (#1874)
    • Adds decode API for HuggingFace tokenizer (#1843)
    • Adds padding and truncation support for HuggingFace tokenizer (#1870)
    • Support stride in tokenizers (#2006)
  • Introduces time series extension (#1903)
  • Adds new Audio API and improves audio extension (#1974)
  • Adds Android support for ONNXRuntime (#1844)
  • JDK18 support (#1892)
  • Adds python script to import HuggingFace model into DJL model zoo (#1835)
  • DJLServing
    • Adds management console plugin, which allows user manage models with web UI (#205)
    • Adds KServe plugin (#177)
    • Publishes DeepSpeed docker image to dockerhub (#223)

Enhancement

  • Adds a few more built-in Translators:
    • Adds HuggingFace QuestionAnsweringTranslator (#1828)
    • Adds HuggingFace FillMaskTranslator (#1876)
    • Adds HuggingFace TokenClassificationTranslator (#1906)
    • Adds HuggingFace TextClassificationTranslator (#1983)
    • Adds HuggingFace TextEmbeddingTranslator (#1953)
    • Adds speech recognition translator (#1899)
  • Adds new models to DJL model zoo:
    • Adds PyTorch deeplabvs model into DJL model zoo (#1818)
    • Adds MobileNetV1 into model zoo (#1817)
  • Image handling enhancement:
    • Improves ImageFactory to allow convert float32 NDArray to Image. (#1814)
    • Handle both HWC and CHW image (#1833)
  • DJL API improvements:
    • Adds NDArray normalize() operator (#1924)
    • Adds DeferredTranslatorFactory to let serving.properties take effect (#1868)
    • Makes PtBertQATranslator compatible with huggingface model (#1827)
    • Improves debug log for model loading options. (#1825)
    • Allows to load block only model for PyTorch (#1831)
    • Adds IdentityBlockFactory for demo/test purpose (#1854)
    • Support queryString for JarRepository (#1842)
    • Set arguments in serving.properties as model properties (#1853)
    • Allow overriding special token flags in encode and decode methods (#1855)
    • Adds support for intermediate sequential block results (#1943)
    • Adds load SentencePiece model from InputStream (#1949)
    • Allows use cached PyTorch native libraries in offline mode by caching "files.txt". (#1982)
    • Makes Encoding class constructor protected. (#1945)
    • Adds string tensor support for PyTorch (#1968)
    • Adds Loss function: Coverage.java (#1653)
    • Adds Loss function: QuantileLoss.java (#1652)
    • Validate data type for NDArray.set(Buffer) API (#1975)
    • Adds offline mode to to ensure not download engine files from network (#1987)
    • Adds encodeDual support for HuggingFace tokenizer (#1826)
    • Bulk batch creation and array indexing on mxnet engine (#1869)
    • Adds NDArray gammaln and sample distribution function support. (#1990)
    • Padding when the size of input is 2 in LSTM (#2000)
    • Creates a SystemNDManager interface (#1888)
  • Adds python script to import huggingface model into DJL model zoo
    • Added fill-mask support for converting huggingface model to model zoo (#1849)
    • Adds support for converting huggingface token-classification models (#1902)
    • Adds support for converting huggingface sentence-similarity models (#1913)
    • Adds support for converting huggingface text-classification models (#1972)

Documentation and Examples

  • Adds Neural machine translation example (#1851)
  • Adds New Bert example using Goemotions (#1682)
  • Adds Semantic Segmentation example (#1808)
  • Adds tokenizer readme for usage (#1981)
  • Updates troubleshooting.md to remove -native-auto package (#1793)
  • Document PYTORCH_PRECXX11 usage in README (#1807)
  • Immutable array output from InferenceMode PyTorch (#1822)
  • Fixes NDIndex javadoc issue (#1841)
  • Updates pose estimation example to detect joints for all people (#2002)
  • Adds Semantic segmentation and Speech recognition to README (#2003)
  • Updates links in README (#2005)
  • Adds an example of time series model inference (#1971)

Breaking change

  • NDManager.vaildateBufferSize() has been renamed to NDManager.validateBuffer()
  • Remove unnecessary DeviceType interface (#1978)

Bug Fixes

  • Adds missing text_embedding application in Application.of() (#1917)
  • Fixes capped manager bug for TensorFlow (#1952)
  • Fixes NDArray.set() bug (#1789)
  • Fixes breaking behavior for NDIndex in 0.18.0 (#1801)
  • Backward compatible with Apache MXNet indexing. (#1802)
  • Fixes OrtNDArray double close issue (#1809)
  • Fixes ImageFactory.fromNDArray() bug (#1824)
  • Fixes NDArrayAdapter toDevice() and toType() behavior (#1839)
  • Fixes the parsing issue (#1857)
  • Fixes OrtNDArray double free issue (#1861)
  • Fixes memory leak when using NDManager.newBaseManager() (#1887)
  • Fixes PyTorch download library fallback to CPU bug (#1951)
  • Fixes bug in Criteria (#1964)
  • Fixes issue in TrainMnistWithLSTM (#1965)
  • Fixes closing error in Apache MXNet when indexing results in an empty array (#1966)
  • Fixes path parsing bug on Windows (#1985)
  • Fixes memory leak in Apache MXNet layerNorm() (#1993)
  • Fix DynamicBuffer position error when expand for the first time (#2007)
  • Fix some bugs in pytorch based examples (#2009)

Contributors

  • @925781609
  • @bryanktliu
  • @Carkham
  • @demq
  • @frankfliu
  • @gforman44
  • @JohnDoll2023
  • @KexinFeng
  • @Konata-CG
  • @lanking520
  • @oyy2000
  • @patins1
  • @pdradx
  • @siddvenk
  • @takanori-ugai
  • @warthecatalyst
  • @wxm2018
  • @xyang16
  • @zachgk

New Contributors

  • @bryanktliu made their first contribution in (#1828)
  • @xyang16 made their first contribution in (#1852)
  • @wxm2018 made their first contribution in (#1844)
  • @925781609 made their first contribution in (#1887)
  • @demq made their first contribution in (#1945)
  • @Carkham made their first contribution in (#1903)
  • @gforman44 made their first contribution in (#1653)
  • @takanori-ugai made their first contribution in (#2000)

Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.18.0...v0.19.0

djl - DJL v0.18.0 release

Published by siddvenk over 2 years ago

Key Features

Enhancement

Documentation and Examples

Breaking Changes

The following changes to api.djl.basicdataset.tabular may cause backwards incompatibility:

  • Features and Featurizers have been refactored out of the CSVDataset class. The are now present in ai.djl.basicdataset.tabular.utils
  • CSVDataset now extends a new abstract class, TabularDataset
  • api.djl.basicdataset.utils.DynamicBuffer implementation has moved to api.djl.basicdataset.tabular.utils.DynamicBuffer

Bug Fixes

Contributors

@dandansamax
@DiaaAj
@frankfliu
@WHALEEYE
@patins1
@JohnDoll2023
@KexinFeng
@Konata-CG
@pdradx
@lanking520
@siddvenk
@LanAtGitHub
@warthecatalyst
@zachgk
@freemanliu
@liumingxiy

New Contributors

Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.17.0...v0.18.0

djl - DJL v0.17.0 release

Published by zachgk over 2 years ago

Key Features

  • Adds linux AArch64 support for PyTorch
  • Upgrades Engine Releases
    • XGBoost version 1.6.0 (#1624)
    • PaddlePaddle version 2.2.2 (#1601)
    • ONNXRuntime version 1.11.0 (#1602)
    • PyTorch version 1.11.0 (#1583)
    • Apache MXNet version 1.9.0 (#1429)
  • Newly Added Datasets
    • PennTreebank dataset (#1580)
    • WikiText-2 dataset (#1545)
  • Support freezing parameters for transfer learning (#1544)

Enhancement

  • Upgrade djl-bench to 0.15.0 (#1476)
  • Add SessionOptions support for OnnxRuntime (#1479)
  • Device Name parsing (#1490)
  • arena allocator setter (#1510)
  • Upgrade native build mxnet version (#1517)
  • Refactor ml extension support (#1521)
  • Parse default device with Device.fromName (#1529)
  • Upgrade dependency versions (#1533)
  • release Update document version to 0.16.0 (#1536)
  • Bump up version to 0.17.0 (#1537)
  • Building DJL For aarch64 (#1526)
  • api Make resource loading compatible with java9 module (#1541)
  • pytorch Allows multiple native jars to package into fat jar (#1543)
  • Add document with installation instructions (#1548)
  • Separate AbstractSymbolBlock from AbstractBlock (#1555)
  • Add better error message for libstdc++ tf errors (#1570)
  • basicdataset Add Stanford Question Answering Dataset (#1554)
  • xgb Set default missing value to NaN (#1571)
  • allow empty batch data (#1569)
  • pytorch Allows load libtroch from pip installation package (#1577)
  • HF Tokenizer: get charspans (#1584)
  • benchmark Allows benchmark run on aarch64 for PyTorch (#1591)
  • add pytorch cuDNN acceleration (#1592)
  • add troubleshoot issue (#1600)
  • add paddle CU110 (#1604)
  • Update badge (#1610)
  • ONNXRuntime add tensorRT option (#1611)
  • api Refactor metrics API (#1613)
  • api Fixes metric dimension (#1614)
  • bom Update bom dependency (#1615)
  • pytorch Use precxx11 build for aarch64 native library (#1619)
  • pytorch use precxx11 for aarch64 (#1620)
  • Update inference_performance_optimization.md (#1621)
  • Fix testFreezeParameters for multi-gpu (#1623)
  • Support gather of pytorch (#1622)

Documentation and Examples

  • Update README for 0.15.0 release (#1477)
  • Update README to use 0.16.0-SNAPSHOT version (#1486)
  • typo fix (#1519)
  • docs Fixes dataset document. (#1523)
  • jupyter remove unecessary maven import (#1540)
  • examples Update maven pom file dependency version (#1546)
  • Add release note to README (#1565)
  • update pytorch build instruction on android (#1630)
  • bump up versioning (#1633)
  • update android package versions (#1635)
  • docs Update pytorch and paddle version (#1634)

Breaking Changes

  • Custom symbol blocks should extend AbstractSymbolBlock instead of AbstractBlock

Bug Fixes

  • Fix flaky test for tensorflow 2.7.0 on GPU (#1475)
  • Fixes topK items for DetectedObjects and make it configurable to Classifications (#1478)
  • Fix load native library failure on Android (#1485)
  • Adding huggingface tokenizer extension to BOM (#1487)
  • Fixes #1149, fix NPE bug (#1492)
  • Fix djl site vue version (#1495)
  • Fixes MLP example code in README (#1497)
  • Fixes jni dependency in README document (#1513)
  • Fixes memory leak in hybrid engine (#1518)
  • Fix the version issue reported from get-pip.py (#1530)
  • Fix FastText JNI build (#1531)
  • api Fixes loading BlockFactory bug (#1547)
  • tensorflow Fixes tensorflow session always on gpu(0) bug (#1558)
  • xgb Fixes missing anonymous classes (#1572)
  • examples Fix ImageClassification invalid probability (#1575)
  • fix naming (#1581)
  • ONNXRuntime fix naming (#1608)
  • basicdataset Fixed out of bound limit (#1599)
  • api Avoid NPE in Metric.toString() (#1626)
  • integration Enable gather unit test for windows (#1638)
  • Add rpath fix to Native Publish PyTorch (#1639)
  • api Fixes JDK 18 compiler warnings (#1640)

Contributors

  • @AKAGIwyf
  • @andreabrduque
  • @dandansamax
  • @frankfliu
  • @hd1080p
  • @KexinFeng
  • @lanking520
  • @patins1
  • @sindhuvahinis
  • @WHALEEYE
  • @zachgk

New Contributors

Full Changelog: https://github.com/deepjavalibrary/djl/compare/750c153a7...v0.17.0

djl - DJL v0.16.0 release

Published by frankfliu over 2 years ago

Key Features

  • Upgrades Apache MXNet engine to 1.9.0 with CUDA 11.2 support
  • Improves ONNXRuntime engine memory configurations
  • Improves fastText engine’s API
  • Fixes several critical bugs

Enhancement

  • Upgrades Apache MXNet engine to 1.9.0 with CUDA 11.2 support (#1517)
  • Improves ONNXRuntime engine:
    • Adds arena allocator support to SessionOptions (#1510)
    • Adds SessionOptions support for OnnxRuntime (#1479)
  • Introduces several API improvements:
    • Parse default device with Device.fromName (#1529)
    • Refactor ml extension support to be consistent with DJL api (#1521)
    • Device Name parsing (#1490)
    • Make DetectedObjects topK configurable (#1478)
  • Uses JDK 11 for github actions in CI build (#1489)
  • Adds huggingface tokenizer extension to BOM (#1487)
  • Removes unnecessary github actions workflow (#1484)
  • Upgrades DJL android to 0.15.0 (#1483)

Documentation and examples

  • Fixes outdated dataset document (#1523)
  • Fixes repository README typo (#1519)
  • Fixes PyTorch JNI dependency in README document (#1513)
  • Fixes MLP example code in README (#1497)
  • Fixes djl.ai website vue version (#1495)
  • Updates inferentia demo to DJL 0.15.0 (#210)
  • Updates android README to use 0.15.0 (#1486)
  • Publishes D2L Chinese book with latest chapters https://d2l-zh.djl.ai

Breaking change

  • fastText specific inference and training APIs are removed, use standard DJL API instead

Bug Fixes

  • Fixes memory leak in hybrid engine (#1518)
  • Fixes fastText JNI build (#1531)
  • Fixes the python version bug in the benchmark workflow files (#1530)
  • Fixes sentencepiece NPE bug (#1492)
  • Fixes PyTorch load native library failure on Android (#1485)
  • Fixes topK items for DetectedObjects (#1478)

Contributors

This release is thanks to the following contributors:

New Contributors

Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.15.0..v0.16.0

djl - DJL v0.15.0 release

Published by frankfliu over 2 years ago

DJL v0.15.0 updates the engines PyTorch to 1.10.0, ONNXRuntime to 1.10.0, TensorFlow to 2.7.0, TensorFlowLite to 2.6.2 and introduces several new features:

Key Features

  • Introduces Huggingface tokenizers extension which allows user to leverage high performance fast tokenizer in Java
  • Upgrades PyTorch engine to 1.10.0 with CUDA 11.3 support
  • Upgrades TensorFlow to 2.7.0 with CUDA 11.3 support
  • Upgrades ONNXRuntime engine to 1.10.0
  • Upgrades TensorFlowLite to 2.6.2
  • Provides better PyTorch engine backward compatibility support
  • Adds load model from InputStream support
  • Adds Windows support for SentencePiece
  • Removes -auto packages to simplify DJL dependencies
  • Fixes log4j CVEs

Enhancement

  • Improves PyTorch engine:
    • Adds support to load custom build PyTorch native library from specified location
    • Adds support to use precxx11 version of PyTorch on GPU
    • Provides offline native package for older version of PyTorch to run with latest DJL (#1385)
    • Report correct engine version for custom build PyTorch native library
    • Adds Tuple support to IValue (#1436)
  • Adds support to load model from InputStream for some of the engines (#1400):
    • Adds load from InputStream for PyTorch model
    • Adds load from InputStream for TensorFlowLite model (#1402)
    • Adds load from InputStream for ONNXRuntime model (#1402)
    • Adds load from InputStream for SentencePiece (#1139)
  • Introduces several new features in djl-serving:
    • Automatically detect model’s engine
    • Reduces netty dependencies to minimize package size
    • Installs engine dependency on-demand for non-commonly used engines
    • Adds support for nested folder in model archive file
    • Released djl-serving to homebrew
  • Introduces server new features in djl-bench
    • Release djl-bench to homebrew and snapcraft
  • Improve opencv extension to expose API from opencv to end users
  • Introduces several API improvements:
    • Improves TrainingResult with toString() print out (#1369)
    • Allows to register engine at runtime (#1386)
    • Allows to dynamic add model zoo (#1397)
    • Creates IndexEvaluator and IndexLoss (#1414)
    • Introduces Huggingface Tokenizer (#1406)
  • Publishes several new docker images to dockerhub (https://hub.docker.com/u/deepjavalibrary)
    • djl-serving docker image
    • djl-serving for inferentia docker image
    • DJL windows docker image with jdk11

Documentation and examples

  • Updates inferentia demo to support neuron runtime 2.x
  • Updates jupyter notebooks with NoBatchifyTranslator to simplify example code (#1370)
  • Updates benchmark README (#1410)
  • Fixes rank classification jupyter notebook (#1368)
  • Updates packaging model document (#1471)
  • Fixes d2l book (https://d2l.djl.ai/) image display issue (#183)
  • Translates d2l Chinese book (https://d2l-zh.djl.ai/) chapter 9 to chapter 14 to Chinese

Breaking change

  • N/A

Bug Fixes

  • Fixes several ci issues on Github Actions (#1358, #1362, #1363, #1364)
  • Fixes crash when IValue list input is empty (#1440)
  • Fixes NullPointException bug in getContextClassLoader() (#1445)
  • Fixes model name detection issue in loading model from jar (#1446)
  • Fixes unset numEmbeddings in TrainableWordEmbedding.java (#1450)
  • Fixes protobuf-java CVEs

Contributors

This release is thanks to the following contributors:

New Contributors

Full Changelog

https://github.com/deepjavalibrary/djl/compare/0855108d0...v0.15.0

djl - DJL v0.14.0 release

Published by frankfliu almost 3 years ago

DJL v0.14.0 updates the engines PyTorch to 1.9.1 and introduces several new features:

Key Features

  • Upgrades PyTorch engine to 1.9.1
  • Adds support for Neuron SDK 1.16.1
  • Adds autoscale in djl-serving for AWS Inferentia model
  • Introduces OpenCV extension to provide high performance image processing
  • Adds support for older version of PyTorch engine, user now can use PyTorch 1.8.1 with latest DJL
  • Adds support for precxx11 PyTorch native library in auto detection mode
  • Adds AWS Inferentia support in djl-bench
  • Adds support for TorchServe .mar format, user can deploy TorchServe model archive in djl-serving

Enhancement

  • Introduces several new features in djl-serving:
    • Adds autoscale feature for AWS Inferentia (#31)
    • Creates SageMaker hosting compatible docker image for AWS Inferentia (#36)
    • Adds auto detect number of neuron cores feature for AWS Inferentia (#34)
    • Adds autoscale support for SageMaker style .tar.gz model (#35)
    • Adds support to load torchserve model (#32)
    • Adds support to pip installed dependency per model (#37)
    • Adds custom environment variable support for python engine (#29)
    • Adds nested folder support in model archive file (#38)
    • Improves model status with model version support (#25)
    • Adds model warn up feature for python engine. (#23)
    • Adds WorkLoadManager.unregisterModel (#33)
    • Adds testing tool to test python model locally (#22)
    • Adds set python executable path for python engine (#21)
    • Creates Workflow for ModelServing (#26)
  • Adds OpenCV extension (#1331)
  • Introduces several new features in djl-bench:
    • Adds support for AWS Inferentia (#1329)
  • Introduces several new features in Apache MXNet engine:
    • Implements LayerNorm for Apache MXNet (#1342)
  • Introduces several new features in PyTorch engine:
    • Upgrades PyTorch to 1.9.1 (#1297)
    • Implements padding to bert tokenizer (#1328)
    • Makes pytorch-native-auto package optional (#1326)
    • Adds support to use different version of PyTorch native library (#1323)
    • Adds map_location support for load model from InputStream (#1314)
    • Makes map_location optional (#1312)
  • Introduces several new features in TensorFlow Lite engine:
    • Makes tensor-native-auto package optional (#1301)
  • Introduces several API improvements:
    • Adds support for nested folder in model archive (#1349)
    • Improves translator output error message (#1348)
    • Improves Predictor API to support predict with device (#1346)
    • Improves BufferedImageFactory.fromNDArray performance (#1339)
    • Adds support for downloading .mar file (#1338)
    • Adds debugging toString to Input and Output (#1327)
    • Refactors BERT Translator and Tokenizer (#1318)
    • Makes question answering model serving ready (#1311)
    • Refactors minMaxWorkers from ModelInfo to WorkerPool (#30)

Documentation and examples

Breaking change

  • PyTorch 1.9.1 no longer supports Amazon Linux 2, AL2 user has to use pytorch-native-cpu-precxx11
  • Image.Type is removed and Image.duplicate() function no longer take Image.Type as input
  • Image.getSubimage() is renamed to Image.getSubImage()
  • PaddlePaddle model loading may break due to prefix changes.

Bug Fixes

  • Fixes 2nd inference throw exception bug (#1351)
  • Fixes calculation for SigmoidBinaryCrossEntropyLoss from sigmoid (#1345)
  • Fixes jar model url download bug (#1336)
  • Fixes memory in Trainer.checkGradients (#1319)
  • Fixes NDManager is closed bug (#1308)
  • Fixes PyTorch GPU model loading issue (#1302)
  • Fixes MXNet EngineException message (#1300)
  • Fixes python resnet18 demo model GPU bug (#24)
  • Fixes python engine get_as_bytes() bug (#20)

Contributors

This release is thanks to the following contributors:

djl - DJL v0.13.0 release

Published by frankfliu about 3 years ago

DJL v0.13.0 brings the new TensorRT and Python engines, and updates the engines PyTorch to 1.9.0, ONNXRuntime to 1.9.0, PaddlePaddle to 2.0.2, and introduces several new features:

Key Features

Enhancement

  • Introduces several new features in djl-serving:
    • Improves djl-serving API to make it easy to get HTTP headers (#1134)
    • Loads models on all GPUs at startup for djl-serving (#1132)
    • Enables asynchronous logging for djl-serving
    • Makes djl-serving access log in separate log file (#1150)
    • Adds configuration to support number worker threads for GPU inference (#1153)
    • Improves auto-scale algorithm for djl-serving (#1149)
  • Introduces several new features in djl-bench:
    • Adds a command line option to djl-bench to generate NDList file (#1155)
    • Adds warmup to benchmark (#1152)
    • Improves djll-bench to support djl:// urls (#1146)
    • Adds support to benchmark on multiple GPUs (#1144)
    • Adds support to benchmark onnx on GPU machines (#1148)
    • Adds support to benchmark TensorRT models (#1257)
    • Adds support to benchmark Python models (#1267)
  • Introduces several new features in PyTorch engine:
    • Supports PyTorch custom input data type with IValue (#1208)
  • Introduces several new features in OnnxRuntime:
    • Adds UINT8 support for OnnxRuntime (#1271)
  • Introduces several new features in PaddlePaddle:
    • Adds more model loading options for PaddlePaddle (#1173)
    • Adds load functionalities to PaddlePaddle (#1140)
    • Adds remove pass option to PaddlePaddle (#1141)
  • Introduces several API improvements:
    • Adds missing NDList.get(String) API (#1194)
    • Adds support to directly load models from a TFHub url (#1231)
    • Improves repository API to support passing argument in the URL query string (#1139)
    • Avoids loading the default engine if it is not being used (#1136)
    • Improves IO by adding a buffer to read/write (#1135)
    • Improves NDArray.toString() debug mode performance (#1142)
    • Makes GPU device detection engine specific to avoid confusion when using multiple engines (#1138)

Documentation and examples

  • Adds Style Transfer example with CycleGAN (#1180)

Breaking change

  • Removes support for Apache MXNet 1.6.0
  • Deprecates Device.getDevices() API - Use Engine.getDevices() instead
  • Renames SimpleVocabulary to DefaultVocabulary

Bug Fixes

  • Fixes broken link in documents
  • Fixes TensorFlow NDArray was created on CPU instead of GPU bug (#1279)
  • Fixes default image processing pipeline (#1268)
  • Fixed XGBoost NDArray multiple read bug (#1239)
  • Fixes platform matching bug (#1167)
  • Fixes NullPointerException in NDArray.toString() (#1157)
  • Fixes PaddlePaddle crash due to GC (#1162)
  • Fixes NDArrayAdapter.getSparseFormat() unsupported bug (#1151)
  • Fixes mixed device issue in multiple engine use case (#1123)
  • Fixes handle duplicate plugin issue for djl-serving (#1108)
  • Fixes XGBoost NDArray creation bug (#1109)
  • Fixes runtime exception running benchmark arm machine(#1107)
  • Fixes unregister model regression (#1101)

Contributors

This release is thanks to the following contributors:

djl - DJL v0.12.0 release

Published by frankfliu over 3 years ago

DJL v0.12.0 added GPU support to PaddlePaddle and ONNXRuntime, and introduces several new features:

Key Features

  • Updates PaddlePaddle engine with GPU support.
  • Updates ONNXRuntime engine with GPU support.
  • Upgrades ONNXRuntime engine to 1.8.0.
  • Upgrades XGBoost engine to 1.4.1.
  • Introduces AWS Inferentia support, see our example for detail.
  • Adds FLOAT16 datatype support in NDArray.
  • Support UTF16 surrogate characters in NLP tokenization.
  • Makes benchmark as a standalone tool.
  • Releases djl-serving docker image to docker hub.

Enhancement

  • DJL Benchmark now can benchmark any datatype as input.
  • Makes Grayscale image processing match openCV’s behavior (#965)
  • Improves PyTorch engine to load extra shared library for custom operators (#983)
  • Improves djl-serving REST API to support load model on specified engine (#977)
  • Improves djl-serving to support load multiple version of a model on the same endpoint (#1052)
  • Improves djl-serving to support auto-scale workers based on traffic (#986)
  • Implements several operators:
    • Adds the truncated normal operator (#1005)
    • Adds the one hot operator for PyTorch (#1014)
    • Adds the LayerNorm operator in PyTorch (#1069)
  • Introduces several API improvements
    • Improves Criteria.loadModel() API (#1018)
    • Refactors ModleLoader and TranslatorFactory (#712)
    • Improves BlockFactory API (#1045)
    • Makes SpProcessor public API (#1060)

Documentation and examples

Breaking change

  • Direct access ModelZoo ModelLoader is no longer supported, use Criteria API instead.
  • Deprecates ModelZoo.loadModel() API in favor of using Criteria.loadModel().

Bug Fixes

  • Fixes missing softmax in action_recognition model zoo model (#969)
  • Fixes saveModel NPE bug (#989)
  • Fixes NPE bug in block.toString() function (#1076)
  • Adds back String tensor support to TensorFlow engine (lost in 0.11.0 during refactor) (#1040)
  • Sets ai.djl.pytorch.num_interop_threads default value for djl-serving (#1059)

Known issues

Contributors

This release is thanks to the following contributors:

djl - DJL v0.11.0 release note

Published by frankfliu over 3 years ago

DJL v0.11.0 brings the new engines XGBoost 1.3.1, updates PyTorch to 1.8.1, TensorFlow to 2.4.1, Apache MXNet 1.8.0, PaddlePaddle to 2.0.2 and introduces several new features:

Key Features

  • Supports XGBoost 1.3.1 engine inference: now you can run prediction using models trained in XGBoost.
  • Upgrades PyTorch to 1.8.1 with CUDA 11.1 support.
  • Upgrades TensorFlow to 2.4.1 with CUDA 11.0 support.
  • Upgrades Apache MXNet to 1.8.0 with CUDA 11.0 support.
  • Upgrades PaddlePaddle to 2.0.2.
  • Upgrades SentencePiece to 0.1.95.
  • Introduces the djl-serving brew package: now you can install djl-serving with brew install djl-serving.
  • Introduces the djl-serving plugins.
  • Introduces Amazon Elastic Inference support.

Enhancement

  • Improves TensorFlow performance by reducing GC and fixed memory leaking issue (#892)
  • djl-serving now can run all the engines out-of-box (#886)
  • Improves DJL training by using multi-threading on each GPU (#743)
  • Implements several operators:
    • Adds boolean set method to NDArray (#784)
    • Adds batch dot product operator (#849)
    • Adds norm operator to PyTorch (#692)
    • Adds one hot operator (#684)
    • Adds weight decay to Loss (#788)
  • Adds setGraphExecutorOptimize option for PyTorch engine. (#904)
  • Introduces String tensor support for ONNXRuntime (#724)
  • Introduces several API improvements
    • Creates ObjectDetectionDataset (#683)
    • Improves Block usability (#712)
    • Adds BlockFactory feature in model loading (#805)
    • Allows PyTorch stream model loading (#729)
    • Adds NDList decode from InputStream (#734)
    • Adds SymbolBlock Serialization (#687)
  • Introduces model searching feature in djl central (#799)

Documentation and examples

Breaking change

  • Renames CheckpointsTrainingListener to SaveModelTrainingListener (#686)
  • Removes erroneous random forest application (#726)
  • Deletes DataManager class (#691)
  • Classes under ai.djl.basicdataset packages has been moved into each sub-packages.

Bug Fixes

  • Fixes BufferOverflowException when handling handling subimage (#866)
  • Fixes ONNXRuntime 2nd engine dependency from IrisTranslator (#853)
  • Fixes sequenceMask error when n dimension is 2 (#828)
  • Fixes TCP port range buf in djl-serving (#773)
  • Fixes one array case for concat operator (#739)
  • Fixes non-zero operator for PyTorch (#704)

Known issues

Contributors

This release is thanks to the following contributors:

Package Rankings
Top 4.48% on Repo1.maven.org
Top 6.72% on Proxy.golang.org
Related Projects
machine-learning-curriculum

Learn to make machines learn so that you don't have to struggle to program them; The ultimate list

12 Dec 2016 1,084
kotlindl

High-level Deep Learning Framework written in Kotlin and inspired by Keras

25 Mar 2020 1,411
DLTK

Deep Learning Toolkit for Medical Image Analysis

02 May 2017 1,426
deepxde

A library for scientific machine learning and physics-informed learning

08 Feb 2019 2,634
TensorLayerX

TensorLayerX: A Unified Deep Learning and Reinforcement Learning Framework for All Hardwares, Bac...

03 Dec 2021 537