General

• Diagnostic warnings are emitted when operators fallback to CPU when the caller specified GPU.
• New -logCPUFallback command line argument added. This argument will output addition information about why CPU fallback occured.

Dependencies

• DirectML version bumped to 1.8.2.
• Microsoft.AI.MachineLearning version bumped to 1.11.
• OnnxRuntime version bumped to 1.11.

This change removes the ability of WinMLRunner.exe to load Windows.AI.MachineLearning.dll locally if the DLL is within the same directory as the executable. Instead, MicrosoftMLRunner.exe should be used for loading Microsoft.AI.MachineLearning.dll within the same directory. More details below.

Using Microsoft.AI.Machinelearning NuGet

MicrosoftMLRunner.exe uses Microsoft.AI.MachineLearning.dll in the immediate directory of the executable instead of loading Windows.AI.MachineLearning.dll from System32 (that is for WinMLRunner). MicrosoftMLRunner is useful to compare performance with an older version of WinML or to test a newer version of WinML's NuGet. For more information, please reference Microsoft.AI.MachineLearning NuGet page.

Fixes bug: https://github.com/microsoft/Windows-Machine-Learning/issues/324

• Support to convert to opset10 models
• Updated WinMLRunner

New Features:

• -Tensor [function] : load the input as a tensor, with optional function for input preprocessing

  • Optional function arguments:
    • Identity(default) : No input transformations will be performed
    • Normalize : float scale factor and comma separated per channel means and stddev for normalization.

• Example Usage:

• WinMLRunner.exe -model ./densenet121.onnx -Tensor Normalize 255 0.485,0.456,0.406 0.229,0.224,0.225 -CPU -SaveTensorData First -PerIterationPath <tensorDataPath>

New Features:

• Evaluating folder of images is now available with -inputimagefolder flag.
  example:

.\WinMLRunner.exe -model .\SqueezeNet.onnx -inputimagefolder .\images\

• Standard deviation of metrics are now written to performance files

Updates and fixes:

• DxCore API calls have been updated to latest.

New Features:

• Model Debugging Support
  • You can now capture and visualize intermediate data as it flows through your model when you run it through WinMLDashboard
  • More information can be found here: https://github.com/microsoft/Windows-Machine-Learning/blob/master/Tools/WinMLDashboard/README.md
• Other various UI Changes

Bug Fixes

• Security fixes

New change:

Delayload DXGI and D3d11.dll with DxCore Path

New Features!

• DxCore support to select compute device. Use the functionality with :
  • -GPUAdapterName : run model on GPU specified by its name. NOTE: Please only use this flag on DXCore supported machines.
• Models with Image Denotation as Input are now supported
• Tensors can now be bound backed by GPU memory.
• Performance File metadata can be added with: AddPerformanceFileMetadata(const std::string& key, const std::string& value). This is only exposed when WinMLRunner is consumed as a static library.

Bug Fixes:

• Maximum -iterations now bumped up to 1024 iterations.
• Outputting performance files now has more robust way to specify destination.
  • -BaseOutputPath [<fully qualified path>] : base output directory path for results, default to cwd

New Features:

• Added new flag:
  -TopK <number>: print top <number> values in the result. Default to 1
• PIX programmatic capture!
  • PIX is a tool to capture Direct 3D 12 GPU work. With this new release, WinMLRunner will automatically capture PIX traces if it is launched with PIX.
    • 1st Frame: Will contain PIX captures for Session Creation and 1st iteration Bind, 1st iteration Evaluate
    • Nth Frame: Will contain PIX captures for Nth iteration Bind and Nth iteration Evaluate
  • More information about PIX here.
• .jpeg is now a supported input file format.

Bug Fixes

• If multiple compute devices are specified, WinMLRunner will continue to run the remaining devices even if the the model run fails on one of the devices.
• Print out error message if the input is an unsupported file extension.

x64 and x86 binaries of WinMLRunner as of ecc2b37.

New Features:

• SaveTensorData now takes in a second parameter to allow user to specify path to output tensor data.
• Memory Peak Working Set Usage is printed with "-perf all".
• Float16 output tensors can now be processed with max index / value printed to console and all the tensor values can be saved to CSV.
• Concurrency Scenarios have been added!
  • WinMLRunner supports concurrently loading a model. Currently it is only loading a model for concurrency, but expect to see more changes in the future to support multi-threaded bind and evaluate scenarios
  • "-ConcurrentLoad" will enable multi threaded options where it creates a thread pool of loading a model
  • "-NumThreads" specifies the number of threads in the thread pool for work.
  • "-ThreadInterval" specifies the number in milliseconds in between two works to be pushed into work queue.

Bug Fixes:

• Printing out version number with "-version" flag now has better error checking
• Removed DXCore functionality
• Better invalid commandline args checking
• More efficient output tensor processing
• No longer able to save tensor output if input isn't specified.
• Added extra checking to "-debug" flag which checks if WinMLRunner is under debugger before continuing. This flag got renamed to "-DebugEvaluate".
• Populating tensors with arbitrary values no longer use memset. The memory is arbitrary.

This release is x86 version of WinMLRunner as of commit: bb51359ff9cf827a1869893f5487d0b68533f5f5

Some new changes:

• performance output now separates first iteration performance from the rest of the iteration's performance.
• -perf flag now takes an optional parameter: "all" in which max / min metrics will be displayed in addition to averages. To see max / min metrics, run winmlrunner with iterations >=2 and -perf all
• -version flag prints the version information for this build of WinMLRunner.exe. For example:

>WinMLRunner.exe -version
WinMLRunner.exe
Version: 1.0.0.0.190214.bb51359

• Bug Fix: -CreateDeviceOnClient is now mapped to the GPU device that is specified using -GPUHighPerformance -GPUMinPower or -GPU

This release is x64 version of WinMLRunner as of commit: bb51359ff9cf827a1869893f5487d0b68533f5f5

Some new changes:

• performance output now separates first iteration performance from the rest of the iteration's performance.
• -perf flag now takes an optional parameter: "all" in which max / min metrics will be displayed in addition to averages. To see max / min metrics, run winmlrunner with iterations >=2 and -perf all
• -version flag prints the version information for this build of WinMLRunner.exe. For example:

>WinMLRunner.exe -version
WinMLRunner.exe
Version: 1.0.0.0.190214.bb51359

• Bug Fix: -CreateDeviceOnClient is now mapped to the GPU device that is specified using -GPUHighPerformance -GPUMinPower or -GPU

This release is x64 version of WinMLRunner as of commit: b9229dc7eaf58ec555f7c83bc0c3735343e41f79