Bot releases are hidden (Show)

ILGPU - Release v1.0.0-beta2

Published by m4rs-mt over 3 years ago

This new beta offers significant performance improvements to the generated kernel programs and includes a lot of amazing new features (get the Nuget package).

Please note that this version has some breaking changes compared to previous ILGPU versions.
Refer to the v1.0-beta1 summary for more information.

ILGPU - Release v1.0.0-beta1

Published by m4rs-mt over 3 years ago

This new beta offers significant performance improvements to the generated kernel programs and includes a lot of amazing new features (get the Nuget package).

Please note that this version has some breaking changes compared to previous ILGPU versions.

Breaking changes

The Memory API, involving ArrayView and MemoryBuffer types has been significantly improved to support explicit Stride information (see below).
All IndexX and LongIndexX types have been renamed to IndexXD and LongIndexXD to have a unified programming experience with respect to memory buffers and array views (see below).
The Device API has been redesigned to explicitly enable, filter and configure the available hardware accelerator devices (see below).

Changes

Added new Memory API to support explicit stride information (#421, #475, #483).
Added new Device API to enable, filter and configure the available hardware accelerator devices (#428).
Added support for OpenCL 3.0 API (#464).
Added support for inline PTX assembly instructions (#467).
Added support for multi-dimensional and static constant arrays (#479).
Added support for convenient profiling use ProfilingMarkers (#482).
Improved CPU runtime to support arbitrary Warp/Group/Multiprocessor configurations (#402, #484).
Improved error messages (#466)
Enabled folding of debug assertions in IRBuilder (#477).
Fixed Group helper methods for multi-dimensional kernels (#481).
Fixed invalid code generation of OpenCL kernels in the presence of constant switch conditions (#441).

Summary of the changes related to the new Memory API

The new API distinguishes between a coherent, strongly typed ArrayView<T> structure and its n-D versions ArrayViewXD<T, TStride>, which carry dimension-dependent stride information (The actual logic for computing element addresses is moved from the IndexXD types to the newly added StrideXD types). This allows developers to explicitly specify a particular stride of a view, reinterpret the data layout itself (by changing the stride), and perform compile-time optimizations based on explicitly typed stride information. Consequently, ILGPU's optimization pipeline is able to remove the overhead of these abstractions in most cases (except in rare use cases where strange-looking strides are used). It also makes all memory transfer-related operations explicit in terms of what memory layout the underlying data will have after an operation is performed.

In addition, it moves all copy related methods to the ArrayView instances instead of exposing them on the memory buffers. This realizes a "separation of concerns": One the one hand, a MemoryBuffer holds a reference to the native memory area and controls its lifetime. On the other hand, ArrayView structures manage the contents of these buffers and make them available to the actual GPU kernels.

Example:

// Simple 1D allocation of 1024 longs with TStride = Stride1D.Dense (all elements are accessed contiguously in memory)
var t = accl.Allocate1D<long>(1024);

// Advanced 1D allocation of 1024 longs with TStride = Stride1D.General(2) (each memory access will skip 2 elements)
// -> allocates 1024 * 2 longs to be able to access all of them
var t = accl.Allocate1D<long, Stride1D.General>(1024, new Stride1D.General(2));

// Simple 1D allocation of 1024 longs using the array provided
var data1 = new long[1024];
var t2 = accl.Allocate1D(data1);

// Simple 2D allocation of 1024 * 1024 longs using the array provided with TStride = Stride2D.DenseX
// (all elements in X dimension are accessed contiguously in memory)
// -> this will *not* transpose the input buffer as the memory layout will be identical on CPU and GPU
var data2 = new long[1024, 1024];
var t3 = accl.Allocate2DDenseX(data2);

// Simple 2D allocation of 1024 * 1024 longs using the array provided, with TStride = Stride2D.DenseY
// (all elements in Y dimension are accessed contiguously in memory)
// -> this *will* transpose the input buffer to match the desired data layout
var data3 = new long[1024, 1024];
var t4 = accl.Allocate2DDenseY(data3);

The major changes/features of the new Memory API are:

Index1|Index2|Index3 types have been renamed to Index1D|Index2D|Index3D to match the naming scheme of ArrayViewXD and MemoryBufferXD types.
LongIndex1|LongIndex2|LongIndex3 types have been renamed to LongIndex1D|LongIndex2D|LongIndex3D to match the naming scheme of the ArrayViewXD and MemoryBufferXD types.
Separation of concerns between MemoryBuffer and ArrayView instances:
- ArrayView... structures represent and manage the contents of buffers (or chunks of buffers).
- MemoryBuffer... classes manage the lifetime of allocated memory chunks on a device.
The ILGPU.ArrayView intrinsic structure implements the newly added IContiguousArrayView interface that marks contiguous memory sections.
The ILGPU.Runtime.MemoryBuffer... classes implement the newly added IContiguousArrayView interface that marks contiguous memory sections.
Types implementing the IContiguousArrayView interface provide extension methods for initializing, copying from and to the memory region (not supported on accelerators).
This PR adds the notion of Strides. ILGPU contains built-in common strides for 1D, 2D and 3D views.
- Stride1D.Dense represents contiguous chunks of memory that pack elements side by side.
- Stride1D.General represents strides that skip a certain number of elements.
- Stride2D.DenseX represents 2D strides that pack elements side by side in dimension X (transfers from a to views with this stride involve transpose operations).
- Stride2D.DenseY represents 2D strides that pack elements in the Y dimension side by side.
- Stride2D.General represents strides that skip a certain number of elements in the X and Y dimensions.
- Stride3D.DenseXY represents 3D strides that pack elements in the X,Y dimension side by side (transfers from a to views with this stride involve transposition operations).
- Stride3D.DenseZY represents 3D strides that pack elements in the Z,Y dimension side by side.
- Stride3D.General represents strides that omit a certain number of elements in the X, Y and Z dimensions.
All ArrayViewXD types have been moved to the ILGPU.Runtime namespace.
All ArrayViewXD types do not implement IContiguousArrayView, as they support arbitrary stride information.
Note that the ArrayView1D<T, Stride1D.Dense> specialization has an implicit conversion to ArrayView<T> (and vice versa) for auxiliary purposes.
All CopyFromCPU and CopyToCPU methods are provided with additional hints as to whether they are transposing the input elements or keeping the original layout.
Note that GetAsXDArray(...) always returns elements in .Net standard layout for 1D, 2D and 3D arrays (this may result in transposing the input elements of the buffer on the CPU).
Use view.AsContiguous().GetAsArray() to get the memory layout of the input buffer.

Summary of the changes related to the new Device API

The new Device API removes the enumeration ContextFlags and implements the same functionality in an object oriented way using a Context.Builder class. It offers a fluent-API like configuration interface which makes it easy to set up:

// Enables all supported accelerators (default CPU accelerator only) and puts the context
// into auto-assertion mode via "AutoAssertions()". In other words, if a debugger is attached,
// the `Context` instance will turn on all assertion checks. This behavior is identical
// to the current implementation via new Context();
using var context = Context.CreateDefault();

// Turns on O2 and enables all compatible Cuda devices.
using var context = Context.Create(builder =>
{
    builder.Optimize(OptimizationLevel.O2).Cuda();
});

// Turns on all assertions, enables the IR verifier and enables all compatible OpenCL devices.
using var context = Context.Create(builder =>
{
    builder.Assertions().Verify().OpenCL();
});

// Turns on kernel source-line annotations, fast math using 32-bit float and enables
// *all* (even incompatible) OpenCL devices.
using var context = Context.Create(builder =>
{
    builder
        .DebugSymbols(DebugSymbolsMode.KernelSourceAnnotations)
        .Math(MathMode.Fast32BitOnly)
        .OpenCL(device => true);
});

// Selects an OpenCL device with a warp size of at least 32:
using var context = Context.Create(builder =>
{
    builder.OpenCL(device => device.WarpSize >= 32);
});

// Turns on all assertions in debug mode (same behavior like calling CreateDefault()):
using var context = Context.Create(builder =>
{
    builder.AutoAssertions();
});

// Turns on debug optimizations (level O0) and all assertions if a debugger is attached:
using var context = Context.Create(builder =>
{
    builder.AutoDebug();
});

// Turns on debug mode (optimization level P0, assertions and kernel debug information):
using var context = Context.Create(builder =>
{
    builder.Debug();
});

// Disable caching, enable conservative inlining and inline mutable static field values:
using var context = Context.Create(builder =>
{
    builder
        .Caching(CachingMode.Disabled)
        .Inlining(InliningMode.Conservative)
        .StaticFields(StaticFieldMode.MutableStaticFields);
});

// Turn on *all* CPU accelerators that simulate different hardware platforms:
using var context = Context.Create(builder => builder.CPU());

// Turn on an AMD-based CPU accelerator:
using var context = Context.Create(builder => builder.CPU(CPUDeviceKind.AMD));

Note that by default all debug symbols are automatically turned off when a debugger is attached. If you want to turn on the debug information in all cases, call .builder.DebugSymbols(DebugSymbolsMode.Basic). At the same time, this PR introduces the notion of a Device, which replaces the implementation of AcceleratorId. This allows us to query detailed device information without explicitly instantiating an accelerator:

// Print all device information without instantiating a single accelerator
// (device context) instance.
using var context = Context.Create(...);
foreach (var device in context)
{
    // Print detailed accelerator information
    device.PrintInformation();

    // ...
}

Note that we removed the ability to call the accelerator constructors (e.g. new CudaAccelerator(...)) directly. Either use the CreateAccelerator methods defined in the Device classes or use one of the extension methods like CreateCudaAccelerator(...) of the Context class itself:

using var context = Context.Create(...);
foreach (var device in context)
{
    // Instantiate an accelerator instance on this device
    using Accelerator accel = device.CreateAccelerator();
    // ...
}

// Instantiate the 2nd Cuda accelerator (NOTE that this is the *2nd* Cuda device
// and *not* the 2nd device of your machine).
using CudaAccelerator cudaDevice = context.CreateCudaAccelerator(1);

// Instantiate the 1st OpenCL accelerator (NOTE that this is the *1st* OpenCL device
// and *not* the 1st device of your machine).
using CLAccelerator clDevice = context.CreateOpenCLAccelerator(0);

Context properties that expose types from other (ILGPU internal) namespaces that cannot/should not (?) be covered by the API/ABI guarantees we want to give, has been made internal properties. To access these properties, use one of the available extension methods located in the corresponding namespaces:

using var context = ...

// OLD way
var internalIRContext = context.IRContext;

// NEW way:
// using namespace ILGPU.IR;
var internalIRContext = context.GetIRContext();

Improved CPU runtime to support arbitrary Warp/Group/Multiprocessor configurations

The new CPU runtime significantly improves the existing CPUAccelerator runtime by adding support for user-defined warp, group and multiprocessor configurations. It changes the internal functionality to simulate a single warp of at least 2 threads (which ensures that all shuffle-based/reduction-like algorithms can also be run on the CPU by default). At the same time, each virtual multiprocessor can only execute a single thread group at a time. Increasing the number of virtual multiprocessors allows the user to simulate multiple concurrent groups. Most use cases will not require more than a single multiprocessor in practice.

Note that all device-wide static Grid/Group/Atomic/Warp classes are fully supported to debug/simulate all ILGPU kernels on the CPU.

Note that a custom warp size must be a multiple of 2.

This PR adds a new set of static creation methods:

CreateDefaultSimulator(...) which creates a CPUAccelerator instance with 4 threads per warp, 4 warps per multiprocessor and a single multiprocessor (MaxGroupSize = 16).
CreateNvidiaSimulator(...) which creates a CPUAccelerator instance with 32 threads per warp, 32 warps per multiprocessor and a single multiprocessor (MaxGroupSize = 1024).
CreateAMDSimulator(...) which creates a CPUAccelerator instance with 32 threads per warp, 8 warps per multiprocessor and a single multiprocessor (MaxGroupSize = 256).
CreateLegacyAMDSimulator(...) which creates a CPUAccelerator instance with 64 threads per warp, 4 warps per multiprocessor and a single multiprocessor (MaxGroupSize = 256).
CreateIntelSimulator(...) which creates a CPUAccelerator instance with 16 threads per warp, 8 warps per multiprocessor and a single multiprocessor (MaxGroupSize = 128).

Furthermore, this PR adds support for advanced debugging features that enable a "sequential-like" execution mode. In this mode, each thread of a group will run sequentially one after another until it hits a synchronization barrier or exits the kernel function. This allows users to conveniently debug larger thread groups consisting of concurrent threads without switching to single-threaded execution. This behavior can be controlled via the newly added CPUAcceleratorMode enum:

    /// <summary>
    /// The accelerator mode to be used with the <see cref="CPUAccelerator"/>.
    /// </summary>
    public enum CPUAcceleratorMode
    {
        /// <summary>
        /// The automatic mode uses <see cref="Sequential"/> if a debugger is attached.
        /// It uses <see cref="Parallel"/> if no debugger is attached to the
        /// application.
        /// </summary>
        /// <remarks>
        /// This is the default mode.
        /// </remarks>
        Auto = 0,

        /// <summary>
        /// If the CPU accelerator uses a simulated sequential execution mechanism. This
        /// is particularly useful to simplify debugging. Note that different threads for
        /// distinct multiprocessors may still run in parallel.
        /// </summary>
        Sequential = 1,

        /// <summary>
        /// A parallel execution mode that runs all execution threads in parallel. This
        /// reduces processing time but makes it harder to use a debugger.
        /// </summary>
        Parallel = 2,
    }

By default, all CPUAccelerator instances use the automatic mode (CPUAcceleratorMode.Auto) that switches to a sequential execution model as soon as a debugger is attached to the application.

Note that threads in the scope of multiple multiprocessors may still run in parallel.

Major internal changes:

Added build support for .Net5.0 (#446).
Added support for T4.Build to automatically transform T4 text templates during build (#431).
Restrict net47 unit tests to only run on CI builds (#465).
Avoid duplicate CI runs for pull requests made from the same repo (#485).
Updated InlineList implementation to reduce memory consumption (#478).
Fixed invalid assertion affecting successor blocks in frontend (#445).

Special thanks

Special thanks to @MoFtZ, @Joey9801, @jgiannuzzi and @GPSnoopy for their contributions to this release in form of code, feedback, ideas and proposals. Furthermore, we would like to thank the entire ILGPU community (especially @MPSQUARK, @Nnelg, @Ruberik, @Yey007, @faruknane, @mikhail-khalizev, @nullandkale and @yuryGotham) for providing feedback, submitting issues and feature requests.

ILGPU - Release v0.10.1

Published by m4rs-mt over 3 years ago

The new stable version contains several bug fixes and improves the code quality of the generated kernel programs (get the Nuget package).

It is strongly recommended to upgrade to this version as soon as possible to avoid known bugs and some CPU-buffer deallocation issues.

Changes

Added CopySign intrinsic (#438).
Added intrinsic mappings for BitConverter functions (#437).
Added call stack recording during compilation for error reporting (#436).
Gracefully fail when loading symbols from in-memory assemblies (#435).
Fixed invalid detection of loop bodies (#452).
Fixed incorrect assertion on repeating successors (#447).
Fixed emitting switch statement with constant condition (#442).
Fixed invalid disposal of CPU buffers (#440).
Fixed applications blocking during tear-down by changing Accelerator GC thread to run in the background (#439).
Fixed bounds check on large views (#433).
Fixed retrieving field from structure types (#426).

Special thanks

Special thanks to @MoFtZ, @marcin-krystianc and @jgiannuzzi for their contributions to this release and to the entire ILGPU community for providing feedback, submitting issues and feature requests.

ILGPU - Release v0.10.0

Published by m4rs-mt over 3 years ago

The new stable version offers significant performance improvements of the generated kernel programs and contains critical resource deallocation fixes (get the Nuget package).

It is strongly recommended to upgrade to this version as soon as possible to avoid resource and GC related deallocation issues.

Breaking changes

The inheritance hierarchy of the ExchangeBuffer class has been changed to avoid exposing internal memory buffers. If you previously relied on the immediate inheritance from ExchangeBufferBase on MemoryBuffer, you have to adapt your program to use the intermediate base class MemoryBuffer<T, TIndex> instead (see diff).
Properties exposing internal memory buffers of the high-level MemoryBufferXD classes have been removed to avoid ownership related GC-free issues (see diff).

Why are there breaking changes?

We have decided to remove dangerous properties from several memory buffer classes. The use of these properties can lead to program crashes, since buffers could be disposed asynchronously in the background by the GC without further notice.

Changes

Improved performance of kernel launchers by passing packed argument structures (#358, #372).
Graduated different optimizations from O2 to O1 (release mode) to improve performance in release builds using an additional of stable optimization passes (#344).
Graduated O2 optimizations in the Cuda backend to O1 pipeline to generate vectorized IO operations in release builds (#350).
Added support for managed sizeof IL instruction (#380).
Added PrintInformation method to Accelerator instances to print detailed accelerator information (#389).
Added enhanced assertions and out-of-bounds checks to all ArrayView accesses on GPU devices (Use flag ContextFlags.EnableAsserations or attach a debugger to your application to enable assertion checks. Make sure to use the portable debug information format for detailed source location information) (#375).
Added support for printf-like output in Kernels for CPU, Cuda and OpenCL accelerators (#342).
Added new utility Launch/LaunchAutoGrouped methods to immediately launch kernels using a separate strong-reference cache (#336).
Added new AlignTo alignment methods to explicitly align ArrayView instances to a particular alignment in bytes (#316).
Added enhanced support for local memory via a new LocalMemory class (#316).
Added support for several PopCount, CLZ and CTZ operations (#324).
Added new MemSet functions to all memory buffers (#338).
Added new IfConditionalConversion to fold nested and-also and or-else block chains to O2 pipeline (#328).
Added new local memory optimizations to simplify array accesses (#317).
Added simple 64-bit-based LongGlobalIndex helper to simplify correct computations using 64-bit integers (#337).
Added new CLPlatformVersion and fixed OpenCL 1.2 compatibility issues (#335).
Removed support for .NET Core 2.0 (#353).
Prevent using SharedMemory in implicitly grouped kernels (#354).
Prevent using CudaAccelerator and CLAccelerator instances to run on non-native OS .NET versions (#396).
Fixed critical GC-related resource deallocation issues (#376, #393).
Fixed returning correct length of dynamic shared memory buffers (#357).
Fixed invalid alignment information in the presence of reinterpret casts (#386).
Fixed invalid address computations of fixed array buffers (#361).
Fixed invalid PTX calling convention (#362).
Fixed edge cases in LoopUnrolling (#373).
Fixed invalid printf formats for int64 and uintX types (#391).
Fixed invalid DebugArrayView implementations (#345).
Fixed invalid initializations of local memory arrays (#287).

Major internal changes:

Removed singleton instance of RuntimeSystem to avoid concurrency/reflection-API issues (#393).
Updated default optimizations for ILGPU debug builds (#384).
Added support for unity tests running on. NET Framework 4.7 (#355).
Migrated from FxCop analyzers to .NET analyzers. (#352).
Redesigned internal address-space inference passes (#364).

Special thanks

Special thanks to @MoFtZ, @Ruberik and @jgiannuzzi for their contributions to this release and to the entire ILGPU community for providing feedback, submitting issues and feature requests.

ILGPU - Release v0.10.0-beta2

Published by m4rs-mt over 3 years ago

This new beta version offers important bug fixes and performance improvements of the generated kernel programs and a set of new features (get the Nuget package).

Improved performance of kernel launchers by passing packed argument structures (#358, #372).
Added support for managed sizeof IL instruction (#380).
Added PrintInformation method to Accelerator instances to print detailed accelerator information (#389).
Added enhanced assertions and out-of-bounds checks to all ArrayView accesses on GPU devices (Use flag ContextFlags.EnableAsserations or attach a debugger to your application to enable assertion checks. Make sure to use the portable debug information format for detailed source location information) (#375).
Removed support for .NET Core 2.0 (#353).
Prevent using SharedMemory in implicitly grouped kernels (#354).
Prevent using CudaAccelerator and CLAccelerator instances to run on non-native OS .NET versions (#396).
Fixed critical GC-related resource deallocation issues (#376, #393).
Fixed returning correct length of dynamic shared memory buffers (#357).
Fixed invalid alignment information in the presence of reinterpret casts (#386).
Fixed invalid address computations of fixed array buffers (#361).
Fixed invalid PTX calling convention (#362).
Fixed edge cases in LoopUnrolling (#373).
Fixed invalid printf formats for int64 and uintX types (#391).

Major internal changes:

Removed singleton instance of RuntimeSystem to avoid concurrency/reflection-API issues (#393).
Updated default optimizations for ILGPU debug builds (#384).
Added support for unity tests running on. NET Framework 4.7 (#355).
Migrated from FxCop analyzers to .NET analyzers. (#352).
Redesigned internal address-space inference passes (#364).

Special thanks to @MoFtZ, @Ruberik for their contributions to this release and to the entire ILGPU community for providing feedback, submitting issues and feature requests.

ILGPU - Release v0.9.0