The-Forge

• The Forge now uses the Ozz animation system. This is our first major change to the math library because we needed to add what is necessary for animation. We compared the speed of our math library then with the speed of the Ozz animation system math library and they are about the same. At some point we need to do AVX 1 and ARM specific optimizations to our math library, which is still based on the open-sourced changed Sony math library Vectormath, which is similar to the Bullet math library ...

A screenshot of the PC version:

Here are shots of four of the supported platforms:

PC Windows 10 Vulkan 1.1.77 GeForce 1080 Driver 399.07 with a resolution of 2560x1440 in full-screen:

Linux Vulkan 1.1.82.1 RADEON 480 Driver 18.30 with a resolution of 1920x1080 in full-screen:

iMac with AMD RADEON 580 (Part No. MNED2xx/A) with resolution of 5120x2880 in full-screen:

iPhone 7 with iOS 12.0 (16A366) and a resolution of 1334 x 750 in full-screen:

XBOX One:

We added several unit tests for Ozz. Please find descriptions below.

• Extended the file system for all platforms, making it more consistent to use and more functional
• Tabified - tabified the whole code base ... finally ...
• glTF - we made sure glTF 2.0 support is improved in the latest assimp we use. This should allow us in the future to use glTF 2.0 art assets easier
• Linux / Vulkan - we applaud Khronos for the native Ubuntu Linux package for all the elements of the Vulkan SDK LunarG Vulkan SDK Packages for Ubuntu 16.04 and 18.04 and will support those from now on. This is something that makes gaming on Linux Ubuntu easier to approach for end-users.
• macOS
  • we have a hard time to get the "Wave Intrinsics unit test" running. It runs now on our MacBook Pro test machines with Intel GPUs but not on the iMacs with AMD GPUs. It throws compile errors on the iPhone 7. We are providing Apple with info on their bug tracker.
  • upgraded to Xcode Version: 10.0 (10A255) and MacOS Version: 10.14 (18A389)
• iOS
  • there are numerous improvements for the input system, imGUI and other things
  • support shader includes on iOS like on the other platforms
• Android Pie & Vulkan 1.1 - we are waiting for devkits with better driver support before we move on with this target platform
• XBOX One - see internal GitLab list for improvements as usual ...

• The Forge now has initial support for Android with the same Vulkan 1.1 run-time that is used by the Windows and Linux platforms.

There are only a few phones that support Vulkan 1.1. We developed and support the
Essential Phone.
This phone uses an Adreno 540 GPU and can be obtained relatively cheap and unlocked from Amazon or similar sources. Please check out Vulkan Gpuinfo.org for the supported feature list of this GPU with Android 9.0.
At the moment only the first two unit tests are running but we will extend the support to as many as we can in an upcoming release. We expect to run into driver bugs. We will not actively support Vulkan 1.0.

• Vulkan - mobile phone support see above
• iOS:
  • upgraded to 12.0 (16A366)
  • we extended imGUI support for iOS with touch support and virtual keyboard
• Windows - fixed resizing ... should now work again with Alt+Enter ...
• Windows DXR - The Forge was the first rendering framework to support DXR, now that there will be hardware support, we added a screenshot of one of the unit tests below. We will add support for the other Ray Tracing APIs as well. They are useful for developing Tools. For games we recommend the Hybrid Ray-Traced solutions provided below (so far only shadows).

• Upgraded the Order-Independent Transparency unit test to support Phenomenological Transparency as described in one of Morgan McGuire's papers. Here are screenshots:

Phenomenological Transparency - Refraction

Phenomenological Transparency - Diffusion

Phenomenological Transparency - Diffusion, Refraction, Shadows

• Linux - the wave intrinsics unit test now also works on Linux. We are looking into macOS / iOS next.
• Call for contributors to change IRenderer.h - as a heads-up, we will change the renderer interface to use our own way of "Render Passes and Compute Passes" similar to the Vulkan render passes. We will analyze render and compute calls and use this information to expose data to a "Render Graph" that will be implemented on a higher level. If you want to be part of the group that thinks about how we should adjust IRenderer.h for this, please send Wolfgang a note and your skype ID

• There is a new unit test that evaluates various Order-Independent Transparency methods:
  • Alpha blended transparency
  • Weighted blended Order Independent Transparency (Morgan McGuire)
  • Weighted blended Order Independent Transparency by Volition (GDC 2018 Talk)
  • Adaptive Order Independent Transparency with Raster Order Views (paper by Intel, supports DirectX 11, 12 only)
    There are more details and a comparison in the documentation folder of the unit test.

PC Windows 10 DirectX 12 GeForce 950 Driver 411.63 with a resolution of 1080p in full-screen:

Linux Vulkan 1.1.82.1 GeForce 1080 Driver 390.87 with a resolution of 1080p in full-screen:

MacBook Pro (Model A1706) with a resolution of 1080p in full-screen:

iPhone 7 with a resolution of 1334x750 in full-screen:

• There is a new Wave Intrinsics Unit test. It tests the new wave intrinsic instructions and makes sure they are support on PC Windows (DirectX 12 / Vulkan) for now. We are planning to add support on other platforms. On PC with DirectX it utilizes a "preview" integration of the DirectX Shader compiler. We were running into multiple bugs with DXC and decided to only use it for the new Shader model 6 shaders. So The Forge uses DXC for shaders that request shader_target_6_0 and FXC for everything below. The shader target can be specified during load time.

PC Windows 10 DirectX 12 GeForce 950 Driver 411.63 with a resolution of 1080p in full-screen:

• PC - fixed resizing ... should now work again with Alt+Enter ...
• Vulkan
• upgraded to the latest Vulkan Memory Allocator
• Enabled subgroup extensions for Wave Intrinsics Unit Test
• macOS / iOS
  • on-going work on improving input on macOS and iOS with imGUI
  • upgraded to macOS: 10.14 (18A389), XCode: 10 (10A254a)
• imGUI: improved implementation to make it a native citizen and expose the whole functionality. for tools, now there is more flexibility to create individual UIs for all apps. Check out the new screenshot below.

imGUI support:

• Linux - upgraded AMDGPU-Pro driver to 18.30-641594
• The Confetti Shader Translator received lots of bugs fixes and upgrades over the last six weeks. Head over to the URL and check it out. We have a large backlog with bugs and therefore it is still in alpha. New Features are:
  • include file support
  • preprocessor branching
  • switch statement
  • nested array
  • texture argments for functions
  • pur buffer (Buffer) type
  • Shader Model 6.0 (Wave)
    • SampleGrad()
    • GatherRed()
    • all()
    • gatherRed()
    • f32to16()
    • f16to32()
    • NonUniformResourceIndex()
    • determinate functions
  • GLSL
    • Shader Model 6.0 functions (Subgroup)
    • #extension system (Now, only necessary extensions are written, automatically)
    • textureGather()
    • textureGrad()
  • added loading progress bar
  • and others ...
    Here is a screenshot of the latest version.
    

• This is the biggest release so far :-)
• We added a new unit test -provided by Kostas Anagnostou @KostasAAA- that shows hybrid ray traced (HRT) shadows on all supported GPUs; Windows / Linux / XBOX One / macOS and iOS. This unit test was build to show how to ray trace shadows without using a ray tracing API like DXR / RTX. It should run on all GPUs (not just NVIDIA RTX GPUs) and the expectation is that it should run comparable with a DXR / RTX based version even on a NVIDIA RTX GPU. Kostas wrote a blog post about the details at https://interplayoflight.wordpress.com/2018/09/04/hybrid-raytraced-shadows-part-2-performance-improvements/.

PC Windows 10 Vulkan 1.1.82.1 GeForce 1080 Driver 398.82 with a resolution of 2560 * 1440 in full-screen:

iMac with AMD RADEON 580 (Part No. MNED2xx/A) with resolution of 2560x1440 in full-screen:

Here is a version running on a iPhone 7 with iOS 11.4.1 (15G77) and a resolution of 1334 x 750 in full-screen:

Please note there are no platform specific optimizations for macOS or iOS in the moment. The initial version was developed on PC and then just ported to the other platforms. We might optimize for the other platforms at some point.
Kostas plans to use the San Miguel Scene for this and then also merge it with the Light and Shadow Playground, so that you can pick different shadow techniques and compare them. He wants to implement HRT Reflections next. I wrote a short blog post about the development of this unit test at https://diaryofagraphicsprogrammer.blogspot.com/2018/09/ray-tracing-without-ray-tracing-api.html

• Improved the 09_LightShadowPlayground by calculating the SDF shadow in a more generic way
• Based on customer request and to support Windows 7, we added a DirectX 11 fallback code path. Because this API can not support all unit tests, it is only a choice on a select few
• IRenderer Cleanup for all platforms
  • removed several descriptor related structs, which enabled us to simplify the codebase
  • add render index draw start vertex
• We replaced NuklearUI with imGui; we are going to replace QT with imGui now for our internal tools, so trying to keep this consistent
• Created a new unit test for imGui #13 - it shows the current level of integration of imGui into The Forge
  
• Linux - changed to standard linux define instead of user defined LINUX
• Vulkan
  • supports now the descriptor update template extension: VK_KHR_DESCRIPTOR_UPDATE_TEMPLATE_EXTENSION_NAME
  • added ability to specify instance layers, instance, device extensions from app code
  • upgraded to Vulkan SDK 1.1.82.0
• The Torque 3D engine will use The Forge as its rendering layer in the future

• macOS - one thing notable is that The Forge on macOS can be used with an XBOX controller with this driver: 360Controller

• After evaluating how to approach a unified shader generation system by looking at the DirectX Shader Compiler in GitHub and also implementing it into Lumberyard a whiles ago, we decided to follow a simpler approach by developing our own shader translator based on the work that was done by Thekla with the hlslparser. We are currently extending this shader translator to support a new super HLSL source language (platform specific #defines + material description) and translate that sHLSL into the most current respective shader languages of the target platforms, so that those shaders can then be compiled with the target platform compiler. There is an early alpha version available online at Confetti Shader Translator following the excellent approach of Shader Playground as a base for our online approach (Thanks for your help tgjones!).
  The main motivation to use the shader translator instead of the DirectX Shader Compiler is code maintenance. Maintaining our own code cut of the DirectX Shader Compiler would add too much overhead on the team and it would make it much harder to implement our super HLSL language. We will spend some time testing this system. As soon as it is more reliable, there will be a Visual Studio extension and an extension for XCode to translate within the IDEs in the future.
  
• IRenderer Cleanup for all platforms (more cleanups to come)
  • Remove TextureType, RenderTargetType, TextureUsage, BufferUsage, BufferFeatureFlags
  • Expand DescriptorType enum to hold all descriptor usage patterns
  • Add ability to target mip slice UAV
• macOS
  • upgraded to 10.14 Beta (18A353d)
  • XCode 10.0 beta (10L176w)
• iOS 11.4.1 (15G77)

• Added a unified input system based on Gainput to all platforms (https://github.com/jkuhlmann/gainput). The new input system substantially simplified input management on the application level over all platforms. We also simplified the camera controller. Added also new VirtualJoystick class in UI.

• Added a Light and Shadow Playground unit test. There are two types of shadows supported in the moment exponential shadows and SDF shadows. There is a switch to pick one of them during run-time. In the future there will be more complex lighting and shadow setups.
  

• There is now functionality to create views per mip / per slice for textures and render target through TextureUsage and RenderTargetUsage flags (typecast cube map to 2D texture array etc.)

• UIRenderer was removed and there is now a dedicated text rendering system different from UI rendering and no UI rendering interface anymore

• cmdbindVertexBuffer, cmdBindIndexBuffer now support offsets

• cmdBindRenderTargets now takes color array, mip slices, depth array and mip slice index

• The Pixel-Projected Reflections unit test now uses bindless textures on all platforms except for iOS as its too many to bind in one go.

• iOS: all unit tests run on our iOS test device; Visibility Buffer still doesn't as described below.

• Moved the release notes from this page into the release section and added actual releases

• New unit test Pixel-Projected Projections: this unit test shows reflections that are ray traced. It is an implementation of the papers Optimized pixel-projected reflections for planar reflectors and Implementation of Optimized Pixel-Projected Reflections for Planar Reflections
  Sponza was added to the Art folder for this unit test.
• Updated unit test Material Playground: started out with Materials that can be summed up in the category Metal
• Extended the GPU config system further by adding more GPUs and classifying them with also better support for the GPU Workbench
• DirectX 12 / Vulkan:
  • added support for passing offsets, sizes through arrays in DescriptorData
  • added typed buffer support
  • upgraded to the 1.1.77.0 Vulkan SDK and updated volk for this
• macOS: V-Sync is now switchable
• Added 3D ability to the text rendering library: it was used to render the labels in the Material Playground unit test
• Resolved issues (zeros issues now? ... please keep them coming!):
  • #23 Request: Support V-Sync off on Metal unit tests..
  • #56 getUSec function in WindowsBase.cpp seem buggy

• Added code for our automated test framework
• Removed some old left-over code that was copied into the TinySTL code base. Now the whole codebase is more consistent with TinySTL
• Fixed some visual differences between APIs in the PBR example
• Separated the app interface load / init and unload / exit functions better in the whole code base
• Root signatures can be build for several sets of shaders while pipelines only have one shader set. This way you can use root signatures in several draw calls that will require similar settings except the shaders. We now reduced the number of root signatures in the Visibility Buffer to make that more obvious
• Move the shader cache directory to avoid any conflict
• PC
  • Made buffer / texture naming for PIX debugging more explicit
  • Changed the pipeline creation code to only require a few properties of the render target and not the whole render target
  • Vulkan: remove unnecessary lock in renderpass lookup
• iOS
  • Explicitly set the mCompleted value of a fence to false when submitting
  • Created the presentation command buffer from the main rendering queue to avoid the nil drawable issue on iOS
  • Removed unused CAMetalLayer
  • Removed util_bind_root_constant in favor of the setBytes function for rootConstant binding
  • Removed a bug in our fencing logic which was forcing to wait on all fences (ignoring their status)
• XBOX One: numerous changes and performance optimizations please check the log file of the internal repository

First of all: thanks a lot for the amount of feedback and support we received from the community. I think the amount of code changes in this release is a reflection of the strong feeback. Keep it coming!

• Abstracted the application level by using the interface in IApp.h
• Refactored addShader, it now loads binary shaders and the interface is more abstract. This reduced the number of lines of code in every example. This is an intermediate step to achieving a more unified cross-platform shader system
• Upgraded the BRDF example to PBR
• Removed Panini projection feature to add it into the new Middleware_3 folder
• Added the Fluid Studio memory manager after getting permission from the original author
• Fixed all bugs in issue tracker
• First iOS support. Please see iOS software and hardware requirements below.
  • Added support for on-tile textures
  • In the moment there is no UI and the camera and the input system is basic. We are planning to add https://github.com/jkuhlmann/gainput in the future
• PC:
  • Added a Visual Studio extension that allows to compile shaders with a right mouse click on the shader file (need to be installed from the Tools folder or by clicking on the PRE_BUILD.bat file)
  • Improved GPU recognition: in case a notebook has an integrated GPU and a discreete GPU: it will pick the discreete GPU under certain conditions
  • Now use raw mouse input to workaround the problems with the last Windows 10 update
  • Better synchronization between CPU and GPU
  • Moved to Visual Studio 2017 and Windows SDK 16299.91 (Fall Creators Update) and Vulkan SDK 1.0.68
• XBOX One: numerous performance improvements (check the non-public repository history log)

Any rendering framework needs a test farm that makes sure it is running on all the devices and OS'es supported. Our current testing setup is based on Jenkins and only supports a rather narrow range of hardware devices. We will add more hardware devices over time; this also depends when and how we get ahold of them. Considering that the feature set of The Forge is forward looking with support for Vulkan, DirectX 12 and Metal 2, we will leave out a lot of older hardware. We already started a conversation with Intel and Apple about the challenges we have and we will get in touch with driver teams from Samsung, Qualcomm,Google, AMD, NVIDIA and others to provide feedback on drivers.

• Fixed all the issues mentioned in the issue tracker.
• Removed the NVX commands, we don't use them and they seem to confuse people.
• Dealing with multiple resolutions on PC is now a bit easier. We need to expose this in the GUI, so that you can switch in full-screen between -let's say- 1080p and 4k back and forth
• For macOS the procedural planet unit test works now too. That should bring macOS on par with PC on the macOS platforms we are currently testing. All unit tests and the Visibility Buffer work.
• We improved performance of the Visibility Buffer on macOS a bit more. Now if you have a comparable GPU on the PC, the performance should be on a similar level on macOS and PC.

• Mainly improved the performance of the macOS build. macOS is now using the same art assets as the PC and the performance of the triangle filtering compute shader is improved.
• Reduced the size of the art assets because we only need one version of San Miguel for all platforms now.
• macOS now runs the Hardware Tessellation unit test.
• There is also now a new unit test that shows a procedurally generated planet.

Very first release.

• Added a IMiddleware interface that is now used by Panini Projection. This will be refined and used by more middleware packages in the future
• More unified math library compatible with the PS4 math library
• PC Vulkan
  • Upgraded to Vulkan 1.1.70.1
  • We are now using volk -the metaloader for Vulkan- and load all Vulkan functions dynamically to bypass the loader overhead
  • Due to lack of support of "NonUniformResourceIndex" for Vulkan, we had to create an "interesting workaround" in the Visibility Buffer demo
• XBOX One: major update! Numerous changes and performance optimizations please check the log file of the internal repository

• Addded initial support for Ubuntu 16.04.4 LTS with the AMD Pro Driver and RADEON RX 480 / Vega. Please see the Linux requirement list below for the required software packages and hardware support. Known Issues:
  • Panini Projection not working in 04_ExecuteIndirect
  • In 03_MultiThread the CPU Graph is not drawing
  • Visibility Buffer is not implemented in the moment
• Implemented a common interface for Ray Tracing and the first "user" is the DXR. There are now five simple unit tests for DXR. Please read our critical view on DXR here.
• Published a blog entry that offers details for the Triangle Visibility Buffer implementation in The Forge here.
• Hiding the code for automated testing behind a #define
• Changed the architecture for Load / Unload, now they are independent and graphics pipeline creation was moved to Load / Unload to cover render target format changes, MSAA changes etc.
• Vulkan:
  • Fixed a stencil op translation bug
  • Updated the Vulkan Memory Allocator and the Volk meta loader open-source libraries to latest
  • Updated Vulkan to support Linux Ubuntu 16.04.4 LTS
  • Fix small primitive culling by sending correct sample count for shadow map viewport
  • Add barrier between light cluster clearing and compute to fix race condition on AMD cards in Visibility Buffer
    and others
• macOS / iOS
  • fixed issue #27: fixed the window name for 08_Procedural on macOS
  • upgraded to Xcode 9.3 (9E145), macOS 10.13.4 (17E199), iOS 11.3 (15E216)
• XBOX One: smaller bug fixes, please check the log

• Moved NuklearUI into the middleware folder; still WIP
• Windows:
  • First pass of Linked Multi-GPU support for DirectX 12 and Vulkan; new unit test only supported on Windows. Requires NVIDIA Beta Driver 389.20 or later for support on Vulkan. NVIDIA only, not tested on a set of AMD GPUs so far
  • DXR:
    • Added support for local root signatures
    • Added new unit tests showing usage of local root signature and shadow rendering
    • Update FallbackLayer Release binaries
• Linux: all the unit tests (except: 09) and the Visibility Buffer are now supported on the hardware and software configuration described below
• macOS: upgraded to latest OS
• Fixed issues:
  • #39 "Font Rendering"
  • #38 "Resources.aps"
  • #29 "Problem in ImageFormat::GetBytesPerPixel()"

• We re-architected the rendering interface. If you use an older version, there are a lot of breaking changes in this update
  • replaced begin / endRender with cmdBindRendertargets
  • made the renderer compile into a DLL, so that you can have a Vulkan and a DirectX 12 DLL and switch between them during run-time. There is a new unit test showing this functionality. Removed more C++ constructs to make that happen
• DirectX 12: fixed bug in sampler comparison
• Vulkan (Windows and Linux):
  • upgraded to Vulkan SDK 1.1.73.0
  • added support for VK_EXT_descriptor_indexing and GL_EXT_nonuniform_qualifier
  • fixed minimize bug
• macOS / iOS:
  • updated to macOS 10.13.5 Beta (17F59b) / iOS 11.3.1 (15E302)
  • modified FileSystem so that macOS resembles the windows version in its behaviour
  • logging is working now on macOS / iOS: fixed issue #26 "No logging on macOS". To access iOS Logs after the application finished. Open the Devices and Simulator window in XCode and download the container for the app. The log can be found in the Contents of the container, in Appdata/Documents folder
  • Threads on macOS and iOS were not being joined on destruction, now they are
  • Building for macOS now only outputs errors and warnings, reducing the output to only the important info
• NuklearUI: added a new tree branch system to enable tabs in the future

• The Forge now supports Ubuntu 18.04 LTS and is tested with AMD and NVIDIA cards with latest drivers (see a description of our testing setup below)
• Vulkan (PC and Linux):
  • added VK_EXT_DEBUG_UTILS_EXTENSION_NAME support but excluded it for debugging with RenderDoc because RenderDoc doesn't support the extension currently
  • VK_FEATURE_TEXTURE_ARRAY_DYNAMIC_INDEXING_ENABLED and VK_EXT_DESCRIPTOR_INDEXING_ENABLED are now both added
  • On AMD GPUs, VK_EXT_DESCRIPTOR_INDEXING_ENABLED should be used now by the AMD 18.20 preview driver; so the work around in the Visibility Buffer "shade" shaders should not be necessary anymore
• All variable names were normalized following the naming convention
• Issue fixed: #44 Use VK_EXT_debug_utils

• Added a GPU tracking system that helps to switch on and off features and exclude certain GPUs from running The Forge. There is support on macOS and PC now, We will consider excluding machines based on performance with the GPU tracking system. The database is a simple text file that can be changed at any point in time
• PC Windows - based on user feedback, we changed the run-time switching functionality substantially from using two DLLs to being able to just be statically bound
• Intel-GPU support:
  • Intel updated their drivers for The Forge and the new driver fixed some of the issues we saw. The Vulkan run-time crashes and the DirectX 12 driver has problems with alpha testing. Please see the PC Windows requirement list below for driver version and tested hardware
  • The macOS version of The Forge supports now recent Intel-GPU based MacBook Pro machines. Please see the list of hardware we currently test on below. Fixed issue #28 "Visibility Buffer on MacBookPro13,1 not rendered correctly"

• Replaced the PBR unit test with a Material Playground. In the future this Material Playground will be used in the same way games develop their materials on playgrounds like this. There should be materials like eye, skin, hair, various leather / metal / cloth etc.. This is work in progress.
• Updated assimp for all platforms to 4.1
• Vulkan (Windows & Linux): added the following extensions:
  • VK_KHR_EXTERNAL_SEMAPHORE_EXTENSION_NAME
  • VK_KHR_EXTERNAL_FENCE_EXTENSION_NAME
• macOS / iOS: upgraded to latest beta for testing
• We build a GPU testing PC that can cycle through seven different GPUs; four of them also in Crossfire/SLI. At some point we will publish an article how you can create one on your own. Here is a picture: