Enzyme.jl

Enzyme v0.12.12

Diff since v0.12.11

Merged pull requests:

• Reverse mode apply iterate (#1485) (@wsmoses)
• Make zero in place (#1518) (@wsmoses)

Enzyme v0.12.11

Diff since v0.12.10

Merged pull requests:

• CompatHelper: add new compat entry for StaticArrays in [weakdeps] at version 1, (keep existing compat) (#1469) (@github-actions[bot])
• Update CompatHelper workflow (#1478) (@prbzrg)
• CompatHelper: bump compat for Documenter to 1 for package docs, (keep existing compat) (#1499) (@github-actions[bot])
• CompatHelper: add new compat entry for Literate at version 2 for package docs, (keep existing compat) (#1500) (@github-actions[bot])
• CompatHelper: add new compat entry for Preferences at version 1 for package deps, (keep existing compat) (#1514) (@github-actions[bot])
• CompatHelper: add new compat entry for Scratch at version 1 for package deps, (keep existing compat) (#1515) (@github-actions[bot])
• Avoid undefref error in validation.jl (#1523) (@vchuravy)

Closed issues:

• Dropped gradient only in reverse mode with cached values (#1492)
• LLVM.jl 7.2 seems to break Enzyme (#1496)

Enzyme v0.12.10

Diff since v0.12.9

Merged pull requests:

• Add finalizer under jlcall (#1483) (@wsmoses)
• llvm.julia.gc_preserve_begin splatting (#1486) (@wsmoses)
• Setup benchmarks (#1489) (@vchuravy)
• Test forward mode blas (#1490) (@wsmoses)
• Workaround breakage in llvm.jl (#1497) (@wsmoses)

Closed issues:

• Test faulting in specialfunctions on ARM-based Apple M2 CPU (#610)
• Autodiff fails for solving ODE with tstops keyword specified (#1459)
• autodiff fails when solving ODE if ODEFunction is used (#1460)
• Error in precompiling in julia 1.11beta2 (#1484)
• Jacobian of a function with a StaticVector argument (#1493)

Enzyme v0.12.9

Diff since v0.12.8

Merged pull requests:

• Allow custom rule for constant arg/ret in rev mode (#1371) (@wsmoses)
• rewrite calls (#1453) (@wsmoses)
• Remove closure from active_reg_inner (#1466) (@wsmoses)
• Consider noalias info from julia custom rules (#1467) (@wsmoses)
• Nicer CUDA errors (#1470) (@wsmoses)
• Apply iterate in fwd mode (#1472) (@wsmoses)
• Mark growat as nofree (#1473) (@wsmoses)
• Make nofree errors nicer (#1474) (@wsmoses)
• Fix 1.6 (#1475) (@wsmoses)
• Add broadcast noalias tests (#1477) (@wsmoses)
• Nice union{} error (#1479) (@wsmoses)

Closed issues:

• Performance issue: Likely BLAS fallback? (#435)
• Better error for GPU codes (#1102)
• JET complains about closure in active_reg_inner (#1465)
• Error with local function, keyword args and sparse arrays (#1468)
• Compilation fails with named sparse arrays (#1471)
• Regression in Broadcasting support (#1476)

Enzyme v0.12.8

Diff since v0.12.7

Merged pull requests:

• Type unstable setfield support (#1455) (@wsmoses)
• Batched mixed activity rewrite (#1457) (@wsmoses)
• Fix unused shadow return type (#1462) (@wsmoses)
• Faster type analysis (no JIT) (#1463) (@wsmoses)

Closed issues:

• [KA + Enzyme]: Instruction does not dominate all uses! (#1402)
• Illegal type analysis error for GraphsFlows.jl max flow (#1448)
• Enzyme issue with component arrays (#1451)
• Autodiff for function that builds ODEProblem (#1458)
• Differential use analysis error (#1461)

Enzyme v0.12.7

Diff since v0.12.6

Merged pull requests:

• recursion perf v3 (#1440) (@wsmoses)
• Add jl_simpliy pass (#1445) (@wsmoses)
• Fix type escaping in @import_frule, @import_rrule (#1446) (@mofeing)
• Handle mixed activity of literal 0 constant (#1449) (@wsmoses)
• Fix a broken link and some typos in dev_docs.md (#1450) (@lanceXwq)

Enzyme v0.12.6

Diff since v0.12.5

Merged pull requests:

• Add macro to import rrule from chainrules (#996) (@wsmoses)
• Remove unnecessary val in thunk types (#1432) (@wsmoses)
• Docs: describe differentiable types (#1433) (@wsmoses)
• Document API options (#1435) (@wsmoses)
• Modernize developer instructions to use Preferences.tom (#1436) (@wsmoses)
• Fix static arrays on forward mode gradient call (#1438) (@wsmoses)
• Improve recursion performance (#1439) (@wsmoses)
• Nicer method errors (#1444) (@wsmoses)

Closed issues:

• Modernize developer instructions to use Preferences.toml (#108)
• Add docstrings to Enzyme options in api.jl (#289)
• Unecessary Val in Thunk types (#350)
• Complex pow compilation error (#547)
• How to use ChainRules rrules with autodiff? (#583)
• Add caveat for differentiation through Val (#654)
• Sum of Int with floats fails (#708)
• Discussion: what does Enzyme.jl need for DL in Julia? (#805)
• Catch local version missmatch (#851)
• problem with tullio (#1279)
• IOError creating a development build (#1291)
• Why is autodiff_thunk useful in forward mode? (#1335)
• Why does gradient in forward mode return a tuple? (#1368)
• EnzymeTestUtils : fails on GPU Buffers (#1427)
• StaticArrays + Onehot mismatch (#1437)
• Overly strict enforcement of type equality for Duplicated (#1441)

Enzyme v0.12.5

Diff since v0.12.4

Merged pull requests:

• fix deferred (#1426) (@wsmoses)
• Update Format.yml (#1428) (@CarloLucibello)
• Getfield with reference (#1430) (@wsmoses)
• Optimize away unnecessary recursive forward passes (#1431) (@wsmoses)

Closed issues:

• Illegal type analysis error with setindex! + ifelse + mixed float types (#809)
• Error on forward-mode BatchDuplicated of NamedTuple of complex sin on v1.7 (#904)
• Bounds error when analyzing typeunstablerules.jl (#1317)
• Enzyme execution failed on a basic example? (#1321)
• Illegal instruction error on reverse mode autodiff (#1339)
• Unexpected result when function doesn't return nothing (#1363)
• Autodiff Deferred Thunk is broken (#1417)
• Julia Crashes from Assertion (#1429)

Enzyme v0.12.4

Diff since v0.12.3

Merged pull requests:

• Adjust tests of triangular solve rules to new EnzymeTestUtils capabilities (#1407) (@simsurace)
• CompatHelper: bump compat for LLVM to 7, (keep existing compat) (#1420) (@github-actions[bot])
• No escaping allocation (#1422) (@wsmoses)
• Fewerprints (#1423) (@wsmoses)
• Fix emiterror (#1425) (@wsmoses)

Closed issues:

• Non-Deterministic behaviour from Enzyme Reverse Mode (#1298)

Enzyme v0.12.3

Diff since v0.12.2

Merged pull requests:

• add import_frule (reprised) (#1333) (@CarloLucibello)
• fix add_one_in_place (#1418) (@wsmoses)
• Handle constant gep (#1419) (@wsmoses)
• Correct dead arg elimination (#1421) (@wsmoses)

Closed issues:

• DJ4Earth runtime activity required (#858)
• TypeAnalysisDepthLimit warning with Vector{Vector{Vector{Float64}}} (#866)
• Primal GC preserve end not postdominating begin -> Reverse-mode fails verification (#877)
• triu! Illegal Type Analysis (#1159)
• Error with Flux GRU layer (#1261)
• Long Enzyme compile times (#1296)
• Unexpected zero derivative (#1304)
• Julia segfaults when using maps over a segmented array (#1312)

Enzyme v0.12.2

Diff since v0.12.1

Merged pull requests:

• More debug info on error (#1396) (@wsmoses)
• WIP opt out of types (#1413) (@wsmoses)
• Fix test (#1414) (@wsmoses)
• Special case make_zer(array) for perf (#1415) (@wsmoses)

Closed issues:

• JIT error (#1021)
• Error when differentiating LinearAlgebra.copy_similar (#1320)
• make_zero is type-unstable (#1359)
• Illegal type analysis (#1400)
• Reverse mode is type-unstable in v0.12 (#1401)
• Segfault with Assertion 'isa<LoadInst>(orig) == isa<LoadInst>(val)' failed (#1408)
• Reverse mode gradient is up to 100x slower in Enzyme v0.12 (#1409)

Enzyme v0.12.1

Diff since v0.12.0

Merged pull requests:

• [EnzymeTestUtils] Vectorize function for FiniteDifferencesCalls (#1327) (@sethaxen)
• Enzyme rules for partialsort! (#1373) (@jgreener64)
• Identify and attempt fix for gc bug (#1386) (@wsmoses)
• Try narrowing inactive rules for rand (#1388) (@gaurav-arya)
• Remove vararg tuple warning (#1395) (@wsmoses)
• [EnzymeTestUtils] Add rng keyword to testing functions and test with seed (#1398) (@sethaxen)
• stabilize default_adjoint (#1403) (@wsmoses)
• Null addr cast GC handler (#1404) (@wsmoses)
• Fix autodiff_deferred_thunk abi (#1412) (@wsmoses)

Closed issues:

• partition, type analysis mismatch (#880)
• julia_type_rule callback performance bottleneck (#1299)
• Reverse-mode on complex function fails (#1319)
• Intimidating docs (#1344)
• Enzyme segfault (#1353)
• Type analysis depth limit / task switch not allowed from inside staged nor pure functions (#1354)
• Add more details to docs (#1355)
• Basic SparseMatrixCSC calculations do not work. (#1381)
• Overly broad inactive marker for Random.rand (#1387)
• Gradient returns all zeros (#1389)
• StackOverflowError depending on loaded packages (#1390)
• Julia crash on v1.7.3 with batch mutation of triangle of Hermitian array (#1394)
• EnzymeTestUtils nondeterminstic failure (#1397)
• enumerate not working in 0.12 (#1405)

Enzyme v0.12.0

Diff since v0.11.20

Merged pull requests:

• TA pass julia type info via param attr (#1300) (@wsmoses)
• Memoize typetree calls (#1302) (@vchuravy)
• add complex sqrt (#1324) (@wsmoses)
• Remove invariant loads on primal computation (#1325) (@wsmoses)
• Add Julia formatter (#1326) (@simsurace)
• Define compiler_job_from_backend (#1328) (@vchuravy)
• Add FAQ section to docs (#1343) (@gdalle)
• Set parallel to false if parent_job is not nothing (#1348) (@giordano)
• Bump Enzyme_jll to 0.0.103 (#1351) (@vchuravy)
• Skip intermittant builds (#1352) (@vchuravy)
• Use make_zero and Ref inside gradient (#1356) (@mcabbott)
• Add additional docs (#1357) (@wsmoses)
• Set type warning false by default (#1360) (@wsmoses)
• Fix assertion on nocapture llvm.enzymejl.sret (#1361) (@wsmoses)
• Fix CUDA tests (#1364) (@vchuravy)
• Support GPUCompiler 0.26 (#1365) (@vchuravy)
• Load Adapt unconditionally for Julia 1.8 (#1369) (@vchuravy)
• Backport EnzymeCore Adapt fix to 0.6 (#1374) (@vchuravy)
• Set development branch to 0.12 (#1375) (@vchuravy)
• Fix undefined memory in faq (#1376) (@wsmoses)
• Add differential use handler (#1377) (@wsmoses)
• Mark randn! as inactive (#1378) (@wsmoses)
• [EnzymeTestUtils] Mark 1.8 batch test as failing (#1379) (@wsmoses)
• Restore 1.9+ setfield (#1383) (@wsmoses)
• Update Project.toml (#1384) (@wsmoses)

Closed issues:

• Why does Duplicated(x, dx) assume x and dx have the same type? (#1329)
• Forward over reverse for variadic function (#1336)
• gradient (and gradient!) fails on basic example (#1340)
• Warning with Flux.Bilinear (#1341)
• Matrix exponential fails (#1346)
• EnzymeTestUtils tests failing on main (#1349)
• Using Julia's LLVM and Enzyme.jl's artifacts to differentiate C code (#1367)
• Enzyme requests reverse rule on Const returns (#1380)
• Gradient returns wrong result for static arrays (#1382)

Enzyme v0.11.20

Diff since v0.11.19

Merged pull requests:

• fix checknan (#1338) (@wsmoses)
• Disable ojit on windows (#1342) (@wsmoses)

Closed issues:

• StackOverflow in forward mode (#1330)

Enzyme v0.11.19

Diff since v0.11.18

Merged pull requests:

• Loosen new struct requirements for jl_new_struct (#1323) (@wsmoses)

Enzyme v0.11.18

Diff since v0.11.17

Merged pull requests:

• Add tape_type query given a parent compiler job context (#1104) (@wsmoses)
• Add rule for triangular solves (#1264) (@simsurace)
• Describe activity stable (#1265) (@wsmoses)
• Fix triangular solve rule for Adjoints (#1306) (@simsurace)
• Err and correctly handle complex returns in reverse mode (#1309) (@wsmoses)
• Add Attributor flag in optimize.jl (#1311) (@dchang10)
• Mixed activity docs (#1314) (@wsmoses)
• Fix docs (#1318) (@wsmoses)
• Update compiler.jl (#1322) (@wsmoses)

Closed issues:

• Incorrect sign for imaginary part of derivative of functions returning complex numbers (#1308)

Enzyme v0.11.17

Diff since v0.11.16

Merged pull requests:

• Explicitly precompile register_llvm_rules (#1303) (@vchuravy)

Enzyme v0.11.16

Diff since v0.11.15

Merged pull requests:

• fix rendering (#1276) (@ZuseZ4)
• Mark julia.call as nofree (#1297) (@wsmoses)

Closed issues:

• Enzyme inexact error (#1294)

Enzyme v0.11.15

Diff since v0.11.14

Merged pull requests:

• Enable abs interp on fn args (#1212) (@wsmoses)
• Throw descriptive AsssertionError instead of @show + @assert inside of create_abi_wrapper (#1262) (@devmotion)
• Try to fix the incorrect math rendering (#1270) (@ZuseZ4)
• Logabsgamma support (#1271) (@wsmoses)
• Widen cholesky rule to Hermitian and Symmetric matrices (#1273) (@devmotion)
• Absint boxint (#1278) (@wsmoses)
• Improve / remove errors (#1280) (@wsmoses)
• Update Project.toml (#1281) (@wsmoses)
• Mark blas as on by default (part 2) (#1282) (@wsmoses)
• Add phi erasure test (#1283) (@wsmoses)
• Use reinterpret fn attr index (#1285) (@wsmoses)
• EnzymeTestUtils: check that args are mutated correctly (#1286) (@sethaxen)
• Improve new struct error message (#1288) (@wsmoses)
• Support complex knownops (#1292) (@wsmoses)
• Turn no shadow error into runtime (#1293) (@wsmoses)

Closed issues:

• Implement equivalent of gradcheck from PyTorch (#642)
• Allow for setting preferences to turn off warnings (#643)
• Float16 argument errors (#870)
• Hanging and task switch errors when differentiating DynamicExpressions.jl (#1018)
• Enzyme fails to autodiff if this particular function isn't inlined , MWE attached (#1233)
• ceil or % error in cuda backpropagation (#1259)
• Enzyme execution failure (#1260)
• Error of custom type interaction with StaticArrays (type unstable update of an immutable variable) (#1263)
• @threads breaks Enzyme? (#1266)
• Incorrect derivative for sparse matrix sum (#1267)
• Generator messes with activity analysis (#1268)
• Wrong gradient when using bitcode_replacement!(false) in neural ODE (#1269)
• Differentiating cholesky(::Symmetric) throws an error (#1272)
• Out of memory error (#1284)
• Issue with Enzymanigans (#1287)

Enzyme v0.11.14

Diff since v0.11.13

Merged pull requests:

• [WIP] First stab at Cholesky (#1220) (@michel2323)
• Handle batched inactive (#1247) (@wsmoses)
• Mark computesparms as inactivenoinl (#1248) (@wsmoses)
• [BLAS] Enable tablegen blas by default for reverse mode gemm/gemv/axp… (#1249) (@wsmoses)
• Cleanup and simplify cholesky handler (#1251) (@wsmoses)
• Mark gensym as inactive (#1252) (@wsmoses)
• Fix typo "appliable" (#1253) (@devmotion)
• Fix cholesky part 2 (#1256) (@wsmoses)
• Restrict ldiv! rules to Cholesky (#1257) (@devmotion)
• Custom rules overwritten fix (#1258) (@wsmoses)

Closed issues:

• Gradient correctness issues in expression trees (#1241)
• Trouble writing custom rule in Enzyme: AssertionError: !(overwritten[end]) (#1242)
• Error with Duplicated{<:Cholesky} (#1250)
• Wrong gradient for cholesky (#1254)
• Found unhandled active variable in tuple splat (#1255)