sile

Summary of Improvements

This release addresses a few pain points discovered using the v0.14 series in production.

For 3rd party package developers, it's now easier to find non-Lua resources you installed along with your package. Among several contributions from @Omikhleia, the output backend now returns useful information about images it embeds directly. The same work also allows specific pages of PDFs to to be emdeded as images, not just the first page. He also helped refactor existing interfaces to make them easier to extend, such as the table of contents package. A long standing issue with Turkish hyphenation has been addressed, eliminating the need for preprocessing or other workarounds. More long standing issues with the book class and chapter opening spreads have been addressed. Some priority and interactions between new CLI flags has been worked out. A few more misc bugs were also squished and relevant tests added.

Features

• languages: Handle hyphenation of inter-word apostrophes in Turkish (50ae936)
• packages: Add \open-spread function with more features that \open-double-page (c2ba579)
• packages: Add ability to select a page in PDF images (a477d94)
• packages: Allow for customized content on otherwise blank filler pages (5ae97bf)
• packages: Provide base directory to packages (#1529) (f9ae994)
• utilities: Return image resolution with libtexpdf backend (a9c11d3)

Bug Fixes

• cli: Actually apply cli provided class options (505919e)
• cli: Allow CLI option to override document specified class (5232ce8)
• languages: Make Turkish hyphenation less bad around intraword apostrophes (008d4c4)
• nodes: Correct calculating width of postbreak discretionaries (ea7912c)
• nodes: Work around discressionaries being output when not wanted (c7dc439)
• packages: Fix over-aggressive eject in \open-double-page (5620556)
• packages: Homogenize image width and height as measurements (b91cfbb), closes #1506
• packages: Make sure PDF initialized before PDF package does anything (#1550) (ebc3748)
• packages: Resolve src= relative to document for SVG images (b55fc98), closes #1532

Reverts

• Revert "chore(build): Avoid mktemp during build, breaks opensuse packaging (#1542)" (bca007f), closes #1542
• Revert "chore(cli): Output header before doing anything that might throw warnings" (58da8ad)

Summary of Improvements

It’s a slow news day, but these few little fixes touchup glitches introduced in the v0.14.0 release and generally improve the robustness of the new API interfaces. Writing (and porting) classes should go just a bit smoother and behave more as expected now.

Bug Fixes

• classes: Allow package option declarations to be reset (215e83a)
• classes: Bring back space after subsection numbering (70a3304)
• packages: Check for user supplied commands before setting noops (54b5071)

Summary of Improvements

These few quick touch-ups smooth over last minute bugs and ease the transition process from v0.13.x. Besides squashing a pointless warning messages and some bug fixes, this changes the order deprecated includes are processed to facilitate the transition of old projects to new APIs by making it easier to update them one bit at a time.

Features

• inputters: Expand postamble functionality for parity with preambles (#1518) (eb09eb3)

Bug Fixes

• cli: Suppress deprecation message for internal shims (b339e27)
• cli: Swap order of new --uses and legacy --include (ef0087e)
• packages: Correct (and improve scope of) exported testcase functions (fd438e9)
• tooling: Make sure Git version detection only picks up semver tags (25d669a)

Summary of Improvements

Work to refactor SILE's internals into a more regular API has been in progress for some time. This release brings a big chunk of that work to light. Many/most of the changes are under the hood and won't show up in the appearance of documents. They do affect anybody writing classes or packages for SILE, hopefully for the better.

Most notably, 3rd party modules have much more robust support. A module may be a class, a package, a typesetter, a shapper, an inputter, an outputter, etc. This makes it easy for developers to extend SILE without needing to modify the core distribution. Modules can be packaged and installed using LuaRocks. LuaRocks.org may be used to host and distribute SILE addons. Rocks may also be distrusted independently. Either way they are easy to install with luarocks install and such installs will make the modules available for loading into SILE. The old built in package manager is still functional for now but deprecated.

The ergonomics of packages has changed a little. The overloaded \script command has gained some more specialized peers. For example Lua code may be passed in \lua commands. To load a package or other module, use \use[module=package.name]. This syntax also allows passing options to modules.

The inputters have been normalized a bit, and XML, SIL, and Lua are all first class input languages. Passing some content in XML format from a SIL format document is as easy as placing it in an \xml command. The same goes for SIL format content in an XML document.

3rd party developers may add new input formats as well! All that is required is an inputter module that returns an AST. Expect to see new inputters soon. @Omikhleia has already been working on a new Markdown inputter module. Other XML based formats such Docbook have been supported for a while, but the new class and inputter APIs make them easier to add and use.

The CLI has gained several new options. As usual sile --help and man sile both have relevant information. Notably the -I/--include argument has been deprecated in favor of more explicit -u/--use, -p/--preamble, and -P/--postamble. As you might expect from those options this makes it possible to load an inputter, class, or package from the command line without having it set in the document. The preamble/postamble system has been cleaned up so content or code can easily be injected at the start or end of a document. This is a change from the previous 'preamble' system that actually became the master document and then included the original master as effectively a postamble.

Many old shims and deprecated functions have been completely removed. Most will still throw relevant warning messages with hints about how to update.

While I was busy moving cheese, several contributers were chipping in new features and fixes. @rolfmblindgren pitched in Norwegian localization and improved hyphenation. @matteodelabre contributed quite a few new features and fixes to the math package. @Omikhleia did some data mining and came up with localization strings for a whole slew of languages. He also pitched in fixes to a number of packages. He also has pending contribution for many new packages and features. Be sure to checkout the PR scene on GitHub if you are looking for new features.

⚠ BREAKING CHANGES

• packages: The primary use was probably internal to SILE, but if by chance you have bibtex databases with formatting commands in SIL markup format rather than just plain text content the markup will cease to function and will need to be converted to XML syntax instead. This enables the use of declarative markup in Fluent localizations.

• classes: Each SILE package now inherits some interfaces from a common base package. This model allows packages easier access to SILE internals while at the same time tracking what they do so it is easier to enable/disable them. The package knows which document class instance it is attached to, and the document class knows which packages are loaded at any given time. Legacy style packages will continue to work for the time being but will not be tracked in the same way.

• core: The role of document commands has always been tightly scoped to classes. For example the book class has a \footnote command while plain does not—unless you manually load the package and setup the frames. In spite of this obvious functional scope, registering commands has been a global operation that stored them in a global registry. In order to allow SILE to be used more programmatically as a library with potentially more than one document and class being processed at at once, these need to be moved out of the global scope. This will also facilitate things like being able unload packages and revert to previous functionality for anything they over-rood on load. For now the functionality is shimmed, but code using the SILE.registerCommand() function should switch to the method of the same name on the current class, i.e. class:registerCommand().

• core: Some internal files and APIs got renamed with more structured name spaces. In particular the inputter, shaper, and outputter libraries all have a common naming scheme now and sensible inheritance chains. No functionality was harmed, but if you are overriding undocumented internal Lua methods you might have to update your name spaces to match.

• cli: The -I / --include option was overloaded for more than one purpose and is now deprecated in favor of more specific replacements: -r / --require for loading code into SILE before input processing, -p / --preamble for processing content prior to a document and -P / --postamble for processing content after a document.

• packages: The original package manager POC that used Git to clone packages into the SILE installation directory has been deprecated. It will continue to function for a while, but all new 3rd party packages should use the LuaRocks based installation process. Whether or not they use luarocks as a package manager or LuaRocks.org as a distribution channel they should install themselves to any usable the system or user Lua library path under a top level "sile" namespace.

• classes: The shims allowing classes designed for SILE releases v0.12.x and prior have now been removed and documents using them will now throw errors when rendering. Only the refactored class system introduced in SILE v0.13.0 is supported going forward.

• deps: We previously deprecated all use of stdlib. This release stops providing it entirely. If you use it in your own projects you will now need to provide and require() it directly.

• build: The C modules compiled as shared libraries (.so files on Linux, .dll on Windows) are now installed to the project root shared directory instead of it's 'core' subdirectory. Distro packages that split the library into its own package will need to adjust this path. People installing from their distro packages or from source should be unaffected, but this will bring us one step closer to being able to install and use SILE as a library.

Features

• classes: Add \use command to help deconflate \script usage (eb298c3)
• classes: Track loaded packages per document class (32bd87b)
• cli: Add CLI argument -E for evaluating Lua code after input (5948aca)
• cli: Add usage hints and cleanup output of errors (cc58824)
• cli: Allow loading custom inputters from -r option (a212e83)
• cli: Allow passing options to any modules specified from --use (4cdcae7)
• cli: Change --require to --use to match declarative markup (2411328)
• core: Add ability to pass args to modules via \use and other commands that load modules (9e54bad)
• core: Add ability to pass args to modules via \use and other commands that load modules (e64ce0f)
• core: Add inline-escaping in SIL-language (f09b135)
• core: Support loading classes/packages installed with luarocks (232e72b)
• i18n: Add more Russion localizations (350cf14)
• i18n: Add support for as many languages as possible (da57577)
• i18n: Fallback to messages from 'und' language if no localized (9f47715)
• i18n: Parse XML style SILE commands in Fluent messages (989290b)
• inputters: Allow arbitrary root elements from XML input without a preamble (ad46a92)
• inputters: Allow CLI to mandate inputter used for master document (1b9009f)
• inputters: Promote Lua to first class input filetype, improve input type detection (3540943)
• languages: Add Norwegian localizations (76b8f84)
• languages: Add Norwegian Nynorsk hyphenation exceptions (520cd3f)
• languages: Handle 'nb' code for Norwegian Bokmål, linked to 'no' rules (373bd17)
• math: Add modulus operator support (429b162)
• math: Allow forcing the atom type of an operator (14d384c)
• math: Express lengths in “mu” (math units) (39c7efc)
• math: Macros no longer wrap their replacement into s (d1f24b3)
• math: Print resulting mbox tree to debug log (f2e7c33)
• math: Support relative units in spaces and add standard spaces (4f2bee2)
• packages: Add new method to export package functions to class (07a28a4)
• packages: Allow configuring target folio frame from options (74e3924)
• packages: Provide API for registering commands linked to packages (4875972)

Bug Fixes

• build: Update Flake to work with Nix >= 2.10 (effb0dc)
• classes: Reset default font direction if document direction changed (11bb0f9)
• cli: Avoid throwing extra error on error without message (0d530a5)
• core: Avoid error when outputting overflow warnings with specific measurements (49ef650), closes #945
• debug: Flatten content if necessary to process and debug location (c753bd2)
• debug: Re-implement option display in trace stacks lost with std (01d2379)
• inputs: Drop Lua path handling duplicated in core (8abb0f2)
• math: Fix caching of getMathMetrics (3332698)
• math: Fix debug logs in TeX-like parsing (a686f90)
• math: Fix spacing before integral operators (bc847b3)
• math: Fix tostring functions in mbox subclasses (7a7c6bc)
• math: Set math elements to inherit hbox node properties (0279556)
• math: Turn font name printing into debug log (068ec4b)
• outputter: Non-RGB colors shall work with the debug outputter (#1469) (e68dee3)
• packages: Correct URL formatting when backend is not libpdftex (fc4212d)
• packages: Ensure grid hook is ineffective when grid is off (b99482b), closes /github.com/sile-typesetter/sile/issues/1174#issuecomment-1173141699
• packages: Make \script command properly initialize packages (#1479) (9723d0d)
• packages: Parse height argument to \raise / \lower as measurement (#1506) (7196fda)
• packages: Stop legacy package manager from adding empty paths (cf9b9fa)
• packages: Stricter color parsing and improved color documentation (f7b919a)
• packages: The autodoc package could choke on some inputs (#1491) (c7db5d5)
• utilities: Correct traceback output for SILE.error() to show parent, not itself (16b8900)

Miscellaneous Chores

• build: Move C modules to same relative location in source directory as installed (55ad795)
• classes: Remove stdlib class shims (c4210da)
• cli: Deprecate CLI argument -I in favor of -r, -p, and -P (d63a484)
• deps: Stop providing Lua stdlib (8a8c0e9)
• packages: Deprecate legacy package manager (b72653c)

Code Refactoring

• core: Move inputters/shapers/outputters to isolated classes (14329ce)
• core: Move registerCommand() out of global to classes (bc527ea)
• packages: Process bibtex content as XML not SIL (a259b32)

Summary of Improvements

The upcoming v0.14.0 is chock full of changes (hopefully all for the better!) but several small issues have cropped up in the v0.13.x series and there doesn't seem to be any point in inconveniencing existing users with bugs we've already fixed while we work on new features. This should fix up some issues with includes not activating exported functions and getting blocked when both directories and files of the same name exist. The Nix Flake will now work with the new Nix 2.10.x releases.

Additionally @Omikhleia contributed some fixups and features to the Bibtex package to make it much more usable.

Features

• packages: Add minimal support for usual BibTeX types (bibtex) (292a2f2)

Bug Fixes

• build: Update Flake to work with Nix >= 2.10 (3d5a18c)
• core: Avoid duplicate paths blocking directory searches (7a7209f)
• core: Avoid error when outputting overflow warnings with specific measurements (cb51ed5), closes #945
• outputter: Non-RGB colors shall work with the debug outputter (#1469) (41fbdf4)
• packages: BibTeX types/tags are case-insensitive, etc (61c1fc6)
• packages: Make \script command properly initialize packages (9ded7e1)

Summary of Improvements

This patch release brings both bug fixes and new goodies.

@Omikhleia contributed a new package for formatting lists. Features include bullet, numbered, and mixed list types with many styling options.

@ctrlcctrlv fixed zenkaku unit handling and helped refactor the jplain/jbook classes so we now have tate-capable but language-agnostic tplain/tbook classes in addition to the Japanese specific variants. He also contributed full support for Esperanto including hyphenation, number formatting, and localized output strings. Even if you don't have a use case for Esperanto directly the issues that rooted out with language and class handling should benefit everyone.

The papersize option provided by the base class is now case insensitive and allows dashes. If you prefer A4 to a4 or ANSI-A to ansia feel free to use them. A couple new paper size presets were added in the process.

Additionally the font fallback system got an overhaul. A change from previous behavior is that TOFU handed out by the last available fallback font will be rendered instead of skipping the characters completely. Switching back and forth between the color fonts package and the font fallback package is now possible. Finally it should stop throwing bogus warnings when it successfully shapes a segment via the last available fallback.

Other minor fixes include interactions with color plus dropcaps, some bibtex errors, and errors in the pecha class.

Features

• core: Add presets for some ANSI paper sizes and ArchE variants (0f26756)
• languages: Add full Esperanto language support (b740709)
• packages: Add 'lists' package (bullets and enumerations) (6af3c62)
• packages: Add more options for custom 'lists"' styling (3167410)
• packages: Handle font fallback when glyph named null returned on shape falure (09c0a86)
• packages: Pass through font-specific options to fallback fonts (fb29442)

Bug Fixes

• classes: Clarify the scopes of tate and jplain (db83e9e)
• classes: Fix circular reference in pecha class (4501ec0)
• classes: Fix diglot and triglot class instantiation (71af1a9)
• core: Make paper size parser case insensitive, e.g. 'a4' or 'A4' (af441c8)
• measurements: Move the zenkaku width (zw) unit into core (cfe5060)
• packages: Correct fall-back font processing (d3cc59b)
• packages: Correct package load path for colored dropcaps (41a0c17)
• packages: Fix coding errors in untested corners of bibtex package (804b1a5)
• packages: Fix loading TOC twice resetting pdf links (97797b8)
• shaper: Handle switching between color & fallback shapers in single document (04f2d5d)
• utilities: Raise Lua error instead of manually aborting if inside pcall() (6e70a17)

Summary of Improvements

Hot on the heals of a major release we have small one to patch up some odds and ends. Thanks to @ctrlcctrlv for correcting my oversight in not distributing some of the new localization files. If by chance you've on a new fangled hardware platform like RISC-V things should build out of the box now. The documentation for installation on various systems has also been cleaned up. Thanks to @Omikhleia for some new rule functions and fixes to old ones plus leader alignment corrections.

Features

• build: Update libtexpdf to support new hardware platforms (da1182e)
• packages: Add hrulefill command to the "rules" package (ccd3371)
• packages: Add strikethrough command to the rules package (#1422) (f230a3a)
• packages: Use new strikethrough when rendering Panndoc's SILE writer (20d19eb)

Bug Fixes

• build: Make sure i18n/ dir is actually distributed (#1445) (61ed8e1)
• packages: Add more props to keep CJK from tipping over, per #1245 (381b9f1)
• packages: Leaders shall be an explicit (non-discardable) glue (631ba21)
• packages: The fullrule now extends over a full standalone line (8fe57c8)

Summary of Improvements

In the last patch release I hinted we had bigger things coming. This release finally surfaces major internal changes I've been working on since 2019. If our releases had code names this one might be "class wars". The SILE code base has two distinct usages for 'class'. One usage is our document classes, cohesive bundles of features and styles used to render a document. The second usage is the object-oriented programming paradigm of class inheritance. In serendipitous naming ripe for conflation, the document class feature is itself written using programming classes, enabling document classes to inherit from each-other.

This release refactors the implementation of both types of classes. Since nearly its inception, SILE has depended on the stdlib project for many of its Lua abstractions. Unfortunately this library is not as standard as its name suggests, has not been developed or even well maintained, and has been a constant source of problems. It even tinkers with core Lua functionality making it very difficult to keep SILE working across interpreter versions and to debug problems. For some time we've been gradually replacing it with abstractions from the Penlight library (loosely based on Python's standard library). While not perfect, that library of functions is much more robust and predictable. This release finally removes the last of SILE's usage of stdlib. For this release cycle the core program will still provide the stdlib library to ease transition for downstream projects that currently assume it is present, but it is no longer used internally. Beginning with the next major release cycle, v0.14.0, it will not be provided at all. (You will of course be free to include it as a direct Lua dependency in your own projects.)

We've attempted to shim most of the API changes so that most old SILE documents (and in particular, custom classes) continue to work out of the box. Most simple projects should still render without intervention, but you may see many warnings about deprecated functions. A few more advanced projects may run into trouble and fail to render at all, especially if they messed around with classes (of either sort) very much. As far as possible we've tried to add warning notices with hints about how to use the new class models correctly. Please don't hesitate to open issues if you are having trouble getting anything to work. For package authors, a new hook system should make it a lot easier to write packages that do more without also having to write a dedicated class to use them.

Additionally the localization system for all language specific strings that may be rendered by SILE has been changed from the home-grown system of nested SILE commands to a more flexible i10n system based on Fluent. The Lua implementation is not at 100% feature parity with Project Fluent reference implementation, but it is 100% interopperable with other implementations for features implemented and is a much more robust localization system than just substituting strings. Besides using it to easily customize the limited set of embedded localizations (such as the Table of Contents header or chapter titles), the tooling for localizing messages and rendering them in context aware functions is exposed for package developers and document authors to use at will.

A small collection of bug fixes to the typesetter rounds out this release. Justification of lines with ligatured characters is much improved. Rules with a depth property no longer throw off baseline calculations. Empty documents now generate blank PDFs. Adding a new master frameset layout no longer destroys the current page's frameset.

⚠ BREAKING CHANGES

• settings: All the functions under SILE.settings.*() should now be called using the instance notation SILE.settings:*(). Usage should be shimmed with a warning for now.

  Changing this in your code is relatively easy with a search and replace. As an example with a project in Git, you could use perl like this:

   funcs="pushState|popState|declare|reset|toplevelState|get|set|temporarily|wrap"
   git ls-files | xargs -n1 perl -i -pne "s#(SILE\.settings)\.($funcs)#\1:\2#g"
  

• typesetter: Making a new instance of the typesetter should now be done by calling SILE.defaultTypesetter() instead of copying the object. It has been changed from a std.object to a Penlight class. As such the correct initialization function is also now _init() instead of init(). A shim is in place to catch legacy usage, but this will be removed in the future.

• deps: All calls to the Lua default string library have been using a version monkey-patched by stdlib. This has created all sorts of issues including not being able to properly use some of Lua's default features and conflicts with out explicit meta methods. Also we're busy dropping dependency stdlib altogether.

  If you were relying on it for any of your string operations, replace string.func() with std.string.func(). For now std is being provided by SILE, but if you use it in your projects please add it as a direct dependency yourself since that will eventually be removed as well.

  By the way in case anything ever git bisects back to here, one way to test if your problem is related to this change or not (especially if you have downstream code that might have built on the assumption SILE's Lua strings were monkey patched) is to load it manually yourself:

   $ sile -e 'require("std.string").monkey_patch()' your_file.sil
  

• classes: This changes the way classes are represented as Lua objects and the mechanism used for inheritance. While shims will be in place to catch most cases that use old syntax it is not possible to grantee 100% API compatibility. If you have classes that do anything remotely fancy (i.e. not just copy/paste from SILE examples) they may or may not work at all; and even if they do they should be updated to
  explicitly use the new API.

Features

• classes: Add hook system for more versatile packages (9287721)
• languages: Add \ftl command to make adding fluent localizations easy (b331456)
• languages: Add fluent() command to output translations (ad87995)
• languages: Validate languages against CLDR database (f96a331)

Bug Fixes

• backends: Add Pango shaper when selecting Cairo backend (bbc2817)
• backends: Always output pdf on finish() even if no content (3af7a94)
• backends: Correct image sizing in Cairo and Podofo backends (f2785ad)
• core: Avoid throwing deprecation errors when just inspecting SILE's internals (b303059)
• core: Justify lines with ligatures (workaround) (cf2cb3a)
• core: Patch Penlight 1.9.0 compatibility issue (092fbd3)
• languages: Correct bogus usage of resource loading / error catching (fb1fd7f)
• packages: An hrule with depth shall not affect current baseline (c759892)
• packages: Don't destroy frames when defining masters, only when switching to one (b7de7ca)
• packages: Fix autodoc parsing, typeset string not series of bytes (14f6126)

Miscellaneous Chores

• deps: Drop std.string.monkey_patch() (e8b2bdf)

Code Refactoring

• classes: Convert class inheritance from stdlib to Penlight (f7dafe0)
• settings: Change settings object to be self referential (dd97d05)
• typesetter: Change typesetter instancing to Penlight model (a9400ad)

Summary of Improvements

Today we have a fairly small release motivated primarily by wanting to patch a few small issues before some bigger things in the pipeline land. We cleaned up a few small scripting issues with the configure and install process that should make it easier to install from sources on BSD platforms or Linux distros such as Alpine that do not default to bash and GNU tooling.

Besides the minor tooling fixes, @Omikhleia has continued to pour in contributions.
This release features his feature overhaul to the leaders package so that dot fills align vertically. It also includes his fixes to whitespace handling around punctuation in French and better extraction of strings such as chapter titles to PDF bookmarks.

Features

• packages: Align leaders vertically (#875) (8b5d418)

Bug Fixes

• build: Catch and complete unfinished library builds (91ff438)
• build: Use BSD compatible scripting in make selfcheck (319e0c5)
• build: Use POSIX compatible shell syntax in configure (55e64ab)
• deps: Correct include to work with current LuaRocks packages (#1357) (b584be5)
• languages: French punctuation spacing must honor current font options (724daf4)
• packages: Better TOC title extraction for PDF bookmark (#1029) (5a65701)
• packages: The dotfill must stretch as an hfill (#1343) (c94a4b5)

Summary of Improvements

This patch release fixes a bug introduced while fixing other bugs. While enabling BiDi by default for all typesetters was working, enabling it manually on typesetters would result in an infinite loop. Given that many people worked around it not being enabled by default by turning it on manually, this meant a lot of documents started taking lots (∞) of time to render. Apologies for the wasted cycles, whether CPU or brain wave.

Bug Fixes

• docker: Fix GHCR → Docker Hub copy used when releasing (e5d83d0)
• packages: Avoid infinite loop when re-enabling BiDi (b4d691b)

Summary of Improvements

This patch release changes the way new frames are generated to inherit settings differently. These changes should qualify as "fixes", but could affect workarounds for anybody that had trouble before. Loading the BiDi package (which happens by default) now changes the default typesetter in addition to the current one. This affects some features such as the commands to typeset into a different frame than the current one when RTL languages are involved. Additionally, the way frames are created has changed so that direction settings such RTL set on the class are inherited by default to default and new frames unless otherwise specified. This affects folio, footnote, and other default frames as well as any created programmatically.

Both changes are likely the behavior most users would expect anyway –hence the minor patch release– but could cause unexpected results in documents that had other ways of dealing with the previous unexpected behavior.

Bug Fixes

• frames: Inherit class direction setting in new frames (35c8a25)
• packages: Make boxaround respect shrink/stretch (rules package) (9d8f708)
• packages: Make underline respect shrink/stretch (rules package) (a5d99f0)
• typesetter: Enable bidi for default typesetter on package load (6a8d7f4)

Summary of Improvements

This patch release is to fix a regression in handling negative width spaces — something that turn up when typesetting Nastaliq. Thanks to first-time contributor @mimbrown for both the report and fix. Also the documentation for the dropcap package added in the last release is now in the manual. The only other thing possibly worth noting is that the prebuilt Docker images now run on the recently released Lua 5.4.4.

Bug Fixes

• shaper: Fix line length calcs with negative width word spacing (685d12d), closes #579

Summary of Improvements

The first minor release in the 12.x series brings several new features relevant to paragraph shaping. @Omikhleia has fleshed out the linebreak mechanism with full parity of LaTeX's settings including hangingIndent and parShape. Having fully featured paragraph-wise hanging indent settings enabled a new package, dropcaps, to provide much more robust dropcap functionality than the previous floating frame mechanism we used for this. (Note the dropcap package documentation did not make it in the manual, see the example on the website for usage.) Additionally the paragraph shape callback function can now be used to create fancy paragraph shapes using any geometry you can specify or calculate. Bugfixes in this release cover several issues with French language, PDF metadata encoding, color fonts, book class headers, and more.

Features

• build: Accommodate SOURCE_DATE_EPOCH for reproducible builds (16c81a8)
• classes: Add \noop function for versatile SILE.call() use (2b04507)
• core: Add OpenType post (v1) table parser (a985aed)
• core: Implement Knuth's hangAfter and hangIndent (5417189)
• core: Implement paragraph duration hanging indent settings (18ee23b)
• core: Implement paragraph shaping (parshape) (c2c0235)
• packages: Add dropcaps package (cb9105a)
• packages: Add shift, raise, and size options to dropcaps (0a88948)
• packages: Implement color option for dropcaps (d042bcf)
• packages: Use font's post table to determine underline position (ae1b929)

Bug Fixes

• backends: Move Lua 5.1 macro so covers whole file (9b40772)
• classes: Reset state when calling running headers (ec0a7b8)
• classes: Unnumbered book sections shall not display a number in running headers (4afde42)
• Fixes name of accented math symbol, eliminates duplicate newStandardHspace assignment (4c38f1a), closes #1274
• languages: Correct hyphenation after apostrophe in French and Catalan (4c93891)
• languages: Correct synchronisation between indexes in French word breaking (94ca931)
• languages: Repair broken French hyphenation patterns (c25d9d7)
• packages: Add \pdf:metadata support for dates (1b87305)
• packages: Apply OpenType x and y offsets to color fonts (d66dc5f), closes #1147
• packages: Correct rebox to not output duplicate content (2802d9b)
• packages: Don't over-isolate functions run in Pandoc imports (#1254) (84507a5)
• utilities: Fix UTF-16 encode/decode utility functions (7180081), closes #1280
• utilities: Set language of roman numerals to Latin to avoid casing issues (#1253) (95c4e2c)

Summary of Improvements

While this release follows close on the heels of previous releases it represents over 2 years of work from community contributors. @harrysummer started work on a native math feature in April of 2019. While this initial phase was never completed, it did get as far as a working prototype. In November of 2020, @OlivierNicole picked up on the work and transformed the prototype into the shape as found in this release. Additional thanks to @akavel for input throughout the process.

On a conceptual level the math support is based on MathML and relies on OpenType math fonts. The input notation may be MathML (either directly using the native XML syntax or using an 1-to-1 equivalent SILE syntax) or an alternative math shorthand similar to the math notation used by TeX. See section 5.29.5 in the v0.12.0 manual for detailed documentation on the new math package. Examples are given for various formulas and syntax options. A math example has also been added to the packages example gallery on this website.

Additionally @Omikhleia contributed several new features and fixes including much more control over Table of Contents, styling options for PDF links, correct French punctuation treatment, and more.

Thanks to the contributions of @doronbehar, as of this release SILE is usable as a Nix Flake (in addition to the usual Nix package). This is an easy way to test run the latest Git HEAD version at any time from almost any OS. One you have flake support enabled, just run nix run github:sile-typesetter/sile -- <sile orguments>. This website uses this to regenerate some of the examples in the gallery.

Even though the math package is a major new feature, ultimately the major version bump was caused by a breaking change elsewhere. Previous to this release, the font used for footnotes was the SILE default unless specifically set. Setting changes were relative to the SILE default rather than the document default. This behavior was workable, but usually came as a surprise to most users. With this release, footnote and folio frame font settings default to being relative to the current document font. The font family, size, leading, and all other aspects of footnotes and folios are still configurable with the same tools as before. But this is a breaking change because those settings are now relative to a different base which might cause many documents to need updating.

⚠ BREAKING CHANGES

• packages: Previous to this release footnote and folio frames took
  their font settings from a new typesetter with default settings. With
  this release the settings are now derived from the typesetter in the
  default frame, hence inheriting font family, size, leading, and other
  settings. Values can still be set using the same functions, but relative
  values such as font sizes are based on a different base.

Features

• core: Add MATH variants table parser (b6c554e)
• core: Add OpenType MATH table parser (835da21)
• math: Add ‘debug’ option to math command (58cc9dc)
• math: Add “big operator” support (5b9a150)
• math: Add fixes to support less complete fonts (1c22af3)
• math: Add italic correction to superscript; correct subscript size (d81fdee)
• math: Add math.font.filename setting (522d70b)
• math: Add math.font.size setting (5077d1c)
• math: Add operator defaults (14bdf1a)
• math: Add parameter to draw debug boxes around math components (2458d18)
• math: Add parameters and support mathvariant param for mi tag (869dca8)
• math: Add plain text support (3a09e9d)
• math: Add subscript and superscript; add math constants (0489c04)
• math: Add support for “symbol macros”, expanding to strings (27658f5)
• math: Add support for fractions (6f4fc24)
• math: Add tags for some mathematical symbols (b9fd771)
• math: Add tex-like math parser (edceaf7)
• math: Allow vertical stacking of top-level ‘mrow’s (56b553c)
• math: Center display math neatly (8951378)
• math: Implement and use munder and mover (61eac7a)
• math: Implement generic bbox shaper (9c86aff)
• math: Output error if rending with non-math font (c79617b)
• math: Replace leading - with − in numbers (f8d490c)
• math: Support double-struck identifiers (29674bf)
• math: Support for simple macros (5b4ecf7)
• math: Support italic (c9b2884)
• math: Support more integral-like operators (90a6c44)
• math: Support of UTF-8 in texmath, support of mo, mi and mn in-grammar (959d1cc)
• math: Turn "-" (hyphen) into "−" (minus) in math (fbed523)
• packages: Add border style and color to hyperlinks (bb880be)
• packages: Add function to remove last added fallback font (acf987b)
• packages: Add linking support to toc entries (e589cb9)
• packages: Add toc depth option and hooks for showing section numbers (c48fcde)
• packages: Allow URLs to have many breakpoints (#1233) (b145605)
• packages: Warn if toc contents have changed (5b6eed8)
• tooling: Enable use as a Nix flake (8b503bb)

Bug Fixes

• classes: Don't increment counters on unnumbered book sections (6cfca4d)
• core: Correct --help output to reflect required values (da487ec)
• languages: Add test 704 for French punctuations, fix expected 621 and 702 results (8e9b056)
• languages: Correct Armenian support to use ISO 639 code ‘hy’ (ffafbe6)
• languages: Correct punctuation rules for French (95c2398)
• languages: Don't initialize Japanese unless actually called for (3aba931)
• languages: Shortcut ICU soft breaks in French (ed8734c)
• math: Fix underover error with sub wider than base but no sup (bc87393)
• packages: Don't replace shaper unless actually initializing color-fonts package (269ca59)
• packages: Fix deprecation warning command in package docs (a69d774)
• packages: Reset footnote and folio settings top level state (3795a4e)
• shaper: Fix memory leak in Harfbuzz library (#1243) (035dcc8)

Summary of Improvements

Today's small patch release is mostly to help out downstream packagers with some small issues that cropped up. First it's a stab in the dark hoping we made things easier to get Homebrew packaging working on ARM based macOS. Secondly we added a build time check for git, which is required in order to use SILE's internal package manager. This dependency may change in the future, but since we use it now it's helpful that system compatibility is checked at build time. Lastly we fixed up some internal usage of an API we deprecated 6 releases ago and documented as going away in v0.11.0, but forgot to stop using ourselves. Notably the French language support module was affected.

Bug Fixes

• build: Avoid implied line continuation in makefile (f2af48f)
• build: Require Git even building tarballs, used by package manager (aba8662)
• languages: Update deprecated syntax in language options (3fb1719)

Summary of Improvements

This release brings a significant fix to the vertical grid typesetting system that was causing extra full grid spaces between paragraphs where they didn't belong. The new behavior is objectively more correct, but it does break previous workarounds. If you previous used a workaround to get the right output in spite of the bug, those documents may not render incorrectly. They can be fixed by removing whatever other workaround was in place or adding a shim that makes the paragraph spacing emulate the old behavior. Some utility functions also received bug fixes. Thanks to contributor @ctrlcctrlv, color fonts now behave themselves much better in TTB layouts and the counter display style Alpha works as documented.

Not much has been added on the feature front, just a few conveniences such as traditional units of (small) measurement and handling more attributes in documents converted via the Pandoc writer (as-yet unreleased).

Beginning with this release Docker images will now be available from GitHub Packages (also known as the GitHub Container Registry). This is an alternative to Docker Hub which has recently started limiting the features available to open source projects.

⚠ BREAKING CHANGES

• packages: Previous to this release any and all leading between
  paragraphs (as set with document.parskip) –even a 0 height skip– would
  result in the skip of one full empty grid space — as if parskip had been
  set to something approximating a full line height. This change corrects
  the calculation so if a 0 height skip is added and everything fits, the
  next line or paragraph will continue uninterrupted in the next grid
  slot. To get the previous layout behavior back, document.parskip must
  be explicitly set to be something larger than 0. Even a minimal 1pt
  skip will result in paragraph spacing that includes one full grid height
  left blank as before:

    \set[parameter=document.parskip,value=1lh]
  

• utilities: Previous return value for breadcrumbs:contains() was
  just an depth index with -1 indicating no match. This made sense when
  I wrote it, but coming back to it for a new project I expected a boolean
  return value. Returning two values seems like the best option, but given
  the function naming it seemed to make sense to return the boolean first,
  hence the API breakage.

Features

• actions: Use tagged images for faster CI job spin up (6a00388)
• build: Add configure flag --disable-dependency-checks (5caf413)
• docker: Build, tag, and push images to GHCR (3988339)
• measurements: Add ‘hm’ (himetric) unit (f4b6b62)
• measurements: Add ‘twip’ unit (cf9d5a7)
• packages: Map unnumbered class to legacy opts in Pandoc package (#1167) (2868da2)

Bug Fixes

• core: Avoid crash on warn by using correct function (b403ad9)
• packages: Avoid crash on warn by using correct function (5d05be1)
• packages: Avoid unnecessary skips to next grid space (6424369)
• packages: Correctly handle color fonts on TTB pages (9b35d6a), closes #1171

Code Refactoring

• utilities: Change breadcrumbs:contains() to return <bool, index> (a987394)

Summary of Improvements

This release fixes a couple of regressions and adds a couple small features. On the regression front, access to the interactive prompt has been restored and the table of contents metadata in PDFs is properly encoded again. For new features, COLR fonts are now automatically detected and the necessary support package is loaded when they are used, a new package allows spaces to be drawn using font-provided glyphs rather than assuming empty space, and another new package allows access to OpenType font features through more verbose fontspec names rather than their OpenType feature code names.

Features

• fonts: Allow for code to be run when a font is first loaded (bdf05ab)
• packages: Add \font-feature command (e2cf008)
• packages: Add complex-spaces package (#1148) (b7451ae)

Bug Fixes

• cli: Re-enable access to repl, input argument not required (a6434ee)
• core: Allow builtin Lua bitwise operators on Lua 5.4 (5f0c2c7)
• docker: Switch to BuildKit and make Docker Hub cooperate (783b104)
• docker: Use patched glibc to work around outdated hosts (fa2532c)
• docker: Use patched glibc to work around outdated hosts (#1141) (bf74417)
• docker: Work around libtexpdf build having side effects (33510d9)
• packages: Add CharacterVariant to features (929eca2)
• utilities: Correct UTF-8/UTF-16 conversions (4863ed6)

Summary of Improvements

This release introduces a new dependency on an external Lua UTF-8 string handling library. By introducing proper string handling instead of our home grown functions we knocked out several bugs including invalid encoding in the output when error tracing was turned on. We also added some (ironically, home grown) UTF-16 string handling to sanitize output of Font meta data.

Features

• core: Make luautf8 library available in global scope (ab7e745)

Bug Fixes

• build: Run autoupdate to fix autoconf issues (ab8307b)
• core: Decode UTF-16BE strings in Windows platform name entries (e7662f8)
• debug: Use UTF8 safe substring function in trace stack (495a5bf)
• manual: Small error (d738b62)

Summary of Improvements

This release brings a few bug fixes, most notably unscrambling Arabic (and some other complex script) shaping. Since current work in process includes significant changes to the document class API and these fixes have been sitting around for a while I figured it was time to give them a release number before anything potentially more disruptive lands.

Feature-wise the only significant improvement is complete coverage for all element types in the Pandoc document AST, enabling any converted documents to compile without errors. They might not be pretty out of the box yet, but at least they render and can be styled by adding or modifying commands.

Features

• classes: Allow footnotes in plain class if package loaded (42c1ceb)
• classes: Run deferred package init() on late load (0224fe3)

Bug Fixes

• backends: Add complex shaping data to debug backend (a1a6509)
• backends: Don't crash if debug output precedes regular (19c21f2)
• build: Don't abuse libtool internals (for NetBSD packaging) (#1084) (048c8b5)
• classes: Define \strong weight=700, not 600 (#1097) (68abf91)
• packages: Add default options to simpletable (1f10c97)
• packages: Correct math operations on grid spacing (5286188)
• packages: Turn off complex flag for items in \latin-in-tate (b20690f)

Summary of Improvements

This release fixes just two bugs. The simpler of the stops custom styles in verbatim blocks. Much more urgently, a backend fix correctly accounts for offsets calculated during the shaping process and stops resetting the PDF cursor position. This issue primarily broke Arabic, but typesetting any fonts with cursive positioning or mark placement rules were affected.

Bug Fixes

• backends: _drawString should take an offset (#1079) (594ae03), closes #1078
• packages: \verbatim:font can process text (#1076) (eb4fb1a)