sile

Summary of Improvements

This is a relatively minor maintenance release. Thanks again to ongoing work by @Omikhleia, support for bibtex format bibliographies is improving. Date and author parsing and rendering has been improved. Support for CSL is still brewing in the bibliography project.

Additionally the --quiet flag now correctly suppresses all parts of warning messages for cleaner output when you don't want to debug your project. Even more behind the scenes, the entire code base has also been spell-checked, with hundreds of words in comments, code, and documentation being corrected.

New Features

• packages: Support biblatex date field and improve date formatting (5169d67)

Bug Fixes

• build: Fix distribution of unit tests clean up dist file list (00abf9b)
• classes: Make document state available during class's post init (71336b9)
• cli: Suppress trace info in quiet for warnings but do show errors (e12dcc9)
• packages: Parse and split all bibtex name fields (e3a7dc1), closes #2052
• packages: Use non-breakable space in bibTeX name splitting, not tilde (f4ea4ed)

Summary of Improvements

The dust finally settled on the big v0.15.0 release. This patch does include a few more build system touch ups affecting the source tarball that made in require automake when it didn't really need it. But the impetus for this release is actually new features and typesetter bug fixes.

On the language front, @jodros has jumped in with some more domain expertise and Portuguese should now fully support hyphenation of Brazilian locale specific patterns. Additionally we fixed two bugs in the localization system. The locale set for Fluent localizations now properly tracks the document language at any given moment whether set by a font or even a Latin interjection in a bibliography.

Speaking of bibliographies, @Omikhleia is in the middle of a deep overhaul of our bibliography handling. This release has a number of features and bug fixes that lay some groundwork for better parsing and handling bibtex files. More fixes and features are also in progress and even ready for testing. If you have any interest in bibliographies feel free to review ongoing issues and PRs in the bibliography cleanup project.

New Features

• build: Add a developer target to test build dist without automake (7bdaf16)
• languages: Add Brazilian Portuguese specific hyphenation points (895c575)
• packages: Biblatex data inheritance and field mapping (646e3a4)
• packages: Support @string syntax in bibTeX bibliography (63083ad), closes #2051
• packages: Support @xdata entry type and xdata field in bibTeX bibliography (ca906f0)
• packages: Support crossref field in bibTeX bibliography (c4ae919), closes #2021
• tooling: Add developer target to build Rust API docs (384fa46)

Bug Fixes

• build: Avoid building sile binaries if not using them (bbde8e6)
• build: Avoid the perceived need for an extra automake cycle in dist tarball (f634a60)
• build: Do not distribute main man page, requires Rust tooling (26a6346)
• build: Don't distribute distfiles list with configure option specific output (b9d95fd)
• build: Flag transitive dependency on Git if not using system LuaRocks (ea2d501)
• languages: Always set Fluent locale when setting document.language (283fdc3)
• packages: Correct handling of ampersands and tildes in bibtex (aaa613f), closes #2050 #1860
• packages: Ignore @preamble in bibTeX bibliography (742a0c4), closes #2051

Summary of Improvements

This release fixes a regression in v0.15.0 involving 3rd party modules. We were not referencing the LUA_PATH (and LUA_CPATH) environment variables at run time, only at build time. This made it unreasonably difficult to install and use 3rd party modules to a user's $HOME directory and use them in SILE. We were finding modules installed at the system level or locally to a project, but other trees were not being seen.

With this fix, installing modules via luarocks --local install should be a viable workflow again. Before running SILE, make it aware of these modules with eval $(luarocks --local path). The same goes for any arbitrary luarocks --tree locations.

Bug Fixes

• core: Allow LUA_PATH env var to take effect at runtime (e573c2d)

Summary of Improvements

The big v0.15.0 release had some rough edges. But we mentioned that in the v0.15.1 release notes already. The truth is we fumbled some of the sources in the source tarball again. Here is to being able to build cleanly from the source tarball again.

Bug Fixes

• build: Bundle all assets in source distribution (5198641)

Bug Fixes

• build: Distribute vendored compat-5.3.c source file (3c469d6)
• build: Remove Git from configure dependencies (352a125)
• classes: Correct order of operations when paragraphs are ended manually (08f1a7b)
• packages: Change retrograde package so 'target' version works as expected (6ad9b76)

Summary of Improvements

In the works for over a year with over 500 commits and 100 issues closed, please welcome the biggest single release in SILE history. This is the big “Rewrite it in Rust”! Caveat lector. In truth this is a big release but it is not a rewrite. All the typesetting internals of SILE are still written in Lua and 100% user modifiable at runtime. From an end user or 3rd party module developer standpoint little has changed. However SILE itself is now a compiled Rust application that includes its own Lua interpreter. The build process can (optionally) embed all the Lua and other resources files that makeup SILE and its dependencies in a single binary. This opens the door for improvements such as being able to leverage Rust libraries (including exposing their functions to Lua), write some parts of core functions in Rust for performance or preference, write modules in languages other than Lua or C, package SILE for platforms where Lua is not easy to get running, and much more.

The language change mostly affects building and packaging SILE itself. Once running, relatively little has changed with the way SILE interacts with documents. That being said quite a number of default settings have been changed. See the Usage section of these release notes and the retrograde module documentation for tips on how to transition smoothly.

For a transition period, the Lua based CLI is still available as sile-lua. This may be useful for scripting environments that generate inputs and/or parse the output of SILE itself. These should be transitioned to the new Rust CLI sile which has a few minor differences in argument handling and output message formatting. The Lua CLI will only be available for a limited number of future releases. Please do report any issues using the new CLI.

Extra thanks to @Omikhleia for lots contributions and input during the development cycle; and also to @ctrlcctrlv for generous sponsorships that enabled me to commit quite a bit more time to development.

Installation: For Anyone Installing From Packages

If you install SILE from your distro's package manager or other packaging, nothing about your process needs to change. Update via your system tools and enjoy.

Installation: For Distro Packagers and Source Installations

If you install from source or package SILE, the build command sequence is the same but there are new prerequisite dependencies. It now requires Rust tooling (cargo, rustc) as well as some more utilities (jq) to be available at build time. No new dependencies are needed at run time. In fact it is now no longer necessary to use a Lua VM available at runtime for the new CLI. SILE brings its own Lua VM along. (The legacy sile-lua CLI of course still uses Lua as before.) Optionally the build process can also be setup to embed all of its runtime dependencies in a single binary. See ./configure --enable-embedded-resources and ./configure --enable-static if you want to pursue the single binary route. Most users should use the defaults that install the Lua files and other assets separately as an easy reference for tinkering with and overriding them.

Another notable change is that the default Lua VM has been switched from whatever the system supplied to to LuaJIT. This has been an option for a while, but the default has been whatever the newest PUC Lua version was on the host system. LuaJIT is roughly equivalent to Lua 5.1 and doesn't have some small niceties from 5.4, but it is much much faster. Some of the differences are papered over since SILE depends on and provides the compat53 library compatibility layer. Of course build time SILE can still be configured to used any version of Lua of your choice. This can be used to match compatibility with 3rd party modules or other system components.

Usage

A number of command behaviours and default settings have been changed. This will likely cause documents to render with a different flow. It will also break support for some 3rd party modules which will need to be updated to match. Many (but not all) of the changes can be temporarily disabled to cause as few changes when rendering old documents as possible. A new module called retrograde can be loaded at run time that will reset defaults and even revert some commands and functions to their previous behavior. A target argument can be passed for the version of SILE your document was designed for. Any default setting changes and as much other functionality as practical that may have changed since that release will be reverted.

$ sile -u 'packages.retrograde[target=v0.14.17]' <INPUTS>

Since the Lua VM version used by default is different, you may need to reinstall 3rd party modules with a matching Lua version. You can first query SILE to understand what version of Lua it is using. With that information you can specify the Lua version you want when you install modules as LuaRocks to match. As a demonstration we'll install a Markdown input module in a project-local directory where SILE will find it without extra path configuration, then use it to render a PDF file:

$ sile -q -e 'print(SILE.lua_version); os.exit()'
5.1
$ luarocks --lua-version 5.1 --tree lua_modules install markdown.sile
[…]
$ echo 'Test *Markdown* rendering.' > test.md
$ sile -u inputters.markdown test.md
[…]

Without further ado, here is the nitty–gritty.

⚠ BREAKING CHANGES

• packages: Lists now respect the input document spacing and normal settings with regard to paragraphs breaks before, after, and inside lists. This is place of overriding the paragraph skip settings to match the list item spacing setting and always forcing paragraph breaks before and after lists.

• classes: Hitherto SILE has cleared the current.parindent setting as soon as it used it at the beginning of a paragraph. With this release, the setting is not being cleared until a paragraph is explicitly ended. This will not have an affect on many documents, but could completely blow up layout code that implicitly relied on the effect. Normal paragraphs (e.g. separated by a blank line in the input) and any use cases that explicitly called far ending a paragraph (e.g. by calling \par) will be unaffected. But anywhere a paragraph break was simulated by adding vertical space, the indentation will not be applied the next start of a line. This means that calling any variant of \skip inline in a paragraph will result in content beginning on a new line without using the parindent setting.

  Fixing this change in behaviour requires either explicitly resetting the current.parindent setting after it is initially used or explicitly ending a paragraph before or after placing a vertical skip.

• inputters: Input documents using the SIL language will now retain whitespace more consistently. Whitespace following environment blocks is no longer swallowed in differently than space following command syntax. Consecutive line breaks in the input will consistently trigger new paragraphs no matter what they follow.

  Note that this change cannot be patched over via the retrograde package settings because by the time your document could specify what packages to load or settings to set, the input document has already been parsed. To achieve the same rendering results where environments could be ended leaving any amount of blank lines and still joined to the following content as part of the same paragraphs, you will need to remove the extraneous whitespace.

• core: Several top level instance creators of various names have been re-organized under SILE.types.

  Specifically SILE.color, SILE.measurement, and SILE.length have the same names, just under SILE.types.. Additionally SILE.nodefactory is now SILE.types.node and SILE.units is not SILE.types.unit.

  This brings a little bit of sanity to the naming schemes so that you can guess how to use something from the name, but it also makes room for 3rd party add ons to more easily extend or replace these functions. It also makes it easier to start substituting Rust bits where desired.

• core: Use SILE.papersize() instead of SILE.paperSizeParser()

• classes: The "center", "raggedleft" and "raggedright" environments formerly reset the margins (left or right skips), meaning they'd take the full frame width. They all cancelled the paragraph indent. The new behaviour honors the fixed part of the parent context's margins, meaning that if you have an environment playing with margins such as an epigraph or an indented quote, those margins are not lost. The raggedleft and raggedright environment also now no longer cancel the paragraph indent.

• classes: The \script function was heavily overloaded to have many different functions at once and more targeted tools were introduced in SILE v0.14.0 To load 3rd party modules designed for use with SILE, use \use[module=...] instead of \script[src=...]. To run arbitrary Lua code inline use \lua{}; Lua code may be provided inline or externally via either a require= option to load a regular (non-SILE) Lua module using the Lua module path or src= option to load a file by file path.

• core: For ... reasons ... the default width of spaces in SILE has been a highly opinionated and non-standard 1.2 spaces. While it can be argued that this makes some fonts and some documents look better, it is a very strange thing to have as a global default. Unfortunately setting it back to a more conventional 1 space is a major change and will cause many/most documents to re–flow.

  The old default can be recovered either in documents with:

  \set[parameter=shaper.spaceenlargementfactor,value=1.2,makedefault=true]
  

  ...or even from the CLI when rendering a document:

  $ sile -e 'SILE.settings:set("shaper.spaceenlargementfactor", 1.2, true)'
  

• core: The previous default paragraph indent was hard coded with a point size (20pt) that did not adapt will to different fonts or page layouts. The new default uses a relative unit that will adjust based on the leading (1bs). This will cause most documents to re–flow. To keep them the same the setting may be reset to the old default either in the document:

  \set[parameter=document.parindent,value=20pt,makedefault=true]
  

  ...or even from the CLI when rendering a document:

  $ sile -e 'SILE.settings:set("document.parindent", "20pt", true)'
  

• utilities: For modules that rely on SILE.utilities (SU), and in particular raw content handling functions subContent(), walkContent(), stripContentPos(), hasContent(), and contentToString(), these and similar functions have been moved into SILE.utilities.ast (SU.ast). The subContent() implementation also no longer adds id="stuff" attributes to everything.

• packages: The default rendering of Ruby readings has changed from just using a bold weight to using the OpenType +ruby feature. Fonts that support this should work with no change, but documents rendered in fonts that do not support it will need to set the ruby.opentype feature to false to get the same rendering method as before.

• classes: The former implementation of the "em" command did not support nesting and was just setting the font style to italic. The command now alternates italic and regular when nested. * packages: The current (pseudo) idempotent behaviour when loading a package potentially clobbers anything that has been modified since the last load. Loading a package, then modifying a function it provides, then loading the same package again will clobber the modification. This is good for idempotency but not very good for user experience when you may not be modifying all aspects of a document render pipeline at once, as in when using templates.

  This change makes the default behaviour to run setting, raw handler, and command registrations only once. An alternative to :loadpackage() called :reloadpackage() can be used to force all these registrations to be rerun when the goal is to make sure of a specific state.

• classes: Remove obsolete/broken native markdown class

• core: The internal package manager that installed stuff to the system from inside SILE was deprecated back in v0.13.2. It is now completely removed. External 3rd party packages are fully supported using LuaRocks and are much more robust.

Features

• build: Make vendoring Lua sources controllable at build time (773245b)
• build: Set default Lua version to prefer LuaJIT (#1873) (8a8b506)
• classes,outputters,packages: New approach to full bleed printing, cropmarks, background (aa538e2)
• classes: Make text alignment commands honor nesting and margins (7cde8a3)
• classes: Provide a "blockquote" environment in the plain class (75a8bdf)
• classes: Support emphasis nesting (0c1c017)
• cli: Add configure time option to embed Lua resources in binary (9f11100)
• cli: Allow more than one input document (Rust edition) (bdfeecd)
• core: Allow multiple runtime SILE_PATH segments (b329923)
• core: Allow multiple runtime SILE_PATH segments in Lua binary as well as Rust (5e05fa9)
• core: Automatically include project-local lua_modules in module loader path (fd940a9)
• core: Make it easier to add project-specific LuaRocks trees as dependencies (796b344)
• docker: Add all default fonts and package management tooling to container (6bd5dc0)
• docker: Automatically adjust Docker user to owner of mounted data (4ee535f)
• docker: Deploy upcoming major version branches to a develop tag on GHCR (bd2c6f3)
• docker: Enable data for all provided tools that support system locales (c01203f)
• docker: Make it easier for users to install stuff into the images (71aa6f2)
• inputters: Add formal grammar specification for SIL files (5f78af3)
• inputters: Output source comments in SIL debug info (672226f)
• math: Support numbering display math equations (d2e348f)
• outputters: Add hook system to outputters (b5422a8)
• outputters: Add support for pdf elements to debug outputter (a33e5c7)
• packages: Add package to reset some defaults similar to previous releases (fa8d401)
• packages: Backport default font change from v0.9.5 to retrograde package (6b5f52d)
• packages: Expand retrograde package to handle behavior shims (a549b44)
• packages: Expand retrograde package to shim paragraph indentation changes (bead177)
• packages: Make most package _init() actions run only once (ed3331a)
• packages: PDF links can now span multiple lines (842cb56)
• packages: Remove arbitrary six-level maximum to list nesting (ee76eab)
• packages: Strikethrough and underline can span multiple lines (3eb1a1f)
• packages: Use OpenType +ruby feature for ruby, toggle with setting (624e3a8)
• settings: Add hook callback system for settings (efade76)
• settings: Allow \set with no parameter as a temporary wrapper (59755c9)
• tooling: Add direnv support (a52e6e5)
• tooling: Add DRAFT flag to allow rendering docs without regenerating TOC (f581b54)
• tooling: Add nix builds with various lua versions (c28cd94)
• tooling: Add nix packages building sile with clang (5b30695)
• tooling: Add tooling to generate Lua API documentation from sources (3995e14)
• types: Add __tostring to colors for easier debugging (e1106a7)
• typesetter,shaper: Add automated italic correction heuristics (540b0cf)
• typesetters: Add base typesetter (multi-)liner support (e645dbd)
• utilities: Add Greek alphabetical (non-arithmetic) numbering (34e2e53)
• utilities: Automatically cast anything passed to SU.debug to a string (da80c56)
• utilities: Make it easier to search breadcrumbs excluding the current command (64e8d3b)
• utilities: Refactor and extend AST-handling utilities (fc7e7fb)

Bug Fixes

• backends: Fix plain text backend to work in LuaJIT/Lua 5.1 (d0a2a1e)
• build: Adjust macfonts module to more robustly import AppKit (c0571be)
• build: Avoid race condition by making sure binary exists before use (d37da35)
• build: Correct configure flag so debug builds are not release mode (d931849)
• build: Fix build configuration for --with-system-libtexpdf (d22ea34)
• build: Move build-time dependency checks out of runtime dep check configure flag (5e587bd)
• build: Stop copying libraries places autotools doesn't normally handle them (68d9008)
• build: Switch from xz to zst compression for official artifacts (685780d)
• classes: Reset current parindent at end of paragraphs, not beginning (84c70fd)
• core: Output makedepends file after class finish and snippets (f59670c)
• core: Use unique function name so static linking doesn't clobber Lua-5.1 (ccdb8a2)
• docker: Use LuaJIT not Lua5.1 for docker builds (7687bf6)
• inputters: Allow empty string as option value in SIL parser (160501c)
• inputters: Correct typo in SIL parser, disallow conflicting commands (923b11e)
• inputters: Make whitespace after environments behave the same as commands (6d51ee3)
• languages: Handle UTF8 properly in Liang hyphenation (b33c11c)
• languages: Load default en language support if none explicitly requested (#2036) (d02b063)
• packages: Color changes shall not affect other content at page breaks (9d6d409)
• packages: Correct bogus mixup of variable names in package inits (8dfe3c3)
• packages: Fix broken font fallback clear function (2b15de5)
• packages: Use current pagebuilder API not deprecatod one internaly (730150f)
• tooling: Add npx to nix' devShell (624137f)
• tooling: Fix Nix build for rustier sile (f53904c)
• tooling: Really ignore all nix symlinks (34b0493)
• typesetters: Debug hbox could show incorrectly offset boxes (c5113b7)
• utilities: Cast empty to default and only ever return a bool from SU.boolean() (ba2e37a)
• utilities: Fix broken breadcrumbs:contains() (f1972bd)

Performance Improvements

• utilities: Leave tostring() casting to after debug invocations, not before (72965ad)

Miscellaneous Chores

• classes: Deprecate \script in favor of more explicit options (b738507)
• classes: Remove obsolete/broken native markdown class (133567e)
• core: Change default paragraph indent to 1 baseline skip (542c869)
• core: Change default space width to 1 space (66cdbad)
• core: Deprecate SILE.paperSizeParser() (4486a19)
• core: Remove deprecated package manager (4836a93)

Code Refactoring

• core: Move assorted classes from oddball names in core to types (8287a0f)
• packages: Stop overriding paragraph handling in and around lists (8b2117c)

Summary of Improvements

The "big v0.15" is still brewing, but we keep coming up with little improvements that easily fit in the v0.14 series. It turns out the special hyphenation handling we added to Polish (for explicitly hyphenated words) is used by quite a few languages. At this time we've applied it by default to Croatian, Czech, Portuguese, Slovak. and Spanish. Thanks to @Omikhleia for much of the research on this as well as @jodros, @DavidLRowe, @jakubkaczor, and @tomas-vl for domain expertise.

Additionally João contributed an option for columns command to more easily set up column frames with balancing turned on or off.

Features

• languages: Enable explicit hyphen repetition handling in Croatian (c29545d)
• languages: Enable explicit hyphen repetition handling in Czech (b05d621)
• languages: Enable explicit hyphen repetition handling in Portuguese (2a58d96)
• languages: Enable explicit hyphen repetition handling in Slovak (82640b0)
• languages: Enable explicit hyphen repetition handling in Spanish (8db7f23)
• packages: Add balancing option to makecolums command (#1950) (b5ce8e6)

Summary of Improvements

If goofed up some of the merges involved in the release process for v0.14.15. This doesn't bring any user facing features or fixes, only a bit of refactoring intended to be in the last release. The main purpose is to straighten out the repository so the expected branch contains the release tags. Sorry for the noise.

Bug Fixes

• tooling: Re-merge so release tag is on master branch (e8eaf11)

Summary of Improvements

In today's minor release we tinker with language support.

@Omikhleia contributed improvements to non-breaking space handling in French. He also setup handling of Catalan hyphenation at ela geminada and Polish hyphenation at existing hyphens. Turkish gained a new setting (on by default per the current Turkish Language Association guidelines) for handling hyphenation at apostrophes. The previous behaviour (used by some publisher style guides) is available by toggling off the new setting.

All languages gained an (on by default) setting that makes spaces after em-dashes at the start of paragraphs be fixed width. This is used in at least French and Turkish typography to typeset dialogue and the fixed spacing keeps everything lining up per expectations. Other languages may benefit as well, but also the feature can be disabled if flexible spacing is actually desired.

Details for each of these languages and their related settings are in the SILE manual.

Features

• languages: Accomodate alternate Turkish hyphenation guidelines at apostrophes (92904fc)
• typesetters: Support for speaker change introduced by em-dash (2afa4cb)

Bug Fixes

• languages: Break at ela geminada in Catalan cancels the punt vola (f8c4c1e)
• languages: Break at explicit hyphen in Polish repeats the hyphen (0f8c7aa)
• languages: Remove multiple repetitions of U+00A0 nbsp in French (0356a49)
• languages: U+00A0 nbsp in French causes unexpected word breaks (b93975c)

Summary of Improvements

Merry Christmas!

Not this isn't the Rusty release you might want in your stocking. That is coming along nicely though, we're just including an abitioun rollup of breaking changes and going slow to do it right. You can preview the release notes or follow issues in the milestore for teasers. More notably there are already lots of ways to run the development release: Homebrew has --HEAD support, Docker images are available, the Nix flake works, Arch Linux has VCS packages, and more. If you want to start putting it through the paces go ahead.

Until the cookies are fully baked, this minor release has a few goodies that are not breaking changes. @Omikhleia contributed some updates to the dropcaps package to better handle fonts with descenders in their capitals. He also contributed some improvements to Unicode handling: soft hyphen and non-breaking spaces work as expected. SILE has perhaps more robust ways of defining custom hyphenation and glue nodes, but the Unicode handling is convenient is handy if you input content has such data anyway. For bonus points some CLI error messages have been dramatically improved to be more informative and less repetative.

Features

• cli: Change module load error to suggestion for how to install 3rd party modules (c280050)
• languages: U+00A0 non-breaking space must be stretchable by default (b769a63)
• typesetters: Support U+00AD soft hyphen as discretionary break (285507e)

Bug Fixes

• backends: Fix plain text backend to work in LuaJIT/Lua 5.1 (b185d49)
• cli: Stop outputting error messages twice on failure to process main file (da5d609)
• core: Use nil-safe and Lua-version-robust table utils (2405b23)
• packages: Adjust dropcap logic for letters with a depth (fd6963d)

Summary of Improvements

This minor release brings a couple improvements from a new contributor, @jodros. We have one new feature, a boolean option flag on our base class that enables landscape mode. Any paper orientation has been supported already, but the preset paper sizes only had the standard portrait orientation and landscape orientations required entering custom dimensions. This flag makes it a bit easier to swap the X and Y dimensions of the standard (or custom) value parsed from the papersize option. Additionally he fixed a bug in the frametricks package such that using the \makecolumns function doesn't break relative frame constraints used by, for example, footnote or folio frames. This fix only works for TTB-LTR automatic column setups for now.

Finally, some build system fixes and features were backported from v0.15.0 development work. Most significantly the handling of the SILE_PATH environment variable now allows multiple paths, which in turn allows a single project to utilize multiple external collections of 3rd party packages rather than having to collate them all into one location.

Features

• classes: Add landscape option to base class (#1892) (0fb9ade)
• cli: Allow multiple runtime SILE_PATH segments (backport from develop branch) (e7c8fe2)

Bug Fixes

• frames: Update frame constraints with new frame IDs after \makecolumns (b2d6b4f)

Summary of Improvements

This is another minor maintenance release. We throw a few less warning messages having dropped a few that were not actually informative. A few memory management bugs have been fixed, most notably affecting Lua 5.1. For the most part we hope you are on LuaJIT anyway (or your system's default Lua, hopefully newer than 5.1) but we do try to keep everything working smoothly on any platform. A few other minor improvements are noted below.

In other news, the v0.15.0 release is coming soon too. It is already stable enough to be used for real work and should play nice with most systems. Issue #1864 has some notes on how to run the development branch for it before it is released.

Features

• i18n: Add Portuguese localizations for bibtex package (#1859) (f716c35)
• utilities: Add utility function for console messages without trace info (18526ce)

Bug Fixes

• build: Make sure vendored luarocks isn't a phony target that runs repeatedly (713434d)
• core: Allocate exactly what we use, not a guess with an extra just in case (640ded0)
• core: Correct usage of HarfBuzz when passing a filtered list of shapers (f488643)
• core: Fixup class loader so cache is all Lua module specs (#1863) (7efff5b)
• packages: Don't warn on TOC content change if not actually used (87c443d)

Summary of Improvements

Today we only fix the things we broke yesterday. This is a very minor release with only a couple of touch-ups to actual code. The biggest motivation for the release is actually the documentation. Our documentation changes don't show up in the automatic change logs, but we fixed a number of problems with the layout of the manual. Over the last couple releases we introduced several mistakes in the documentation code causing a bit of mess. The manual still isn't perfect, but at least no bits are drawn over other bits any more!

Bug Fixes

• core: Leave legacy masterFilename alone but use first input filename internally (29667a7)
• core: Make masterFilename actually a filename (759131e)
• packages: Avoid mix-and-matching indents in fixed-width specimin blocks (de41cac)
• utilities: Use real semver parser for deprecation warnings (5f0fed5)

Summary of Improvements

Not all releases bring groundbreaking changes. This one is a roll-up of bits and bops. A few small quality of live improvements, some localizations, some bug fixes, some conveniences for 3rd party package developers, and so forth. Enjoy.

Notes about the Future

If you're looking for messy fun, check out the riir branch and PR#1762. As the name suggests, this is a major project overhaul based in Rust. Have no fear! Everything is still fully customizable in Lua. We have no plans of taking away that flexibility. The current proof of concept is a CLI binary is Rust that provides it's own Lua interpreter (optionally linked to the system one or completely vendored). This normalizes the environment, allows targeting environments where Lua is hard to get running, allows parts of the system to be selectively coded in Rust for speed benefits, etc. The PoC already passes 100% of the existing tests, loads 3rd party packages as usual, and is a full drop in replacement for the current Lua based CLI. As of this writing the plan is to land this new CLI in v0.15.0, potentially enabling easy(er) installation on Windows in follow up releases.

If you are a 3rd party developer, by sure to keep an eye on your Lua support. SILE already supports Lua versions 5.1.x through 5.4.x and well as LuaJIT. The same range of support is already achievable at build time in the Rust CLI. The LuaJIT provides the usual massive improvement in speed. We plan to push for this to be the default option in distro packages in the future even on platforms where the default Lua interpreter is 5.4. Making sure your packages run under the existing LuaJIT support should future proof them to easily transition to the Rust based SILE builds.

Features

• cli: Allow more than one input document (d20cbd8)
• i18n: Add localized strings for Cantonese and Chinese (cb67d36)
• packages: Add document class styling in autodoc (e70fa50)
• packages: Provide API for registering raw handlers linked to packages (45cd3ac)

Bug Fixes

• build: Avoid build artifacts being listed for installation (29c2ccd)
• core: Avoid stack overflow in Harfbuzz module (#1793) (5001efe)
• outputters: Setup --makedeps to play along without explicit --output (6ff2e16)
• packages: Converters package no longer worked after 0.13.0 (433795c)
• packages: Correct chord line height and chord font use (65961c6), closes #1351

Summary of Improvements

Today we have a substantial rollup of small fixes. No one thing here is going to knock your socks off, but lots of things are just better.

On the documentation front a new contributor, @jslabovitz, stepped in with a huge copy-edit of the entire manual.

For shiny new features, @Omikhleia added a new scalebox package for reshaping other output. He also taught some existing packages new tricks, such as adding style hooks to the url package and allowing the infinitely stretchy fill glue nodes to be initialized with a starting length. The CLI gained a new --quiet flag to suppress info and warning messages for those times when you just don't want to know what the engine is telling you.

In bug extermination news, the list of squashed ones is long, and some of them had been around for a long time. Didier did a lot of work under the hood with hboxes and discretionary node handling. The upshot for end users is that a lot of weirdness with parindent settings and they way content that didn't fit on one page is pushed to the next has been smoothed over. Your indents shouldn't ever get applied twice, underlines won't apply to the indentation space, unnumbered chapters won't sometimes disappear from your ToC, and so forth.

Features

• classes: Add Picas unit to cover all units speced in Docbook (88f03fa)
• classes: Implement the\code command in the plain class (0d371ba)
• cli: Add -q / --quiet flag to reduce output to essential errors (#1759) (f69ed20)
• core: Support initializing fill glues with a width (#1765) (5bc372a)
• packages: Introduce urlstyle hook in the url package (8f6235d)
• packages: New scalebox package for arbitrary box re-scaling (a11f61e)
• packages: Support migrating content in re-wrapped hboxes (da3ab6d)
• typesetters: Implement hbox building logic in the typesetter (0f5bc69)

Bug Fixes

• build: Distribute SVG and FTL source files in packages (7cef0ea)
• classes: Avoid justification issues with relative parindent (3ffd272)
• classes: Make sure un-numbered chapters make it in the ToC (e5af292)
• classes: Parse bare number and percentage units in docbook images (8b965b9)
• classes: Setting current.hangIndent is a measurement (e213d6e)
• cli: Return success if --help explicitly requested (#1737) (35a229d)
• core,typesetter: Discretionary nodes are incorrectly handled (dd7d05c)
• core: Ensure restoring settings top-level state does not error (fce8447)
• outputters: Update Cairo/Podofo hbox debug API to match libtexpdf (#1703) (607dcf7)
• packages: Correct image aspect preservation logic (6ace5b1)
• packages: Fix output of debug breaks in infonode package (#1725) (c8a1467)
• packages: Make sure pullquotes start in block mode (#1774) (00151bc)
• packages: Strip content position in ToC entries (#1739) (23345ea)
• packages: Text conversion in bookmarks has spacing issues (7ef2bb4)
• typesetter: Account for discretionary dimensions in hbox building (91cb950)
• typesetter: Avoid initializing new line during hbox creation (ae455a1)
• typesetter: New typesetter instances shall not reset settings (16d8a6a)
• typesetter: Skip lines containing only discardables without ignoring next lines (9c3dc65)
• typesetter: Top glues shall be skipped when streching/shrinking a page (8818a24)
• typsetter: Hack around scoping issues for parindent setting (fc85298)
• utilities: Enforce stricter type casts (SU.cast, SU.boolean) (a325eb7)

Summary of Improvements

If you have extra cake in storage just waiting for something worth celebrating, this might be a good release to bust it out for. @khaledhosny has contributed a major new feature: support for variable fonts! This contribution was supported by a bounty grant from the MFEK Foundation. Many thanks to both Khaled and @ctrlcctrlv for making this possible.

Enabling this significant step forward requires the font instancing support introduced in HarfBuzz 6 (part of the harfbuzz-subset library). This is a new default minimum requirement for SILE. At the time of writing this is relatively new, although many platforms already have updates available. It is possible to build against older HarfBuzz versions by using the configuration option --disable-font-variations. When built this way SILE will continue to run on platforms with old HarfBuzz releases but will throw an error if you attempt to render a document using variable font features.

Features

• build: Pass build time configuration into Lua environment (c5d8789)
• core: Add variations support to font command (a37e7bc)
• shapers: Instanciate variable fonts (d50881f)
• shapers: Support named instances with FontConfig (29119b9)
• shapers: Support named instances with macfonts (39a3242)

Bug Fixes

• build: Package license file for vendored lunamark fork (#1686) (13df3c1)
• classes: Coerce option values to booleans (#1696) (8368cb4)
• packages: Absolutize parskip heights on use (1ac793f)
• packages: Pass style & weight values to the math font loader (c92712f)
• packages: Quote option values in documentation when necessary (41e47bb)
• tooling: Use luaEnv properly (#1679) (a34e1c1)

Summary of Improvements

Some of the changes that came out in v0.14.6 were … less than optimal. The build system changes were for a good purpose, but made life a bit difficult for some distro packagers. I also managed to introduce a parsing regression. It also came to light that a previous API deprecation didn't come with the usual warning and shim for transitional support.

This release is notable for distribution packagers for two reasons. First, there is now an easier way to skip the font dependency check at build time. Second, this release is the first time using LuaJIT is at 100% parity with regular Lua. If your distro has LuaJIT available there is now little reason not to make the switch.¹ The speed increase is substantial, especially for large documents.²

While I was tinkering with smoothing out the build and fixing regressions a couple other fixes landed. @khaledhosny tackled a long standing issue with the font loader. The opsz axis can now be used as expected so fonts with OpentType FeatureVariations are usable in SILE. @raphCode also spotted and fixed some outdated code in the manual.

¹: As far as SILE core and all its Lua dependencies are concerned there is no downside. Previously some of the debug and tracing features did not give as much useful information when using LuaJIT. Now the only caveat is related to end users and their documents. If people write custom Lua code in their projects and/or use other 3rd party Lua libraries they will need to consider interpreter support.

²: As a rough comparison on my desktop machine, rendering the 115 page SILE manual takes about 48 seconds with Lua 5.4, 22 seconds with LuaJIT, and 17 seconds with OpenResty.

Features

• build: Allow easy skip of font checks with FCMATCH=true (5c0cef6)

Bug Fixes

• build: Only check tooling to bulid manual if really needed (e166e00)
• inputters: Rework SIL input to handle both junk outside of document tag and fragments (4c51c55)
• outputters: Patch up error message when failing to load font (#1671) (771d87f)
• shaper: Correct font-variants using opsz axis (#1666) (a929583)
• shaper: Respect variations when shaping (#1265) (#1662) (f50ae77)
• tooling: Keep all Lua packages in same env for Nix (8fc8670)
• utilities: Tweak breadcrumbs to work under LuaJIT limitations (32f744c)

Summary of Improvements

This release has tons of fixes and even a few features, but perhaps the most interesting bit to end users might just be the documentation. Thanks to @hegjon, official SILE packages are in the works for Fedora. The procedure for installing them via COPR is documented and will be updated when they land in default repositories. Also @ctrlcctrlv added some useful notes about installing 3rd-party packages. This is really important because very useful actions like typesetting Markdown are best accomplished with external packages.

Anyone working with the source code for SILE itself should note the build system has been split so only run time dependencies are checked at configure time by default. This means no-change is needed for distro packagers and end users that are just wishing to install and move on. However checking for dependencies needed in order to develop SILE itself (including running tests) now requires different configuration. Using --enable-developer should be added as a go-to configure option for anyone hacking or wishing to contribute the to SILE sources. Also of interest to developers, using nix develop should now be a lot more seamless as far as providing a ready-to-use environment.

@Omikhleia has rebuilt the formatNumber() utility function using ICU. This increases language support from a handful of manually implemented ones to almost complete CLDR coverage. In the process a new utility function collatedSort() was added to sort Lua data with locale aware collation. He also fixed a number of little typsetter bugs and overhauled some more documentation.

First-time contributer @raphCode submitted a fix for calling citation keys in the bibtex package.

Debug output is now both faster and more robust since it can't crass the typesetter. When errors are encountered, location reports for where in the document an error was triggered are more accurate. Bundled in this release are also a batch of other small bug fixes to the typesetter, page builder, and shaper.

Features

• build: Add ./configure --enable-developer to ease setup for SILE developers (e8a56ae)
• core: Add SU.collatedSort for language-dependent table sorting with collation (ea7446d)
• core: SU.formatNumber has more options and language support (ed0db29)
• packages: Add package loaded that can later be used to track package dependencies (d48633a)
• packages: Code block environment and raw handler for autodoc (7661330)

Bug Fixes

• classes: Apply page/framebreak in hmode but warn the user (809cbba)
• cli: Deduplicate Lua module loading paths when adding segments (e0f75b1)
• cli: Escape possible path character in replacement (0161f9a)
• cli: Make user system root not added to resource search path (4305850)
• debug: Correct filename in debug info after includes (#1652) (4990ecc)
• debug: Fix pagebuilder debug functions in absence of luastd (ab46bf7)
• debug: Fix typesetter:debugState() in absence of luastd (42f6b0b)
• inputters: Correct Lua inputter AST expectations to match others (6177b0b)
• inputters: Work around SIL parser returning tags as part of content (ef4efb7)
• languages: Replace custom EN/TR ordinals with ICU (82b6709)
• nodes: Ignore empty node properties when debugging breaks (f034e05)
• packages: Correct content position reporting in inputfilter (bb53d77)
• packages: Don't discard grid makup vboxes at top of new pages (22b899c)
• packages: Fix \cite{key} in bibtex package (#1655) (648bb5d)
• packages: Use casting to restore shaper state after fallbacks (351fc68)
• shapers: Apply tracking settings even in font-fallback shaper (55f0c9c)
• tooling: Exempt LuaJIT from external bit32 library requirement (#1654) (d094f1b)
• typesetter: Ensure being in horizontal mode after pushback (a82b604)
• utilities: Correct logic in AST debugging output, also protect (97c82f0)
• utilities: Protect debug functions so they can't crash SILE (319b96a)

Summary of Improvements

We're just squishing bugs today and making the manual a little neater. Working with STDIN streams should be a bit more robust now as the content type detection isn't so picky about whitespace and isn't so prone to false positive detections. Document file restrictions have been relaxed a little too allowing comments and whitespace before the leading document tag. Package developers should have a little bit easier time with in the event their package is loaded twice. Also a number of small typesetting issues in the user manual were cleaned up by @Omikhleia.

Bug Fixes

• inputters: Correct false positive detection of STDIN as Lua content (d54946b)
• inputters: Don't duplicate passthrough content in AST (07c8e87)
• inputters: Permit content outside of the document note, e.g. comments or blanks (#1596) (f1a508a)
• inputters: Relax SIL format sniffing to allow valid syntax (43fc4bc)
• languages: Remove superfluous line (848b91f)
• languages: Tidy up variable scope in languages/unicode.lua (78b453d), closes #699
• measurements: Allow redefinition of existing units (#1608) (8d81018)
• packages: Ensure a page switch does not break boustrophedon (#1615) (64abaf9)

Summary of Improvements

Today we have a small rollup of fixes and improvements. @Omikhleia overhauled the counters packages as well as fixed up bugs in several other packages. The default center of rotation has been changed to give a more expected result. Text casing functions are accessible again from the Lua side of things, changing masters during a page output doesn't break page breaking, and using the twoside package doesn't force the use of mirroring. @OlivierNicole also stepped in to fixup some math bugs. Also thanks to @ThatGeoGuy and @snan for pointing out and fixing small issues in our documentation.

Features

• packages: Add boolean noleadingzeros option to counter formatter (e4f8133)
• packages: Add new command \set-multilevel-counter (11578a8)

Bug Fixes

• classes: Always break out of hmode before processing \chapter headings (0c44d8e)
• core: A typo in a variable prevents using -u with a class (b8f5c40), closes #1569
• languages: Make 'und' an exception to language name canonicalization (52e9b79)
• math: Fix insertion order of MathML children (738e9e6)
• packages: Account for depth when calculating rotation center (289dd2a)
• packages: Avoid forcing mirrored masters in twoside package (#1562) (8cdf6ed)
• packages: Combine \unichar'ed chars with same font only (91a8d40)
• packages: Correct rotation origin calculation back to pre v0.10.0 (3521936)
• packages: Don't inhibit page breaking after switching masters mid-page (6b20f73)
• packages: Make sure PDF initialized before rotate package directly calls it (449b2a6)
• packages: Rework simple and multilevel counters (1e6e91a)
• packages: Textcase package name typo preventing using methods from code (7f68766), closes #1568

Reverts

• Revert "docs(packages): Fixup unichar documentation, work around known bug (#1549)" (03d1b11), closes #1549