sile

Summary of Improvements

This release supports versions of Lua Penlight newer than 1.8.0. Previous releases of Penlight have contained a long-standing bug in the class inheritance system that we were side-stepping with our own workaround. In one case our workaround was actually relying on a side-effect of said Penlight bug and never should have worked. With a proper fix finally implemented in Penlight, our workaround started to fail. This release includes an emergency patch so that SILE can be made to work on systems with the latest Penlight release. Future releases will likely remove our workarounds entirely and require a recent version of Penlight with the fixed class inheritance system.

Additionally, this is the first release that officially supports running SILE as a GitHub Action. You can now call SILE directly in CI workflows on your own projects.

Other cleanup work now allows classes to specify their own framesets without inheriting the default ones. This has no visible effect for most documents but it does make debugging framesets a lot easier because there aren't a bunch of empty frames behind the scenes inherited from other classes.

With special thanks to contributions from @ctrlcctrlv, SVG support is much improved in this release. Specifically, shapes with holes now actually render as holes. Also, thanks to @jmaibaum's contributions and careful eye, many examples from the manual which had fallen behind or had typos now actually work as documented.

Features

• actions: Add configuration file to run as GitHub Action (ee2d509)
• backends: Modify setCursor() to handle relative movements (7caa9c8)
• classes: Make it possible to not use parent class framesets (99b9f50)
• cli: Add Lua interpreter info to --version (bf5210d)

Bug Fixes

• backends: Properly switch between normal and debug fonts (b53896e)
• classes: Identify triglot class as triglot not diglot (495654a)
• classes: Make declareFrames() workable by passing ids (27b6b4a)
• classes: Move class setup code into deferred class:init() (6f470d7)
• core: Patch Penlight 1.9.0 compatibility issue (1eb4290)
• packages: Allow Hanmen frame creation to use optional ID arg (7853d5a)
• packages: Fix hole drawing from svg in PDF (6521fd0)
• packages: Remove extra space in \code in url (b90cd37), closes #1056
• tooling: Expand variables so fonts are known dependencies of tests (88ac888)

Performance Improvements

• backends: Reuse variables instead of recalculating values (02cce40)

Summary of Improvements

This release is the first to fully support LuaJIT. For systems with a LuaJIT interpreter available this brings a full 2× speed improvement to rendering your documents over standard Lua interpreters. This speed increase is not without downsides though. Most available JIT versions do not have full Lua debugging functionality built in, limiting the usefulness of some of SILE's warning messages and especially the trace functionality. A LuaJIT build is probably what you want to use when working on content and re-rendering your documents over and over, but it may not be the best choice for doing development work, coding up your classes and functions, and so forth. To make the switch, add the configuration flag ./configure --with-luajit, then compile and install as usual. In the event you want to keep both versions around, it is actually possible to install both. SILE fully supports GNU Automake's program name transformations, you can specify a prefix, suffix, or complete alternate name to install under. See ./configure --help for details, but --program-suffix=-luajit is one way to get a sile-luajit executable that could be used alongside sile.

A few functions have been renamed and their old versions deprecated. If any of your code calls outputter functions directly these calls will need to be modified to be uniformly serf-referential method calls (i.e. :method() instead of .method()). This makes the outputter API consistent between backends, but may be a breaking change for some. Several other functions have been renamed and their old names deprecated. For these warning messages will be thrown for now, and the next version series (0.11.x) will deprecate them entirely. The most notable rename is \include-svg-file is now just \svg. In addition to the rename you can now specify a desired size by width instead of only by height.

In smaller news a large number of small fixes now draw frame debugging lines in exactly the right place instead of next to the right place, mark PDF bookmark locations at the top of the current line instead of the bottom, draw rules in the correct writing and advance directions for RTL and BTT frames. Other fixes include keeping Japanese language and tate functions from taking over documents. The \center command now disables line indentation so content is centered as expected, and attempting to set indentation after the start of a paragraph will throw a useful warning.

Features

• build: Detect and use luajit first (601dfc4)
• build: Detect LuaJIT if explicitly configured to want it (c3e8089)
• classes: Add warning to \noindent if called after input (f29b9d9)
• packages: Allow scaling SVGs by width or height (44588b5)
• settings: Add a way to reset single setting to defaults (f318cdf)
• settings: Bring Lua settings.set to parity with \set (d73b08c)

Bug Fixes

• classes: Reset parindent's inside \center command (7b62f74)
• core: Always compare like-types so LuaJIT can run (c608090)
• core: Don't read zero-length name table entries (bcd9a9e), closes #1015
• examples: Properly center title in showoff document (55717fb)
• frames: Discard content (usually whitespace) inside \pagetemplate (3b7085b)
• frames: Draw frame debug lines exactly on frame lines (db92edc)
• languages: Stop Japanese resetting global chapter post macro (836f199)
• packages: Align pullquote ending mark with outside margin (8b808db)
• packages: Draw rules in the writing direction (18bca68)
• packages: Error if asked to add bogus dependencies (59e2b56)
• packages: Fix indentation of second paragraph in pullquotes (a8525e5)
• packages: List \include files in makedeps (bf670ab)
• packages: Orient rules for all 8 directions (bc4a33a)
• packages: Place PDF bookmarks at top of current line (ce30d83)
• utilities: Use deterministic sort for sorted pairs (99e2b59)

Summary of Improvements

This is the first version of SILE to officially support Lua 5.4! If your OS or distro is migrating to the latest Lua, this release is for you. The changes in this Lua release are not very significant in relation to SILE, but it took a bit of time and effort for all the modules we use to be compatible. Additionally in this release how we handle bundling LuaRocks modules has changed. We no longer include the full source and prebuilt rocks in the source packages. For most platforms and installations this will not change anything at all, using --with-system-luarocks still skips anything to do with bundling and just requires the dependencies be available on the host system. This is used by all platforms to date that have SILE packages. For people building from source using the default configuration of --without-system-luarocks the bundling still happens using Luarocks to download and build all the required dependencies when you run make. Previously it copied the sources for your version of Lua out of the source package, but with 4 supported versions of Lua and many variation in how different platforms build them this was getting too cumbersome. LuaRocks does this job already, we just leverage it.

Still on the topic of packaging, a minor change is that make install will not install the PDF manual and examples by default, but if you add --with-manual and/or --with-examples it will not only install them, but it will install them to your platform’s default location for PDF documentation (previously they were just stuffed in alongside the SILE code, wherever that was). The PDF documentation directory is the same as for other documentation on most platforms, but it does go to the right place for those system that have special handling for PDF documentation.

As for the typesetter itself, if you ever used inputfilter() functions and your content randomly came out in the wrong order, we fixed that non-determinism. Additionally Unicode combining diacritic marks can be now be added with \unichar{}. If you’re trying to figure out what is what on a page, --debug hboxes also works again after being broken some time back.

For developers, adding multiple debug flags from the CLI is easier than ever, and fewer things need to be rebuilt from scratch every time you touch something in SILE’s source code.

Features

• build: Install manual to $(pdfdir) if configure --with-manual (ee33ff7)
• core: Allow adding --debug flag multiple times (9ac2838)

Bug Fixes

• build: Correct typo in dependencies for building docs (ad548a5)
• build: Ship blank lua_modules install list in source packages (7939970)
• build: Touch Makefile.in to avoid automake errors (e7f4627)
• build: Work around src/libtexpdf subdirs using side-effects (26d6769)
• core: Iterate on sequential data with ipairs() or SU.sortedpairs() (9db0a28)
• debug: Fix math in hbox debugging (6c0029d)
• packages: Combine unichar output with existing unshaped node (712bc92)
• packages: Use sortedpairs to avoid non-determinism (a28ef06)
• utilities: Add sorted pairs function (5aad397)

This release addresses the Docker builds which came out broken in the previous release. Some Lua modules shuffled to make way for Lua 5.4 support (which is almost ready, just waiting on one more upstream dependency). Additionally some build issues on BSD have been smoothed over.

Note I accidentally pushed this release tag along with a bad commit that was never supposed to be released. Hence the v0.10.8 tag was briefly attached to a different commit (but with no packages attached). I apologize for any inconvenience for any inconvenience this caused.

Features

• build: Output hints about how to compile from repo snapshots (596cd9f)

Bug Fixes

• build: Avoid possible race condition on first bulid (b937c95)
• build: Use BSD compatible find syntax (c96683e)
• build: Use BSD compatible touch syntax (25eb6fd)
• docker: Make sure Lua modules installation works on the first pass (f0c3e26)

Performance Improvements

• build: Save a ./configure cycle by bootstraping the version (2997d05)

Reverts

• Revert "chore(build): Save a double-configure on first download/build" (ef56de4)

This release is substantially the same as v0.10.6, except the release packages were published fully automatically. This was supposed to happen in the last release too, but some issues just don't surface until you do the real thing. This time it actually worked.

Bug Fixes

• build: Merge Github Actions release step with build (b2d77ab)

Most of the changes in this maintenance release are related to the build and packaging system, with a notable exception of the manual. Many more packages have been documented and the manual has seen quite a bit of copy-editing. For users of the docbook class, the default monospace font has changed from Dejavu Sans to Hack.

One new feature added to the TeXlike input language now allows single quotes to be escaped (\") to appear in the value of options passed to a command.

In the installation department, two new option flags for ./configure have been added, --with-manual and --with-examples. Both are off by default. Enabling these flags will cause a copy of the manual and examples respectively to be installed to your system's $docsdir. These PDFs are included pre-built in the source package, but not installed anywhere by default.

Features

• build: Add --with-examples option to configure & make (245e8a6)
• build: Add --with-manual option to configure & make (3415b3a)
• inputs: Allow (escaped) quote mark in quoted command options (2e9d1b5)

Bug Fixes

• build: Always distribute Lua_modules even if build uses system (e75ece7)
• build: Correct typo in test dependencies causing no font downloads (ad49a85)
• build: Correct typo in test dependencies causing no font downloads (09a653a)
• build: Explicitly filter packaging *.lua and *.sil to avoid cruft (a89773d)
• build: Fix conflations between Lua source types (163959f)
• build: Handle any combo of --with-manual and --with-examples flags (145a86e)
• build: Mark make busted as PHONY so it always runs (23b81ac)
• build: Move dynamically generated file lists out of automake (f626867)
• classes: Use Hack as default monospace font (0e61067)
• core: Handle empty content in macros using \process (2dc6d66)
• frames: Reset font to Gentium to output frame IDs (102dd09), closes #915
• inputs: Disallow 'begin' and 'end' as environment names (b13b99a)
• inputs: Only allow reserved characters as 1-char commands (2a4c095)
• packages: Assure PDF initialization first-output can be rotated (0613ab1)
• packages: Cast measurements to numbers before use in PDF functions (5f2d2e3)
• packages: Fix measurement-to-number issue in SVG (168dffc)
• packages: Improve multi-paragraph pullquotes (7d3f355), closes #865
• packages: Ruby class should not affect document language (#926) (8034aa1)
• packages: Tate should not affect document language (#932) (193fded)
• tooling: Allow make dist on systems without native lua packages (5758085)

This maintenance release fixes a number of small bugs that have cropped up in production use of the v0.10 series. Using the frametricks package no longer causes crashes, your table of contents justification won't be wonky, and passing functions as content is working as expected again. As an added bonus it is possible to process the content passed to functions / macros more than once if desired.

One minor new feature slipped in as well. In addition to the letter-space system that added glue nodes between every glyph there is now a setting for true tracking (that won't break hyphenation or other features). Packages might also appreciate the improved dependency detection and quick self-check that verifies everything is in working order without running the whole test suite.

Note that while Lua 5.4.0 was released a couple days ago, this release officially only supports Lua through the latest 5.3 release. It may or may not work on Lua 5.4 yet, we haven't tested.

Features

• build: Add make check fast self-check target, fixes #835 (89cefef)
• shaper: Add tracking setting and implement for harfbuzz (9e1dec7)

Bug Fixes

• build: Check for luarocks if not configured --with-system-luarocks (e8770ce)
• core: Account for possibility that there are no working fallbacks (391f44e)
• core: Gracefully do nothing when SILE.process() passed nothing (1085049)
• core: Revamp macro system to fix #535 (47a0af8)
• frames: Avoid possible infinite loop when looking for a frame (157dfc8)
• frames: Rely on __tostring() meta method, toString() is no more (77b8956)
• nodes: Fix calling non-existent nodefactory function (#864) (9580a15)
• packages: Center dotfill in the event only one dot fits (95181d2)
• packages: Don't let dotfill content be stretchy (079ff97)

Most of the changes in this release are either related to the tooling to build and package it smoothly or minor. Some English users might appreciate that their Table of Contents headers for otherwise English books aren't localized in Turkish, and Japanese users will have an easier time with their Tate layouts because they cooperate with the new measurement systems introduced in v0.10.

Not properly mentioned in the ChangeLog is a tweak that allows Glues to be cast to Lengths. This makes it much easier to create measurements based on existing nodes as references.

Bug Fixes

• build: Fix version detection in sparse git checkouts (#803) (#818) (dcd0023)
• core: Return correct length from icu.bidi_runs with surrogate pairs (000515f), closes #839
• docker: Work around fresh GNU coreutils bombing Docker Hub (#851) (ed49fbb)
• languages: Localize TOC title functions (#849) (1ab4345)
• packages: Update PDF package to use correct measurement types (79e24ca)
• packages: Update Tate package to use correct measurement types (180024f)
• tooling: Add missing lua-cosmo dependency for Markdown class (#822) (ea81598)
• typesetter: Make typesetter.breakwidth a measurement (721280d)

This release is essentially the same as v0.10.2, but with all the Lua module dependencies bundled again. The previous release would only build either with system provided Lua dependencies or with internet access to download them via luarocks; it was not fully self contained and hence could not be built on some systems.

Bug Fixes

• tooling: Make sure Lua modules get included in source tarball (ef5bb53)
• tooling: Unblock standard-version release number bumping (7b18cd5), closes #816

This release primarily fixes minor packaging issues. The biggest change is this will be the first release with an officially supported Ubuntu PPA!

• build: Don't include build *.so modules in dist (4eb2a73)
• build: Fix version detection in sparse git checkouts (#803) (e46091f)
• build: Include modules for all supported Lua versions in dist (a4e9f03)
• build: Look for Lua 5.3 executables with the version in their name (3952bf8)

See the included CHANGELOG.md or review the commit history for more explicit details.

Summary of Improvements

This is a small follow up to the v0.10.0 release last week to fix a regression that cropped up and add deprecation warning messages. The last release went out with several major changes, particularly to the node and length APIs. Compatibility shims are in place so that old code still works for now, but this adds warning messages to make it easier to find and track down things that need updating.

Additionally since the last release cycle Docker images have been made available, and now support mapping system fonts into the Docker image.

Notable Bug Fixes

• backends: Implement cursor tracking to roughly simulate glues (26afcec)
• core: Actually deprecate old nodefactory instantiators (774f0fc)
• measurements: Actually deprecate old constructors (bfdb1b8)
• nodes: Fix pushHbox() regression, recognize zerohoxes (#789) (dae51f1)

New Features

• backends: Approximate space and break in text output to PDF (9577ae4)
• docker: Add dockerfile and setup to build an image (4424d44)
• docker: Add method to inject fonts into Docker container (104124a)

See the included CHANGELOG.md or review the commit history for more explicit details.

Summary of Improvements

Most of the many, many improvements in this release are under the hood.
The build and test systems have been significantly overhauled, the release process has been streamlined and partially automated, continuous integration now tests code quality and commit messages, and there has been an enormous amount of code refactoring.
All of these things make for a more readable code base and a more reliable and maintainable development environment, which we hope will facilitate SILE development in the future.
Creating installers should be easier for most platforms than in previous releases, and we hope future releases will come with more pre-built packages.

There are also some user-facing improvements:

• Installation is made easier, as SILE will now download and bundle the Lua modules that it requires.
  For those who have their own Lua installation and wish to make use of dependent modules installed with luarocks or their system's package manager, use the --with-system-luarocks argument to ./configure.
• Error and warning reporting has been improved; SILE will now tell you where in the document any errors occurred and (with the --trace option) the stack of commands it took to get there.
• Language support:
  • Fixes to hyphenation in Danish, Greek, Finnish, and Ukrainian.
  • French punctuation conventions are followed, with (optional) non-breaking spaces before high punctuation marks.
  • Numbers can now be formatted as ordinals (at least for English and Turkish), by using the counter display type nth, and as words using the string display type.
• A new text backend is available to dump a text-only version of the rendering process.
  This can much more accurate that extracting text versions from a PDF later for search indexing or similar purposes.
• Improvements to tracking & letter spacing.
• Dependency system: sile -m will output a Makefile snippet showing all the files required to build a SILE document.
• Adding the toc=false option to a book section or subsection will cause it not to be added to the table of contents.
• Added a man page for quick reference of CLI options.

Notable Bug Fixes

• Boustrophedon, grid, and simpletable packages now work again.
• Temporarily switching away from fonts specified by filename into a verbatim environment won't crash any more.

⚠ BREAKING CHANGE

This removes the auto-guessing file extension mechanism that allowed *.sil files to be loaded without specifying the full file name with extensions.

A command like sile test will no longer find and build sile.sil, you must run sile test.sil.
The mechanism that was doing this was a hack than only worked in some scenarios anyway, and dropping it instead of trying to cover all the edge cases will make it that much easier to use and document.
Importantly it avoids edge cases where both *.xml, *.sil, and/or *.lua files all have the same name and the loader really has so idea which one you mean.

Note that packages may still be loaded with no file extension, this does not affect the require() mechanism that looks for *.lua and various other incantations by default.

Scheduled Deprecations

This release renames and deprecates many internal functions and classes.
Most notably the default library used for class models has changed from std to penlight.
Old methods are still available for now, but those developing external Lua packages and classes will want to check the wiki for instructions on how to update their code.
The next release cycle will start throwing warnings when these deprecated functions are called, and the following cycle will remove them entirely.

No code changes. Fix tarball. Some unrelated files (including some binaries) crept into the release.

New in v0.9.5:

• Experimental package manager.

• The "smart" bare percent unit (where SILE guessed whether you meant height or width) has now moved from deprecated to error. Replace with %pw etc.

• Language support: variable spaces in Amharic (and other languages if enabled with the shaper.variablespaces setting), improvements to Japanese Ruby processing, Uyghur hyphenation revisited and improved, Armenian hyphenation added.

• You can now set the stretch and shrink values of a space using the shaper.spaceenlargementfactor, shaper.spaceshrinkfactor and shaper.spacestretchfactor settings.

• You can use - as input filename to pipe in from standard input, and - as output filename to pipe generated PDF to standard output.

• New letter class.

• New commands: \neverindent and \cr

• New units: ps (parskip) and bs (baselineskip)

• Links generated via the url package are hyperlinked in the PDF.

• You can now style folios (page numbers) by overriding the \foliostyle macro.

• Languages may define their own counting functions by providing a counter function; you may also lean on ICU's number formatting to format numbers.

• ICU is now required for correct Unicode processing.

• Experimental support for SVG graphics and fonts. (see tests/simplesvg.sil)

• Users may select the Harfbuzz subshaping system used (coretext, graphite, fallback etc.) by setting the harfbuzz.subshapers setting.

• Fix typos in documentation (Thanks to Sean Leather, David Rowe).

Most other changes in this release are internal and non-user-visible, including:

• Introduced vertical kern nodes.

• Various fixes to pushback (end of page) logic, bidi implementation. ICU is now used for bidi.

• Updated various examples to work with current internals.

• Many and varied internal fixes and speedups, and improved coding style.

Nearly 600 changes, including:

• New packages include: letter spacing, multiple line spacing methods, Japanese Ruby, font specimen generator, crop marks, font fallback, set PDF background color.
• Fixed handling of font weight and style.
• Hyphenation: Correct hyphenation of Indic scripts, words with non-alphabetic characters in them, and allow setting hyphen character and defining hyphenation exceptions.
• Relative dimensions ("1.2em") are converted to absolute dimensions at point of use, not point of declaration. So you can set linespacing to 1.2em, change font size, and it'll still work.
• Default paper size to A4.
• Changes to semantics of percent-of-page and percent-of-frame length specifications. (width=50% etc.)
• Much improved handling of footnotes, especially in multicolumn layouts.
• Support for: the libthai line breaking library, color fonts, querying the system font library on OS X, multiple Amharic justification conventions.
• Added explicit kern nodes.
• Changed to using Harfbuzz for the text processing pipeline; much faster, and much more accurate text shaping.
• Rewritten and more accurate bidirectional handling.
• Removed dependency on FreeType; use Harfbuzz for font metrics.
• Fixed the definition of an em. (It's not the width of a letter "m".)

and much more besides.

Version 0.9.3 (2015-10-09)

• Support for typesetting Japanese according to the JIS X 4051 standard, both horizontally and vertically.
• Unicode line-breaking support; scripts now line-break correctly even if they don't have specific language support. Optionally uses the ICU library if installed.
• Font designers rejoice: you can now say \font[filename=...] to use uninstalled fonts.
• Pango/Cairo support is now officially deprecated. Stop using it!
• Improvements to USX Bible processing.
• Experimental support for Structured PDF generation.
• Support for Opentype kerning.
• Support for custom frame direction (e.g. "TTB-LTR" for Mongolian).
• Support for many-way parallel texts across pages or spreads.
• Line breaking support for Myanmar, Javanese and Uyghur.
• Support for boustrophedon Greek. No, really.
• Various fixes to bidirectionality, discretionary hyphens, insertions, footnotes, grid typesetting, alignment.
• Under-the-hood advancements for Harfbuzz.

Version 0.9.2 (2015-06-03)

• New packages for: rotated content, accessing OpenType features and ligatures, alternative input of Unicode characters, PDF bookmarks and links, input transformation.
• Packages to help with typesetting chord sheets and bibles.
• Experimental packages for bibliography management, typesetting URLs, Japanese vertical typesetting, balanced columns, and best-fit page breaking.
• Support for quoted strings in the parameters to TeX-like commands.
• Language support: Many fixes to Arabic; support for Tibetan and Kannada; hyphenation for many languages; much improved bidirectional typesetting.
• Warn when frames are overfull.
• Support for older versions of autotools, for Lua 5.3 and mingw32 environments.
• Continuous integration and testing framework
• Fixes to long-standing bugs in grid support, centering, ligatures, insertions and page breaking.
• Better font handling and substitution.
• Valid PDFs will still be generated on error/interruption.
• Improved error handling and error messages.
• Many miscellaneous bug fixes.

Version 0.9.1 (2014-10-30)

• The main change in this release is a new shaper based on Harfbuzz and a new PDF creation engine. This has greatly improved the output quality on Linux, as well as bringing support for multilingual typesetting and allowing future support of interesting PDF features. (It's also much faster.)
• The new PDF library also allows images to be embedded in many different formats, rather than just PNG.
• Documents can now be written in right-to-left languages such as Hebrew or Arabic, and it's possible to mix left-to-right and right-to-left text arbitrarily. (Using the Unicode Bidirectional Algorithm.)
• Initial support for languages such as Japanese which have different word/line breaking rules.
• Frames can be grouped into a set called a "master", and masters can be used to set the frame layout of a given page.
• Hopefully a much easier installation process, by bundling some of the required Lua modules and using the standard autoconf ./configure; make strategy.
• Support for Lua 5.2.