sile

Summary of Improvements

This release brings a few bug fixes, most notably unscrambling Arabic (and some other complex script) shaping. Since current work in process includes significant changes to the document class API and these fixes have been sitting around for a while I figured it was time to give them a release number before anything potentially more disruptive lands.

Feature-wise the only significant improvement is complete coverage for all element types in the Pandoc document AST, enabling any converted documents to compile without errors. They might not be pretty out of the box yet, but at least they render and can be styled by adding or modifying commands.

Features

• classes: Allow footnotes in plain class if package loaded (42c1ceb)
• classes: Run deferred package init() on late load (0224fe3)

Bug Fixes

• backends: Add complex shaping data to debug backend (a1a6509)
• backends: Don't crash if debug output precedes regular (19c21f2)
• build: Don't abuse libtool internals (for NetBSD packaging) (#1084) (048c8b5)
• classes: Define \strong weight=700, not 600 (#1097) (68abf91)
• packages: Add default options to simpletable (1f10c97)
• packages: Correct math operations on grid spacing (5286188)
• packages: Turn off complex flag for items in \latin-in-tate (b20690f)

Most of the changes in this maintenance release are related to the build and packaging system, with a notable exception of the manual. Many more packages have been documented and the manual has seen quite a bit of copy-editing. For users of the docbook class, the default monospace font has changed from Dejavu Sans to Hack.

One new feature added to the TeXlike input language now allows single quotes to be escaped (\") to appear in the value of options passed to a command.

In the installation department, two new option flags for ./configure have been added, --with-manual and --with-examples. Both are off by default. Enabling these flags will cause a copy of the manual and examples respectively to be installed to your system's $docsdir. These PDFs are included pre-built in the source package, but not installed anywhere by default.

Features

• build: Add --with-examples option to configure & make (245e8a6)
• build: Add --with-manual option to configure & make (3415b3a)
• inputs: Allow (escaped) quote mark in quoted command options (2e9d1b5)

Bug Fixes

• build: Always distribute Lua_modules even if build uses system (e75ece7)
• build: Correct typo in test dependencies causing no font downloads (ad49a85)
• build: Correct typo in test dependencies causing no font downloads (09a653a)
• build: Explicitly filter packaging *.lua and *.sil to avoid cruft (a89773d)
• build: Fix conflations between Lua source types (163959f)
• build: Handle any combo of --with-manual and --with-examples flags (145a86e)
• build: Mark make busted as PHONY so it always runs (23b81ac)
• build: Move dynamically generated file lists out of automake (f626867)
• classes: Use Hack as default monospace font (0e61067)
• core: Handle empty content in macros using \process (2dc6d66)
• frames: Reset font to Gentium to output frame IDs (102dd09), closes #915
• inputs: Disallow 'begin' and 'end' as environment names (b13b99a)
• inputs: Only allow reserved characters as 1-char commands (2a4c095)
• packages: Assure PDF initialization first-output can be rotated (0613ab1)
• packages: Cast measurements to numbers before use in PDF functions (5f2d2e3)
• packages: Fix measurement-to-number issue in SVG (168dffc)
• packages: Improve multi-paragraph pullquotes (7d3f355), closes #865
• packages: Ruby class should not affect document language (#926) (8034aa1)
• packages: Tate should not affect document language (#932) (193fded)
• tooling: Allow make dist on systems without native lua packages (5758085)

This maintenance release fixes a number of small bugs that have cropped up in production use of the v0.10 series. Using the frametricks package no longer causes crashes, your table of contents justification won't be wonky, and passing functions as content is working as expected again. As an added bonus it is possible to process the content passed to functions / macros more than once if desired.

One minor new feature slipped in as well. In addition to the letter-space system that added glue nodes between every glyph there is now a setting for true tracking (that won't break hyphenation or other features). Packages might also appreciate the improved dependency detection and quick self-check that verifies everything is in working order without running the whole test suite.

Note that while Lua 5.4.0 was released a couple days ago, this release officially only supports Lua through the latest 5.3 release. It may or may not work on Lua 5.4 yet, we haven't tested.

Features

• build: Add make check fast self-check target, fixes #835 (89cefef)
• shaper: Add tracking setting and implement for harfbuzz (9e1dec7)

Bug Fixes

• build: Check for luarocks if not configured --with-system-luarocks (e8770ce)
• core: Account for possibility that there are no working fallbacks (391f44e)
• core: Gracefully do nothing when SILE.process() passed nothing (1085049)
• core: Revamp macro system to fix #535 (47a0af8)
• frames: Avoid possible infinite loop when looking for a frame (157dfc8)
• frames: Rely on __tostring() meta method, toString() is no more (77b8956)
• nodes: Fix calling non-existent nodefactory function (#864) (9580a15)
• packages: Center dotfill in the event only one dot fits (95181d2)
• packages: Don't let dotfill content be stretchy (079ff97)

Most of the changes in this release are either related to the tooling to build and package it smoothly or minor. Some English users might appreciate that their Table of Contents headers for otherwise English books aren't localized in Turkish, and Japanese users will have an easier time with their Tate layouts because they cooperate with the new measurement systems introduced in v0.10.

Not properly mentioned in the ChangeLog is a tweak that allows Glues to be cast to Lengths. This makes it much easier to create measurements based on existing nodes as references.

Bug Fixes

• build: Fix version detection in sparse git checkouts (#803) (#818) (dcd0023)
• core: Return correct length from icu.bidi_runs with surrogate pairs (000515f), closes #839
• docker: Work around fresh GNU coreutils bombing Docker Hub (#851) (ed49fbb)
• languages: Localize TOC title functions (#849) (1ab4345)
• packages: Update PDF package to use correct measurement types (79e24ca)
• packages: Update Tate package to use correct measurement types (180024f)
• tooling: Add missing lua-cosmo dependency for Markdown class (#822) (ea81598)
• typesetter: Make typesetter.breakwidth a measurement (721280d)

This release is essentially the same as v0.10.2, but with all the Lua module dependencies bundled again. The previous release would only build either with system provided Lua dependencies or with internet access to download them via luarocks; it was not fully self contained and hence could not be built on some systems.

Bug Fixes

• tooling: Make sure Lua modules get included in source tarball (ef5bb53)
• tooling: Unblock standard-version release number bumping (7b18cd5), closes #816

This release primarily fixes minor packaging issues. The biggest change is this will be the first release with an officially supported Ubuntu PPA!

• build: Don't include build *.so modules in dist (4eb2a73)
• build: Fix version detection in sparse git checkouts (#803) (e46091f)
• build: Include modules for all supported Lua versions in dist (a4e9f03)
• build: Look for Lua 5.3 executables with the version in their name (3952bf8)

See the included CHANGELOG.md or review the commit history for more explicit details.

Summary of Improvements

This is a small follow up to the v0.10.0 release last week to fix a regression that cropped up and add deprecation warning messages. The last release went out with several major changes, particularly to the node and length APIs. Compatibility shims are in place so that old code still works for now, but this adds warning messages to make it easier to find and track down things that need updating.

Additionally since the last release cycle Docker images have been made available, and now support mapping system fonts into the Docker image.

Notable Bug Fixes

• backends: Implement cursor tracking to roughly simulate glues (26afcec)
• core: Actually deprecate old nodefactory instantiators (774f0fc)
• measurements: Actually deprecate old constructors (bfdb1b8)
• nodes: Fix pushHbox() regression, recognize zerohoxes (#789) (dae51f1)

New Features

• backends: Approximate space and break in text output to PDF (9577ae4)
• docker: Add dockerfile and setup to build an image (4424d44)
• docker: Add method to inject fonts into Docker container (104124a)

See the included CHANGELOG.md or review the commit history for more explicit details.

Summary of Improvements

Most of the many, many improvements in this release are under the hood.
The build and test systems have been significantly overhauled, the release process has been streamlined and partially automated, continuous integration now tests code quality and commit messages, and there has been an enormous amount of code refactoring.
All of these things make for a more readable code base and a more reliable and maintainable development environment, which we hope will facilitate SILE development in the future.
Creating installers should be easier for most platforms than in previous releases, and we hope future releases will come with more pre-built packages.

There are also some user-facing improvements:

• Installation is made easier, as SILE will now download and bundle the Lua modules that it requires.
  For those who have their own Lua installation and wish to make use of dependent modules installed with luarocks or their system's package manager, use the --with-system-luarocks argument to ./configure.
• Error and warning reporting has been improved; SILE will now tell you where in the document any errors occurred and (with the --trace option) the stack of commands it took to get there.
• Language support:
  • Fixes to hyphenation in Danish, Greek, Finnish, and Ukrainian.
  • French punctuation conventions are followed, with (optional) non-breaking spaces before high punctuation marks.
  • Numbers can now be formatted as ordinals (at least for English and Turkish), by using the counter display type nth, and as words using the string display type.
• A new text backend is available to dump a text-only version of the rendering process.
  This can much more accurate that extracting text versions from a PDF later for search indexing or similar purposes.
• Improvements to tracking & letter spacing.
• Dependency system: sile -m will output a Makefile snippet showing all the files required to build a SILE document.
• Adding the toc=false option to a book section or subsection will cause it not to be added to the table of contents.
• Added a man page for quick reference of CLI options.

Notable Bug Fixes

• Boustrophedon, grid, and simpletable packages now work again.
• Temporarily switching away from fonts specified by filename into a verbatim environment won't crash any more.

⚠ BREAKING CHANGE

This removes the auto-guessing file extension mechanism that allowed *.sil files to be loaded without specifying the full file name with extensions.

A command like sile test will no longer find and build sile.sil, you must run sile test.sil.
The mechanism that was doing this was a hack than only worked in some scenarios anyway, and dropping it instead of trying to cover all the edge cases will make it that much easier to use and document.
Importantly it avoids edge cases where both *.xml, *.sil, and/or *.lua files all have the same name and the loader really has so idea which one you mean.

Note that packages may still be loaded with no file extension, this does not affect the require() mechanism that looks for *.lua and various other incantations by default.

Scheduled Deprecations

This release renames and deprecates many internal functions and classes.
Most notably the default library used for class models has changed from std to penlight.
Old methods are still available for now, but those developing external Lua packages and classes will want to check the wiki for instructions on how to update their code.
The next release cycle will start throwing warnings when these deprecated functions are called, and the following cycle will remove them entirely.

No code changes. Fix tarball. Some unrelated files (including some binaries) crept into the release.

New in v0.9.5:

• Experimental package manager.

• The "smart" bare percent unit (where SILE guessed whether you meant height or width) has now moved from deprecated to error. Replace with %pw etc.

• Language support: variable spaces in Amharic (and other languages if enabled with the shaper.variablespaces setting), improvements to Japanese Ruby processing, Uyghur hyphenation revisited and improved, Armenian hyphenation added.

• You can now set the stretch and shrink values of a space using the shaper.spaceenlargementfactor, shaper.spaceshrinkfactor and shaper.spacestretchfactor settings.

• You can use - as input filename to pipe in from standard input, and - as output filename to pipe generated PDF to standard output.

• New letter class.

• New commands: \neverindent and \cr

• New units: ps (parskip) and bs (baselineskip)

• Links generated via the url package are hyperlinked in the PDF.

• You can now style folios (page numbers) by overriding the \foliostyle macro.

• Languages may define their own counting functions by providing a counter function; you may also lean on ICU's number formatting to format numbers.

• ICU is now required for correct Unicode processing.

• Experimental support for SVG graphics and fonts. (see tests/simplesvg.sil)

• Users may select the Harfbuzz subshaping system used (coretext, graphite, fallback etc.) by setting the harfbuzz.subshapers setting.

• Fix typos in documentation (Thanks to Sean Leather, David Rowe).

Most other changes in this release are internal and non-user-visible, including:

• Introduced vertical kern nodes.

• Various fixes to pushback (end of page) logic, bidi implementation. ICU is now used for bidi.

• Updated various examples to work with current internals.

• Many and varied internal fixes and speedups, and improved coding style.

Nearly 600 changes, including:

• New packages include: letter spacing, multiple line spacing methods, Japanese Ruby, font specimen generator, crop marks, font fallback, set PDF background color.
• Fixed handling of font weight and style.
• Hyphenation: Correct hyphenation of Indic scripts, words with non-alphabetic characters in them, and allow setting hyphen character and defining hyphenation exceptions.
• Relative dimensions ("1.2em") are converted to absolute dimensions at point of use, not point of declaration. So you can set linespacing to 1.2em, change font size, and it'll still work.
• Default paper size to A4.
• Changes to semantics of percent-of-page and percent-of-frame length specifications. (width=50% etc.)
• Much improved handling of footnotes, especially in multicolumn layouts.
• Support for: the libthai line breaking library, color fonts, querying the system font library on OS X, multiple Amharic justification conventions.
• Added explicit kern nodes.
• Changed to using Harfbuzz for the text processing pipeline; much faster, and much more accurate text shaping.
• Rewritten and more accurate bidirectional handling.
• Removed dependency on FreeType; use Harfbuzz for font metrics.
• Fixed the definition of an em. (It's not the width of a letter "m".)

and much more besides.

Version 0.9.3 (2015-10-09)

• Support for typesetting Japanese according to the JIS X 4051 standard, both horizontally and vertically.
• Unicode line-breaking support; scripts now line-break correctly even if they don't have specific language support. Optionally uses the ICU library if installed.
• Font designers rejoice: you can now say \font[filename=...] to use uninstalled fonts.
• Pango/Cairo support is now officially deprecated. Stop using it!
• Improvements to USX Bible processing.
• Experimental support for Structured PDF generation.
• Support for Opentype kerning.
• Support for custom frame direction (e.g. "TTB-LTR" for Mongolian).
• Support for many-way parallel texts across pages or spreads.
• Line breaking support for Myanmar, Javanese and Uyghur.
• Support for boustrophedon Greek. No, really.
• Various fixes to bidirectionality, discretionary hyphens, insertions, footnotes, grid typesetting, alignment.
• Under-the-hood advancements for Harfbuzz.

Version 0.9.2 (2015-06-03)

• New packages for: rotated content, accessing OpenType features and ligatures, alternative input of Unicode characters, PDF bookmarks and links, input transformation.
• Packages to help with typesetting chord sheets and bibles.
• Experimental packages for bibliography management, typesetting URLs, Japanese vertical typesetting, balanced columns, and best-fit page breaking.
• Support for quoted strings in the parameters to TeX-like commands.
• Language support: Many fixes to Arabic; support for Tibetan and Kannada; hyphenation for many languages; much improved bidirectional typesetting.
• Warn when frames are overfull.
• Support for older versions of autotools, for Lua 5.3 and mingw32 environments.
• Continuous integration and testing framework
• Fixes to long-standing bugs in grid support, centering, ligatures, insertions and page breaking.
• Better font handling and substitution.
• Valid PDFs will still be generated on error/interruption.
• Improved error handling and error messages.
• Many miscellaneous bug fixes.

Version 0.9.1 (2014-10-30)

• The main change in this release is a new shaper based on Harfbuzz and a new PDF creation engine. This has greatly improved the output quality on Linux, as well as bringing support for multilingual typesetting and allowing future support of interesting PDF features. (It's also much faster.)
• The new PDF library also allows images to be embedded in many different formats, rather than just PNG.
• Documents can now be written in right-to-left languages such as Hebrew or Arabic, and it's possible to mix left-to-right and right-to-left text arbitrarily. (Using the Unicode Bidirectional Algorithm.)
• Initial support for languages such as Japanese which have different word/line breaking rules.
• Frames can be grouped into a set called a "master", and masters can be used to set the frame layout of a given page.
• Hopefully a much easier installation process, by bundling some of the required Lua modules and using the standard autoconf ./configure; make strategy.
• Support for Lua 5.2.