Modern and easy syntax highlighting using tree-sitter
GPL-3.0 License
syntastica
Modern and easy syntax highlighting using tree-sitter
Note
If viewing this file on GitHub or crates.io, some links might not be working. Go to the custom docs page or the docs.rs page instead, which additionally include the Features section.
To use syntastica
, you probably want to depend on three crates:
syntastica
crate for all the logic.So for example:
syntastica = "<version>"
syntastica-parsers = { version = "<version>", features = ["some"] }
syntastica-themes = "<version>"
syntastica
has three main ways of highlighting code, for three different use
cases:
highlight
] andProcessor::process_once
], [render
], andProcessor
], [render
], andBesides the main syntastica
crate, many other crates for different purposes
were developed and are included in the repository. This section aims to provide
a good overview.
The main syntastica
crate provides no tree-sitter parsers and queries by
itself. However, the project does provide three different parser collections
with different advantages and drawbacks each. All three collections depend on
syntastica-queries
for the tree-sitter queries. Choose
one, and add it as a dependency next to syntastica
itself.
All three parser collections also provide the same public API and provide
features for all supported languages, as well as the three feature groups
some
, most
, and all
. Take a look at the respective crate documentation for
more information.
If you want to additionally use languages that are not in any of these parser collections, one approach is shown in the custom languages example.
syntastica-parsers
isgit
command to be accessible andsyntastica-parsers-gitdep
syntastica-parsers-git
however,To render highlighted code to end users, a
theme is needed, which specifies the colors to use for
which theme key. The syntastica
project comes with a
separate crate containing a few default themes:
syntastica-themes
.
If you wish to create your own theme, have a look at the
custom theme example and the documentation for the
[theme!
] macro.
The syntastica
repository/workspace also includes some crates which are not
meant for outside use, but are instead used internally. These are listed below.
Note: There are no guarantees about the public API of these crates! If, for any reason, you have to depend on one of them, then pin the exact version using
<crate> = "=<version>"
.
syntastica-core
defines types,syntastica
crate re-exports all those items transparently, so thatsyntastica-core
however, to avoid cyclic (dev-)dependencies inside thissyntastica-macros
defineslanguages.toml
syntastica-highlight
is atree-sitter-highlight
,syntastica
. It containsLanguageSet
. Unlike the previous crates in thisThis list includes crates which were developed for syntastica
but have no
direct association with the main project and can be used completely separately.
rsexpr
is a generic S-expression parserdprint-plugin-sexpr
forsyntastica
this crate isqueries
cargo xtask codegen queries
and result ingenerated_queries
syntastica-queries
.lua-pattern
is a parser for Luasyntastica
this is used, as many of the source#lua-match?
predicates for matching with Lua patterns.cargo xtask codegen queries
), all Lua patterns are replaced with regularsyntastica-query-preprocessor
; inherits <lang>
comments, conditional skipping of nodes with comments,lua-match?
, contains?
and any-of?
,syntastica
in the name, the crate can be usedsyntastica-
crates. Insyntastica
it is used in thecodegen queries
xtask,syntastica
can be used with WebAssembly, although the current support is a bit
lacking. There are currently two primary ways to use syntastica
in a
WebAssembly context.
tree-sitter-c2rust
runtimeIn order to make syntastica
compile to wasm32-unknown-unknown
targets,
feature flags can be used to use the
c2rust transpilation of tree-sitter
instead of the
official C implementation. This is only
supported by the syntastica-parsers-git
parser collection, and only parsers
that don't use an external C++ scanner are available.
To use this approach, simply set default-features = false
and enable the
runtime-c2rust
feature for all syntastica
dependencies. An example using
this approach for use of syntastica
in a Dioxus project can be found
here.
syntastica-js
packagesyntastica
can also be compiled to wasm32-unknown-emscripten
which has much
better support for C and C++ interop. But annoyingly, basically the entire Rust
Wasm ecosystem is built around the wasm32-unknown-unknown
target (e.g.,
wasm-pack
and wasm-bindgen
can only be used with wasm32-unknown-unknown
),
which makes it very cumbersome to use Emscripten for Rust. In the attempt to
make using syntastica
on the web a bit easier, the
syntastica-js
crate
and accompanying
syntastica
NPM package provide a
JavaScript/TypeScript wrapper around an Emscripten build of syntastica
.
There are three examples using syntastica-js
:
Note
The
syntastica
NPM package is currently not being updated and uses an old version ofsyntastica
, because the current implementation always includes all parsers in one big binary, which would be over 60 MB big with all currently supported parsers. The eventual plan is to find a way to split the package into multiple binaries that can be fetched from a server on-demand, and to provide multiple NPM packages for manual selection of the parsers.
This section contains some basic usage examples. More specific examples can be
found in the documentation of some items such as the [Processor
] type or the
[render
] function. Additionally, the
examples
directory contains a few complete examples.
This is the list of examples found here:
This example shows the easiest and quickest way to use syntastica
. See the
section about use cases for when it is appropriate to use
syntastica
this way.
use syntastica::renderer::TerminalRenderer;
use syntastica_parsers::{Lang, LanguageSetImpl};
let output = syntastica::highlight(
// the code to highlight
r#"fn main() { println!("42"); }"#,
// the input's language
Lang::Rust,
// use `syntastica-parsers` language set
&LanguageSetImpl::new(),
// use the TerminalRenderer with no background color
&mut TerminalRenderer::new(None),
// use the gruvbox dark theme from `syntastica-themes`
syntastica_themes::gruvbox::dark(),
)
.unwrap_or_else(|err| panic!("highlighting failed: {err}"));
println!("{output}");
This example shows how to render the same input with two different themes using two different renderers.
use syntastica::{Processor, style::Color, renderer::*};
use syntastica_parsers::{Lang, LanguageSetImpl};
// process the input once, but store the raw highlight information
let highlights = Processor::process_once(
// the code to highlight
r#"fn main() { println!("42"); }"#,
// the input's language
Lang::Rust,
// use `syntastica-parsers` language set
&LanguageSetImpl::new(),
)
.unwrap_or_else(|err| panic!("highlighting failed: {err}"));
// render the highlights to the terminal using the
// gruvbox dark theme on a dark gray background
println!("{}", syntastica::render(
&highlights,
&mut TerminalRenderer::new(Some(Color::new(40, 40, 40))),
syntastica_themes::gruvbox::dark(),
));
// render the same input to HTML using the onelight theme
let html = syntastica::render(
&highlights,
&mut HtmlRenderer::new(),
syntastica_themes::one::light(),
);
// you could for example write that to a file called `index.html`:
// std::fs::write("index.html", html).unwrap();
This example shows how a [Processor
] can be reused if multiple different
inputs should be highlighted.
use syntastica::{Processor, style::Color, renderer::*};
use syntastica_parsers::{Lang, LanguageSetImpl};
// create a language set and a `Processor`
let language_set = LanguageSetImpl::new();
let mut processor = Processor::new(&language_set);
// Note: `language_set` has to be stored in a variable, because the processor
// is bound to the lifetime of the reference passed to `new`
// process some input
let highlights_rust = processor.process(
// the code to highlight
r#"fn main() { println!("42"); }"#,
// the input's language
Lang::Rust,
)
.unwrap_or_else(|err| panic!("highlighting failed: {err}"));
// process some other input in another language
let highlights_js = processor.process(r"console.log('42')", Lang::Javascript)
.unwrap_or_else(|err| panic!("highlighting failed: {err}"));
// render the rust code to the terminal using the
// gruvbox dark theme on a dark gray background
println!("{}", syntastica::render(
&highlights_rust,
&mut TerminalRenderer::new(Some(Color::new(40, 40, 40))),
syntastica_themes::gruvbox::dark(),
));
// render the same rust code to HTML using the onelight theme
let html = syntastica::render(
&highlights_rust,
&mut HtmlRenderer::new(),
syntastica_themes::one::light(),
);
// you could for example write that to a file called `index.html`:
// std::fs::write("index.html", html).unwrap();
// now render the javascript code to the terminal using the
// onedark theme and no background color
println!("{}", syntastica::render(
&highlights_js,
&mut TerminalRenderer::new(None),
syntastica_themes::one::dark(),
));
This is an alteration of the first example showing how to detect the language to use based on a file type. See that first example for explanations of the rest of the code.
syntastica
uses tft
for file types which
provides automatic detection.
use syntastica::{renderer::TerminalRenderer, language_set::{LanguageSet, SupportedLanguage}};
use syntastica_parsers::{Lang, LanguageSetImpl};
// detect the file type given a file's path and content.
// this requires a dependency on `tft`
let ft = tft::detect("main.rs", "");
let language_set = LanguageSetImpl::new();
let output = syntastica::highlight(
r#"fn main() { println!("42"); }"#,
// the `SupportedLanguage` trait provides a `for_file_type` function
// which returns an `Option<Lang>`
// make sure to have the trait in scope
Lang::for_file_type(ft).unwrap(),
&language_set,
&mut TerminalRenderer::new(None),
syntastica_themes::gruvbox::dark(),
)
.unwrap_or_else(|err| panic!("highlighting failed: {err}"));
println!("{output}");
This is an alteration of the first example showing
how to create a simple custom theme. See that first example for explanations of
the rest of the code, and see the documentation of the [theme!
] macro for more
information.
use syntastica::{renderer::TerminalRenderer, theme};
use syntastica_parsers::{Lang, LanguageSetImpl};
let theme = theme! {
// specify colors using hex literals
"purple": "#c678dd",
"blue": "#61afef",
"green": "#98c379",
// link to other keys using a `$` sign
"keyword": "$purple",
"function": "$blue",
// specify more styling options in curly braces
// (note that currently this order is required by the macro)
"string": {
color: None,
bg: None,
underline: false,
strikethrough: false,
italic: true,
bold: false,
link: "green",
},
};
let output = syntastica::highlight(
r#"fn main() { println!("42"); }"#,
Lang::Rust,
&LanguageSetImpl::new(),
&mut TerminalRenderer::new(None),
theme,
)
.unwrap_or_else(|err| panic!("highlighting failed: {err}"));
println!("{output}");
All crates in this workspace whose names start with syntastica
share the same
version. The typical semantic versioning rules are used across the public APIs
of all of these, except for
the ones listed as internal. The
other crates in this workspace have their own separate
versions.
Versions are specified as MAJOR.MINOR.PATCH
. As long as the MAJOR
version
specifier is still at 0
, changes to the MINOR
version may also be breaking
changes. The PATCH
part is only incremented if the public API stays exactly
the same.
The entire idea of this project started out as a way to use tree-sitter code
highlighting in a LaTeX project. While working with
@MikMuellerDev on
our paper on rush I
created a CLI app called lirstings
.
The initial sketch simply called out to the tree-sitter-cli
and converted the
output HTML to LaTeX code. However, not long after that I already implemented
some of the logic myself and made a
first public commit.
This version of lirstings
(called ts2tex
at the time) already laid out some
groundwork like
query pre-processing
and theming that is
still present in syntastica
today. Towards the end of our project we wanted to
use the same highlighting on our rush playground,
which would require lirstings
to become more general and support WebAssembly.
Work on that started in the
generalize branch just
enough to suffice for our needs at the time.
After the entire rush project was done and after taking a break for a while, I
started syntastica
with the intent to be a library from the ground up, and a
possible replacement for syntect
. The main
difference from lirstings
at the start was the parser collection(s), providing
a rigid set of parsers and queries for users. Over time syntastica
then grew
to the big project it is today.