Tokenize.jl

Tokenization for Julia source code

OTHER License

Stars
99
Committers
16
View Code on GitHub View on X
Ecosystems: The Julia Language

Bot releases are visible (Hide)

Tokenize.jl - v0.5.7

Published by julia-tagbot[bot] almost 5 years ago

v0.5.7 (2019-11-27)

Diff since v0.5.6

Merged pull requests:

  • disallow float w/ trailing . juxt w/ op (#152) (ZacLN)
Tokenize.jl - v0.5.6

Published by julia-tagbot[bot] about 5 years ago

v0.5.6 (2019-08-13)

Diff since v0.5.5

Closed issues:

  • number parsing error (#148)

Merged pull requests:

  • mark suffixed ops (needed for CSTParser) (#151) (ZacLN)
  • Fix outdated links in README (#150) (janEbert)
Tokenize.jl - v0.5.5

Published by julia-tagbot[bot] over 5 years ago

v0.5.5 (2019-07-16)

Diff since v0.5.4

Merged pull requests:

  • fix float parsing with trailing dotted operator (#149) (ZacLN)
  • add colon ops (#147) (ZacLN)
  • add hat suffix (#146) (ZacLN)
  • fix dotop starting position (#145) (ZacLN)
Tokenize.jl - v0.5.4

Published by julia-tagbot[bot] over 5 years ago

v0.5.4 (2019-06-11)

Diff since v0.5.3

Closed issues:

  • Dot operators have inconsistent location information (#139)
  • escaping within cmds (#135)
  • Fails to precompile on master (#134)

Merged pull requests:

  • update travis (#144) (KristofferC)
  • add Project file (#143) (KristofferC)
  • add errortoken for invalid numeric constants (#142) (ZacLN)
  • add new kw kind (#141) (ZacLN)
  • fix startpos for dotted ops (#140) (KristofferC)
  • update cmd lexing (#138) (ZacLN)
  • don't peek Chars (#137) (ZacLN)
Tokenize.jl -

Published by KristofferC almost 6 years ago

Tokenize.jl -

Published by KristofferC about 6 years ago

Tokenize.jl -

Published by KristofferC over 6 years ago

Tokenize.jl -

Published by KristofferC over 6 years ago

Tokenize.jl -

Published by KristofferC almost 7 years ago

Tokenize.jl -

Published by KristofferC about 7 years ago

Tokenize.jl -

Published by KristofferC about 7 years ago

Tokenize.jl -

Published by KristofferC about 7 years ago

Tokenize.jl -

Published by KristofferC about 7 years ago

Tokenize.jl -

Published by KristofferC about 7 years ago

Tokenize.jl -

Published by KristofferC about 7 years ago

Tokenize.jl -

Published by KristofferC over 7 years ago

Tokenize.jl -

Published by KristofferC over 7 years ago

Tokenize.jl -

Published by KristofferC over 7 years ago

Tokenize.jl -

Published by KristofferC over 7 years ago

Fixed two small bugs

Tokenize.jl -

Published by KristofferC over 7 years ago

Package Rankings
Top 4.74% on Juliahub.com
Badges
Extracted from project README
Build Status
Related Projects
EnglishText.jl

Utilities for English-language quirks in Julia

22 Dec 2015 17
Automa.jl

A julia code generator for regular expressions

01 Jan 2017 176
TreeSitter.jl

Julia bindings for tree-sitter.

15 Jul 2020 18
BitFlags.jl

BitFlag.jl provides an Enum-like type for bit flag option values.

09 Sep 2018 16
SymbolicUtils.jl

Symbolic expressions, rewriting and simplification

09 Feb 2020 502
JSONLines.jl

JSONLines reader/writer for Julia

13 Aug 2020 9
atom-language-julia

A Julia language support package for the Atom editor

12 Jun 2015 51
K.jl

K programming language dialect embedded in Julia

12 May 2022 18
JuliaSyntaxHighlighting.jl

21 Oct 2023 9
JuliaSyntax.jl

The Julia compiler frontend

22 Nov 2021 263
SimpleStringScanners.jl

Julia package for lexical scanning operations on a string

16 Mar 2024 0
JSONWebTokens.jl

Secure your Julia APIs with JWT.

16 Jul 2018 22
PackageAnalyzer.jl

23 Jan 2021 59
JET.jl

An experimental code analyzer for Julia. No need for additional type annotations.

26 Dec 2019 699
Typstry.jl

The Julia to Typst interface

29 Jul 2023 28