Builds stenography lessons for Typey Type for Stenographers
MIT License
This repo houses the Typey Type Command Line Interface (CLI) to build stenography (steno) lessons.
The Typey Type CLI is used to build Typey Type stenography lessons by using source data in this repo to produce lessons and dictionaries in the Typey Type data repo.
You can support Di’s efforts on Patreon. A monthly donation helps Di build more lessons and features to help you fast-track your steno progress.
Note: I've tested this on macOS and Linux. It mostly uses POSIX-compatible tools. Please let me know what other systems you run it successfully on.
Note: both the source and output lesson data is committed in git so that it's easy to git diff
and compare the impact of dictionary changes across all lessons.
Requirements:
This project includes a Git submodule for steno dictionaries. If you want to clone this repository as well as its submodules, you can use the --recursive
parameter:
git clone --recursive [email protected]:didoesdigital/typey-type-cli.git
Alternatively, if you've already cloned the repository without the --recursive
parameter, you can load its submodules using submodule update
:
git submodule update --init --recursive
Change directory into the cloned repository:
cd typey-type-cli
Yarn install the dependencies:
yarn install
You'll also need the Typey Type data repo somewhere nearby to produce lessons, which you can clone by itself but I recommend setting up the Typey Type app with the data repo included as a submodule so you have everything you might want in the right place:
cd ../
git clone https://github.com/didoesdigital/typey-type.git
cd typey-type
git submodule update --init --recursive
If you want to install the Typey Type app dependencies:
yarn install
cd typey-type-cli
git pull
yarn
cd ../
cd typey-type
git pull
yarn
git submodule update
For the following commands, make sure you're in the right repo:
cd typey-type-cli
To build the lessons, lesson index, and dictionaries, and compile, test, and lint everything:
time make -j 8 build-everything
-j 8
builds as many jobs in parallel as possible (i.e. for 4 CPUs, use -j 4
).time
times the whole process including making prerequisitesWhen you're happy with the results of build-everything
, sync the misstrokes.json
and emoji.json
files to the steno-dictionaries
submodule and copy all the finalised lessons into the Typey Type data repo at the path that you need to specify ahead of publishing to production:
make sync-typey-type-data TYPEY_TYPE_DATA_REPO_PATH="~/projects/typey-type/typey-type-data/"
Warning You need to change the value of TYPEY_TYPE_DATA_REPO_PATH to the path where you have cloned the
typey-type-data
repo with a trailing slash.
For testing, try an alternative test path like this:
time make -j 8 sync-typey-type-data TYPEY_TYPE_DATA_REPO_PATH="~/projects/test-typey-type/test-typey-type-data/"
To build all the lessons one at time:
make lessons
Note To create a new lesson, see the new lesson section below.
To build the lesson index:
make lesson-index
To rebuild a specific lesson and always build it, even if dependencies have not changed:
time make --always-make faux-typey-type-data/lessons/fundamentals/numbers/lesson.txt
To build the Typey Type dictionary with one preferred entry for every word:
time make typey-type-dict
To copy source dictionaries (Plover, Jade's phrasing dictionary, Di's steno-dictionaries from the submodule that aren't auto-generated, etc.) to dictionaries target:
time make copy-dictionaries
To build recommendations courses:
time make build-recommendations-courses
To generate emoji dictionary as emoji.json
:
time make emoji-dict
To generate modified emoji strategy in order to build emoji dictionary:
time make emoji-modified-strategy
To collect misstrokes into misstrokes.json
:
time make collect-misstrokes
To check for outlines that are duplicated across Typey Type dictionaries:
yarn dev help check-duplicate-outlines
yarn dev check-duplicate-outlines
yarn dev check-duplicate-outlines --unique-translations
While you can run CLI commands individually, for lessons, just use make
. If you want to run a command though, do this:
yarn dev <command>
yarn dev help
For example, yarn dev build-emoji-dictionary --target didoesdigital/steno-dictionaries/dictionaries/emoji.json
.
For interactive debugging, here are some options:
yarn build && node --inspect-brk ./build/index.js build-lesson --target=faux-typey-type-data/lessons/drills/homophones/lesson.txt --metadata=faux-typey-type-data/lesson-source-data/drills/homophones/meta.json
yarn build && node --inspect-brk ./build/index.js build-typey-type-dictionary --target=faux-typey-type-data/lesson-intermediate-data/typey-type-standard-dict-set-combined.json --intermediate
yarn build && node --inspect-brk ./build/index.js build-lesson-index
For interactive debugging of tests, edit the tsconfig.json
file to compile tests and edit the jest.config.js
file to include the build
directory:
yarn build && node --inspect-brk ./node_modules/jest/bin/jest.js --runInBand build/shared/utils/transformingDictionaries/rankOutlines/rankOutlines.test.js
time make lint-and-test
… or:
yarn lint:check
yarn lint:fix
yarn test
yarn test:watch
yarn dev validate-lessons
This cleans the build
directory, compiles all the TypeScript to JavaScript and makes the ./build/index.js
executable:
yarn build
To remove all intermediate data (which is git ignored) and target data (which is tracked in git) for lessons and dictionaries, and remove the build
directory where compiled JavaScript is built:
make clean
These terms are not necessarily technically accurate, but are used frequently throughout the code. "Word" is often used where something like "phrase" might be more precise but "word" is more representative of typical content.
Word is often used to refer to an item of material in a lesson. It is often a plain, single word, but it could also be something like pick-axe
, sister-in-law
, bread and butter
, React.Component
, <img src="
, or I really think the
.
Outline is generally used to refer to all the steno strokes needed to write a "word". For example, TKOPB/KEU KW-BG
to write "donkey," (with a comma). It is also used to refer to the outline as it appears in a dictionary. For example, TKOPB/KEU
for "donkey" and KW-BG
for {,}
(a comma with dictionary formatting symbols).
Translation is generally used to refer to the word that will be output by translating a dictionary outline to text. For example, donkey,
. It is also used to refer to that part of the dictionary entry, including original dictionary formatting symbols, for example, donkey{,}
.
Entry or Dict Entry is loosely used to refer to a combination of outline and word or outline and translation. Lookup Entry refers to a word and its array of possible outlines.
Lookup dict refers to a word-first lookup dictionary, usually a Map, containing lookup entries with an array of possible outlines.
Steno dict refers to a steno dictionary that could be used by a steno engine, usually an object, containing outlines and their translations.
To create a new lesson, you need to create the new lesson's source files outlined in the next section, add the lesson to the index (lesson-source-data/sourceLessonIndex.json
), then run the make lessons
command to build the new lesson using those new source files.
You can also add lessons directly to the community’s lessons spreadsheet. See Typey Type lessons to learn more.
Use material that is in the public domain or otherwise legal to use in Typey Type. Check out the Wikipedia List of countries' copyright lengths.
Aim for lessons that are around 50–300 words long. Short lessons less than 10 words often result in heavily skewed speed scores and are over too quickly. Longer lessons drag on, and not everyone is aware or comfortable to hit "Stop" before reaching the end.
Split the lesson up into one word per line. In other words, split on spaces. For example, split the sentence "Only a little--not very."
like this:
"Only
a
little--not
very."
One exception to that suggestion is phrasing briefs. If you intend to write a pair of words or a few words in a single stroke, keep them on one line. For example, you might split the sentence I struck a match a little bit on the wall; probably it came a little too close to the thatched roof.
like this:
I
struck
a
match
a little bit
on the
wall;
probably
it
came
a little
too
close
to the
thatched
roof.
For lessons specifically to practice affixes, try to make the entire lesson use the same affixes (all prefixes or all suffixes), preserve the dictionary formatting in the words list, and use the replaceAffixCurlies
presentation option in the meta file. For example:
{^s about}
{^s is}
{^er and}
{^ing and}
{^s and the}
Check for non-ASCII characters that might be difficult to type and consider possible amendments. For example, lots of Project Gutenberg material uses back tick (grave) characters as quotation marks, which you might change to actual quotation marks. Here's a regular expression to quickly highlight non-ASCII and non-printable control characters: [^\x20-\x7f]
The lesson source files live in lesson-source-data
in a category such as Fundamentals
, Drills
, Collections
or Stories
and, for Collections and Stories, a subcategory.
Each lesson must have:
meta.json
(more details below)A lesson may optionally have:
lesson-overview.html
(more details below)words.txt
(more details below)In rare cases, you may also create a lesson-hints dictionary, but that's usually not necessary. For examples of that, see the meta files for one of these lessons: numbers, steno party tricks, markdown, or medical-prefixes. It's a last resort, try to avoid it.
The source lesson metadata file is called meta.json
and can include:
["punctuation.json"]
to use for overriding vocabulary outlines (e.g. 1-R
for "I") and expanding vocabulary words e.g. "sankey" from data-viz.json
{}
from affix entries like {pre^}
:
replaceAffixCurlies
replaceGlueCurlies
replaceArrowNavigation
replaceCapitalisationFormatting
replaceSuppressedSpaces
["pos", "sol", "spas", "pros"]
"slug": "ux-vocabulary"
defines ux-vocabulary.json
in lessons/collections/user-experience/ux-vocabulary/ux-vocabulary.json
.The source lesson overview file is called lesson-overview.html
. If this exists, Typey Type will show an "Overview" link to describe the lesson's concepts. To add a new overview to an existing lesson, you may need to run make --always-make lesson-index
to re-build the lesson index to link to the new overview and make it show up in Typey Type.
The source lesson index file contains an ordered list of the lesson paths.
didoesdigital
— Typey Type's main dictionaries based on Plover plus Di's own dictionariesindividual
— dictionaries made by individuals such as Jade's phrasing dictionary or Emily's symbols dictionaryplover
lesson-hints
includes dictionaries that only exist to make lessons work better, such as numbers-lesson.json
, that aren't intended for steno usage.Dictionaries referenced by recommendedDictionarySet
may be in any of those places.
The source dictionaries index file contains a list of the source dictionaries to copy straight over to target data directories.
Intermediate data is produced from source files but before creating target files. It's necessary to create the target files, but can be safely deleted at any time and it will be recreated by the source files.
typey-type.json
dictionary.To scaffold some files to add a new rule for lessons that are generated according to rules:
yarn dev add-new-rule --rule <ruleName>
yarn dev add-new-rule --rule testRule # example
See the CONTRIBUTING guidelines.
This project and everyone participating in it is governed by the Code of Conduct. By participating, you are expected to uphold this code. Please report unacceptable behaviour to [email protected].
src/**
) is licensed under MIT License, as shown in LICENSE.md.
stemoji.js
script from stemoji to this Typey Type CLI repo under MIT License.faux-typey-type-data/dictionaries/plover/
and faux-typey-type-data/dictionary-source-data/plover
faux-typey-type-data/dictionaries/plover/
and faux-typey-type-data/dictionary-source-data/plover
faux-typey-type-data/dictionaries/plover/
and faux-typey-type-data/dictionary-source-data/plover
faux-typey-type-data/
) are licensed under GPL-2.0 as shown in LICENSE_DATA.vendor/emoji_strategy_c8900a0.json
renamed from emoji_strategy.json
from the original emojione project.vendor/emoji_strategy.json
from the updated emoji-toolkit project.vendor/emoji_strategy_reduced_to_c8900a0_chars.json
derived from vendor/emoji_strategy.json
and vendor/emoji_strategy_c8900a0.json
.Typey Type was created by Di.