Caterpillar

Caterpillar is a programming language with a dual goal:

• Create an immediate connection between you and the code you're working on,
  using interactive programming.
• Bring this experience to many places: browsers, servers, desktops, phones,
  watches, microcontrollers; CPUs and GPUs.

Interactive programming is the practice of directly manipulating your running program, instead of going through an extended loop of re-compiling, re-starting, then navigating to where you can test your change.

The core premise of the project is the hypothesis, that interactive programming is underutilized. That using it as the central element the language and its tooling are built around, can make development more fun, productive, and intuitive.

Status

There's a very basic language, and a very basic game implemented in it. So far, not much thought has been given to using Caterpillar outside of this specific scenario. If you want to experiment with it, your best bet is to clone this repository and play around with what's already here (see Usage below).

Interactive programming is currently being implemented (it was already present in a previous prototype). Right now, the most basic cases work, but most changes will result in undefined behavior. Fixing this is an ongoing process.

The other part of the goal, bringing the interactive development experience to many places, is not being worked on yet. Right now, the focus is to create a basic solution for a single use case (game development) on a single platform (browsers).

You can keep up with the project by reading my daily thoughts, which include development updates.

Usage

As per Status above, not much thought has been given to using Caterpillar outside of this repository. If you want to play around with it, playing around with this repository is the way to do it:

1. Clone this repository.
2. Run cargo run (requires Rust to be installed).

That second step will start a build tool that should compile everything in here, start a development server, and eventually output a link.

Open that link in your browser. You should see the Snake game running, alongside the Caterpillar debugger.

You can modify the code of the game or any of the Caterpillar code itself. The builder should automatically re-compile every change, and the web page should reload, if necessary.

Please note that there currently is a substantial number of bugs and other basic problems.

Inspiration

Caterpillar draws inspiration from many sources. The following is an incomplete list:

• Inventing on Principle
• Tomorrow Corporation Tech Demo
• Stop Writing Dead Programs

Design

This section aims to document the decisions that have gone into the language design. Due to the current state of the project, this isn't (yet) a long list, nor is anything here going to be final.

For more information on future design directions, please follow my daily thoughts.

Developer experience is the priority, but not an absolute one

Caterpillar's core premise is to improve developer experience using interactive programming. But this does not make developer experience an absolute priority. It must always be weighed against other factors.

Where developer experience conflicts with performance, the decision will be made configurable, if practical. Where possible, designs are chosen that are good for both.

And while a focus on developer experience might decrease peak performance, it might improve performance in general. Simply because developers have the tools and spare resources to make their programs run fast.

The following daily thoughts provide more context: 2024-08-30, 2024-09-01, 2024-09-02, 2024-09-03, and 2024-09-04.

Experimentation first; conservative decisions later, as necessary

I want Caterpillar to be adopted. That could mean that I need to focus innovation where that provides the most benefit, and keep other aspects of the language conservative and familiar.

Before that becomes necessary, I want to experiment though. At least give the language to chance to be more than an incremental improvement over the status quo.

The following daily thoughts provide more context: 2024-06-18 and 2024-06-19.

Continued evolution over backwards compatibility

I'm not targeting a 1.0 release after which the language is expected to have few or no breaking changes. Right now, everything is early-stage and experimental anyway. But even long-term, I don't want to commit to backwards compatibility. The continued evolution of the language and the costs of ongoing maintenance will be prioritized instead.

As the language matures, there will be a growing focus on making any given upgrade easy. But each release might introduce changes that require updates to Caterpillar code. Where possible, users will be given ample time to make those changes, or they will be automated outright.

The following daily thoughts provide more context: 2024-05-28, 2024-05-29, 2024-05-31, 2024-06-01, 2024-06-02, 2024-06-03, and 2024-06-05.

Simplicity over familiarity and short-term convenience

Caterpillar strives to be simple, even where that is unfamiliar to most developers (because it's different to other languages), and even where that might be inconvenient in some situations.

Simplicity makes the language easier to understand. For the developer, but also for tooling, which can then provide a better experience to the developer. Unless this advantage is clearly outweighed by other factors, the simple design should be chosen.

An example of this is not supporting early returns. Functions in Caterpillar (or rather their branches, to be precise) have a single exit point, which can make code easier to follow for the developer, but specifically allows the debugger to do call stack reconstruction.

The following daily thoughts provide more context: 2024-08-25 and 2024-08-26.

Minimalism over convenience, for now

For the time being, to make iterating on the language easier, simplicity trumps convenience. This means that a small number of concepts is used to cover a wide range of use cases.

To give a concrete examples, pattern-matching functions are the single tool used to represent all control flow (both conditionals, via the pattern matching; and iteration, via recursion).

This is sometimes more verbose than more specific constructs, which makes the language less convenient to write and harder to read. For now, I consider this to be an acceptable trade-off.

Later on, it will make more and more sense to add syntax sugar, thereby increasing convenience, while under the hood still mapping the new syntax to simple primitives.

The following daily thoughts provide more context: 2024-07-24, 2024-08-08, 2024-08-10, 2024-08-12, 2024-08-13, 2024-08-14, 2024-08-15, 2024-08-16, 2024-08-17, 2024-08-18, 2024-08-20, 2024-08-21, and 2024-08-22.

Untyped now, statically typed later

Caterpillar is currently untyped. This means there is only a single data type, 32-bit words, and all values are represented using those. The developer often has to choose a specific operation, depending on what specific type they expect (add_s8 vs add_s32, for example).

This isn't a final decision. It just was the easiest place to start in. The goal is to make the language statically typed. I expect this to be a gradual process, where the compiler understands more and more about types over time, until it can select the correct operation itself (and the developer can just call add or +).

The following daily thoughts provide more context: 2024-07-16, 2024-07-17, and 2024-08-23.

Caterpillar code is embedded into a host

Caterpillar code does not execute I/O operations directly. Whenever it needs to do something that affects the outside world, it triggers an effect. That effect is executed by a platform-specific piece of code called the "host", which then passes the result back to Caterpillar.

At this point, this is just an implementation detail, but I hope to use this concept to realize a number of benefits in the future: Mainly to represent all I/O resources as values provided by the host, making Caterpillar code sandboxed by default, and allowing side effects to be tracked through those values, making all functions pure.

A lot of this is inspired by the "platform" concept in Roc.

The following daily thoughts provide more context: 2024-07-02, 2024-07-03, 2024-07-04, 2024-07-05, 2024-07-06, 2024-07-07

Postfix operators

The language uses postfix operators, like arg1 arg2 do_thing or 1 2 +, as opposed to prefix (do_thing(arg1, arg2), (+ 1 2)) or infix (1 + 2) operators.

To keep the language simple, I want to (at least initially) restrict it to one type of operator. I believe postfix operators are the best option under that constraint, due to their combination of simplicity, conciseness, and natural support for chaining operations. That comes at the cost of familiarity.

The following daily thoughts provide more context: 2024-05-03, 2024-05-04, 2024-05-05, 2024-05-06, 2024-05-07, 2024-05-08, 2024-05-09, 2024-05-10, and 2024-05-11.

Stack-based evaluation, but not a stack-based language

Caterpillar, featuring postfix operators, has a stack-based evaluation model. But it is not a stack-based language. There is no single data stack that is used to pass arguments and return values between functions.

Instead, Caterpillar uses a much more conventional model, with a regular stack and explicit function arguments. Each operand stack is local to a (function) scope.

This approach is less error-prone, but also less flexible and more verbose. It seems to make sense right now, but as the language grows other features that make it less error-prone (like static typing and better tooling), this decision can be revisited.

The following daily thoughts provide more context: 2024-05-10, 2024-05-11, 2024-06-20, 2024-06-21, 2024-06-22, 2024-06-23, 2024-06-24, and 2024-06-25.

Designed to be used with tooling

Caterpillar is designed to be used with tooling and makes various trade-off that benefit this intended use case, at the cost of other cases where tooling is not available.

The following daily thoughts provide more context: 2024-07-21, 2024-07-22, and 2024-07-23.

Compilation doesn't stop on errors

The compiler doesn't stop, when encountering an error. Instead it encodes the error into the representation it is currently generating, still creating an executable result in the end. If code generated from one of the build errors is hit, this results in a runtime panic.

This has advantages, like allowing you to run tests or debug a piece of code, even while something else doesn't currently typecheck, for example. This could make day-to-day development easier. It also makes sure that the enriched version of the source code that the debugger displays, is always available.

The following daily thoughts provide more context: 2024-09-05 and 2024-09-06.

Acknowledgements

I'd like to thank Martin Dederer for suggesting the name!

License

This project is open source, licensed under the terms of the Zero Clause BSD License (0BSD, for short). This basically means you can do anything with it, without any restrictions, but you can't hold the authors liable for problems.

See LICENSE.md for full details.