partykit

#346 58fe7e2 Thanks @mellson! - pass custom port to dev from CLI correctly

#345 9990c2c Thanks @threepointone! - add a lint rule for not using deprecated APIs

#347 3d05975 Thanks @threepointone! - use the right config/options in dev

We weren't reading config correctly in dev, this fixes it

#348 08a1caa Thanks @jevakallio! - Fix partykit login with new bundled imports

#339 0206dd5 Thanks @threepointone! - partykit: bundle dependencies

This brings back packaging of dependencies for partykit (and introduces it for create-partykit). We had previously disabled it for deps because of esm/cjs nonsense, but we've now figured out how to make them play decently together.

I had to fork ink-testing-library because of https://github.com/vadimdemedes/ink-testing-library/pull/23, I'll remove it once that PR is merged.

This also updates ink to 4.4.1, which fixes our previous issue where it was exiting early.

#339 0206dd5 Thanks @threepointone! - partykit: bundle dependencies

This brings back packaging of dependencies for partykit (and introduces it for create-partykit). We had previously disabled it for deps because of esm/cjs nonsense, but we've now figured out how to make them play decently together.

I had to fork ink-testing-library because of https://github.com/vadimdemedes/ink-testing-library/pull/23, I'll remove it once that PR is merged.

This also updates ink to 4.4.1, which fixes our previous issue where it was exiting early.

#329 9a55f69 Thanks @threepointone! - change naming of /server exports

This changes the naming of our exported types from partykit/server. The motivation here was aesthetic/simplification. Most of the exports los the Party prefix; i.e. PartyServer becomes Server, PartyConnection becomes Connection, and so on. If you look at the class example, this doesn't drastically change the code; the import becomes a lot shorter (import type * as Party from 'partykit/server' gets all the required types) and required types are taken by Party.*.

I also did a run on all the docs/templates and the blog post, but lemme know if I missed anything!

#333 dc324d9 Thanks @threepointone! - fix: init doesn't install the right versions of packages

This fix should pick up the latest versions of partykit /partysocket when installing. We were using the previous beta logic when every package was on the same version. This patch picks up the latest versions of both and uses it.

(also sneaking in a quick types fix for usePartySocket)

#334 9e46880 Thanks @jevakallio! - Fix create-partykit crash issue by downgrading "ink" dependency to last known good version

#323 2ca369a Thanks @threepointone! - useWebSocket

a hook version of our reconnecting websocket

#274 30fabc9 Thanks @threepointone! - don't compile node builtins that the platform supports

We support some node builtins, so we shouldn't try to compile them into the bundle.

#307 ad3ea55 Thanks @threepointone! - fix: route sub party requests correctly

sigh. we were always fetching from the main party. This fix routest he request to the correct sub party.

#296 57f71c9 Thanks @jevakallio! - Add partykit token generate to create API tokens

#305 8e087cc Thanks @jevakallio! - Fix edge case bugs in partykit init

#299 2209de9 Thanks @threepointone! - fix: call oBC/oBR on sub parties

We hadn't wired up onBeforeConnect/onBeforeRequest for multi parties. To fix this, I did a refactor where every request goes through a common codepath now. Additionally, this also means that /parties/main/:id is equivalent to party/:id. I also massaged out some differences between the platform/cli facade.

#300 6bd4609 Thanks @threepointone! - add new lines when hitting return in dev

fairly common behaviour with other CLIs, just adding it to ours as well.

#293 53eb0b5 Thanks @threepointone! - partykit init: add partykit to existing projects

The common usecase is to add partykit to an existing project, usually running on another stack/provider. This adds an init command that simply adds dependencies, a partykit.json file, and an entry point. If it's not run inside an existing project, it defers to running npm create partykit instead.

#303 6b773ae Thanks @jevakallio! - Improve type documentation

#251 049bcac Thanks @threepointone! - small tweaks to init

• replace process.env.PARTYKIT_HOST with just PARTYKIT_HOST
• add a tsconfig.json
• add partykit to devDependencies in init
• strip protocol from host (partysocket, y-partykit)

#27 1f384a0 Thanks @threepointone! - feat: y-partykit persistence support

This adds persistence support for y-partykit (based on the work at y-workers). It... just works haha. Pass persist:true to onConnect to store it. This still bumps into DO's 128 kb limit, but we'll fix that asap.

#62 1410f06 Thanks @mellson! - Node 16 compatibility.

#18 f255e12 Thanks @threepointone! - y-partykit: callback/persistence configuration

This patch now properly forks y-websocket, and beings in functionality for passing configuration for callbacks/persistence (that was previously using process.env vars). We also fix builds/type generation for the package.

#17 bf2e66f Thanks @threepointone! - y-partykit: fix builds, usage

This patch fixes y-partykit so it can actually be published and used in the platform.

#228 1d259bc Thanks @threepointone! - fix: properly sort chunking in y-partykit for big values

So, out persistence layer has a 128kb limit on values, but y-partykit should/could be used for documents bigger than that. So we implemented a chunking strategy that break up values across multiple keys. When making keys for these, we didn't pad the generated indexes, which meant that for sufficiently large values, we might have assembled them back in the wrong order (because lexicographical sorting). This is a fix for that. It's a breaking change, but oddly I haven't heard from people who've faced the problem at all.

#28 f42b1e0 Thanks @threepointone! - better types for storage keys in y-partykit

#59 5edde06 Thanks @threepointone! - remove y-partykit's onCommand config

now that we have onRequest, we don't need to set up a command channel within y-partykit itself, so let's remove the code for it.

#192 18035c2 Thanks @threepointone! - y-partykit: better default for gc

It's better to default to gc: true with partykit, so memory is better used in partykit servers. We should also ensure gc and persist are never used together. So this PR adds a little logic to find a better default, and ensuring the 2 never clash.

#34 0fdf7a3 Thanks @threepointone! - y-partykit: remove vendored libs

We'd previously vendored the libs used by y-partykit (yjs, lib0, etc) to workaround a bug in edge-runtime https://github.com/vercel/edge-runtime/issues/243, but it makes using other libs that include those libs difficult. So instead this patch removes the vendoring, and applies the other workaround (which is to set minify:true). The tradeoff for this workaround is that any "dev mode" code (i.e. code wrapped with if (process.env.NODE_ENV !== 'production')) will be removed. This is temporary and we'll remove it once the bug is fixed by edge-runtime.

#88 4d448bb Thanks @threepointone! - y-partykit: wait for bindState to finish before connection fulfills

#211 fffe721 Thanks @threepointone! - update dependencies

#105 f576783 Thanks @threepointone! - fully fork y-websocket

This fully forks y-websocket for y-partykit. Previously, we were still extending the client provider, but this now inlines that implementation so we can make our own changes/fixes as required.

#33 37986a8 Thanks @threepointone! - y-partykit: remove document size limits

By chunking values, we can workaround DO's 128 kb value size limit. This patch implements that, and adds a couple of tests too.

#182 a7370aa Thanks @iojcde! - fix: change type of conn to PartykitConenection

#226 9a4b594 Thanks @threepointone! - configure party in y-partykit

This adds a party parameter to the options bag for YPartyKitProvider

#22 825bb02 Thanks @threepointone! - y-partykit: export YPartyKitProvider

#54 244de6a Thanks @threepointone! - fix: don't crash in some secure contexts

This uses a weaker implementation to generate conneciton IDs if crypto.randomUUID() isn't available in the browser (re: https://github.com/WICG/uuid/issues/23) Fixes https://github.com/partykit/partykit/issues/53

#181 7bdfcf2 Thanks @iojcde! - feat: add support for headers in callback options

#16 13c4acf Thanks @threepointone! - y-partykit: first cut

This lands a first implementation of yjs for partykit. It uses patch-package to create a modified build of y-websocket, and re-exports it for usage in a partykit server. This also lands a small example of lexical+y-partykit.

#285 53ba5ac Thanks @jevakallio! - Make YProvider work in workers runtime

#207 ac76dbc Thanks @threepointone! - expose connection id on YpartyKitProvider

available as provider.id

#31 984a9ec Thanks @threepointone! - [y-partykit] fix: write state to storage on connection close

we'd forgotten to implement writeState() so data wasn't being saved when everyone left a room. the fix is to implement, seems fine now.

#30 323fce1 Thanks @threepointone! - chore: update esbuild, edge-runtime, and co.

of note, esbuild had breaking changes, had to rewrite the rebuild logic.

#35 7eb3b36 Thanks @threepointone! - y-partykit (feat): onCommand as a server configuration option

y-partykit currently assumes all messages on the wire are yjs messages (usually all binary data), but we have usecases where we want to send arbitrary commands to the server and act on them (usually still with the context of a doc). So now y-partykit accepts a handler for all string messages that we're calling 'commands' - onCommand(string, Y.Doc).

Additionally, partysocket now also exports it's base websocket class as partysocket/ws

#251 049bcac Thanks @threepointone! - small tweaks to init

• replace process.env.PARTYKIT_HOST with just PARTYKIT_HOST
• add a tsconfig.json
• add partykit to devDependencies in init
• strip protocol from host (partysocket, y-partykit)

#208 de4e507 Thanks @jevakallio! - Allow connecting directly to a multiparty using a party field on partysocket

#215 cc88615 Thanks @threepointone! - fix: make PartySocket work inside cloudflare/partykit

The implementation of ReconnectingWebSocket tests whether a WebSocket class is valid by the presence of a static field that's not available on CF's/PK's WebSocket class. It's a bit much anyway, so we just remove the test.

#239 e57200d Thanks @threepointone! - partysocket: warn if EventTarget is not available

#71 4273c76 Thanks @mellson! - Fix included files in published build.

#233 8981945 Thanks @turkerdev! - partysocket: check if crypto is defined

#211 fffe721 Thanks @threepointone! - update dependencies

#191 39cf5ce Thanks @jevakallio! - Improve PartySocket types and React hooks API:

• Add websocket lifecycle event handlers to usePartyKit options to reduce need for effects in userland
• Allow usePartySocket to provide startClosed option to initialize without opening connection
• Fix types for PartySocket#removeEventListener

#220 e7e14a7 Thanks @threepointone! - expose .id on PartySocket

#21 f01ad2b Thanks @threepointone! - partysocket

partysocket is a for of reconnecting-websocket (which appears to be abandoned), that adds a few missing features and fixes a few bugs.

#54 244de6a Thanks @threepointone! - fix: don't crash in some secure contexts

This uses a weaker implementation to generate conneciton IDs if crypto.randomUUID() isn't available in the browser (re: https://github.com/WICG/uuid/issues/23) Fixes https://github.com/partykit/partykit/issues/53

#69 9c3418d Thanks @mellson! - Makes partysocket compatible with both esm and cjs.

#160 2753bc8 Thanks @threepointone! - fix: configure _pk with PartySocket with options.id

This adds an id config for new PartySocket() that's uses as the value of _pk, that becomes the connection id in the backend.

Fixes https://github.com/partykit/partykit/issues/159

#183 acca510 Thanks @threepointone! - partysocket: export ReconnectingWebSocket class from root

#35 7eb3b36 Thanks @threepointone! - y-partykit (feat): onCommand as a server configuration option

y-partykit currently assumes all messages on the wire are yjs messages (usually all binary data), but we have usecases where we want to send arbitrary commands to the server and act on them (usually still with the context of a doc). So now y-partykit accepts a handler for all string messages that we're calling 'commands' - onCommand(string, Y.Doc).

Additionally, partysocket now also exports it's base websocket class as partysocket/ws

#63 2910f99 Thanks @mellson! - PartySocket commonjs compatibility and React hook test.

#241 a5b3dda Thanks @threepointone! - generate a json schema and publish it

#251 049bcac Thanks @threepointone! - small tweaks to init

• replace process.env.PARTYKIT_HOST with just PARTYKIT_HOST
• add a tsconfig.json
• add partykit to devDependencies in init
• strip protocol from host (partysocket, y-partykit)

#270 db40a08 Thanks @threepointone! - fix: serve newer assets on changes

We had a bug where newly created assets weren't being served on dev, this should fix that.

#246 30e4dbd Thanks @threepointone! - feat: define a default process.env.PARTYKIT_HOST

We can actually make a pretty good guess to what the host is for both dev and deploy, so let's define it when compiling, and remove a bunch of boilerplate in the process.

#107 e0135cd Thanks @threepointone! - fix: don't receive messages until onConnect completes

#201 b993cb5 Thanks @threepointone! - fix: correct room id/parties in hibernation mode

We weren't hydrating the room id and parties correctly in hibernation mode, this patch fixes that.

#8 208c67b Thanks @threepointone! - update esbuild (and misc).

#196 7aeeff6 Thanks @threepointone! - Expose parties inside onBeforeConnect and onBeforeRequest.

#10 6864be8 Thanks @threepointone! - Error when cli is used in older versions of node

We now throw an error when the CLI is used on node < v18.12.1 (We could probably make this work on node 16, but we'll see in the future of required)

#104 ede48ab Thanks @threepointone! - npx partykit env remove to delete all env vars for a project at once

#19 167a677 Thanks @threepointone! - lazy load heavy deps in cli

This imports modules like edge-runtime, esbuild, etc lazily / on demand in the cli, which should make the cli startup marginally faster.

Additionally, we're modifying prereleases to use changeset --snapshot instead of our custom thing

#145 66fb515 Thanks @threepointone! - feat: onAlarm()

This exposes DO's alarm functionality. We already added the ability to set/remove alarms when we moved to workerd, this adds the api onAlarm()

#62 1410f06 Thanks @mellson! - Node 16 compatibility.

#194 87a3c46 Thanks @threepointone! - feat: unstable_onFetch as a catch-all for other requests

This introduces unstable_onFetch(req, lobby, ctx) as a catch-all for requests that don't match /party/:id or /parties/:party/:id.

#156 46d871d Thanks @threepointone! - feat: enable multiparty support when deploying

We'd previously landed multiparty support in https://github.com/partykit/partykit/pull/151, this enables us to publish as well.

#283 3d3bddb Thanks @threepointone! - unload listeners on mf server / fix live reload console logging

We were adding a bunch of listeners on the miniflare server that we weren't cleaning up, and the dependency array was wrong. This PR fixes those up, while also conveniently fixing the issue where console.log wasn't working after the first compile failure.

#224 5ed8917 Thanks @threepointone! - enable inspector in non-tests 😖

In https://github.com/partykit/partykit/pull/222, we didn't actually enble the inspector outside of tests. Oops. This fixes that.

#261 6c8504d Thanks @jevakallio! - # New class-based PartyServer API

PartyKit now supports a new ES6 Class-based API.

TL;DR;

Before:

import type {
  PartyKitServer,
  PartyKitRoom,
  PartyKitConnection,
} from "partykit/server";

export default {
  onBeforeConnect(request: Request) {
    request.headers.set("X-User", getUser(request.headers.Authorization));
    return request;
  },
  onConnect(connection: PartyKitConnection, room: PartyKitRoom) {
    room.broadcast(`Someone joined room ${room.id}!`);
  },
} satisfies PartyKitServer;

After:

import type {
  Party,
  PartyConnection,
  PartyRequest,
  PartyServer,
  PartyWorker,
} from "partykit/server";

export default class MyParty implements PartyServer {
  constructor(public party: Party) {}

  static onBeforeConnect(request: PartyRequest) {
    request.headers.set("X-User", getUser(request.headers.Authorization));
    return request;
  }

  onConnect(connection: PartyConnection) {
    this.party.broadcast(`Someone joined room ${this.party.id}!`);
  }
}

MyParty satisfies PartyWorker;

The old API remains supported for the time being, but we highly recommend starting all new projects with the new API, as the old API may be deprecated in the future.

New Class-based API

Previously, you created PartyKit servers by exporting an plain object that defines handlers for different events that occur in a room. This was nice and terse, but we found a lot of room for improvement:

• This API didn't accurately convey PartyKit's mental model of each party being a stateful object (backed by a CloudFlare Durable Object)
• It was hard to manage derived state safely
• It was hard to distinguish between code that runs in the Edge worker near the user (e.g.onBeforeConnect) and the Party worker that runs where the room was first created (e.g. onConnect).
• The naming of different concepts (parties, rooms, etc) was ambiguous

With this feedback in mind, we've redesigned PartyKit's primary server interface with a new ES6 Class-based API.

The following code sample demonstrates the new API, with comments describing what's changing:

import type {
  Party,
  PartyConnection,
  PartyRequest,
  PartyServer,
  PartyWorker,
  PartyServerOptions,
  PartyConnectionContext,
} from "partykit/server";

// PartyKit servers now implement PartyServer interface
export default class Main implements PartyServer {
  // onBefore* handlers that run in the worker nearest the user are now
  // explicitly marked static, because they have no access to Party state
  static async onBeforeRequest(request: PartyRequest) {}
  static async onBeforeConnect(request: PartyRequest) {}

  // onFetch is now stable. No more unstable_onFetch
  static async onFetch(req: PartyRequest) {}

  // Opting into hibernation is now an explicit option
  readonly options: PartyServerOptions = {
    hibernate: true,
  };

  // Servers can now keep state in class instance variables
  messages: string[] = [];

  // PartyServer receives the Party (previous PartyKitRoom) as a constructor argument
  // instead of receiving the `room` argument in each method.
  readonly party: Party;
  constructor(party: Party) {
    this.party = party;
  }

  // There's now a new lifecycle method `onStart` which fires before first connection
  // or request to the room. You can use this to load data from storage and perform other
  // asynchronous initialization. The Party will wait until `onStart` completes before
  // processing any connections or requests.
  async onStart() {
    this.messages =
      (await this.party.storage.get<string[]>("messages")) ?? [];
  }

  // You can now tag connections, and retrieve tagged connections using Party.getConnections()
  getConnectionTags(
    connection: PartyConnection,
    ctx: PartyConnectionContext
  ) {
    return [ctx.request.cf?.country as string];
  }

  // onConnect, onRequest, onAlarm no longer receive the room argument.
  async onRequest(req: PartyRequest) {}
  async onAlarm() {}
  async onConnect(connection: PartyConnection, ctx: PartyConnectionContext) {
    // You can now read the room state from `this.party` instead.
    this.party.broadcast(JSON.stringify({ messages: this.messages }));

    const country = ctx.request.cf?.country as string;

    // room.connections is now called room.getConnections(tag?)
    // that receives an optional tag argument to filter connections
    for (const compatriot of this.party.getConnections(country)) {
      compatriot.send(
        JSON.stringify({
          message: `${connection.id} is also from ${country}!`,
        })
      );
    }
  }

  // Previously onMessage, onError, onClose were only called for hibernating parties.
  // They're now available for all parties, so you no longer need to manually
  // manage event handlers in onConnect!
  async onError(ws: PartyConnection, err: Error) {}
  async onClose(ws: PartyConnection) {}
  async onMessage(message: string, connection: PartyConnection) {}
}

// Optional: Typecheck the static methods with a `satisfies` statement.
Main satisfies PartyWorker;

In addition to moving from an object syntax to class syntax, we've introduced multiple improvements to developer ergonomics:

onStart handler

PartyKit servers are convenient, because they're stateful, but you still need to make sure to store the state into room storage for when the room restarts due to inactivity or exceeding its maximum CPU time.

There's now a new lifecycle method onStart which fires before first connection or request to the room. You can use this to load data from storage and perform other asynchronous initialization, such as retrieving data or configuration from other services or databases.

  // Servers can now keep state in class instance variables
  messages: string[] = [];
  async onStart() {
    this.messages = (await this.party.storage.get<string[]>("messages")) ?? [];
  }

The Party will wait until onStart completes before processing any connections or requests to the party.

Better support for Hibernatable WebSockets

In order to support scaling parties to tens of thousands of concurrent connections, PartyKit supports CloudFlare Durable Object Hibernatable WebSockets API.

When opting into hibernation, the server goes to sleep between messages, and only comes alive when there is work to be performed, making it more cost-effective and easier to scale. Hibernation comes with tradeoffs: for certain type of applications, you may want to keep the server in memory for longer between requests.

Explicit hibernation opt-in

Previously, we automatically opted you into "hibernation mode" when you defined an onMessage handler in your server. Now, you can define an options.hibernate field, which defaults to false:

  readonly options: PartyServerOptions = {
    hibernate: true,
  };

More convenient message handling

Previously the so-called "hibernation-mode handlers" onMessage, onError, onClose were only called when you opted into hibernation. They're now available for all servers, so you no longer need to manually manage event handlers in onConnect:

  async onMessage(message: string, connection: PartyConnection) {}
  async onError(ws: PartyConnection, err: Error) {}
  async onClose(ws: PartyConnection) {}

No more manually registering event handlers in onConnect! (Unless you want to, of course.)

Better connection management

Previously, PartyKit managed connections in memory in a big Map, available in room.connections. This meant that every time your server woke up from hibernation, PartyKit needed to rehydrate all connections, which was both slow and expensive.

Now, instead, you can access connections on Party as follows:

// get connection by id (previously room.connections.get(id))
const connection = this.party.getConnection(id);
// iterate over all connection (previously room.connections.values())
for (const c of this.party.getConnections()) {
}

Tagged connections

You can set additional metadata on connections by returning them from a getConnectionTags callback on PartyServer:

  getConnectionTags(connection: PartyConnection) {
    return ["some tag"];
  }

You can then filter connections by tag, removing the need to wake up hibernated sockets unnecessarily:

for (const c of this.party.getConnections("some tag")) {
}

Naming changes

While designing the new API, we also wanted to be thoughtful about naming.

The biggest problem we wanted to solve was the distinction between "parties" and "rooms", which was confusing for many. From now on, we'll refer to PartyKit server instances as "parties", and will no longer use the "room" terminology.

To reflect this, we made the following names in our TypeScript types:

• PartyKitRoom is now Party, and refers to a single server instance (i.e. Durable Object)
• PartyServer refers to the instance code definition of the server (i.e. Durable Object)
• PartyWorker refers to the static code definition of the server which runs in a separate worker before connecting to the Party.
• PartyKit*-prefixed types are now shortened to Party* by dropping the "Kit". It's cleaner.
• room.parties ➡️ party.context.parties — Represents the taxonomy and relationship between parties more clearly.

The old names are deprecated, but will continue to work. The deprecated names are decorated with JSDoc @deprecated pragmas, so you can find the types you need to rename.

Breaking changes

There are no breaking runtime changes. Existing PartyKit projects should continue working as expected.

Incoming request types

However, while making changes to our TypeScript types, we discovered that the type definitions for on(Before)Request and on(Before)Connect were incorrectly defined to receive a Fetch API Request type, whereas at runtime they would always receive a PartyKitRequest type, which is an instance of CloudFlare Workers -specific Request object.

We decided to fix this issue, which means you'll need to make the following change:

+ import { PartyRequest } from "partykit/server";

export default {
-  onBeforeRequest(request: Request) {
+  onBeforeRequest(request: PartyRequest) {
    return new Request(request.url, { headers: { "x-foo": "bar" } });
  }
}

Note that you can still return a normal Request -- only the input type has changed.

#58 c4bc1f2 Thanks @threepointone! - feat: onRequest/onBeforeRequest

This feature allows you to optionally configure onBeforeRequest/onRequest on a room. just like oBC/oC, the former runs in a worker closest to you, and the later runs in the room. This also makes defining onConnect fully optional. This should open the door to some interesting integrations.

#113 f176d8d Thanks @threepointone! - feat: tail for live logs

running partykit tail --name <project> will now hook into a project's production logs and stream them live. This is a full clone of wrangler tail. Two notes:

• there's a bug with cf where logs on websocket connections don't come through until the websocket disconnects
• the filter aren't tested yet

That said, this is a good start, so let's land it and see what to fix after.

#9 24f8641 Thanks @threepointone! - add client id (_pk) to connection requests

#245 158c239 Thanks @threepointone! - rename assets -> serve

#249 ac55fd0 Thanks @threepointone! - npx partykit init - the basics

This implements the basics for an init command: it creates a package,.json if needed with dependencies, a simple server, a client entry and an index.html file (using the new static assets, woo)

5cc8734 Thanks @threepointone! - Let onBeforeRequest return a Response

#7 97c26e7 Thanks @threepointone! - export default {onConnect}

changing the server's export signature so it's easier to add types and validate.

#87 a3f7b72 Thanks @threepointone! - add update-notifier

#186 d3c6c72 Thanks @threepointone! - feat: expose the 'main' party as room.parties.main

d2c4878 Thanks @threepointone! - fix: configurable API endpoint name

#177 5c5b729 Thanks @threepointone! - fix: without in room.broadcast() is optional

#281 a1cfe42 Thanks @threepointone! - run multiple instances simultaneously

We weren't looking for free ports when starting up partykit dev. We run a few servers; the runtime, the assets server, and an inspector port. This PR uses preferred ports when starting up, and finds free ports when not.

#282 38d3975 Thanks @threepointone! - fix asset server reload

#123 0387b1a Thanks @TrySound! - replace express with node http server

#2 83570bf Thanks @threepointone! - read PARTYKIT_API_BASE from process.env, with a production default

#289 8bf5eb8 Thanks @threepointone! - partymix: first commit

partymix is a remix.run adapter for deploying on to partykit. This PR includes an implementation, as well as example app.

#255 7aab91f Thanks @threepointone! - expose external for asset bundling config

#13 28ada99 Thanks @threepointone! - feat: initial implementation of partykit env

This patch adds an initial implementation of the partykit env list/pull/add/remove commands. This doesn't actually use the env vars anywhere yet, but it's a start.

#288 5caf200 Thanks @threepointone! - fix: timing issues with assets/build server

In dev we have some issues with the way we initialise/restart servers

• if an external process is generating assets, and it does it quickly, then we end up crashing the assets server
• we were restarting the assets folder watcher whenever the assets watcher restarted
• we weren't cleaning up the assets folder watcher on effect rerun
• we weren't cleaning up the custom build folder watcher on effect rerun

So, the fixes:

• debounce setting the assets map (100ms, is short enough to not be noticeable, but long enough to let esbuild manager a restart)
• close the assets build watcher correctly
• don't restart the assets build watcher incessantly
• close the custom build watcher correctly

#204 2a43d58 Thanks @threepointone! - add a user agent to all our fetch calls

Good manners, and will be useful when we start debugging/o11ying

#198 fd49ef8 Thanks @threepointone! - cache parties object arter creating it

Let's not create the parties object on every request/connection, and instead cache it after first creating it.

#25 31c95a7 Thanks @threepointone! - s/unstable_onValidate/onBeforeConnect

This changes the behaviour of unstable_onValidate. Instead of return a boolean, this function now expects an error to be thrown if it's an invalid connection. Further, you can now return a json-serialisable object that gets passed on to onConnect (currently inside room.connections..unstable_initial, but we'll expose it on the connection soon.). This is particularly cool because onBeforeConnect will usually be run on a different machine from onConnect, but you'll still be able to pass data like session info etc on to onConnect.

misc: remove serve, fix builds.

#168 59218ee Thanks @threepointone! - feat: persist state locally in dev

Currently, .storage is only held in memory. This is fine for most usecases, but it means if you shutdown the dev process, we lose any state you may have been holding. This PR adds a config/flag persist to store this locally. We also turn it on by default. You can pass --persist false in the cli, or persist: false in config to turn it off (or true for default path). You can also pass --persist some/custom/path in the cli, or `persist: "some/custom/path" in the config, to use a custom path for the state.

Fixes https://github.com/partykit/partykit/issues/161

#111 4a808d5 Thanks @mellson! - Ensure that commander expects port to be a number.

#290 e4fb27a Thanks @threepointone! - fix: actually fix the listeners/crash scenario

Ok I figured it out. We were adding listeners to the inspector websocket server. I also found some stray listeners we weren't cleaning up. I also removed the debouncing setup for the assets server. This is looking a lot better now.

#212 0bca793 Thanks @threepointone! - add update-notifier again

This time it's integrated into the banner, so it's not as intrusive. It'll show up all the time in dev, but rarely in prod builds, so it's not annoying either.

#144 314e7ce Thanks @threepointone! - add ctx to onBefore* fns

importantly, this lets you call ctx.waitUntil(promise) in the onBefore fns.

#129 a4a8568 Thanks @threepointone! - fix: extract room id correctly

For onRequest, we were just using the part of the url after /party as the room id, which isn't true when we have query params. This quickly fixes that. Thanks @mellson for the catch.

#275 0261a76 Thanks @threepointone! - implement single page mode

setting singlePageApp: true in partykit.json#serve enables, well, single page app mode.

#244 f25c3f3 Thanks @threepointone! - feat: a better assets dev/deploy story

This enhances the static assets story, with features and utility:

• First, we remove sirv/polka, and use esbuild's server to serve assets in dev, since we already have it.
• We then introduce a config for actually building static assets. It's a tiny subset of esbuild's configuration; it takes (all fields optional):
  • entry: a path to an entry, or an array of paths, or even no entry at all (and simply serve the assets folder)
  • bundle: whether to bundle dependencies (default true)
  • outdir: where to put the output (default assets dir + '/dist')
  • minify: whether to minify (default false in dev, true for deploy)
  • splitting: whether to split up lazy import (default true)
  • format: one of esm | cjs | iife (default esm)
  • sourcemap: default true
  • define: expressions to substitute in the source; it merges the top level define here coneveniently, which means you can also pass it as a cli arg (--define key:value)
  • loader: corresponding to esbuild's loader args

Lovely. the config can also be a plain string, which it's use as the entry point instead.

I updated all the examples to use this new config, it drastically simplifies (imo) the boilerplate to get started.

#286 8b345ab Thanks @threepointone! - export lobby types

710460a Thanks @threepointone! - remove update-notifier for now

#234 2d5311b Thanks @threepointone! - delay login browser popup for 5 seconds

When we're logging in, we open up a browser for a user to paste a code into github. We currently open it immediately, so users aren't sure what's happening. This PR adds a delay for a few seconds, with clearer instructions on what's happening.

#141 f1dd644 Thanks @threepointone! - introduce ink

This uses ink to render a couple of commands (login, logout). Introducing this in a small PR since it requires some changes to how we build stuff. Unfortunately this means we're not bundling packages into the package anymore, but we'll revisit that later. After this, we'll rewrite dev to use ink as well.

#229 d2e26a4 Thanks @threepointone! - feat: serve config.assets / --assets

This PR adds support for deploying static assets.

• You can configure config.assets or --assets to point to a directory of static assets
• You can also configure config.assets to { path, browserTTL, edgeTTL, include, exclude, serveSinglePageApp }
• In dev, it serves these assets "locally"
• It can deploy these assets to PartyKit and serve them from the edge

f26bb08 Thanks @threepointone! - fix: import from partykit/server when moduleResolution: "nodenext"

fixes https://github.com/partykit/partykit/issues/167

#216 ac6e892 Thanks @threepointone! - feat: pass --compatibility-date --compatibility-flags via CLI for dev/deploy

#269 2af99fd Thanks @threepointone! - fix: type signatures on onBeforeRequest/onBeforeConnect

The types on onBeforeRequest/onBeforeConnect were causing errors, this should fix 'em

#122 5620808 Thanks @threepointone! - fix: in dev, close the socket with a 1011 when starting error

#195 839ebff Thanks @threepointone! - partykit: remove unused packages. move dev only deps to devDependencies

Also added a banner for all commands. Because why not.

#26 5eb8f9d Thanks @threepointone! - feat: persistence / storage (phase 1)

This implements persistence / storage for partykit. It hijack's cloudflare's DO storage api (without config options). This doesn't implement DO's i/o gates yet, but that's kinda fine, because it means you have to write code that's good in dev (but production will automatically be better). We'll implement them later. Also this currently does in-memory storage. We'll fix that in the future by (optionally) using disk for persistence.

#84 c138aa2 Thanks @threepointone! - read configuration from package.json#partykit

#132 206315d Thanks @threepointone! - connection ids aren't mandatory anymore

2d08282 Thanks @threepointone! - show banner for npx partykit env

e5832a0 Thanks @threepointone! - export "server" type definition

#57 9cd9b8d Thanks @threepointone! - feat: wasm modules

This adds support for importing and using .wasm modules. It requires a ?module prefix, ala vercel. Gives a compiled. WebAssembly.Module instance.

#214 0e56c2e Thanks @threepointone! - feat: better errors/warnings

Copied some code from wrangler for better looking errors/warnings. This also enables sourcemaps only in dev.

#34 0fdf7a3 Thanks @threepointone! - y-partykit: remove vendored libs

We'd previously vendored the libs used by y-partykit (yjs, lib0, etc) to workaround a bug in edge-runtime https://github.com/vercel/edge-runtime/issues/243, but it makes using other libs that include those libs difficult. So instead this patch removes the vendoring, and applies the other workaround (which is to set minify:true). The tradeoff for this workaround is that any "dev mode" code (i.e. code wrapped with if (process.env.NODE_ENV !== 'production')) will be removed. This is temporary and we'll remove it once the bug is fixed by edge-runtime.

#15 d2959d5 Thanks @threepointone! - integrate dotenv, expose .env in local dev on room

#205 8eec1a6 Thanks @threepointone! - fix: choose a recent compatibility date for dev

#140 6ce60e2 Thanks @threepointone! - local dev assets + custom builds

• This adds support for serving static assets (in dev only). This makes onboarding a little simpler; starting a new project doesn't require (too much) extra tooling. It's fairly simple, you make a field called assets in partykit.json, or pass it via cli with --assets <path>. In the future we may also deploy these assets to partykit directly, we'll see.
• This also adds support for custom build commands. You may be using some other compile-to-js language, or need to run some codegen, or even compile your client side app; you can now define a command that runs. It looks like this (all fields are optional):

build: {
  command: "...", // the command to run
  cwd: "...", // the directory to run it "in",
  watch: "..." // the directory to "watch" for changes
}

#218 ffd3434 Thanks @threepointone! - fix: better messages for env commands

This adds better messages for env commands. We were previously just spitting out some json, this cleans it up.

#56 a0d1bbd Thanks @threepointone! - feat: read env vars from command line

We hadn't implemented actually taking input for env vars from the command line, this patch fixes that. You can either type/copy paste a value, or pipe it in from a file.

#211 fffe721 Thanks @threepointone! - update dependencies

#5 e4b721f Thanks @threepointone! - queryparams for client, onConnect in server

client: new PartySocket now accepts query: {...} that gets encoded as query params in the url when connecting.

server: export onConnect(){...} to better reflect what's happening (and opening the door to other descriptive exports, like onAuth, etc)

#157 6927442 Thanks @threepointone! - feat: optional hibernation api

This introduces cloudflare's new hibernation api for durable objects. (see https://developers.cloudflare.com/workers/runtime-apis/durable-objects/#hahahugoshortcode-s2-hbhb)
It kicks in when you specify onMessage(){} in the export. This should let us scale to thousands of connections on a single object, while also not charging for idle objects. Very nice.

#291 fa8d73a Thanks @threepointone! - feat: add hot keys to open a browser, clear console, exit

Stealing this idea from wrangler: this adds a floating bar with hotkeys to open a browser, clear the console, or simply exit the process.

#147 e7aa4cd Thanks @threepointone! - feat: refactor PartyKitConnection / PartyKitRoom

This adds a couple of APIs based on common usage patterns:

• room.broadcast(message, without) lets you broadcast a message to all connections to a room, optionally excluding an array of connection IDs
• the "socket" now includes .id

This shouldn't break any code, so I'm comfortable landing it.

#179 e49223f Thanks @jevakallio! - Fix Response types for onBeforeRequest and onRequest callbacks.

When you respond from onBeforeRequest or onRequest, you have to construct a new response using the new Response().

Unless you've overridden the global Response type to refer to "@cloudflare/workers-types", the type is assumed a Fetch API Response. This is a pain in the ass, especially if you have your PartyKit server code as part of your frontend project.

Because the actual return value will be an instance of whatever Response class is defined in the environment, the type does not matter to us, so let's just allow the user to return either type.

#252 6534490 Thanks @threepointone! - init: add a default favico, styles.css

#280 3228764 Thanks @jevakallio! - Separate modes for local dev against local dev vs production

#66 9044fb4 Thanks @jevakallio! - Add room-like parameter to onBeforeConnect/onBeforeRequest

#151 3f260d6 Thanks @"user.ts",! - feat: multiparty support

Not all architectures can be represented with a single 'type' of entity. The interesting ones have multiple types of entities, interconnected by business logic. For example, you could model a chat system with a chatroom entity, a user entity, and a rate limiter entity.

This patch introduces "multiparty" support. You can define multiple modules with the same PartyKitServer interface in different modules, and configure them in partykit.json, like so:

main: "./src/main.ts" // your "main" entity, usually performing supervisory work
parties: {
  chatroom: "chatroom.ts",

  limiter: "rate-limiter.ts"
}
// ...

You can then reference these entities via room.parties.<user/chatroom/limiter>.get('some-id') and then make an http request to it with .fetch(req) or open a WebSocket to it with .connect().

This needs more examples and documentation (and potentially iterating on the api), but let's land it to experiment with the model a little.

#127 1d7d600 Thanks @mellson! - Add preview to deployed URL output.

#271 4624a2d Thanks @threepointone! - fix: wasm modules

Looks like importing wasm modules was broken, this should fix it

#260 f4add35 Thanks @threepointone! - create-partykit: move implementation from partykit init

#178 deaa474 Thanks @threepointone! - remove unstable_initial, and onBeforeConnect can now return a Request/Response

We were using unstable_initial to pass info between onBeforeConnect and onConnect. But it just felt so alien. Now that onConnect also receives request (via https://github.com/partykit/partykit/pull/166), let's just let onBeforeConnect return a Request or a Response. If a Request, it gets passed on to onConnect. Else, it just returns.

This is a breaking change, but it's a minor one, and it's worth it to get rid of unstable_initial.

#230 278833e Thanks @threepointone! - don't minify code in dev

This a remnant from our previous arch, we don't need this anymore. (and it's getting overwritten anyway, so it's not functional)

#217 04e5bed Thanks @threepointone! - feat: add --minify and whoami

• This adds a --minify / config.minify flag for both dev and deploy commands.
• This also adds a partykit whoami command

#190 cc7d994 Thanks @jevakallio! - Clean up hibernatable connection before invoking event handler

#152 a12b450 Thanks @threepointone! - fix: dev on windows doesn't resolve correctly

fixes https://github.com/partykit/partykit/issues/150

#3 7cd9edc Thanks @threepointone! - partykit list

very simply, lists all deployed parties.

#272 9d532f8 Thanks @threepointone! - fix: only fetch static assets for get/head methods

#240 77b897c Thanks @threepointone! - default to hibernation mode when onConnect isn't defined

ee72c1f Thanks @threepointone! - configure bundledDependencies

#206 f11c487 Thanks @threepointone! - feat: use compatibilityDate/compatibilityFlags if available

This lets you pass a compatibility date and/or flags when uploading a worker. See https://developers.cloudflare.com/workers/configuration/compatibility-dates for more info.

#292 8cbd4ee Thanks @threepointone! - fix: show hot keys only when possible

#14 b97b75b Thanks @threepointone! - feat: pass room to onConnect handler

#108 9e8a062 Thanks @threepointone! - deploy --with-vars, and fixes for env commands

• running deploy --with-vars now reads values from config files
• env pull now writes to partykit.json
• env commands don't fail if a main isn't specified

#103 90ca2e1 Thanks @threepointone! - fix: pass --var values to deploy

#80 232340d Thanks @threepointone! - feat: configuration file / flags

This adds a configuration file for partykit projects. It looks for partykit.json by default, but you can pass -c / --config to a custom path if you'd like. Fields:
name: corresponds to project name
main: corresponds to path to script
port: local dev port number
vars: a bag of key values, where values can be any serialisable object. this can be overriden by .env files, or --var X=A flags
define: a bag of key values, where values can be strings representing js expressions.

This PR also refactors the user login/config logic, but shouldn't have broken anything.

This also adds a env push command to upload a bunch of env vars at one, collecting values partykit.json and .env

#237 77b8ca3 Thanks @threepointone! - add a verbose mode for debugging dev

#225 e1bebe6 Thanks @threepointone! - fix: don't forward request to room without an onRequest handler

#112 5aaa7be Thanks @mellson! - Read port number from config file.

#121 d338a56 Thanks @threepointone! - fix: don't validate user tokens

This comments out validateUserConfig. We already had a bug where we weren't using it, but it's actually not that useful since we validate on the server anyway. So this patch comments out the implementation and usage.

#279 55444c1 Thanks @threepointone! - create: don't initialise git repo inside another

When we create a project, we shouldn't try to create a git repo if we're initialising inside another git repo (like a monorepo). This PR detects whether there's a git folder and skips the git repo creation part.

Additionally:

• we switch away from find-config to find-up everywhere, since it works on both files and directories
• I also took the opportunity to enable verbatimModuleSyntax in our tsconfig, which makes vscode import types correctly when we auto-import

#6 b04abec Thanks @threepointone! - lint for no-floating-promises and consistent-type-imports

enables some more linting rules (and immediately caught some errors). also fixes caching rules for actions.

#138 4d484bc Thanks @threepointone! - Allow deploys by users with capitals in username

This required a fix on the platform side, which I've deployed. It also required a cosmetic fix in the cli, which this patch does.

Fix https://github.com/partykit/partykit/issues/137

#264 2293ed5 Thanks @threepointone! - log a message when it's first publish

#222 25be6e8 Thanks @threepointone! - disable inspector for tests

The server we spin up makes our tests flaky. we don't have tests for dev logs anyway right now, so let's disable them.

#166 2f3676c Thanks @threepointone! - feat: add ctx to onConnect

onConnect(connection, room, context){}

Developers want access to additional metadata available on every connection. For example, to implement a rate limiter, they need access to the IP address of the client. They may want access to the user agent, some cookie values, or any of the special cf properties like geo information, etc. Most of this information is available on the request object when the connection is made, but we don't actually expose that to the developer right now. I propose we do that.

We can expose a third parameter to the onConnect method that is the context object. For now it'll just be an onject with a single property request that is the request object. In the future we can add more properties to this object.

onConnect(connection, room, context) {
  const { request } = context;
  const { headers } = request;
  const ip = headers['x-forwarded-for'];
  // do something with the ip
}

Now, the hibernation api variant doesn't include onConnect at all. So, we'll need to add that to the hibernation api. So, I also propose that we let folks add an optional onConnect method when defining a party. Calling connection.addEventListener with that variant will silently do nothing, but maybe we could override that to throw an error instead. Importantly, folks should be able to call .serializeAttachment in the onConnect method, and recover that data in the onMessage method.

export default {
  onConnect(connection, room, context) {
    const { request } = context;
    const { headers } = request;
    const ip = headers["x-forwarded-for"];
    connection.serializeAttachment({ ip });
    // do something with the ip
  },
  onMessage(message, connection, room) {
    const { ip } = connection.deserializeAttachment();
    // do something with the ip
  },
};

This PR adds the context object to onConnect. After we land the hibernation api, we can do the additional work mentioned.

#209 a6b8b46 Thanks @threepointone! - update miniflare/workers-types

#4 d2c5c51 Thanks @threepointone! - fix exports/typings

This patch fixes how we generate types (by actually doing so), configuring exports in package.json, and making sure it points to the right thing. I had to write a script that moves the generated types to the root for... javascript reasons ™ but at least it works now. good enough.

#11 b1e47ae Thanks @threepointone! - add unstable_onValidate

This adds unstable_onValidate to server's default export, alongside onConnect. Users are expected to implement this and return a boolean, that will reject the connection when false.

#250 f579143 Thanks @threepointone! - apply cli args correctly

we weren't applying overrides to config via the cli (eg --name xyz, when config.name was already defined). This fixes the logic.

#202 437458b Thanks @threepointone! - cache hydrated connections

When using hibernation, we should cache initialised connections, or else we end up doing it for every message, which seems wasteful.

#30 323fce1 Thanks @threepointone! - chore: update esbuild, edge-runtime, and co.

of note, esbuild had breaking changes, had to rewrite the rebuild logic.

#12 115c094 Thanks @threepointone! - rename publish -> deploy

#187 450e139 Thanks @threepointone! - allow using onConnect with onMessage

We now allow using onConnect with onMessage. This lets you get access to the context {request}. Warning: You can't use .addEventListener when using onMessage, but it currently just silently fails. We'll make this an error later.

#85 8b13a3c Thanks @threepointone! - read from .env.local for local dev

#29 05e647f Thanks @threepointone! - fix: add key/value size limits for storage api

This adds checks for size of key/values for the persistence api (mirroring DO's limits). (2kb keys. 128 kb values)

#213 528a117 Thanks @threepointone! - feat: render list of projects in a table

#263 17cc254 Thanks @threepointone! - tweaks to create-partykit

• ensure instructions are logged on new project
• don't log long git commit info
• output path where project was made

#262 cab1365 Thanks @threepointone! - a better create-partykit

• questions for every choice
• flags for every choice: --git, --typescript, -- install
• implement -y to skip questions
• js/ts versions of the template
• use package manager of choice
• --dry-run to skip writing stuff to disk
• better messaging

#276 0aec7b3 Thanks @threepointone! - initialise git repo with 'main' branch

#277 6e124c4 Thanks @threepointone! - create: always make a package.json

we were trying got be clever and not make a package.json if there was a parent (that's wasn't a monorepo root). Turns out folks have package.json's all the time. So this just always makes a package.json, and only runs the installer in the root package,json dir f it's a monorepo/has workspaces

#266 36cd904 Thanks @threepointone! - create: Add READMEs to generated projects

#260 f4add35 Thanks @threepointone! - create-partykit: move implementation from partykit init

#279 55444c1 Thanks @threepointone! - create: don't initialise git repo inside another

When we create a project, we shouldn't try to create a git repo if we're initialising inside another git repo (like a monorepo). This PR detects whether there's a git folder and skips the git repo creation part.

Additionally:

• we switch away from find-config to find-up everywhere, since it works on both files and directories
• I also took the opportunity to enable verbatimModuleSyntax in our tsconfig, which makes vscode import types correctly when we auto-import