hotwheels

Erlang messaging server optimized to send 1 message to 40k subscribers to a topic in < 1s

Stars
79

h1. Janus

Janus is a messaging server optimized to unicast over TCP to thousands of clients subscribed to topics of interest.

The ultimate goal is to maintain a latency of less than 2 seconds for 20 thousand clients on Amazon EC2 (small instance).

h1. License

Janus is available under the MIT license.

h1. Bounty and rules

  1. Unicast to 20K clients in < 1s (max latency) nets you a $1000 bounty if you are the first person to achieve this. Unicast in < 2s (max latency) nets you a $500 bounty.

  2. Final proof has to be from EC2, one small instance for the server and one for the bots.

  3. {packet, 0}, -smp disabled

  4. TCP, not UDP.

  5. No change of the protocol unless I agree to it.

Note that SMP is disabled since Amazon EC2 small instances have a single core.

h1. Installation

Janus requires at least OTP R13B01.

Create a soft link from janus/mochiweb to your Mochiweb installation.

h1. Running

  1. 'make run1' to start the server
  2. 'make sh' in a different window to run clients
  3. 'bot:test(flashbot, 20000).' to run 20k bots on the same machine
  4. 'bot:test(flashbot, 20000, 'host', 8081).' if the server is somewhere else

You should see output like this if everything goes well (MacBook Pro Core2Duo, 2.93Ghz, SSD):

h1. Notes

On Leopard:

Also, you won't be able to go past 1024 clients with Erlang on Mac OSX, even with kernel poll enabled (+K true).

Edit FD_SETSIZE in /usr/include/sys/select.h, i.e. change __DARWIN_FD_SETSIZE in /usr/include/sys/_structs.h and recompile Erlang with --enable-kernel-poll.

On Linux: