actionhero

** Redis-powered Cluster & major refactor **

Details

This version realizes the dream of a true cluster for actionHero. There is no longer a need for a master process, and every node in the cluster can work alone or in a group. This release also enables using the node.js cluster module to make the most of your server(s).

This version is likely to be incompatible with prior versions. Running an actionCluster now requires redis (running a single node does not require redis and is still a pure node.js implementation).

Using a redis backend, actionHero nodes can now share memory objects and have a common queue for tasks. Philosophically, we have changed from a mesh network to a queue-based network. This means that no longer will every node talk to every other node, but rather all nodes will talk to redis. Now, I know that you are thinking "isn't that bad because it creates a single point of failure?" Normally, the answer is yes, but redis already has mesh networking support! The suggested method of deployment is to setup a redis instance on each server, and they will handle the mesh networking for you.

api.cache now works in a shared way if you are part of an actionCluster. All cache actions refer to redis and in this way, all peers can have access to shared objects. To avoid conflicts, you will have access to 'lastReadAt' as part of api.cache.load responses. actionHero will also no longer store its own cache to disc periodically as redis does this already.

The task system also has undergone some major refactoring. All tasks are now stored in a shared queue within redis. All peers will periodically check the queue for unfilled tasks, and drain the queue one at a time. In this manner, you can add more task capacity by spinning up more actionHero nodes which may or may not also handle web/socket traffic. This also means that tasks will not get lost if a node crashes as they will remain in the redis queue until drained. Each peer also has a 'personal' task queue for "all" actions.

For periodic tasks ("any" and "all"), the peer which most recently completed the task while hold the semaphore for that task (in a actionHero:tasksClaimed shared list) until the proper amount of time has elapsed, then they will re-enqueue the task. This does not mean that a specific node will always preform tasks of the same type.

There are new requirements to config.json to configure redis. Here is an example:

"redis" : {
"enable": true,
"host": "127.0.0.1",
"port": 6379,
"password": null,
"options": null
},

All methods under the api.actionCluster namespace have been removed for simplicity. Just use the normal cache methods, and if you are in a cluster, you will operate in a shared memory space.

Notes

• all peers now share the same api.cache data
• api.tasks.enqueue is now api.tasks.enqueue(api, taskName, runAtTime, params) Set runAtTime to 0 to run the task as soon as possible
• using redis cache save; no longer saving cache periodically
• all nodes are created equal; there is no need for a master
• the entire actionCluster namesapace has been removed
• there are new requirements to config.json to setup redis
• every node will try to handle requests and process one job pending in the task queue at a time
• shared tasks will be preferred over per-node tasks
• the 'status' action has some new output types to reflect 'global' stats in comparison to 'local' stats (IE: count of web requests that this node has served vs total)

Summary: Bug Fixes & Examples

Details
Check the notes for the bug fixes. There are now also more examples in the project, showing off how you can connect from various environments. This also coincides with the release of the actionHeroClient NPM package. More coming soon.

Notes

• Updates to how socket client requests are tracked to enforce proper message IDs. If you directly used api.processAction before, the interface has changed.
• running the test suite will now use a separate redis db. You can also set this in config.json for your app.
• better logic to ensure that tasks are enqueued properly
• cluster members will remove stale tasks they were working on when the app is started (in case of a crash)

Summary: Bug Fixes, Examples, and REPL

• actionHero now has a REPL! You can use the command line to use all the api-namespaced functions and define your own. Great for debugging or checking on the status of the cluster.
• Checkout the /examples folder for how to use the actionHeroClient package, browser javascript, and curl to interface with actionHero. More languages coming soon.

Notes

• cache actions now take milliseconds like everything else.
• you can now have non-expiring cache entires (just pass a null value for expireTimeMS)
• there are no no default periodically-run tasks. Look in /examples for the tasks that used to be there and copy them into your own project's /tasks folder. This was done to reduce the 'magic' in the framework. the runTask task still remains, as it was not run periodically.

Note:
** actionHeroClient released! Connect node to node! **

WebSockets and a better configuration system

This is a major release. Sections of older actionHero projects will be incompatible with this release.

This release adds web socket support to the project, and abstracts the chat room system to be available generically to all types of persistently-connected clients. This release also updates the configuration system to allow developers to use any/all/none of the connection methods. This should make it easier to add more and more communication protocols to the project.

Notes

General

• default file route changed from '/file' to '/public'; /file is the "action" called file which is provided as an example
• new configuration system. Check config.js for updated structure
• all transports can be enabler or disabled based on config.js
• api.id now reflects protocols in use
• you can now specify which IPs actionHero will listen on for each protocol. Use "0.0.0.0" for "all""

Actions

• randomNumber action update to show how to inspect different HTTP verbs to create different responses

Chat

• new initializer to make chat available to all persistent-socket clients
• new api.chatRoom namespace for chat functions (IE: api.socketServer.socketRoomBroadcast => api.chatRoom.socketRoomBroadcast)

Web Sockets

• socket.io a dependency of the project
• socket.io client JS symlikned to /public
• websocket example created
• status counts for webSockets
• websockets can be bound to http or https

Tests

• Spec Helper & Tests updated to reflect new configuration system

Examples

• updated to reflect new configuration system

Project Generator

• A project Generator!
  • You can start up a new project in an empty directory with npm install actionHero && npm run-script actionHero generate. This will create project structure and copy in some default actions and tasks.
• Only tasks present in your project will be loaded (like actions). Now that there is a generator which will copy in some default actions, loading tasks within actionHero is not needed.
• Project reorganization per the above
• remove hredis from the project
  • hredis is awesome, but this makes the project have less complex compiled dependencies. You should add it to your project if you want fast redis communication, but it isn't required for actionHero to function
• This release adds more configuration options to webSockets in the form of api.configData.webSockets.logLevel (integer) and configData.webSockets.settings (which is an array strings which will be applied to socketIO's 'set' command)

General Bug Fixes

• fixed bugs where some tasks would not allow the task queue to continue even if they had been completed
• fixed a bug where non-redis tasks wouldn't be re-enqueued
• spelling

Notes

• Configuration to set how many task workers each actionHero node has. You now are required to set api.configData.general.workers
• Added generators for actions and tasks
  • npm run-script actionHero generateAction
  • npm run-script actionHero generateTask

General Bug Fixes

• fixed bugs regearding load order of initializers and user-added initializers
• fixed a bug introduced in the previous version which may double-enqueue tasks

File Server

• Files will now be served with a default cache-duration of 0 seconds. Setting added to configure this
• Default files for directories is still "index.html", but this is now configurable.
• accessing directories works without the trailing slash (IE: http://localhost:8080/public resolves the same as http://localhost:8080/public/)
• the Cache-Controll is returned along with Expires header

Better Logging

• separation between file and action logging
• better shared syntax for all types of connections
• files log the path they were accessed from and duration

Project Organization

• You can now orginize your tasks and actions in sub-folders
• You can now symlink actions/taks into your project
• You can now symlink files in /public

General Bug Fixes

• better logic for including base config.json when you don't provide one
• updates for travis.ci

Chat Rooms

• Events Emitted when folks enter/leave the room you are in

Socket / WebSocket Clients

• New Method detailsView to retrieve information about yourself (including your public ID)

Action Cluster

• a new global hash, actionHero:peerPings, will store the last pings from each peer. They should ping every ~500ms. This can be used to detect when peers disappear
• disconnected peers are removed from the global list, and any tasks they were working on are re-enqueued

Bugs

• fixed a bug where using the generator to create a new action with no inputs would generate invalid syntax
• refactor redis name-space to be actionHero: rather than actionHero:
• actionHero.restart now returns the api object on success actionHero.restart(true, api)

CLUSTER

• A new, production-ready cluster example (complete with unix signal handling and 0-down time reloads for code changes)

Bugs

• Stopping an actionHero node will remove himself from actionHero:peerPings
• Cleaned up the colorizer within the logger
• Shutting down the actionHero when no servers were active to begin with returns true on actionHero.stop()
• final message sent to TCP clients on disconnect or shutdown now matches, "Bye!"

Development Mode

• actionHero now has a development mode! files in /actions and /tasks will be watched for changes, and reloaded on the fly.
• this uses fs.watchFile() and doesn't work on all OSs / file systems.
• new files won't be loaded in just yet, only existing files when the app was booted will be monitored
• as deleting a file might crash your application, we will not attempt to re-load delted files
• if you have changed the task.frequency of a periodic task, you will the old timer value will be in use until the event fires at least once after the change
• Don't use this in production!

Bugs

• default config does not use redis again

Route parsing

You can now extract prams per HTTP(s) request from the route requested by the user via an included utility.
URL routes remain not being a source of RESTful parameters by default, however, you can opt to parse them:

  var urlMap = ['userID', 'gameID'];
  connection.params = api.utils.mapParamsFromURL(connection, urlMap);

• this is still left up to the action as the choice of which to choose as the default: query params, post params, or RESTful params is a deeply personal mater.
• if your connection is TCP or webSockets, api.utils.mapParamsFromURL will return null
• map is an array of the param's keys in order (ie: /:action/:userID/:email/:gameID => ['userID', 'email', 'gameID'])
• the action itself will be omitted from consideration in the mapping
• these are equivalent: [ localhost:8080/a/path/and/stuff?action=randomNumber ] && [ localhost:8080/randomNumber/a/path/and/stuff ]

Socket Communiaction

You can now pass a stringified-JSON message over TCP sockets to request an action to be preformed.

• {"action": "myAction", "params": {"key": "value"}} is now a valid request over TCP
• params passed within an action's hash will NOT be 'sticky' to the connection, unlike paramAdd which remains 'sticky'
• general refactoring to the TCP codebase which will ensure that the state of params at the time of the action will be used when computing an action, and changes to the connection's state will not effect the action's execution
• these changes do not break the previous API, and sending single-line 'action' still works!

Bugs

• fixed bug where websocket clients would not return after a actionHero reboot; also allows for more than one socket.io binding point
• Fixed default room assignment for webSocket clients
• only respond with messageCount to TCP/webSocket clients when it is in response to a request they made
• tests for webSockets finally writen and passing
• added connection idle timeout to aid with HTTP(s) server shutdown
• fixed a bug where action generators with 0 params wold generate invalid js

RESTful Routes

• Building off of api.utils.mapParamsFromURL, you can now define named routes to your actionHero projects to help out http and https clients
• routes remain optional
• actions defiend in params directly action=theAction or hitting the named URL for an action /api/theAction will always override RESTful routing
• the hierarchy of the routes object is prefix --> REST verb -> data
• data contains the 'action' to map to, and then an optional urlMap (api.utils.mapParamsFromURL)
• only single depth routes are supported at this time
• generating a new project will create a template routes.js, but it will contain no content

An example routes.js

  exports.routes = {

    users: {
      get: {
        action: "usersList", // (GET) /api/users
      }
    },

    user : {
      get: {
        action: "userAdd",
        urlMap: ["userID"], // (GET) /api/user/123
      },
      post: {
        action: "userEdit",
        urlMap: ["userID"] // (POST) /api/user/123
      },
      put: {
        action: "userAdd",
        urlMap: ["type", "screenName"] // (PUT) /api/user/admin/handle123
      },
      delete: {
        action: "userDelete",
        urlMap: ["userID"] // (DELETE) /api/user/123
      }
    }

  };

chat & redis

• The redis pub/sub channel used for interclient communiaction is now suffiexed by your redis DB
• This means that differenet actionHero clusters using the same redis instance will no longer intercept eachother's messages

non-breaking Exceptions (node >= v0.8.0 only)

Thanks to node.js domains, exceptions in Actions and Tasks will no longer crash the application.

• Exceptions within actions will be logged, and a new api.configData.general.serverErrorMessage will be rendered to clients within an action
  • web clients will be sent the 500 (server error) header
• Exceptions created in tasks will also be logged, and the task will return
• If the Exception occured within a periodic task, the task will be re-enqueud.
• keep in mind that any applicaton-wide settings which may have been modified in this erronious action/task will not be rolled-back

Readme and Wiki

• The readme.md was gettting really long and hard to searh, so we started a github wiki!

Other

• Documentation and Tests for XML output

Connections

• Connections have been refactored to all live within api.connections, and matain an type refering to thier connection method (web, websocket, tcp)
• web clients can now also be sent messages. Web clients can retrieve pending messages with the chat api action and the method messages. The old say action has been merged into this example action.
• all connections (including web clients) will be assigned a connection.public.id to facilitate the above. actionHero will attempt to save this id within in a cookie. Options can be set in api.configData.commmonWeb.fingerprintOptions
• http client's messages will be pruned every so often as determined by api.configData.commonWeb.httpClientMessageTTL

Strict Mode

• actionHero now conforms to strict JS mode.

Bugs

• Relaxed the necessity to provide public information to mock connections when using socketRoomBroadcast problematically.

Cleanup and changing our test suite to mocha

Connections

• Version 3.0.12 introduced http(s) message queues, but this created a lot of overhead for each request. We have changed this feature to "opt-in" if you want to use it.
• api.configData.commonWeb.httpClientMessageTTL will default to null (disable http client http(s) message queues). Setting it an integer (ms) will enable it again.

Other

• routes.js will now be reloaded when in development mode as the file changes

Bugs

• Fixed a header issue with mime times on flat files
• Added a helper which will ensure no duplicate headers are sent to http clients (like mime/type) except for setting cookies
• ignore the developer mode test in node < 0.7.0

Refactor of servers

• Rather than having both a HTTP and HTTPS server, you now configure both the web server and socket server if you want them to be secure (tls, https) or not (tcp, http)
• the TCP server can now be upgraded to a TLS server, similar to how the https server can be enabled
• as there is now only one http server, webSockets will be bound appropriately if enabled

Other

• changing your config.js file in development mode will now reboot the server

Bugs

• reloading of tasks in development mode crashed before... now it doesn't!
• providing both a custom config.js and params.configChanges when booting actionHero didn't work (only config.js was used). Fixed
• in development mode, causing a parser error in a task or action will no longer crash the program. The action will be ignored until you fix it.

This is likley to cause many minor, but breaking changes

Pattern Consistency

• actionHero.start, actionHero.stop, and actionHero.restart's callbacks now all callback with (error, api) rather than just (api)
• Actions now have a default error of 'null' rather than 'false' and actions will not return an error object unless there is an error. Client-side checks should now look for errors with if(error != null), which makes more sense
• api.cache.save and api.cache.destroy now return with callback(error, didSave) and callback(error, didDestroy). api.cache.load now responds with next(error, cacheObj.value, cacheObj.expireTimestamp, cacheObj.createdAt, cacheObj.readAt)
• a number of other functions have also been updated to properly follow the (err, data) pattern. Check them out in the "internal methods" wiki page
• Tasks should now callback(data, toContinue) where 'toContinue' is a boolean indicating if the task ran sucessuflly, and is able to be run again
• the extra methods for socket/webSocket connections (say, roomView, etc) have been removed from the actionsView action, as they aren't really actions
• default session ID renambed from "__browser_fingerprint" to "sessionID"
• you can now pass "x-sessionID" headers as well as cookies to continue a web session with the same ID

Tasks

• api.tasks.inspect can be used to inspect all the tasks in the queue
• api.tasks.enqueue now has a callback(err, enqueued) which can be used to check if there were any errors adding your task to the queue
• anquing a non-periodic "all" task now will be sent to all servers in the actionCluster properly

custom matchers on api.chatRoom.socketRoomBroadcast

• api.chatRoom.socketRoomBroadcast() can now send messges to only some clients that match certain params, IE: {auth: true} or {id: 123}. This will let you segment the users in your rooms who hear you broadcasts.
• connection.params.roomMatchKey and connection.params.roomMatchVale are set by the broadcaster to determine who they wish to message, along with the room.