swift-distributed-actors

Peer-to-peer cluster implementation for Swift Distributed Actors

APACHE-2.0 License

Stars
587
Committers
24
View Code on GitHub Visit Website
Ecosystems: Swift

Bot releases are hidden (Show)

swift-distributed-actors - 1.0.0-beta.3 Latest Release

Published by ktoso almost 2 years ago

Highlights

This release fixes a number of bugs preventing stable usage of the beta.2 release in a long running cluster (https://github.com/apple/swift-distributed-actors/pull/1059).

It also performs a number of destructive renames while we still have the ability to do so.

  • Module rename (!) The DistributedActors module has been renamed to DistributedCluster as it explains more precisely what this module is offering (the ClusterSystem) type.

  • The Node and UniqueNode types have been changed to Cluster.Endpoint and Cluster.Node respectively. See https://github.com/apple/swift-distributed-actors/pull/1077 for a more detailed explanation.

Internally we also adopted a new integration testing framework - offering MultiNode tests. This multi node testing framework will initially be matured internally before making it available to users outside the package. https://github.com/apple/swift-distributed-actors/pull/1055

What's Changed

New Contributors

Full Changelog: https://github.com/apple/swift-distributed-actors/compare/1.0.0-beta.2...1.0.0-beta.3

swift-distributed-actors - 0.7.1

Published by ktoso about 2 years ago

This release is primarily aimed at supporting Swift 5.7 - due to the updated Swift syntax dependency.

What's Changed

Full Changelog: https://github.com/apple/swift-distributed-actors/compare/0.7.0...0.7.1

swift-distributed-actors - 1.0.0-beta.2

Published by ktoso about 2 years ago

This beta release marks major improvements in all surface APIs and completes most of the public surface move to distributed actors and async await. We also added more documentation and will continue adding more examples and guides about using and deploying distributed actor clusters.

A few API changes may still happen like making more methods async etc.

Documentation

Latest beta documentation is hosted on github pages.

Milestone

The 1.0.0-beta.2 milestone contains all closed issues for your reference.

What's Changed

New Contributors

Thank you to all contributors!

Full Changelog: https://github.com/apple/swift-distributed-actors/compare/1.0.0-beta.1.1...1.0.0-beta.2

swift-distributed-actors - 1.0.0-beta.1.1

Published by ktoso over 2 years ago

Require macOS 13 / iOS 16

With macOS 13 beta announced, we're able to express the minimum deployment targets in Package.swift and allow developers to use this library without the -disable-availability-checking flags.

Use macOS 13 Ventura and Xcode 14.0 beta to build this project, and develop using distributed actors 🎉

What's Changed

Full Changelog: https://github.com/apple/swift-distributed-actors/compare/1.0.0-beta.1...1.0.0-beta.1.1

swift-distributed-actors - 1.0.0-beta.1

Published by ktoso over 2 years ago

First Beta

This is the first beta release of the distributed actors and cluster library.

Supported features

  • a ClusterSystem implementation, enabling clustered distributed actor instances,
  • cluster events system.cluster.events, i.e. nodes joining, becoming down and leaving,
  • observe actor lifecycle and termination with LifecycleWatch,
  • a cluster-wide DistributedReceptionist implementation enabling actor discovery,
  • and more!

Refer to the reference documentation for more details

swift-distributed-actors - 0.7.0

Published by ktoso over 2 years ago

Support Swift and Swift Syntax 5.6

The upcoming Swift 5.7 release will not be supported on the release/0.x branch and we will move on towards the language provided distributed actor features.

What's Changed

New Contributors

Full Changelog: https://github.com/apple/swift-distributed-actors/compare/0.6.10...0.7.0

swift-distributed-actors - 0.6.10

Published by ktoso over 2 years ago

What's Changed

Full Changelog: https://github.com/apple/swift-distributed-actors/compare/0.6.9...0.6.10

swift-distributed-actors - 0.6.8

Published by ktoso about 3 years ago

Minor

  • adjust for Swift 5.5
swift-distributed-actors - 0.6.7

Published by ktoso about 3 years ago

Patch

  • Fix building in release mode in Swift 5.4
  • Update some code that was causing warnings / errors
swift-distributed-actors - 0.6.6

Published by ktoso over 3 years ago

More details: https://github.com/apple/swift-distributed-actors/milestone/22?closed=1

swift-distributed-actors - 0.6.4

Published by ktoso over 3 years ago

Swift 5.4-development updates

May have some issues on linux + 5.4 still.

swift-distributed-actors - 0.6.3

Published by ktoso over 3 years ago

  • Swift 5.4 compatibility
swift-distributed-actors -

Published by ktoso about 4 years ago

Bugfix

  • =serialization rm 5.2.4 workaround, fix LWWMap with Codable Value (Codable in Proto nesting) #782
    • this would manifest when using deeply nested CRDTs such as LWWMap which stores an LWWRegister internally, both being encoded using protocol buffers, and using a value which is Codable/JSON encoded -- it would have wrong assumptions about how it should store the serialized bytes.

Stats

A total of 1 tickets (including 1 bug 🐞) were resolved in this patch release.
You can view the full changes made on the 0.6.2 milestone.

swift-distributed-actors - 0.6.1 - Cluster/SWIM metrics

Published by ktoso about 4 years ago

Swift Metrics support

SWIM

Available metrics are:

// actor lifecycle counters
first.actors.lifecycle["root:/user", "event:start"] // counter
first.actors.lifecycle["root:/user", "event:stop"] // counter
first.actors.lifecycle["root:/system", "event:start"] // counter
first.actors.lifecycle["root:/system", "event:stop"] // counter

// swim metrics
first.cluster.swim.message.count["direction:out"] // counter

first.cluster.swim.probe.ping["type:successful"] // counter
first.cluster.swim.probe.ping["type:failed"] // counter
first.cluster.swim.probe.pingRequest["type:successful"] // counter
first.cluster.swim.probe.pingRequest["type:failed"] // counter

first.cluster.swim.lha[] // gauge

first.cluster.swim.roundTripTime.pingRequest[] // timer
first.cluster.swim.roundTripTime.pingRequest["type:all"] // timer
first.cluster.swim.roundTripTime.pingRequest["type:firstAck"] // timer

// cluster/swim membership

first.cluster.swim.members["status:alive"] // gauge
first.cluster.swim.members["status:suspect"] // gauge
first.cluster.swim.members["status:unreachable"] // gauge
first.cluster.swim.removedMemberTombstones[] // counter

Opt-in per actor (group) metrics

It is now possible to enable metrics per actor and report them as a groups etc.

For example, one may want to monitor a specific actor and how long deserialization of the messages it receives takes, as well as monitor it's mailbox count while it is active. This can be now done via:

        let one: ActorRef<String> = try first.spawn(
            "measureMe",
            props: .metrics(group: "measureMe", measure: [.mailbox, .deserialization]),
            someBehaviorOrActorable
        )

resulting in metrics like:

actorSystemName.measureMe.mailbox.count // gauge
actorSystemName.measureMe.deserialization.size // gauge (bytes)
actorSystemName.measureMe.deserialization.time // timer (nanoseconds)

Refer to:

for detailed documentation on the metrics.

Stats

A total of 4 tickets (including 1 bug 🐞) were resolved in this patch release.
You can view the full changes made on the 0.6.1 milestone.

commits  added  removed
      2   1079      253 Konrad `ktoso` Malawski
      1      0        4 Tomer Doron

Thanks as usual for all the feedback and help to our users and contributors!

swift-distributed-actors - 0.6.0 - OSS SWIM

Published by ktoso about 4 years ago

Use OSS Swift Cluster Membership's SWIM

The SWIM+Lifeguard cluster membership was extracted from Distributed Actors and released as a stand-alone module as part of the Swift Cluster Membership.

See the blog announcement here: https://swift.org/blog/swift-cluster-membership/

Other Notable Changes

New API

  • Implement Actorable version of ActorTestKit.ensureRegistered() ga:test-kit t:actorable t:testkit #742

General

  • Patterns around storing Actor relates-to-swift-feature t:actor t:docs user-feedback #618

  • ActorSystem.shutdown() does not invoke callback if system is already shutdown bug 🐞 #760, thank you @budde!

  • +test actorable call during shutdown should not crash ga:actorable-feature-parity t:actorable #730

  • =test ensureNodes is too strict; a down may already become a removal ga:cluster t:tests#732

  • exact nio ssl version for compat with specific project which requires it #738

  • Actorable and generic actorable types source-gen breaks down 3 - in progress ga:actorable-feature-parity relates-to-swift-feature t:actorable #658

  • (Make it) Crash process if failure escalates to a Guardian (esp. /system) t:actor t:isolation https://github.com/apple/swift-distributed-actors/issues/31

  • =build bump to using stable 5.3 swift-syntax #777

  • SWIM: Do not ensureAssociated, just get associations for unknown nodes 1 - triaged ga:swim t:cluster:swim #428

Test fixes and hardening

  • Ensure we close all associations when we terminate / receive RIP ga:cluster t:cluster:all #600

  • [radar 68467435] Swift 5.3-dev compilation broken on Linux bug 🐞 failed 💥 relates-to-swift-feature

  • Foundation.JSONEncoder issue in Swift 5.2.4 Linux bug 🐞 t:serialization #729

  • !serialization assume _getMangledTypeName is not available #776

  • =test harden test_gossip_shouldEventuallyStopSpreading failed 💥 t:crdt #771

  • FAILED: SWIMShellClusteredTests.test_swim_shouldIncreasePingTimeout_whenHighMultiplier failed 💥 ga:swim t:cluster:swim #727

  • flaky test: 'ActorableExampleRouterTests.test_refRouter_shouldWork' failed 💥 #740

  • SKIP one integration test because #758 compilation breaks until swift… #773

  • FAILED: test_interceptor_shouldInterceptSignals failed 💥 t:actor #614

  • FAILED: ActorableExampleRouterTests test_actorRouter_shouldWork failed 💥 #736

  • FAILED: DistributedActorsTests.ShootTheOtherNodeClusteredTests test_shootOtherNodeShouldTerminateIt failed 💥 t:cluster:all #390

  • !!tmp disable test_actorOwned_ORSet_add_many_times #747

  • =ios fix availability requirements for iOS #731

Stats

A total of 27 issues was closed since the 0.5.1 release, including 4 bugs 🐞.

You can view all resolved tickets on the 0.6.0 milestone.

commits  added  removed
     27   4890     5354 Konrad `ktoso` Malawski
      2     90        0 Adam Budde
      1    130       47 Yim Lee
swift-distributed-actors - 0.5.1

Published by ktoso over 4 years ago

Bug Fixes

CRDT

  • Fixed a bug with unfortunate interplay of CRDT.ORMap serialization and Foundation's JSONEncoder as implemented in 5.2.4 on Linux (only) #720

Stats

A total of 1 issues was closed since the 0.5.1 release, including 1 bug 🐞.

Thank you to Yim Lee for spotting and fixing the issue!

You can view all resolved tickets on the 0.5.1 milestone.

commits  added  removed
      1    130       47 Yim Lee
      1     27        1 Konrad `ktoso` Malawski
swift-distributed-actors - 0.5.0

Published by ktoso over 4 years ago

New Features

Core / Actors

Serialization

Actorable

(Reusable) Gossiper / GossipLogic

  • Extracted shared needs of various gossip implementations into Gossiper imeplementation, and re-used it in CRDT gossip as well as re-phrasing Membership gossip in it's terms. This already spotted minor issues which we would have missed in the ad-hoc implementations otherwise. https://github.com/apple/swift-distributed-actors/issues/28
  • Gossiper can in the near future become a public type offering "if you need any gossip protocol, just plug in your GossipLogic and it just works" for end-users, including peer discovery, gossip round management etc.

CRDTs

Cluster Singleton Actors

Instrumentation and Instruments(.app) instruments

  • Introduced a few initial Instrumentation cut-points in the system https://github.com/apple/swift-distributed-actors/pull/475
    • These will be used for future tracing efforts as well as general measurements and other instrumentation (eagerly awaiting swift-baggage-context). See Sources/DistributedActors/Instrumentation, for the current Instrumentation types.
  • Implemented need-driven custom instruments for the following use-cases:
    • ReceptionistInstrumentation,ActorInstrumentation, ActorTransportInstrumentation (incl. Serialization events), and ActorMailboxInstrumentation; See example screenshot.
    • These tools are becoming tremendously helpful in deciding about implementation tradeoffs and inspecting runtime behavior, we expect users to value them as well, in order to avoid treating the actor system as a black box.
  • ActorInstruments package definition is fully implemented in type-safe Swifty Instruments DSL, meaning there is no possibility to mis-configure the instrumentation for non-existing fields/tables/schemas:
  • Introduced SwiftyInstruments DSL, which enabled efficiently building specialized custom instruments for Instruments.app using plain Swift.
    • Definitions cover most of the Instruments package definition schema, some missing bits easy to fill-in in the upcoming future on an as-needed basis. See: SwiftyInstruments, discussed with Instruments team a little bit.
    • Brought up some function-builder limitations which are a bit annoying for this API on the forums:

ActorTestKit

Improvements

Core

Transport

  • 🔥 Hardened and re-worked the Association design as well as connection management
    • This was a long outstanding task which we knew was important to revisit; New implementation guarantees no dropped messages even during racy connection establishments (e.g. nodes A and B joining "eachother" simultaniously -- neither A's or B's sent messages will be dropped when they face such racy handshake and multiple similar races were fixed).
  • 🔥 Hardened RemotePersonality for ActorRefs, lessening the use of locks tremendously as well as improving correctness at the same time https://github.com/apple/swift-distributed-actors/issues/382 & https://github.com/apple/swift-distributed-actors/issues/383
  • 🔥 Hardened Handshake handling with non-existing or dead nodes, cluster performs exponential backoff when attempting to join such nodes and eventually gives up (rathe than trying to join a dead node forever) https://github.com/apple/swift-distributed-actors/issues/389

Membership Gossip

SWIM Membership / Failure Detector

Receptionist

  • 🎉 Cluster Receptionist: dropped previous placeholder implementation (full-state sync based) and replaced with operation sync based one. Similar to op-based CRDTs however not using the CRDT infra (long story why not, happy to discuss).
  • 🚀 Performance work resulting in API redesign for greater reusability and exploiting CoW semantics correctly for Receptionist.Listing avoiding copying of read-only data to subscribers. Greatly improved scalability of some sample apps, speeding up spawning/registering/looking up 15,000 actors to ~300ms, from previous patological cases of >1minute.

Serialization

  • Serialization infrastructure re-design around Codable (docs: https://github.com/apple/swift-distributed-actors/issues/538)
    • This means that Actor messages MUST be Codable, leading to an easier way between "hello world" and "now my hello world is distributed!"
    • Introduced NonTransportableActorMessage as way to "cheat" when non-codable types intended for local only actor messaging need to be sent around.
    • We are aware of some usability issues and limitations with that approach, e.g. the lack of automatically synthesized Codable conformances for enums with associated values, and we intend to fix those limitations in the language itself.
  • New serialization infrastructure makes use of Serialization.Manifest and (optionally, when available) _getMangledTypeName ⚠️ Need to discuss future direction with security aspects of this feature and alternative tradeoffs we can take. Had discussions planned but things got rescheduled a bit.

CRDT

Actorable

  • ⚠️ To re-gain the fine-grained control over what functions should be exposed as actor messages we introduced the "comment annotation" of // @actor.
    • Previously magic method names were used for this, resulting in terribly looking code like __notAnActorFunction(hello: Hello)
  • Continued work on full feature-parity with Behavior API
  • swift run GenActors is now compatible with Swift 5.3 (as well as 5.2)
  • Minor: Removed custom argument parser implementation and using Swift Argument Parser from OSS ecosystem 💕

Examples

  • Investigated and fixed performance issues uncovered by some internal sample apps.
  • New CRDT sample app: SampleDistributedCRDTScoreGame
  • Sample app spawning 10,000 actors and registering with Receptionist: SampleReceptionist used to stress test some of the new features

Stats

A total of 128 issues was closed since the 0.4.0 release, including 21 bug 🐞 and 26 failed 💥 tickets, resulting in the best and most stable release we ever made 🎉

We'd also like to thank Anton and Artem for their contributions, thanks!

You can view all resolved tickets on the 0.5.0 milestone.

commits  added  removed
    147  38076    23089 Konrad `ktoso` Malawski
     15   3534     1474 Yim Lee
      6   1804      829 Anton Volokhov
      5    737      608 Tomer Doron
      1     17        7 Artem Redkin
swift-distributed-actors - Cluster Hardening, new Membership, Plugins and Singletons

Published by ktoso over 4 years ago

Main features

  • Introduction of Membership and Cluster.Event types and infrastructure
  • Tons of cluster hardening and edge case handling
  • Cluster documentation and initial feature set
  • New feature: Actor (Cluster) Singleton
  • DiningPhilosophers also now in Actorable APIs, thanks to Erik Hornberger! 🎉

Stats

A total of 41 issues was closed since the 0.3.0 release 🎉
You can view all resolved tickets on the 0.4.0 milestone.

commits  added  removed
     72  21653    16625 Konrad `ktoso` Malawski
     11   2449      620 Yim Lee
      1    566        0 Erik Hornberger
      1      4        1 Tom Doron
swift-distributed-actors - 0.3.0 PoC XPC transport and Actorable interop

Published by ktoso over 4 years ago

Main features

Developed an XPC integration Proof of Concept (NOT production ready!), showing how we could utilize the Actor concept's similarity to XPC Services as well as the GenActors infrastructure to generate statically type checked Actor<>s for interacting and defining XPC services using Actorables.

If or not this PoC will become something serious remains to be seen however -- if this seems compelling (esp. the ability to easily move or interact with code in XPC Services, Remote Nodes and/or Linux isolated processes) all using the same programming model, let us know.

  • Introduced XPCTransport in DistributedActorsXPC module
  • Introduced ActorableTestProbe
  • Expanded capabilities of GenActors to generate Stubs and Codables.

Stats

A total of 9 issues was closed since the previous minor, all of them here: https://github.com/apple/swift-distributed-actors/milestone/7?closed=1

swift-distributed-actors - 0.2.4 Make settings public

Published by ktoso almost 5 years ago

Main features

  • SWIM settings were internal but should have been public -- fixed.

Stats

A total of 2 issues were resolved since 0.2.3, see here for details: https://github.com/apple/swift-distributed-actors/milestone/10?closed=1

Package Rankings
Top 11.79% on Swiftpackageindex.com
Related Projects
swiftpm-on-llbuild2

28 May 2020 39
ConcurrencyRecipes

Practical solutions to problems with Swift Concurrency

04 Dec 2023 1,121
ReactiveSwift

Streams of values over time

21 May 2016 2,964
swift-sample-distributed-actors-transport

Distributed actors transport example, for feature review

25 Aug 2021 95
Swift-Guide

Swift Guide

20 Sep 2020 6
promises

Promises is a modern framework that provides a synchronization construct for Swift and Objective-C.

19 Jan 2018 3,760
swift-async-queue

A queue that enables ordered sending of events from synchronous to asynchronous code

23 Nov 2022 142
SwiftCode

Swift2.2 ~Swift 5.x

28 May 2016 9
SwiftConcurrencyExample

Experiments with Swift async/await and structured concurrency

08 Jan 2021 42
arachne

Networking layer for apps using Swift Concurrency.

18 Oct 2021 4
websocket-actor-system

A client/server actor system for Swift distributed actors

31 Oct 2023 82
swift-tools-support-async

Common infrastructural helpers on top of NIO for llbuild2 and swiftpm-on-llbuild2 projects. This ...

13 Jun 2020 29
ReactiveCocoa

Cocoa framework and Obj-C dynamism bindings for ReactiveSwift.

02 Mar 2012 19,925
coherence

Coherence: a comprehensive resource management framework.

30 Oct 2015 3
swift-distributed-tracing

Instrumentation library for Swift server applications

24 Sep 2020 213