SlimCluster has the Raft distributed consensus algorithm implemented in .NET. Additionally, it implements the SWIM cluster membership list (where nodes join and leave/die).
- Membership list is required to maintain what micro-service instances (nodes) constitute a cluster.
- Raft consensus helps propagate state across the micro-service instances and ensures there is a designated leader instance performing the coordination of work.
The library goal is to provide a common groundwork for coordination and consensus of your distributed micro-service instances. With that, the developer can focus on the business problem at hand. The library promises to have a friendly API and pluggable architecture.
The strategic aim for SlimCluster is to implement other algorithms to make distributed .NET micro-services easier and not require one to pull in a load of other 3rd party libraries or products.
This a relatively new project!
The path to a stable production release:
- âś… Step 1: Implement the SWIM membership over UDP + sample.
- âś… Step 2: Documentation on Raft consensus.
- âś… Step 3: Implement the Raft over TCP/UDP + sample.
- ⬜ Step 4: Documentation on SWIM membership.
- ⬜ Step 5: Other extensions and plugins.
| Name | Description | NuGet |
|---|---|---|
SlimCluster | The core cluster interfaces |  |
| Core abstractions | ||
SlimCluster.Membership | The membership core interfaces |  |
SlimCluster.Serialization | The core message serialization interfaces |  |
SlimCluster.Transport | The core transport interfaces |  |
SlimCluster.Persistence | The core node state persistence interfaces |  |
| Plugins | ||
SlimCluster.Consensus.Raft | Raft consensus algorithm implementation |  |
SlimCluster.Membership.Swim | SWIM membership algorithm implementation |  |
SlimCluster.Serialization.Json | JSON message serialization plugin |  |
SlimCluster.Transport.Ip | IP protocol transport plugin |  |
SlimCluster.Persistence.LocalFile | Persists node state into a local JSON file |  |
SlimCluster.AspNetCore | ASP.NET request routing to Leader node |  |
Check out the Samples folder on how to get started.
Setup membership discovery using the SWIM algorithm and consensus using Raft algorithm:
builder.Services.AddSlimCluster(cfg => { cfg.ClusterId = "MyCluster"; // This will use the machine name, in Kubernetes this will be the pod name cfg.NodeId = Environment.MachineName; // Transport will be over UDP/IP cfg.AddIpTransport(opts => { opts.Port = builder.Configuration.GetValue<int>("UdpPort"); opts.MulticastGroupAddress = builder.Configuration.GetValue<string>("UdpMulticastGroupAddress")!; }); // Protocol messages (and logs/commands) will be serialized using JSON cfg.AddJsonSerialization(); // Cluster state will saved into the local json file in between node restarts cfg.AddPersistenceUsingLocalFile("cluster-state.json"); // Setup Swim Cluster Membership cfg.AddSwimMembership(opts => { opts.MembershipEventPiggybackCount = 2; }); // Setup Raft Cluster Consensus cfg.AddRaftConsensus(opts => { opts.NodeCount = 3; // Use custom values or remove and use defaults opts.LeaderTimeout = TimeSpan.FromSeconds(5); opts.LeaderPingInterval = TimeSpan.FromSeconds(2); opts.ElectionTimeoutMin = TimeSpan.FromSeconds(3); opts.ElectionTimeoutMax = TimeSpan.FromSeconds(6); // Can set a different log serializer, by default ISerializer is used (in our setup its JSON) // opts.LogSerializerType = typeof(JsonSerializer); }); cfg.AddAspNetCore(opts => { // Route all ASP.NET API requests for the Counter endpoint to the Leader node for handling opts.DelegateRequestToLeader = r => r.Path.HasValue && r.Path.Value.Contains("/Counter"); }); }); // Raft app specific implementation builder.Services.AddSingleton<ILogRepository, InMemoryLogRepository>(); // For now, store the logs in memory only builder.Services.AddSingleton<IStateMachine, CounterStateMachine>(); // This is app specific machine that implements a distributed counter builder.Services.AddSingleton<ISerializationTypeAliasProvider, CommandSerializationTypeAliasProvider>(); // App specific state/logs command types for the replicated state machine // Requires packages: SlimCluster.Membership.Swim, SlimCluster.Consensus.Raft, SlimCluster.Serialization.Json, SlimCluster.Transport.Ip, SlimCluster.Persistence.LocalFile, SlimCluster.AspNetCoreThen somewhere in the micro-service, the ICluster can be used:
// Injected, this will be a singleton representing the cluster the service instances form. ICluster cluster; // Gives the current leader INode? leader = cluster.LeaderNode; // Gives the node representing current node INode self = cluster.SelfNode; // Provides a snapshot collection of the current nodes discovered and alive/healthy forming the cluster IEnumerable<INode> nodes = cluster.Nodes; // Provides a snapshot collection of the current nodes discovered and alive/healthy forming the cluster excluding self IEnumerable<INode> otherNodes = cluster.OtherNodes;The IClusterMembership can be used to understand membership changes:
// Injected: IClusterMembership ClusterMembership ClusterMembership.MemberJoined += (target, e) => { Logger.LogInformation("The member {NodeId} joined", e.Node.Id); PrintActiveMembers(); }; ClusterMembership.MemberLeft += (target, e) => { Logger.LogInformation("The member {NodeId} left/faulted", e.Node.Id); PrintActiveMembers(); }; ClusterMembership.MemberStatusChanged += (target, e) => { if (e.Node.Status == SwimMemberStatus.Suspicious) { Logger.LogInformation("The node {NodeId} is suspicious. All active members are: {NodeList}", e.Node.Id, string.Join(", ", ClusterMembership.Members.Where(x => x.Node.Status == SwimMemberStatus.Active))); } };- The service references SlimCluser NuGet packages and configures MSDI.
- Nodes (service instances) are communicating over UDP/IP and exchange protocol messages (SWIM and Raft).
- Cluster membership (nodes that form the cluster) is managed (SWIM).
- Cluster leader is elected at the beginning and in the event of failure (Raft).
- Logs (commands that chage state machine state) are replicated from leader to followers (Raft).
- State Machine in each Node gets logs (commands) applied which have been replicated to majority of nodes (Raft).
- Clients interact with the Cluster (state mutating operations are executed to Leader or Followers for reads) - depends on the use case.
cd src dotnet build dotnet pack --output ../distNuGet packaged end up in dist folder
To run tests you need to update the respective appsettings.json to match your cloud infrastructure or local infrastructure.
Run all tests:
dotnet testRun all tests except integration tests which require local/cloud infrastructure:
dotnet test --filter Category!=Integration