Skip to content

Commit

Permalink
Doc: make README shorter
Browse files Browse the repository at this point in the history 
Add wechat
  • Loading branch information
@drmingdrmer
drmingdrmer committed Dec 13, 2023
1 parent 48d06d4 commit b073c64
Show file tree
Hide file tree
Showing 2 changed files with 63 additions and 64 deletions.
122 changes: 58 additions & 64 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,15 @@ This project intends to improve raft as the next-generation consensus protocol f
Currently, openraft is the consensus engine of meta-service cluster in [databend](https://github.com/datafuselabs/databend).


- **Get started**: [The guide](https://docs.rs/openraft/latest/openraft/docs/getting_started/index.html) is the best place to get started,
followed by [the docs](https://docs.rs/openraft/latest/openraft/docs/index.html) for more in-depth details.
- 🚀 **Get started**:
- [Openraft guide](https://docs.rs/openraft/latest/openraft/docs/getting_started/index.html) is the best place to get started,
- [Openraft docs](https://docs.rs/openraft/latest/openraft/docs/index.html) for more in-depth details,
- [Openraft FAQ](https://docs.rs/openraft/latest/openraft/docs/faq/index.html) explains some common questions.

- [Openraft FAQ](https://docs.rs/openraft/latest/openraft/docs/faq/index.html) explains some common questions.

- 🙌 Questions? Join the [Discord channel](https://discord.gg/ZKw3WG7FQ9) or start a [discussion](https://github.com/datafuselabs/openraft/discussions/new).
- 🙌 **Questions**?
- Join the [Discord channel](https://discord.gg/ZKw3WG7FQ9),
- or start a [Discussion](https://github.com/datafuselabs/openraft/discussions/new),
- or join our wechat group: `drmingdrmer`.

- Openraft is derived from [async-raft](https://docs.rs/crate/async-raft/latest) with several bugs fixed: [Fixed bugs](https://github.com/datafuselabs/openraft/blob/main/derived-from-async-raft.md).

Expand All @@ -47,7 +50,7 @@ Currently, openraft is the consensus engine of meta-service cluster in [databend
Check our [change-log](https://github.com/datafuselabs/openraft/blob/main/change-log.md). A commit message starts with a keyword to indicate the modification type of the commit:

- `DataChange:` on-disk data types changes, which may require manual upgrade.
- `Change:` if it introduces incompatible changes.
- `Change:` if it introduces incompatible changes.
- `Feature:` if it introduces compatible non-breaking new features.
- `Fix:` if it just fixes a bug.

Expand All @@ -57,33 +60,28 @@ Currently, openraft is the consensus engine of meta-service cluster in [databend
The main branch is for the [release-0.9](https://github.com/datafuselabs/openraft/tree/release-0.9).

- **Branch [release-0.8](https://github.com/datafuselabs/openraft/tree/release-0.8)**:
Latest published: [v0.8.8](https://github.com/datafuselabs/openraft/tree/v0.8.8) | [Change log v0.8.8](https://github.com/datafuselabs/openraft/blob/release-0.8/change-log.md#v088) |
⬆️ [0.7 to 0.8 upgrade guide](https://docs.rs/openraft/0.8.4/openraft/docs/upgrade_guide/upgrade_07_08/index.html) | ⬆️ [0.8.3 to 0.8.4 upgrade guide](https://docs.rs/openraft/0.8.4/openraft/docs/upgrade_guide/upgrade_083_084/index.html) |
Latest: ( [v0.8.8](https://github.com/datafuselabs/openraft/tree/v0.8.8) | [Change log](https://github.com/datafuselabs/openraft/blob/release-0.8/change-log.md#v088) );
Upgrade guide: ⬆️ [0.7 to 0.8](https://docs.rs/openraft/0.8.4/openraft/docs/upgrade_guide/upgrade_07_08/index.html), ⬆️ [0.8.3 to 0.8.4](https://docs.rs/openraft/0.8.4/openraft/docs/upgrade_guide/upgrade_083_084/index.html);
`release-0.8` **Won't** accept new features but only bug fixes.

- **Branch [release-0.7](https://github.com/datafuselabs/openraft/tree/release-0.7)**:
Latest published: [v0.7.6](https://github.com/datafuselabs/openraft/tree/v0.7.6) | [Change log v0.7.6](https://github.com/datafuselabs/openraft/blob/release-0.7/change-log.md#v076) |
⬆️ [0.6 to 0.7 upgrade guide](https://docs.rs/openraft/0.8.4/openraft/docs/upgrade_guide/upgrade_06_07/index.html) |
Latest: ( [v0.7.6](https://github.com/datafuselabs/openraft/tree/v0.7.6) | [Change log](https://github.com/datafuselabs/openraft/blob/release-0.7/change-log.md#v076) );
Upgrade guide: ⬆️ [0.6 to 0.7](https://docs.rs/openraft/0.8.4/openraft/docs/upgrade_guide/upgrade_06_07/index.html);
`release-0.7` **Won't** accept new features but only bug fixes.

- **Branch [release-0.6](https://github.com/datafuselabs/openraft/tree/release-0.6)**:
Latest published: [v0.6.8](https://github.com/datafuselabs/openraft/tree/v0.6.8) | [Change log v0.6](https://github.com/datafuselabs/openraft/blob/release-0.6/change-log.md) |
Latest: ( [v0.6.8](https://github.com/datafuselabs/openraft/tree/v0.6.8) | [Change log](https://github.com/datafuselabs/openraft/blob/release-0.6/change-log.md) );
`release-0.6` **won't** accept new features but only bug fixes.

# Roadmap

- [x] **2022-10-31** [Extended joint membership](https://docs.rs/openraft/latest/openraft/docs/data/extended_membership/index.html)

- [x] **2023-02-14** Minimize confliction rate when electing;
See: [Openraft Vote design](https://docs.rs/openraft/latest/openraft/docs/data/vote/index.html);
Or use standard raft mode with [feature flag `single-term-leader`](https://docs.rs/openraft/latest/openraft/docs/feature_flags/index.html).

- [x] **2023-04-26** Goal performance is 1,000,000 put/sec.

- [ ] Reduce the complexity of vote and pre-vote: [get rid of pre-vote RPC](https://github.com/datafuselabs/openraft/discussions/15);

- [ ] Support flexible quorum, e.g.: [Hierarchical Quorums](https://zookeeper.apache.org/doc/r3.5.9/zookeeperHierarchicalQuorums.html)

- [ ] Consider introducing read-quorum and write-quorum,
improve efficiency with a cluster with an even number of nodes.

Expand All @@ -99,62 +97,39 @@ Currently, openraft is the consensus engine of meta-service cluster in [databend
The benchmark is focused on the Openraft framework itself and is run on a
minimized store and network. This is **NOT a real world** application benchmark!!!

Benchmark history:
| clients | put/s | ns/op |
| --: | --: | --: |
| 256 | **1,014,000** | 985 |
| 64 | **730,000** | 1,369 |
| 1 | 70,000 | **14,273** |

| Date | clients | put/s | ns/op | Changes |
| :-- | --: | --: | --: | :-- |
| 2023-04-26 | 256 | **1,014,000** | 985 | |
| 2023-04-25 | 64 | **730,000** | 1,369 | Split channels |
| 2023-04-24 | 64 | **652,000** | 1,532 | Reduce metrics report rate |
| 2023-04-23 | 64 | **467,000** | 2,139 | State-machine moved to separate task |
| | 1 | 70,000 | **14,273** | |
| 2023-02-28 | 1 | 48,000 | **20,558** | |
| 2022-07-09 | 1 | 45,000 | **21,784** | Batch purge applied log |
| 2022-07-07 | 1 | 43,000 | **23,218** | Use `Progress` to track replication |
| 2022-07-01 | 1 | 41,000 | **23,255** | |


To access the benchmark, go to the `./cluster_benchmark` folder and run `make
bench_cluster_of_3`.

The benchmark is carried out with varying numbers of clients because:
- The `1 client` benchmark shows the average **latency** to commit each log.
- The `64 client` benchmark shows the maximum **throughput**.

The benchmark is conducted with the following settings:
- No network.
- In-memory store.
- A cluster of 3 nodes in a single process on a Mac M1-Max laptop.
- Request: empty
- Response: empty

For benchmark detail, go to the [./cluster_benchmark](./cluster_benchmark) folder.

# Features

- It is fully reactive and embraces the async ecosystem.
It is driven by actual Raft events taking place in the system as opposed to being driven by a `tick` operation.
Batching of messages during replication is still used whenever possible for maximum throughput.

- Storage and network integration is well defined via two traits `RaftStorage` & `RaftNetwork`.
This provides applications maximum flexibility in being able to choose their storage and networking mediums.

- All interaction with the Raft node is well defined via a single public `Raft` type, which is used to spawn the Raft async task, and to interact with that task.
The API for this system is clear and concise.
- **Async and Event-Driven**: Operates based on Raft events without reliance on periodic ticks, optimizing message batching for high throughput.
- **Extensible Storage and Networking**: Customizable via `RaftLogStorage`, `RaftStateMachine` and `RaftNetwork` traits, allowing flexibility in choosing storage and network solutions.
- **Unified Raft API**: Offers a single `Raft` type for creating and interacting with Raft tasks, with a straightforward API.
- **Cluster Formation**: Provides strategies for initial cluster setup as detailed in the [cluster formation guide](https://docs.rs/openraft/latest/openraft/docs/cluster_control/cluster_formation/index.html).
- **Built-In Tracing Instrumentation**: The codebase integrates [tracing](https://docs.rs/tracing/) for logging and distributed tracing, with the option to [set verbosity levels at compile time](https://docs.rs/tracing/latest/tracing/level_filters/index.html).

- Log replication is fully pipelined and batched for optimal performance.
Log replication also uses a congestion control mechanism to help keep nodes up-to-date as efficiently as possible.
## Functionality:

- It fully supports dynamic cluster membership changes with joint config.
The buggy single-step membership change algo is not considered.
See the [`dynamic membership`](https://docs.rs/openraft/latest/openraft/docs/cluster_control/dynamic_membership/index.html) chapter in the guide.
- ✅ **Leader election**: by policy or manually([`trigger_elect()`][]).
- ✅ **Non-voter(learner) Role**: refer to [`add_learner()`][].
- ✅ **Log Compaction**(snapshot of state machine): by policy or manully([`trigger_snapshot()`]).
- ✅ **Snapshot replication**.
- ✅ **Dynamic Membership**: using joint membership config change. Refer to [dynamic membership](https://docs.rs/openraft/latest/openraft/docs/cluster_control/dynamic_membership/index.html)
- ⛔️ **Wont support**: Single-step config change. <!-- TODO: explain why -->
- ✅ Toggle heartbeat: [`enable_heartbeat`][].
- ✅ Toggle election: [`enable_elect`][].
- ✅ Purge log by policy or manually: [`purge_log()`][].

- Details on initial cluster formation, and how to effectively do so from an application's perspective,
are discussed in the [cluster formation](https://docs.rs/openraft/latest/openraft/docs/cluster_control/cluster_formation/index.html) chapter in the guide.

- Automatic log compaction with snapshots, as well as snapshot streaming from the leader node to follower nodes is fully supported and configurable.

- The entire code base is [instrumented with tracing](https://docs.rs/tracing/).
This can be used for [standard logging](https://docs.rs/tracing/latest/tracing/index.html#log-compatibility), or for [distributed tracing](https://docs.rs/tracing/latest/tracing/index.html#related-crates), and the verbosity can be [statically configured at compile time](https://docs.rs/tracing/latest/tracing/level_filters/index.html) to completely remove all instrumentation below the configured level.
<!--
TODO: 0.9
- ✅ Linearizable read.
-->


# Who use it
Expand All @@ -169,3 +144,22 @@ Check out the [CONTRIBUTING.md](https://github.com/datafuselabs/openraft/blob/ma
# License

Openraft is licensed under the terms of the [MIT License](https://en.wikipedia.org/wiki/MIT_License#License_terms) or the [Apache License 2.0](http://www.apache.org/licenses/LICENSE-2.0), at your choosing.

[`change_membership()`]: https://docs.rs/openraft/latest/openraft/raft/struct.Raft.html#method.change_membership
[`add_learner()`]: https://docs.rs/openraft/latest/openraft/raft/struct.Raft.html#method.add_learner
[`purge_log()`]: https://docs.rs/openraft/latest/openraft/raft/struct.Raft.html#method.purge_log

[`enable_heartbeat`]: https://docs.rs/openraft/latest/openraft/struct.Config.html#structfield.enable_heartbeat
[`enable_elect`]: https://docs.rs/openraft/latest/openraft/struct.Config.html#structfield.enable_elect


<!-- TODO: update these API when 0.9 is published: -->

[`trigger_elect()`]: https://docs.rs/openraft/latest/openraft/raft/struct.Raft.html#method.trigger_elect
[`trigger_snapshot()`]: https://docs.rs/openraft/latest/openraft/raft/struct.Raft.html#method.trigger_snapshot

<!--
TODO: enable when 0.9 is published:
[`ensure_linearizable()`]: xx
-->
5 changes: 5 additions & 0 deletions cluster_benchmark/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ and an in-process network that uses function calls to simulate RPC.
| 64 | **730,000** | 1,369 |
| 1 | 70,000 | **14,273** |

The benchmark is carried out with varying numbers of clients because:
- The `1 client` benchmark shows the average **latency** to commit each log.
- The `64 client` benchmark shows the maximum **throughput**.

The benchmark is conducted with the following settings:
- No network.
Expand All @@ -26,6 +29,8 @@ The benchmark is conducted with the following settings:

## Run it

`make bench_cluster_of_3` in repo root folder, or in this folder:

```sh
cargo test --test benchmark --release bench_cluster_of_3 -- --ignored --nocapture
```

0 comments on commit b073c64

Please sign in to comment.