MTG-661 Add redis pool messenger #81
Conversation
…1.18.*` version + fmt and warnings fix
| MessengerType::Redis => { | ||
| RedisMessenger::new(config).await.map(|a| Box::new(a) as Box<dyn Messenger>) | ||
| } | ||
| _ => Err(MessengerError::ConfigurationError { | ||
| msg: "This Messenger type is not valid, unimplemented or you dont have the right crate features on.".to_string() | ||
| msg: "This Messenger type is not valid, unimplemented or you don't have the right crate features on.".to_string() |
There was a problem hiding this comment.
Nit: Its no longer based on crate features right so could just delete last part of comment?
There was a problem hiding this comment.
I don't think the last part of comment was removed, but its fine it was a nit comment anyways so still approving the PR
| Value::Data(bytes) => bytes, | ||
| Value::BulkString(bytes) => bytes, |
There was a problem hiding this comment.
What causes this change to be needed? I see its handled as Value::BulkString in recv as well but I don't see a config change for this. Will this be a breaking change for someone with existing redis config? Should we match both Value::Data and Value::BulkString?
There was a problem hiding this comment.
Redis crate update causes these changes. It was updated from 0.22.3 to 0.27.2, so Value enum was slightly changed. In previous version Data was representing arbitrary binary data and now BulkString is representing it.
Don't think that we should mathch it somehow because internal data type was not changed, it's still Vec<u8>.
There was a problem hiding this comment.
ok that should be fine then as it sounds like not a breaking change for users of previous versions of plerkle messenger.
| stream.local_buffer_last_flush.elapsed().as_millis() | ||
| ); | ||
| return Ok(()); | ||
| } else { |
There was a problem hiding this comment.
NIT: you not need else statement here as far as inside upper block you call return
|
Great work! |
What
This PR adds the ability for the messenger to stream data to multiple, separate Redis instances (not a cluster). Additionally, the Redis feature was removed since it's redundant, as Redis is the only supported queue service.
Why
With these changes, a single Geyser plugin can stream data to two different Redis instances, allowing one Solana node to feed data to multiple, separate indexers.
How
The
Messengertrait has been slightly refactored, and a new trait calledMessageStreamerwas introduced, which only handles write-related methods. A new entity,RedisPoolMessenger, was added to implementMessageStreamerand enable streaming to multiple Redis instances.