On @tg-x’s suggestion I looked at tjr’s message queuing library. @tg-x could you provide more input on how you would like this to be used?
Does it provide advantages over for example lwt’s streams? The message router is using those at the moment.
can you implement store-and-forward message queues with lwt streams only?
not just between currently connected clients, but asynchronously,
if the recipient connects later should be able to receive the contents of its queue.
Yes I think so but maybe I’m missing something? Not lwt streams only but lwt streams + key-value store.
My understanding is: if a recipient is not connected you put the message in the key-value store. When the recipient connects again you check the key-value store for pending messages.
Also, how long (roughly) do you want to store a message before throwing it away?
We need message queue/dequeue semantics for connected clients,
and indeed lwt-streams provides these as pointed out here.
There’s an alternative library, lwt-pipe that implements similar semantics. It can provide MQ-like persistency even when there’s no consumer connected at the moment.
We should have a look at the issues pointed out in this thread to see if any of that affects our use case (seen some issues mentioned around multiple producers/consumers, for multicast we have the one-to-many use case) and whether lwt-pipe would be more suitable.
For persistency, a key-value store would be used to store the messages (K: msg ID → V: msg),
and per-client message queues (K: client pub key → V: list of msg ids)
When a client disconnects we might want to keep its message queue in memory for a short while in case it reconnects shortly, this we can evaluate/figure out later as an optimization, that’s what an lwt-pipe-like in-memory message queue helps with.
Seems tjr-mq would be similar but the above two may be more stable / more widely used.
The maximum retention time is specified in the message header, and thus we’ll have a periodic garbage collection task throwing away msgs from the KV store.
A local policy can however override this to throw away things sooner to enforce per-node storage limits. (also for later)
Yes but still wondering, how long, roughly?
I understand this concerns connected clients. If the client is disconnected for weeks, it should be able to synchronize when it comes back online, using another mechanism, right?
yes, we need another mechanism for persistence:
messages are stored in the KV store with their ID as key,
and we need to store a list of message ids as the message queue for each disconnected client
1 Like