Allow a Hook to reload an individual client session #402
Conversation
In a clustered environment, client connections are distributed among multiple Server instances on different machines. After a client disconnects, leaving behind a persistent session state, its next login is likely to be on a different node. Because of this, in such a setup an individual Server instance should only keep Client instances corresponding to online client connections, and it should be able to reload an individual client's state (presumably from persistent storage) when that client connects. This commit adds support for such an environment by adding a new hook `StoredClientByID`. An implementation finds and returns any persistent client data for a given session ID. In practice the only necessary information turned out to be the saved subscriptions and in-flight ack messages. The hook also returns the prior 'Remote' property since the server logs that. The Server method `inheritClientSession` is extended to call this hook if there is no matching in-memory Client session. If the hook returns session data, it installs it into the Client object in the same way as the existing code. At the end of the Server method `attachClient`, after disconnection, the existence of a `StoredClientByID` hook is checked; if present, the method expires the Client instance so it won't hang around in memory and so the next connection will go to the hook to reload state.
Pull Request Test Coverage Report for Build 9038356648Details
💛 - Coveralls |
There was a problem hiding this comment.
Hi @snej ! Thanks for the PR, my apologies for the delay I've been caught up with work and life stuff. I'm into the general idea, but I think we need to make a few changes to the implementation to avoid breaking the rules about hooks - they should operate totally independently from the core server code. We can add new hooks and change signatures to reach this, however. See my comments.
Thanks very much :)
| @@ -485,6 +485,11 @@ func (s *Server) attachClient(cl *Client, listener string) error { | |||
| expire := (cl.Properties.ProtocolVersion == 5 && cl.Properties.Props.SessionExpiryInterval == 0) || (cl.Properties.ProtocolVersion < 5 && cl.Properties.Clean) | |||
| s.hooks.OnDisconnect(cl, err, expire) | |||
|
|
|||
| if s.hooks.Provides(StoredClientByID) { | |||
There was a problem hiding this comment.
I have mixed feelings about this. I like the overall idea of the implementation but I'm not sure we should be adding any hook-specific logic to the server code as it defeats the objective of having hooks. In this case the hook is mostly being used as a feature flag, whereas the hook logic should be stored inside the hook.
Maybe a better option here would be to enhance OnDisconnect so that it can overwrite the value of expire:
expire = s.hooks.OnDisconnect(cl, err, expire)
and modify the method signature accordingly:
func (h *Hook) OnDisconnect(cl *mqtt.Client, _ error, expire bool) bool {
| @@ -596,6 +601,42 @@ func (s *Server) inheritClientSession(pk packets.Packet, cl *Client) bool { | |||
| return true // [MQTT-3.2.2-3] | |||
| } | |||
|
|
|||
| // Look up a stored client that's not in memory yet: | |||
| if s.hooks.Provides(StoredClientByID) { | |||
There was a problem hiding this comment.
Same issue here. Possible solution would be to add this logic to the OnSessionEstablish hook and overwrite the values of Client.State directly, but without adding the client to the server. This will avoid the client being picked up in inheritClientSession and overwriting the new networked client values.
| @@ -1669,6 +1694,29 @@ func (s *Server) loadClients(v []storage.Client) { | |||
| } | |||
| } | |||
|
|
|||
| // newClientFromStorage creates a Client from a storage.Client. | |||
| func (s *Server) newClientFromStorage(c *storage.Client) *Client { | |||
There was a problem hiding this comment.
I'm not against it, but what's the intention behind extracting this into an additional method if it's only called from the same place?
In a clustered environment, client connections are distributed among multiple Server instances on different machines. After a client disconnects, leaving behind a persistent session state, its next login is likely to be on a different node. Because of this, in such a setup an individual Server instance should only keep Client instances corresponding to online client connections, and it should be able to reload an individual client's state (presumably from persistent storage) when that client reconnects.
This commit adds support for such an environment by adding a new hook
StoredClientByID. An implementation finds and returns any persistent client data for a given session ID. In practice the only necessary information turned out to be the saved subscriptions and in-flight ack messages. The hook also returns the prior 'Remote' property since the server logs that.The Server method
inheritClientSessionis extended to call this hook if there is no matching in-memory Client session. If the hook returns session data, it installs it into the Client object in the same way as the existing code.At the end of the Server method
attachClient, after disconnection, the existence of aStoredClientByIDhook is checked; if present, the method expires the Client instance so it won't hang around in memory and so the next connection will go to the hook to reload state.