Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

My client sockets disconnect with a "ping timeout" reason after a few hours of connection and can't connect again #5012

Open
maximilien0405 opened this issue Apr 26, 2024 · 10 comments

Comments

@maximilien0405
Copy link

maximilien0405 commented Apr 26, 2024

Hi, i'm having a bit of a issue with my project. I'm building an app that will stay opened at all times in the device, with a non-stop connection through the sockets. However lately, all of my sockets disconnect and can't reconnect. I managed to print some logs from the device and here it is :

Capture d’écran 2024-04-26 à 17 07 56

As you can see on the screenshot, the socket connection starts at around 1:30 AM and then is disconnectd at 7:30 AM, but manages to reconnect each time until 9:14 where it disconnects and then can't reconnect with the reason "ping timeout".

I'm not sure where is the problem here. I'm using Angular 17 with ngx-socket-io version 4.6.1. In my backend i use NestJS with socket-io 4.7.2.

Since this problem that happend this morning i did theses steps to try to fix the problem (i haven't tried it yet) :

  • Upgraded my NestJS from v9 (a year old) to v10 (including the @nestjs/websockets package)
  • Upgraded socket-io in my NestJS from 4.7.2 to 4.7.5
  • Added a pingTimeout of 60000 to my WebSocketGateway (maybe this will fix it ??)

Any idea what could be causing the problem ? Also it's important to note than in my APP i have an interval that tries to connect to the socket every minute just in case it's disconnected, but as you can see on the logs, it can't even reconnect 🤔

Edit: Also important to note that it's only 4 cuccurent connections. With 1 CPU and 2GB of RAM. With NGINX and PM2 for my API.

@darrachequesne

@maximilien0405 maximilien0405 added the to triage Waiting to be triaged by a member of the team label Apr 26, 2024
@maximilien0405 maximilien0405 changed the title My client sockets disconnect with a "ping timeout" reason after a few hours of connection and don't connect again My client sockets disconnect with a "ping timeout" reason after a few hours of connection and can't connect again Apr 26, 2024
@darrachequesne
Copy link
Member

Hi! That's weird indeed. It seems the client is not even able to reach the server.

Is the server reachable? What happens if you restart the server? nginx? You mention using a device, are you on Android? iOS?

@maximilien0405
Copy link
Author

maximilien0405 commented May 1, 2024

Yes the server is reachable since i can connect it from another device or if i just reload the app. If i restart the server nothing changes, they still can't connect, even though they try to reconnect. It happens on IOS and Android. All of my devices are turned on and the app is open h24
Right now i just added a catch to the "ping timeout" error and added a location.reload() to reload the app and reconnect to the server (hopefully). I will also implement a way to do a location.reload() once a day, just to reconnect to the socket freshly and prevent any timeout problem @darrachequesne

Edit : Also important to note that all of them disconnected at the same time (i have some logs if you need)
Edit 2 : Maybe my error on socket.io-admin-ui is related to the problem ? (socketio/socket.io-admin-ui#86)

@Mpratyush54
Copy link

Once try that if you are emitting some random data at fix time interval probably less than 1/2 hrs as it appears that there is a time out error also try connecting it with new devices after this error arises

@GautamJain133
Copy link

yes, i am also facing the same issue if anyone knows how to solve this issue please revert. my application is always open on desktop either in minimize or in focus. but after some random time. on server it is showing that user is disconencted due to pingTimeout. i am using the latest version of socket io client and socket io both. website is build using mern

@GautamJain133
Copy link

@maximilien0405 have u got any permanent solution rather that the hacks like window.reload and all. is this window.reload mechanism works properly

@GautamJain133
Copy link

GautamJain133 commented May 17, 2024

@darrachequesne please help
client code :
socket = io(serverurl.com/react-client, {
path: "/api2/socket.io",
query: { accounts: jsonString },
forceNew: true,
});
server code:
const io = new Server(server, {
cors: {
origin: "*",
},
path: "/api2/socket.io/",
pingTimeout: 60000, // 1 minute
pingInterval: 2000, // 2 second
});

@maximilien0405
Copy link
Author

@GautamJain133 Yes the window.reload works but like you said it's not a great solution.. but also i think it's nice sometime to reload the socket connection if it's connected for long periods of time, so until a solution is found it's not that bad :/

@mainisameer
Copy link

mainisameer commented May 25, 2024

Folks we are facing the same issue on Chrome where this ping pong mechanism is causing a whole bunch of issues. Is there a way to disable this ping pong also what is the client you are using. This issue seems to be more with Chrome than other browser

@darrachequesne darrachequesne added needs investigation and removed to triage Waiting to be triaged by a member of the team labels Sep 17, 2024
@darrachequesne
Copy link
Member

Hi! Were you able to find the culprit?

@maximilien0405
Copy link
Author

maximilien0405 commented Sep 23, 2024

@darrachequesne Hello ! From what I remember that probably came from the fact that my client and server versions where not compatible together.

To keep an active connection at all times I've implemented a few tricks :

  • I reload my app every day to refresh the socket connection and the app (in case it takes more and more ram after x days).

  • I intercept every disconnect event and log it on my own log system to see exactly where the problem was.

  • I implemented an interval loop that checks every second if the sockets is connected, if not connect it.

  • Since I use Nginx i read and applied some changes following the documentation on this page. (that also fixed my issue with me not being able to connect via the Socket IO admin panel.

  • And finally I also changed theses options in my client :

transports: ['websocket', 'polling'],
closeOnBeforeunload: true,
forceNew: true

For now everything works ! I'm still going to implement some things to fix things just in case my users disconnect from the socket and it does not reconnect (for example sending a notification on the app that reloads the whole app).

Anyway that's about it, I hope I helped those who where lost and had the same issues !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants