-
Notifications
You must be signed in to change notification settings - Fork 142
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Important: In latest main, the server can not serve more than 1 user #512
Comments
Note: This version still works well |
I can confirm this issue is happening, it could go un noticed if the Docker container is set to |
Is this still an issue? We have had to revert to older containers due to instability issues. It is not clear it is on the Lorax side, but it could be. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
System Info
Meet this error when more than 1 user request to server (I try to run previous image version and it still work fine)
Information
Tasks
Reproduction
Start server by docker with model microsoft/Phi-3-mini-128k-instruct
Expected behavior
Server can serve many concurrent user at the same time
The text was updated successfully, but these errors were encountered: