Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dynamic/blocking does not work for some services #443

Open
DasDunkel opened this issue Nov 7, 2024 · 5 comments
Open

dynamic/blocking does not work for some services #443

DasDunkel opened this issue Nov 7, 2024 · 5 comments
Labels
bug Something isn't working

Comments

@DasDunkel
Copy link

Describe the bug
Honestly, I have no real idea what the cause of this could even be

One of the services I host (dumb) starts and stops as expected when a request is made, however, if dynamic is enabled the page is not shown, and if blocking is enabled the request is not blocked, instead the user has to refresh the page themselves once the service is ready.

You can test it here, session length is 5 seconds so you don't have to wait around;
https://dumb.dunk.dev

Afaik it's not an issue with my overall configuration as it works for other services (example below)
https://overflow.dunk.dev

Context

  • Sablier version: 1.8.1
  • Provider: Docker 24.0.7
  • Reverse proxy: Caddy 2.8.4
  • Sablier is running in a container beside caddy

Expected behavior
The service starting page is shown/connection hangs until service is ready

Additional context
Setup in caddy;

route {
	sablier http://172.20.0.3:10000 {
       		group dumb
       		session_duration 5s
       		dynamic {
       	    		display_name Dumb
			refresh_frequency 2s
			theme ghost
       		}
	}
	reverse_proxy 172.20.0.107:5555
}
@DasDunkel DasDunkel added the bug Something isn't working label Nov 7, 2024
@acouvreur
Copy link
Member

Hey @DasDunkel

What is the log output of Sablier ?

I think that maybe the session duration is so short that it is never ready.

Currently the session duration starts when calling start.

For the dynamic, you have to set the refresh rate lower than the session duration.

@DasDunkel
Copy link
Author

DasDunkel commented Nov 7, 2024

Hi @acouvreur

Below is the output from sablier for each domain, I have tried increasing the session length to both 1 minute and 5 minutes, but the 'starting service' page still does not show

The service itself (dumb) starts almost immediately, which is the only real difference between other working services (besides the obvious stuff)

https://txt.dunk.dev/dunk/9e84c38c03f74a2b9e76922075cf8b58

Edit;
Enabling debug logging just adds the following lines;

time="2024-11-07T23:14:07Z" level=debug msg="expiring {Name:Dumb CurrentReplicas:1 DesiredReplicas:1 Status:ready Message:} in 1m0s"
time="2024-11-07T23:14:07Z" level=debug msg="map[dumb:[Dumb] overflow:[AnonymousOverflow]]"

@DasDunkel
Copy link
Author

After a bit of poking at random thoughts, and testing a few things, I think I figured it out, and it seems to be a weird edge case, let me try and explain;

The dumb container starts almost immediately, but it might take a second or two for it to be able to process incoming requests, so when sablier starts the container it seems to think it's instantly available, so instantly redirects, but because the service isn't fully ready, it returns an empty response.

I have tried adding a health check to the container which would ensure it's actually serving content, but unfortunately this service's container is so stripped down it doesn't even have bash/sh/ls, so I might need to figure out a way of doing that if something can't be done in sablier, maybe something like a forced initial wait option to give the service chance?

@acouvreur
Copy link
Member

You could try contributing to the image and add a healthcheck command, but I understand the wait option in Sablier to avoid that.

Maybe an option like consider-ready-after: 10s what do you think?

@DasDunkel
Copy link
Author

I'll definitely look into pushing a health check to dumb, but I think something like consider-ready-after would be a great addition, especially for some services where they can process requests while starting up, so would show as healthy, but are incredibly slow/unresponsive until they've finished things like startup tasks

And now that I think about it, I think dumb might be one of these types of services as I never got a 404/other error before it was ready to serve, but that's an issue for elsewhere.

Thanks for your time

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants