From aca36417a767b3677b16d187d8c5191361e480cf Mon Sep 17 00:00:00 2001 From: andie787 <4andie@gmail.com> Date: Tue, 13 Aug 2024 16:48:35 -0400 Subject: [PATCH 1/6] start changes to autostart docs --- apps/fine-tune-apps.html.markerb | 2 +- launch/autostart-stop.html.markerb | 126 ------------------ launch/autostop-autostart.html.markerb | 101 ++++++++++++++ partials/_apps_nav.html.erb | 2 +- partials/_firecracker_nav.html.erb | 1 + reference/configuration.html.markerb | 8 +- .../fly-proxy-autostop-autostart.html.markerb | 53 ++++++++ reference/fly-proxy.html.markerb | 2 +- 8 files changed, 162 insertions(+), 133 deletions(-) delete mode 100644 launch/autostart-stop.html.markerb create mode 100644 launch/autostop-autostart.html.markerb create mode 100644 reference/fly-proxy-autostop-autostart.html.markerb diff --git a/apps/fine-tune-apps.html.markerb b/apps/fine-tune-apps.html.markerb index cfeb3140b1..6343890283 100644 --- a/apps/fine-tune-apps.html.markerb +++ b/apps/fine-tune-apps.html.markerb @@ -1,5 +1,5 @@ --- -title: Tips to fine-tune and (not) benchmark your app on Fly.io +title: Tips to fine-tune your app on Fly.io layout: docs nav: apps redirect_from: /docs/reference/fine-tune-apps/ diff --git a/launch/autostart-stop.html.markerb b/launch/autostart-stop.html.markerb deleted file mode 100644 index d482a29faa..0000000000 --- a/launch/autostart-stop.html.markerb +++ /dev/null @@ -1,126 +0,0 @@ ---- -title: Automatically stop and start Machines -layout: docs -nav: apps -redirect_from: /docs/apps/autostart-stop/ ---- - -
- -
- -Fly Machines are fast to start and stop, and you don't pay for their CPU and RAM when they're in a `stopped` or `suspended` state. For Fly Apps with a service configured, [Fly Proxy](/docs/reference/fly-proxy) can automatically start and stop or suspend existing Machines based on incoming requests, so that your app can meet demand without keeping extra Machines running. You can also set a minimum number of machines to keep running at all times in your primary region. - -The autostop/autostart feature also works well for apps that [shut down automatically when idle](#stop-a-machine-by-terminating-its-main-process). The Fly Proxy can still restart your app when there's traffic. - -## Configure automatic start and stop - -The autostop/autostart settings apply per service, and you set them within the services configured in the `fly.toml` file. See the [[[services]]](/docs/reference/configuration/#the-services-sections) or [[http_service]](/docs/reference/configuration/#the-http_service-section) docs for details. - -Example configuration: - -```toml -... -[[services]] - internal_port = 8080 - protocol = "tcp" - auto_stop_machines = "stop" - auto_start_machines = true - min_machines_running = 0 -... -``` - -For the preceding example, Fly Proxy will automatically stop Machines when the app has excess capacity and start them again when needed. The app will eventually scale all the way down to zero running Machines if there's no traffic. - -Autostop/autostart setting descriptions: - -* `auto_stop_machines`: The action, if any, that Fly Proxy should take when the app is idle for several minutes. Options are `"off"`, `"stop"`, or `"suspend"`. -* `auto_start_machines`: Whether Fly Proxy should automatically start Machines based on requests and capacity. -* `min_machines_running`: The minimum number of Machines to keep running in the primary region when `auto_stop_machines` is set to `"stop"` or `"suspend"`. - -Concurrency limits configured for services also affect [how automatic starts and stops work](#how-it-works). - -### Default settings - -When you create an app using the `fly launch` command, the default settings in `fly.toml` are: - -```toml -... -[http_service] - internal_port = 8080 - force_https = true - auto_stop_machines = "stop" - auto_start_machines = true - min_machines_running = 0 -... -``` - -For apps that don't explicitly specify any autostop/autostart settings in `fly.toml`, the Fly Proxy will automatically start stopped Machines when needed, but won't automatically stop or suspend them. - -### Recommended settings - -If your app already [exits when idle](#stop-a-machine-by-terminating-its-main-process), then you can set `auto_start_machines = true` and `auto_stop_machines = "off"` to have Fly Proxy automatically restart the Machines stopped by your app. - -If your app doesn't exit when idle, then we recommend setting `auto_stop_machines` and `auto_start_machines` so that they are both enabled or both disabled, to avoid having Machines that either never start or never stop. For example, if `auto_start_machines = false` and `auto_stop_machines = "stop"`, then Fly Proxy automatically stops your Machines when there's low traffic but doesn't start them again. When all or most of your Machines stop, requests to your app could start failing. - -To keep one or more Machines running all the time in your primary region, set `min_machines_running` to `1` or higher. `min_machines_running` has no effect unless you set `auto_stop_machines` to `"stop"` or `"suspend"`. - -`min_machines_running` only applies to Machines running in your primary region. If `min_machines_running = 1` and there's no traffic to your app, then Fly Proxy will stop Machines until eventually there is only one Machine running in your primary region. - -There's no "maximum machines running" setting, because the maximum number of Machines is the total number of Machines you've created for your app. - -## How it works - -The Fly Proxy runs a process to automatically start and stop or suspend existing Fly Machines every few minutes. - -
-The autostop/autostart feature only works on existing Machines and never creates or destroys Machines for you. The maximum number of running Machines is the number of Machines you've created for your app using `fly scale count` or `fly machine clone`. Learn more about [scaling the number of Machines](/docs/apps/scale-count/). -
- -### When Fly Proxy stops or suspends Machines - -When `auto_stop_machines` is set to `"stop"` or `"suspend"` in your `fly.toml`, the proxy looks at Machines running in a single region and uses the concurrency [`soft_limit` setting](/docs/reference/configuration/#the-http_service-section) for each Machine to determine if there's excess capacity. If the proxy decides there's excess capacity, it stops or suspends exactly one Machine. The proxy repeats this process every few minutes, stopping or suspending only one Machine per region, if needed, each time. - -If you have the [`kill_signal` and `kill_timeout` options](/docs/reference/configuration/#runtime-options) configured in your `fly.toml` file, then Fly Proxy uses those settings when it stops a Machine. - -Fly Proxy determines excess capacity per region as follows: - -* If there's more than one Machine in the region: - * the proxy determines how many running Machines are over their `soft_limit` setting and then calculates excess capacity: `excess capacity = num of machines - (num machines over soft limit + 1)` - * if excess capacity is 1 or greater, then the proxy stops or suspends one Machine -* If there's only one Machine in the region: - * the proxy checks if the Machine has any traffic - * if the Machine has no traffic (a load of 0), then the proxy stops or suspends the Machine - -### When Fly Proxy starts Machines - -When `auto_start_machines = true` in your `fly.toml`, the Fly Proxy restarts a Machine in the nearest region when required. - -Fly Proxy determines when to start a Machine as follows: - -* The proxy waits for a request to your app. -* If all the running Machines are above their `soft_limit` setting, then the proxy starts a stopped or suspended Machine in the nearest region (if there are any stopped or suspended Machines). -* The proxy routes the request to the newly started Machine. - -## When to use the autostop/autostart feature - -If your app has highly variable request workloads, then you can use `auto_stop_machines` and `auto_start_machines` to manage your Fly Machines as demand decreases and increases. This could reduce costs, because you'll never have to run excess Machines to handle peak load; you'll only run, and get charged for, the number of Machines that you need. - -The difference between this feature and what's typical in autoscaling, is that it doesn't create new Machines up to a specified maximum. It automatically starts only existing Machines. For example, if you want to have a maximum of 10 Machines available to service requests, then you need to create 10 Machines for your app. - -If you need all your app's Machines to run continuously, then you can set `auto_stop_machines` to `"off"` and `auto_start_machines` to `false`. - -If you only need a certain number of your app's Machines to run continuously, then you can set `auto_stop_machines` to `"suspend"` or `"stop"` and `min_machines_running` to `1` or higher. Note that `min_machines_running` only applies to your app's primary region. - -## Stop a Machine by terminating its main process - -Setting your app to automatically stop or suspend Machines when there's excess capacity using `auto_stop_machines` can be a substitute for when your app doesn't shut itself down automatically after a period of inactivity. If you want a custom shutdown process for your app, then you can code your app to exit when idle. - -Here are some examples: - -* [Shutting Down a Phoenix App When Idle](https://fly.io/phoenix-files/shut-down-idle-phoenix-app/): a post by Chris McCord on adding a task to an Elixir app's supervision tree that shuts down the Erlang runtime when there are no active connections. -* For Rails apps, the `dockerfile-rails` generator provides a [--max-idle](https://github.com/rubys/dockerfile-rails#addremove-a-feature+external) option that exits after _n_ seconds of inactivity. -* [A Tired Proxy in Go](https://github.com/superfly/tired-proxy+external) used in [Building an In-Browser IDE the Hard Way](https://fly.io/blog/remote-ide-machines/). [There's a community fork with more recent updates](https://community.fly.io/t/improved-tired-proxy-for-use-with-fly-machines/10584). -* A minimal demo app in TypeScript/Remix: [code](https://github.com/fly-apps/autoscale-to-zero-demo+external) & [demo](https://autoscale-to-zero-demo.fly.dev/+external). - -Fly Postgres also [supports scaling to zero](https://community.fly.io/t/scale-to-zero-postgres-for-hobby-projects/12212). diff --git a/launch/autostop-autostart.html.markerb b/launch/autostop-autostart.html.markerb new file mode 100644 index 0000000000..dcc253b955 --- /dev/null +++ b/launch/autostop-autostart.html.markerb @@ -0,0 +1,101 @@ +--- +title: Autostop/autostart Machines +layout: docs +nav: apps +redirect_from: + - /docs/apps/autostart-stop/ + - /docs/launch/autostart-stop/ +--- + +For Fly Apps with a service configured, [Fly Proxy](/docs/reference/fly-proxy/) can automatically start and stop or suspend existing Machines based on incoming requests, so that your app can meet demand without keeping extra Machines running. Fly Machines are fast to start and stop, and you don't pay for their CPU and RAM when they're in a `stopped` or `suspended` state. + +Learn more about exactly how [Fly Proxy autostops and autostarts Machines](/docs/reference/fly-proxy-autostop-autostart/). + +## When to use autostop/autostart + +You can reduce resource usage and costs by using autostop/autostart to manage your Fly Machines as demand decreases and increases. You'll never have to run excess Machines to handle peak load; you'll only run, and get charged for, the number of Machines that you need. Autostart/autostop works well for apps with highly variable workloads, for smaller apps with low or sporadic traffic, and for most apps that aren't accepting requests continuously. If you need to, you can keep one or more Machines running in your primary region. + +## Configure autostop/autostart + +The autostop/autostart settings apply per service in an app's `fly.toml` file. See the [[[services]]](/docs/reference/configuration/#the-services-sections) or [[http_service]](/docs/reference/configuration/#the-http_service-section) docs for details about services. + +Autostop/autostart settings: + +* **`auto_stop_machines`:** The action, if any, that Fly Proxy should take when the app is idle for several minutes. Options are `"off"`, `"stop"`, or `"suspend"`. If `"off"`, Fly Proxy will never stop Machines. If `"stop"`, Fly Proxy stops Machines when the app has excess capacity. If `"suspend"`, Fly Proxy suspends Machines (if possible) when the app has excess capacity. Starting a Machine from a `suspended` state is faster than starting a Machine from a `stopped` state, but there are [some caveats](https://community.fly.io/t/autosuspend-is-here-machine-suspension-is-enabled-everywhere/20942). +* **`auto_start_machines`:** Whether Fly Proxy should automatically start Machines based on requests and capacity. +* **`min_machines_running`:** The minimum number of Machines to keep running in the primary region when `auto_stop_machines` is set to `"stop"` or `"suspend"`. + +Example configuration: + +```toml +... +[[services]] + internal_port = 8080 + protocol = "tcp" + auto_stop_machines = "stop" + auto_start_machines = true + min_machines_running = 1 +... +``` + +In the preceding example, Fly Proxy will automatically stop Machines when the app has excess capacity and start them again when needed based on traffic to the app. The app will eventually scale down to one running Machine in the primary region if there's no traffic. + +Concurrency limits configured for services in the `fly.toml` file also affect [how automatic starts and stops work](#how-it-works). + +## Recommendations for autostop/autostart configuration + +In general, unless your app shuts itself down when idle, we recommend setting `auto_stop_machines` and `auto_start_machines` so that they are both enabled or both disabled, to avoid having Machines that either never start or never stop. For example, if `auto_start_machines = false` and `auto_stop_machines = "stop"`, then Fly Proxy automatically stops your Machines when there's low traffic but doesn't start them again. When all or most of your Machines stop, requests to your app could start failing. + +### Minimum number of Machines running + +To keep one or more Machines running all the time in your primary region, set `min_machines_running` to `1` or higher. If `min_machines_running = 1` and there's no traffic to your app, then Fly Proxy will stop Machines until eventually there is only one Machine running in your primary region. `min_machines_running` has no effect unless you set `auto_stop_machines` to `"stop"` or `"suspend"`. + +### Maximum number of Machines running + +There's no "maximum machines running" setting, because the maximum number of Machines is the total number of Machines in your app. + +The autostop/autostart feature never creates or destroys Machines for you. The maximum number of running Machines is the number of Machines created for your app on launch, or using `fly scale count` or `fly machine clone`. For example, if you want to have a maximum of 10 Machines available to service requests, then you need to create 10 Machines for your app. Learn more about [scaling the number of Machines](/docs/apps/scale-count/). + +### Keep all Machines running continuously + +If you need all your app's Machines to run continuously, then you can set `auto_stop_machines` to `"off"` and `auto_start_machines` to `false`. + +If you only need a certain number of your app's Machines to run continuously, then you can set `auto_stop_machines` to `"suspend"` or `"stop"` and `min_machines_running` to `1` or higher. Note that `min_machines_running` only applies to your app's primary region. + +### Apps that shut down when idle + +Setting your app to automatically stop or suspend Machines when there's excess capacity using `auto_stop_machines` can be a substitute for when your app doesn't shut itself down automatically after a period of inactivity. If your app already shuts down when idle, then you can set `auto_start_machines = true` and `auto_stop_machines = "off"` to have Fly Proxy automatically restart the Machines stopped by your app. + +If you want a custom shutdown process for your app, then you can code your app to exit when idle. + +Here are some examples: + +* [Shutting Down a Phoenix App When Idle](https://fly.io/phoenix-files/shut-down-idle-phoenix-app/): a post by Chris McCord on adding a task to an Elixir app's supervision tree that shuts down the Erlang runtime when there are no active connections. +* For Rails apps, the `dockerfile-rails` generator provides a [--max-idle](https://github.com/rubys/dockerfile-rails#addremove-a-feature+external) option that exits after _n_ seconds of inactivity. +* [A Tired Proxy in Go](https://github.com/superfly/tired-proxy+external) used in [Building an In-Browser IDE the Hard Way](https://fly.io/blog/remote-ide-machines/). [There's a community fork with more recent updates](https://community.fly.io/t/improved-tired-proxy-for-use-with-fly-machines/10584). +* A minimal demo app in TypeScript/Remix: [code](https://github.com/fly-apps/autoscale-to-zero-demo+external) & [demo](https://autoscale-to-zero-demo.fly.dev/+external). + +Fly Postgres also [supports scaling to zero](https://community.fly.io/t/scale-to-zero-postgres-for-hobby-projects/12212). + +## Default settings + +The defaults differ for new apps created with `fly launch` and for apps with no settings configured for autostop/autostart. + +### Apps created with `fly launch` + +When you create a new app using the `fly launch` command, the default settings in `fly.toml` are: + +```toml +... +[http_service] + internal_port = 8080 + force_https = true + auto_stop_machines = "stop" + auto_start_machines = true + min_machines_running = 0 +... +``` + +### Apps with no autostop/autostart settings + +For apps that don't explicitly specify any autostop/autostart settings in `fly.toml`, the Fly Proxy will automatically start stopped Machines when needed, but won't automatically stop or suspend them. diff --git a/partials/_apps_nav.html.erb b/partials/_apps_nav.html.erb index 0ef1835b65..73b656fecf 100644 --- a/partials/_apps_nav.html.erb +++ b/partials/_apps_nav.html.erb @@ -23,7 +23,7 @@ { text: "Deploy an app", path: "/docs/launch/deploy/" }, { text: "Scale Machine CPU and RAM", path: "/docs/launch/scale-machine/" }, { text: "Scale the number of Machines", path: "/docs/launch/scale-count/" }, - { text: "Autostop/autostart Machines", path: "/docs/launch/autostart-stop/" }, + { text: "Autostop/autostart Machines", path: "/docs/launch/autostop-autostart/" }, { text: "Autoscale based on metrics", path: "/docs/launch/autoscale-by-metric/" }, { text: "Add volume storage", path: "/docs/launch/volume-storage/" }, { text: "Use process groups", path: "/docs/launch/processes/" }, diff --git a/partials/_firecracker_nav.html.erb b/partials/_firecracker_nav.html.erb index b60cff3ab6..f5f1b8b4f8 100644 --- a/partials/_firecracker_nav.html.erb +++ b/partials/_firecracker_nav.html.erb @@ -177,6 +177,7 @@ { text: "Machine migration", path: "/docs/reference/machine-migration/" }, { text: "Multiple Processes in Apps", path: "/docs/app-guides/multiple-processes/" }, { text: "Fly Proxy", path: "/docs/reference/fly-proxy/" }, + { text: "Fly Proxy autostop/autostart", path: "/docs/reference/fly-proxy-autostop-autostart/" }, { text: "Regions", path: "/docs/reference/regions/" } ] }, diff --git a/reference/configuration.html.markerb b/reference/configuration.html.markerb index 6e63e8d697..b25d160bff 100644 --- a/reference/configuration.html.markerb +++ b/reference/configuration.html.markerb @@ -287,9 +287,9 @@ As with the more verbose [`[[services]]`](#the-services-sections), you can speci * `processes`: For apps with multiple processes. The process group that this service belongs to. Define process groups in the [processes](#the-processes-section) section. * `internal_port`: The port this service (and application) will use to communicate with clients. The default is 8080. We recommend applications use the default. * `force_https`: A Boolean which determines whether to enforce HTTP to HTTPS redirects. -* `auto_stop_machines`: Whether to automatically stop or suspend an application's Machines when there's excess capacity, per region. Options are `"off"`, `"stop"`, and `"suspend"`. If there's only one Machine in a region, then the Machine is stopped or suspended if it has no traffic. The Fly Proxy runs a process to automatically stop Machines every few minutes. The default if not set is `"off"`. +* `auto_stop_machines`: The action, if any, that Fly Proxy should take when the app is idle for several minutes. Options are `"off"`, `"stop"`, or `"suspend"`. If `"off"`, Fly Proxy will never stop Machines. If `"stop"`, Fly Proxy stops Machines when the app has excess capacity. If `"suspend"`, Fly Proxy suspends Machines (if possible) when the app has excess capacity. Starting a Machine from a `suspended` state is faster than starting a Machine from a `stopped` state, but there are [some caveats](https://community.fly.io/t/autosuspend-is-here-machine-suspension-is-enabled-everywhere/20942). If there's only one Machine in a region, then the Machine is stopped or suspended if it has no traffic. The Fly Proxy runs a process to automatically stop Machines every few minutes. The default if not set is `"off"`. * `auto_start_machines`: Whether to automatically start an application's Machines when a new request is made to the application and there's no excess capacity, per region. If there's only one Machine in a region, then it's started whenever a request is made to the application. The default if not set is `true`. -* `min_machines_running`: The number of Machines to keep running, in the primary region only, when `auto_stop_machines` is `"stop"` or `"suspend"`. +* `min_machines_running`: The number of Machines to keep running, in the primary region only, when `auto_stop_machines` is `"stop"` or `"suspend"`. The default if not set is `0`.
We recommend configuring `auto_stop_machines` and `auto_start_machines` so that they are both enabled or both disabled, to avoid having Machines that either never start or never stop. Learn more about [automatically starting and stopping Machines](/docs/launch/autostart-stop/), including how Fly Proxy determines excess capacity in a region using the `soft_limit` setting. @@ -445,9 +445,9 @@ Settings: * `processes`: For apps with multiple process groups, the process group or groups that this service belongs to. Define process groups in the [processes](#the-processes-section) section. * `internal_port` : The port this service (and application) will use to communicate with clients. The default is 8080. We recommend applications use the default. * `protocol` : The protocol that this service will use to communicate. Typically `tcp` for most applications, but can also be `udp`. -* `auto_stop_machines`: Whether to automatically stop or suspend an application's Machines when there's excess capacity, per region. Options are `"off"`, `"stop"`, and `"suspend"`. If there's only one Machine in a region, then the Machine is stopped or suspended if it has no traffic. The Fly Proxy runs a process to automatically stop Machines every few minutes. The default if not set is `"off"`. +* `auto_stop_machines`: The action, if any, that Fly Proxy should take when the app is idle for several minutes. Options are `"off"`, `"stop"`, or `"suspend"`. If `"off"`, Fly Proxy will never stop Machines. If `"stop"`, Fly Proxy stops Machines when the app has excess capacity. If `"suspend"`, Fly Proxy suspends Machines (if possible) when the app has excess capacity. Starting a Machine from a `suspended` state is faster than starting a Machine from a `stopped` state, but there are [some caveats](https://community.fly.io/t/autosuspend-is-here-machine-suspension-is-enabled-everywhere/20942). If there's only one Machine in a region, then the Machine is stopped or suspended if it has no traffic. The Fly Proxy runs a process to automatically stop Machines every few minutes. The default if not set is `"off"`. * `auto_start_machines`: Whether to automatically start an application's Machines when a new request is made to the application and there's no excess capacity, per region. If there's only one Machine in a region, then it's started whenever a request is made to the application. The default if not set is `true`. -* `min_machines_running`: The number of Machines to keep running, in the primary region only, when `auto_stop_machines` is `"stop"` or `"suspend"`. +* `min_machines_running`: The number of Machines to keep running, in the primary region only, when `auto_stop_machines` is `"stop"` or `"suspend"`. The default if not set is `0`.
We recommend configuring `auto_stop_machines` and `auto_start_machines` so that they are both enabled or both disabled, to avoid having Machines that either never start or never stop. Learn more about [automatically starting and stopping Machines](/docs/launch/autostart-stop/), including how Fly Proxy determines excess capacity in a region using the `soft_limit` setting. diff --git a/reference/fly-proxy-autostop-autostart.html.markerb b/reference/fly-proxy-autostop-autostart.html.markerb new file mode 100644 index 0000000000..23503e1efd --- /dev/null +++ b/reference/fly-proxy-autostop-autostart.html.markerb @@ -0,0 +1,53 @@ +--- +title: Fly Proxy autostop/autostart +layout: docs +nav: firecracker +--- + +
+ +
+ +For Fly Apps with a service configured, [Fly Proxy](/docs/reference/fly-proxy) can automatically stop (or suspend) and start existing Machines based on incoming requests, so that your app can meet demand without keeping extra Machines running. Fly Machines are fast to start and stop, and you don't pay for their CPU and RAM when they're in a `stopped` or `suspended` state. You can also set a minimum number of machines to keep running at all times in your primary region. + +You can also use the autostop/autostart with apps that [shut down automatically when idle](#stop-a-machine-by-terminating-its-main-process). The Fly Proxy can still restart your app when there's traffic. + +Learn how to [configure autostop/autostart](/docs/launch/autostop-autostart/). + +## How Fly Proxy autostops and autostarts Machines + +The Fly Proxy runs a process to determine whether an app has excess capacity every few minutes. + +
+The autostop/autostart feature only works on existing Machines and never creates or destroys Machines for you. The maximum number of running Machines is the number of Machines you've created for your app using `fly scale count` or `fly machine clone`. Learn more about [scaling the number of Machines](/docs/apps/scale-count/). +
+ +### Fly Proxy process to stop or suspend Machines + +When `auto_stop_machines` is set to `"stop"` or `"suspend"` in your `fly.toml`, the proxy looks at Machines running in a single region and uses the concurrency [`soft_limit` setting](/docs/reference/configuration/#the-http_service-section) for each Machine to determine if there's excess capacity. If the proxy decides there's excess capacity, it stops or suspends exactly one Machine. The proxy repeats this process every few minutes, stopping or suspending only one Machine per region, if needed, each time. + +If you have the [`kill_signal` and `kill_timeout` options](/docs/reference/configuration/#runtime-options) configured in your `fly.toml` file, then Fly Proxy uses those settings when it stops a Machine. + +Fly Proxy determines excess capacity per region as follows: + +* If there's more than one Machine in the region: + * the proxy determines how many running Machines are over their `soft_limit` setting and then calculates excess capacity: `excess capacity = num of machines - (num machines over soft limit + 1)` + * if excess capacity is 1 or greater, then the proxy stops or suspends one Machine +* If there's only one Machine in the region: + * the proxy checks if the Machine has any traffic + * if the Machine has no traffic (a load of 0), then the proxy stops or suspends the Machine + +### Fly Proxy process to start Machines + +When `auto_start_machines = true` in your `fly.toml`, the Fly Proxy restarts a Machine in the nearest region when required. + +Fly Proxy determines when to start a Machine as follows: + +* The proxy waits for a request to your app. +* If all the running Machines are above their `soft_limit` setting, then the proxy starts a stopped or suspended Machine in the nearest region (if there are any stopped or suspended Machines). +* The proxy routes the request to the newly started Machine. + +## Related topics + +- [Fly Proxy features](/docs/reference/fly-proxy) +- [Configure autostop/autostart for Fly Launch apps](/docs/launch/autostop-autostart/) diff --git a/reference/fly-proxy.html.markerb b/reference/fly-proxy.html.markerb index f2a21c6fb8..0f9db3e85c 100644 --- a/reference/fly-proxy.html.markerb +++ b/reference/fly-proxy.html.markerb @@ -51,7 +51,7 @@ Fly Proxy can start and stop existing Machines based on incoming requests, so th Fly Proxy uses the same concurrency settings for autostart/autostop as for load balancing to determine when Machines have excess capacity and can be stopped. -Learn more about [how autostart/autostop works and how to configure it](/docs/apps/autostart-stop/). +Learn more about [how autostart/autostop works](/docs/reference/fly-proxy-autostop-autostart/) and [how to configure it](/docs/apps/autoststop-autostart/). ## Dynamic request routing with `fly-replay` From 8c361197816f8f7149105ea7f215fa9cdc6fa34f Mon Sep 17 00:00:00 2001 From: andie787 <4andie@gmail.com> Date: Tue, 13 Aug 2024 17:01:19 -0400 Subject: [PATCH 2/6] links and adjustments --- launch/autostop-autostart.html.markerb | 6 +++--- reference/fly-proxy-autostop-autostart.html.markerb | 1 + reference/fly-proxy.html.markerb | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/launch/autostop-autostart.html.markerb b/launch/autostop-autostart.html.markerb index dcc253b955..892bc4d3be 100644 --- a/launch/autostop-autostart.html.markerb +++ b/launch/autostop-autostart.html.markerb @@ -13,11 +13,11 @@ Learn more about exactly how [Fly Proxy autostops and autostarts Machines](/docs ## When to use autostop/autostart -You can reduce resource usage and costs by using autostop/autostart to manage your Fly Machines as demand decreases and increases. You'll never have to run excess Machines to handle peak load; you'll only run, and get charged for, the number of Machines that you need. Autostart/autostop works well for apps with highly variable workloads, for smaller apps with low or sporadic traffic, and for most apps that aren't accepting requests continuously. If you need to, you can keep one or more Machines running in your primary region. +You can reduce resource usage and costs by using autostop/autostart to manage your Fly Machines as demand decreases and increases. You'll never have to run excess Machines to handle peak load; you'll only run, and get charged for, the number of Machines that you need. Autostart/autostop works well for apps with highly variable workloads, for smaller apps with low or sporadic traffic, and for most apps that aren't receiving requests continuously. If you need to, you can keep one or more Machines running in your primary region. ## Configure autostop/autostart -The autostop/autostart settings apply per service in an app's `fly.toml` file. See the [[[services]]](/docs/reference/configuration/#the-services-sections) or [[http_service]](/docs/reference/configuration/#the-http_service-section) docs for details about services. +The autostop/autostart settings apply per service in an app's `fly.toml` file. See the [[[services]]](/docs/reference/configuration/#the-services-sections) or [[http_service]](/docs/reference/configuration/#the-http_service-section) docs for details about service configuration. Autostop/autostart settings: @@ -40,7 +40,7 @@ Example configuration: In the preceding example, Fly Proxy will automatically stop Machines when the app has excess capacity and start them again when needed based on traffic to the app. The app will eventually scale down to one running Machine in the primary region if there's no traffic. -Concurrency limits configured for services in the `fly.toml` file also affect [how automatic starts and stops work](#how-it-works). +Concurrency limits configured for services in the `fly.toml` file also affect [how autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/). ## Recommendations for autostop/autostart configuration diff --git a/reference/fly-proxy-autostop-autostart.html.markerb b/reference/fly-proxy-autostop-autostart.html.markerb index 23503e1efd..d79c265172 100644 --- a/reference/fly-proxy-autostop-autostart.html.markerb +++ b/reference/fly-proxy-autostop-autostart.html.markerb @@ -51,3 +51,4 @@ Fly Proxy determines when to start a Machine as follows: - [Fly Proxy features](/docs/reference/fly-proxy) - [Configure autostop/autostart for Fly Launch apps](/docs/launch/autostop-autostart/) +- [Autostop/autostart private apps](/docs/blueprints/autostart-internal-apps/) diff --git a/reference/fly-proxy.html.markerb b/reference/fly-proxy.html.markerb index 0f9db3e85c..9d04f70ba9 100644 --- a/reference/fly-proxy.html.markerb +++ b/reference/fly-proxy.html.markerb @@ -51,7 +51,7 @@ Fly Proxy can start and stop existing Machines based on incoming requests, so th Fly Proxy uses the same concurrency settings for autostart/autostop as for load balancing to determine when Machines have excess capacity and can be stopped. -Learn more about [how autostart/autostop works](/docs/reference/fly-proxy-autostop-autostart/) and [how to configure it](/docs/apps/autoststop-autostart/). +Learn more about [how autostart/autostop works](/docs/reference/fly-proxy-autostop-autostart/) and [how to configure it](/docs/apps/autostop-autostart/). ## Dynamic request routing with `fly-replay` From c8675f94fcb32b30db83236c30bc52571c790d35 Mon Sep 17 00:00:00 2001 From: andie787 <4andie@gmail.com> Date: Wed, 14 Aug 2024 13:07:37 -0400 Subject: [PATCH 3/6] curate links for how to and reference --- about/extensions.html.markerb | 2 +- apps/app-availability.html.markerb | 2 +- apps/concurrency.html.markerb | 2 +- apps/fine-tune-apps.html.markerb | 2 +- blueprints/autoscale-machines.html.md | 8 ++++---- blueprints/resilient-apps-multiple-machines.html.md | 2 +- gpus/getting-started-gpus.html.md | 2 +- launch/autoscale-by-metric.html.markerb | 2 +- launch/autostop-autostart.html.markerb | 2 +- launch/index.html.markerb | 2 +- launch/scale-count.html.markerb | 3 ++- machines/flyctl/fly-machine-run.html.md | 4 ++-- networking/app-services.html.md | 2 +- networking/flycast.html.markerb | 2 +- postgres/managing/scale-to-zero.html.markerb | 2 +- reference/autoscaling.html.md | 3 ++- reference/configuration.html.markerb | 12 ++++++------ reference/fly-launch.html.md | 2 +- 18 files changed, 29 insertions(+), 27 deletions(-) diff --git a/about/extensions.html.markerb b/about/extensions.html.markerb index ea60a4c1d5..e1418ae005 100644 --- a/about/extensions.html.markerb +++ b/about/extensions.html.markerb @@ -52,4 +52,4 @@ To fulfil our promise of a performant application in any Fly.io region, latency Your Fly.io network is essentially a global, encrypted LAN, with DNS service discovery and load balancing built-in. This greatly simplifies configuration of services that cluster and gossip. Customers access your service via a single private IP address that automatically routes traffic to the nearest provider VM. -We also offer cost-cutting features such as [automatic VM stop/start based on incoming request volume](/docs/launch/autostart-stop/). Contact us at [extensions@fly.io](mailto:extensions@fly.io) to learn more - we're happy to help you get started! +We also offer cost-cutting features such as [automatic VM stop/start based on incoming request volume](/docs/launch/autostop-autostart/). Contact us at [extensions@fly.io](mailto:extensions@fly.io) to learn more - we're happy to help you get started! diff --git a/apps/app-availability.html.markerb b/apps/app-availability.html.markerb index 29a0137d25..2b647bfd48 100644 --- a/apps/app-availability.html.markerb +++ b/apps/app-availability.html.markerb @@ -135,7 +135,7 @@ Default settings for new apps created using the `fly launch` command: automatica Default settings for some existing apps (or any apps that don't have these settings in `fly.toml`): automatically start but don't automatically stop Fly Machines. -Get all the details about [automatically stopping and starting Machines](/docs/launch/autostart-stop/). +Learn more about [how Fly Proxy autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/) and [how to configure it](/docs/launch/autostop-autostart/). ## Health check-based routing diff --git a/apps/concurrency.html.markerb b/apps/concurrency.html.markerb index b035264e6d..91b1baa597 100644 --- a/apps/concurrency.html.markerb +++ b/apps/concurrency.html.markerb @@ -5,7 +5,7 @@ nav: apps redirect_from: /docs/reference/concurrency/ --- -Concurrency settings are used by the Fly Proxy for important things like [load balancing](/docs/reference/load-balancing/#load-balancing-strategy) and [autostart/autostop](/docs/launch/autostart-stop/) for Machines. +Concurrency settings are used by the Fly Proxy for important things like [load balancing](/docs/reference/load-balancing/#load-balancing-strategy) and [autostart/autostop](/docs/launch/autostop-autostart/) for Machines. The following concurrency settings apply per Machine and per service in your app: diff --git a/apps/fine-tune-apps.html.markerb b/apps/fine-tune-apps.html.markerb index 6343890283..8a7778d256 100644 --- a/apps/fine-tune-apps.html.markerb +++ b/apps/fine-tune-apps.html.markerb @@ -76,7 +76,7 @@ Refer to our guidelines for [concurrency settings](/docs/reference/concurrency/) ### Auto stop and start feature -[Auto stop and start](/docs/launch/autostart-stop/) can be turned off if you want to test the performance of your app without that variable. +[Auto stop and start](/docs/launch/autostop-autostart/) can be turned off if you want to test the performance of your app without that variable. If you don’t turn off auto stop and start, then you’ll likely have some pretty high p99 values due to the initial cost of the Fly Proxy queueing connections while waiting for the Machine to start. diff --git a/blueprints/autoscale-machines.html.md b/blueprints/autoscale-machines.html.md index 498086a8b2..cb0f1379bc 100644 --- a/blueprints/autoscale-machines.html.md +++ b/blueprints/autoscale-machines.html.md @@ -12,7 +12,7 @@ Machines during the period of low traffic. This blueprint will guide you through the process of configuring the [`fly-autoscaler` app](/docs/launch/autoscale-by-metric/) in conjunction with -the [Fly Proxy autostart/autostop](/docs/launch/autostart-stop/) feature to +the [Fly Proxy autostart/autostop](/docs/launch/autostop-autostart/) feature to always keep a fixed number of stopped Machines ready to be quickly started by Fly Proxy. @@ -103,8 +103,8 @@ $ fly deploy --ha=false ## Read more -- [Autoscale based on metrics](https://fly.io/docs/launch/autoscale-by-metric/) +- [Autoscale based on metrics](/docs/launch/autoscale-by-metric/) -- [Automatically stop and start Machines](https://fly.io/docs/launch/autostart-stop/) +- [Autostart/autostop Machines](/docs/launch/autostop-autostart/) -- [Autostart and autostop private apps](https://fly.io/docs/blueprints/autostart-internal-apps/) +- [Autostart/autostop private apps](/docs/blueprints/autostart-internal-apps/) diff --git a/blueprints/resilient-apps-multiple-machines.html.md b/blueprints/resilient-apps-multiple-machines.html.md index 455196ead1..e3d5413a85 100644 --- a/blueprints/resilient-apps-multiple-machines.html.md +++ b/blueprints/resilient-apps-multiple-machines.html.md @@ -40,7 +40,7 @@ Use autostart/autostop to tell the Fly Proxy to start and stop Machines based on soft_limit = 200 # Used by Fly Proxy to determine Machine excess capacity ``` -Fly Proxy uses the concurrency `soft_limit` to determine if Machines have capacity. Learn more about [autostart/autostop](/docs/launch/autostart-stop/). +Fly Proxy uses the concurrency `soft_limit` to determine if Machines have capacity. Learn more about [how Fly Proxy autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/). **Using the Machines API:** To add or change the autostart/autostop settings with the Machines API, use the settings in the `services` object of the [Machine config](/docs/machines/api/machines-resource/#machine-config-object-properties) in your create or update calls. diff --git a/gpus/getting-started-gpus.html.md b/gpus/getting-started-gpus.html.md index 27f5338f27..ca349588cf 100644 --- a/gpus/getting-started-gpus.html.md +++ b/gpus/getting-started-gpus.html.md @@ -86,7 +86,7 @@ Machine learning tends to involve large quantities of data. We're working with a - The root file system of a Fly Machine is ephemeral -- it's reset from its Docker image on every restart. It's also limited to 50GB on GPU-enabled Machines. - Fly Volumes are limited to 500GB, and are attached to a physical server. The Machine must run on the same hardware as the volume it mounts. -Unless you've got a constant workload, you'll likely want to shut down GPU Machines when they're not needed—you can do this manually with `fly machine stop`, [have the main process exit](/docs/launch/autostart-stop/#stop-a-machine-by-terminating-its-main-process) when idle, or use the Fly Proxy [autostop and autostart](/docs/launch/autostart-stop/#how-it-works) features—to save money. Saving data on a persistent Fly Volume means you don't have to download large amounts of data, or reconstitute a large Docker image into a rootfs, whenever the Machine restarts. You'll probably want to store models, at least, on a volume. +Unless you've got a constant workload, you'll likely want to shut down GPU Machines when they're not needed—you can do this manually with `fly machine stop`, [have the main process exit](/docs/launch/autostop-autostart/#apps-that-shut-down-when-idle) when idle, or use the Fly Proxy [autostop and autostart](/docs/launch/autostop-autostart/) features—to save money. Saving data on a persistent Fly Volume means you don't have to download large amounts of data, or reconstitute a large Docker image into a rootfs, whenever the Machine restarts. You'll probably want to store models, at least, on a volume. ## Using swap diff --git a/launch/autoscale-by-metric.html.markerb b/launch/autoscale-by-metric.html.markerb index 6107b655e3..d9f4c3d75a 100644 --- a/launch/autoscale-by-metric.html.markerb +++ b/launch/autoscale-by-metric.html.markerb @@ -5,7 +5,7 @@ nav: apps redirect_from: /docs/apps/autoscale-by-metric/ --- -The metrics-based autoscaler scales an app's Machines based on any metric, such as pending work items or queue depth. Scaling on metrics other than requests or connections is useful for apps, like background workers, that aren't running web services. Apps with services that need to scale based on HTTP requests can use the built-in Fly Proxy [autostop/autostart feature](/docs/launch/autostart-stop/) for Machines. +The metrics-based autoscaler scales an app's Machines based on any metric, such as pending work items or queue depth. Scaling on metrics other than requests or connections is useful for apps, like background workers, that aren't running web services. Apps with services that need to scale based on HTTP requests can use the built-in Fly Proxy [autostop/autostart feature](/docs/launch/autostop-autostart/) for Machines. The autoscaler works by collecting metrics from different sources, such as Prometheus or Temporal, and computing the number of required Machines based on those metrics. This reconciliation process happens on a loop every 15 seconds by default. The autoscaler uses the [Expr language](https://expr-lang.org/+external) for defining target Machine counts, which gives a rich set of built-in arithmetic functions. diff --git a/launch/autostop-autostart.html.markerb b/launch/autostop-autostart.html.markerb index 892bc4d3be..612af445c0 100644 --- a/launch/autostop-autostart.html.markerb +++ b/launch/autostop-autostart.html.markerb @@ -9,7 +9,7 @@ redirect_from: For Fly Apps with a service configured, [Fly Proxy](/docs/reference/fly-proxy/) can automatically start and stop or suspend existing Machines based on incoming requests, so that your app can meet demand without keeping extra Machines running. Fly Machines are fast to start and stop, and you don't pay for their CPU and RAM when they're in a `stopped` or `suspended` state. -Learn more about exactly how [Fly Proxy autostops and autostarts Machines](/docs/reference/fly-proxy-autostop-autostart/). +Learn more about how [Fly Proxy autostops and autostarts Machines](/docs/reference/fly-proxy-autostop-autostart/). ## When to use autostop/autostart diff --git a/launch/index.html.markerb b/launch/index.html.markerb index 8c5243d1f7..ad6ab18f84 100644 --- a/launch/index.html.markerb +++ b/launch/index.html.markerb @@ -33,7 +33,7 @@ Learn how to work with Fly Launch: - [Scale Machine CPU and RAM](/docs/launch/scale-machine/) - [Scale the number of Machines in an app](/docs/launch/scale-count/) -- [Auto start and stop Machines](/docs/launch/autostart-stop/) +- [Autostop/autostart Machines](/docs/launch/autostop-autostart/) - [Autoscale Machines based on metrics](/docs/launch/autoscale-by-metric/) **Custom deployments** diff --git a/launch/scale-count.html.markerb b/launch/scale-count.html.markerb index e2e654d3f4..701691fa98 100644 --- a/launch/scale-count.html.markerb +++ b/launch/scale-count.html.markerb @@ -19,7 +19,8 @@ Starting and stopping existing Machines is much faster than creating and destroy For bursty workloads, we recommend creating enough Machines to handle your peak load, and adjusting the active capacity of the app by stopping and starting Machines as needed. For more information, see: -* [Automatically Stop and Start App Machines](/docs/launch/autostart-stop/), for adjusting active capacity based on traffic. +* [Automatically Stop and Start App Machines](/docs/launch/autostop-autostart/), for adjusting active capacity based on traffic. +* [Autoscale based on metrics](/docs/launch/autoscale-by-metric/), to use the autoscaler app to scale Machines based on any metric. * [`fly machine` commands](/docs/flyctl/machine/), including [`start`](/docs/flyctl/machine-start/) and [`stop`](/docs/flyctl/machine-start/) subcommands to target individual Machines with flyctl. * Machines API [stop](/docs/machines/api-machines-resource/#stop-a-machine) and [start](/docs/machines/api-machines-resource/#start-a-machine) endpoints.
diff --git a/machines/flyctl/fly-machine-run.html.md b/machines/flyctl/fly-machine-run.html.md index a11ecad3c9..c5817df253 100644 --- a/machines/flyctl/fly-machine-run.html.md +++ b/machines/flyctl/fly-machine-run.html.md @@ -222,9 +222,9 @@ fly machine run . --port 80/tcp:http \
-## Set Fly Proxy auto start and auto stop +## Set Fly Proxy autostop/autostart -The `--autostart` and `--autostop` flags only work on Machines with Fly Proxy services configured. [Read more about Fly Proxy auto start and auto stop](/docs/launch/autostart-stop/#how-it-works). +The `--autostart` and `--autostop` flags only work on Machines with Fly Proxy services configured. Learn more about [how Fly Proxy autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/) and [how to configure it](/docs/launch/autostop-autostart/). In a Machine service's configuration, `autostop` and `autostart` settings are optional. diff --git a/networking/app-services.html.md b/networking/app-services.html.md index adb7a99e09..b6f972cf68 100644 --- a/networking/app-services.html.md +++ b/networking/app-services.html.md @@ -31,7 +31,7 @@ There's a bit more to the whole picture, which is why the rest of this document All services reachable from the public internet via a Fly App's global Anycast address are routed by Fly Proxy. -Fly Proxy can load-balance requests for both public and private ([Flycast](/docs/networking/flycast)) services among a Fly App's VMs. Routing to a service with Fly Proxy also enables other Fly Proxy features, such as [starting and stopping Machines](/docs/launch/autostart-stop/) in response to fluctuations in request traffic. +Fly Proxy can load-balance requests for both public and private ([Flycast](/docs/networking/flycast)) services among a Fly App's VMs. Routing to a service with Fly Proxy also enables other Fly Proxy features, such as [automatically starting and stopping Machines](/docs/launch/autostop-autostart/) in response to fluctuations in request traffic. ### Get the service listening on the right address within the VM diff --git a/networking/flycast.html.markerb b/networking/flycast.html.markerb index 11957cea98..e82db1f598 100644 --- a/networking/flycast.html.markerb +++ b/networking/flycast.html.markerb @@ -8,7 +8,7 @@ Flycast provides network addressing and routing for private apps on Fly.io priva Use Flycast within your organization's private network to: -* Have [Fly Proxy autostop/autostart](/docs/launch/autostart-stop/) Machines based on network requests. +* Have [Fly Proxy autostop/autostart](/docs/launch/autostop-autostart/) Machines based on network requests. * Get Fly Proxy's [geographically aware load balancing](/docs/reference/load-balancing/) for private services. * Connect to a service from another app that can't use DNS. * Connect from third-party software, like a database, that doesn't support round-robin DNS entries. diff --git a/postgres/managing/scale-to-zero.html.markerb b/postgres/managing/scale-to-zero.html.markerb index ff526aeb0d..2c8aeb9cda 100644 --- a/postgres/managing/scale-to-zero.html.markerb +++ b/postgres/managing/scale-to-zero.html.markerb @@ -14,7 +14,7 @@ If you select the Development configuration when you [create your Fly Postgres a After one hour, if there are no open connections, then the database shuts down and waits to restart until something tries to connect again. If there are open connections, then the database stays running and checks for open connections again in one hour.
-You might need to configure any apps that connect to your Postgres app to scale to zero as well, otherwise your Postgres database will never have zero connections and will never scale down. Your app might also need to be able to wait for the database to start back up. You can learn more about [making apps exit when idle](/docs/launch/autostart-stop/#stop-a-machine-by-terminating-its-main-process). +You might need to configure any apps that connect to your Postgres app to scale to zero as well, otherwise your Postgres database will never have zero connections and will never scale down. Your app might also need to be able to wait for the database to start back up. You can learn more about [making apps exit when idle](/docs/launch/autostop-autostart/#apps-that-shut-down-when-idle).
## Turn off the scale to zero feature diff --git a/reference/autoscaling.html.md b/reference/autoscaling.html.md index 09490e1426..aa04e9bdbe 100644 --- a/reference/autoscaling.html.md +++ b/reference/autoscaling.html.md @@ -13,7 +13,8 @@ Autoscaling adjusts the number of running or created Fly Machines dynamically. W The Fly Proxy autostart/autostop feature starts and stops Machines based on load; Machines are never created or deleted. You create a "pool" of Machines in one or more regions and the Fly Proxy starts and stops them as needed. -For a comprehensive overview of how it works, see [Automatically stop and start Machines](/docs/launch/autostart-stop/). +For details about how it works, see [Fly Proxy autostop/autostart](/docs/launch/autostop-autostart/). To configure autostop/autostart for your app, see [Autostop/autostart Machines](/docs/launch/autostop-autostart/). + ## Metrics-based autoscaling diff --git a/reference/configuration.html.markerb b/reference/configuration.html.markerb index b25d160bff..b15cced9bd 100644 --- a/reference/configuration.html.markerb +++ b/reference/configuration.html.markerb @@ -292,12 +292,12 @@ As with the more verbose [`[[services]]`](#the-services-sections), you can speci * `min_machines_running`: The number of Machines to keep running, in the primary region only, when `auto_stop_machines` is `"stop"` or `"suspend"`. The default if not set is `0`.
-We recommend configuring `auto_stop_machines` and `auto_start_machines` so that they are both enabled or both disabled, to avoid having Machines that either never start or never stop. Learn more about [automatically starting and stopping Machines](/docs/launch/autostart-stop/), including how Fly Proxy determines excess capacity in a region using the `soft_limit` setting. +We recommend configuring `auto_stop_machines` and `auto_start_machines` so that they are both enabled or both disabled, to avoid having Machines that either never start or never stop. Learn more about [how Fly Proxy autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/), including how Fly Proxy determines excess capacity in a region using the `soft_limit` setting.
### `http_service.concurrency` -The `http_service` concurrency section configures how to measure load for an application to inform [load balancing](/docs/reference/load-balancing) and [auto start and stop](/docs/launch/autostart-stop/). The `[http_service.concurrency]` section has the same settings as `[services.concurrency]`, which are: +The `http_service` concurrency section configures how to measure load for an application to inform Fly Proxy [load balancing](/docs/reference/load-balancing) and [autostop/autostart](/docs/launch/autostop-autostart/). The `[http_service.concurrency]` section has the same settings as `[services.concurrency]`, which are: * `type` specifies what metric is used to determine when to auto start or stop, or when a given Machine should receive more or less traffic (load balancing). The two supported values are `connections` and `requests`. @@ -306,7 +306,7 @@ The `http_service` concurrency section configures how to measure load for an app **requests**: Load balance and scale based on the number of HTTP requests. This is recommended for web services, because the Fly Proxy can pool and reuse connections for requests. * `hard_limit` : When a Fly Machine is at or over this number of concurrent connections or requests, the system will stop sending new traffic to that Machine. If unset, no `hard_limit` is enforced. -* `soft_limit` : When a Fly Machine is at or over this number of concurrent connections or requests, the system will deprioritize sending new traffic to that Machine and only send it to that Machine if all other Machines are also at or above their `soft_limit`. This setting is also used to determine excess capacity for the [auto start and stop feature](/docs/launch/autostart-stop/). If unset, the default is 20. +* `soft_limit` : When a Fly Machine is at or over this number of concurrent connections or requests, the system will deprioritize sending new traffic to that Machine and only send it to that Machine if all other Machines are also at or above their `soft_limit`. This setting is also used to determine excess capacity for the [autostop/autostart feature](/docs/launch/autostop-autostart/). If unset, the default is 20. For more information about concurrency settings, see [Guidelines for concurrency settings](/docs/reference/concurrency/). @@ -450,7 +450,7 @@ Settings: * `min_machines_running`: The number of Machines to keep running, in the primary region only, when `auto_stop_machines` is `"stop"` or `"suspend"`. The default if not set is `0`.
-We recommend configuring `auto_stop_machines` and `auto_start_machines` so that they are both enabled or both disabled, to avoid having Machines that either never start or never stop. Learn more about [automatically starting and stopping Machines](/docs/launch/autostart-stop/), including how Fly Proxy determines excess capacity in a region using the `soft_limit` setting. +We recommend configuring `auto_stop_machines` and `auto_start_machines` so that they are both enabled or both disabled, to avoid having Machines that either never start or never stop. Learn more about [how Fly Proxy autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/), including how Fly Proxy determines excess capacity in a region using the `soft_limit` setting.
### `services.ports` @@ -589,7 +589,7 @@ One use case is applications using HTTP/2, like gRPC. Fly's edge terminates TLS ### `services.concurrency` -The services concurrency sub-section configures how to measure load for an application to inform [load balancing](/docs/reference/load-balancing) and [auto start and stop](/docs/launch/autostart-stop/). +The services concurrency sub-section configures how to measure load for an application to inform Fly Proxy [load balancing](/docs/reference/load-balancing) and [autostop/autostart](/docs/launch/autostop-autostart/). This section is a simple list of key/values, so the section is denoted with single square brackets like so: @@ -607,7 +607,7 @@ This section is a simple list of key/values, so the section is denoted with sing **requests**: Load balance and scale based on the number of HTTP requests. This is recommended for web services, because the Fly Proxy can pool and reuse connections for requests. * `hard_limit` : When a Fly Machine is at or over this number of concurrent connections or requests, the system will stop sending new traffic to that Machine. If unset, no `hard_limit` is enforced. -* `soft_limit` : When a Fly Machine is at or over this number of concurrent connections or requests, the system will deprioritize sending new traffic to that Machine and only send it to that Machine if all other Machines are also at or above their `soft_limit`. This setting is also used to determine excess capacity for the [auto start and stop feature](/docs/launch/autostart-stop/). If unset, the default is 20. +* `soft_limit` : When a Fly Machine is at or over this number of concurrent connections or requests, the system will deprioritize sending new traffic to that Machine and only send it to that Machine if all other Machines are also at or above their `soft_limit`. This setting is also used to determine excess capacity for the [autostop/autostart feature](/docs/launch/autostop-autostart/). If unset, the default is 20. For more information about concurrency settings, see [Guidelines for concurrency settings](/docs/reference/concurrency/). diff --git a/reference/fly-launch.html.md b/reference/fly-launch.html.md index 75fed57d86..a95a265754 100644 --- a/reference/fly-launch.html.md +++ b/reference/fly-launch.html.md @@ -238,7 +238,7 @@ app e28697ce6d3986 1 ewr stopped 2023-11-29T20:55:33Z ```
-**Note:** Both Machines are in a stopped state, because they were idle for a few minutes and we enable the [auto start and stop feature](/docs/launch/autostart-stop/) by default. +**Note:** Both Machines are in a stopped state, because they were idle for a few minutes and we enable the [autostop/autostart feature](/docs/launch/autostop-autostart/) by default.
I can [scale out](/docs/apps/scale-count/) by adding Machines in other regions if I want to get close to users in more corners of the world. From 518137836d6306028b6f5c3efe7c7fa42821131b Mon Sep 17 00:00:00 2001 From: andie787 <4andie@gmail.com> Date: Wed, 14 Aug 2024 14:06:58 -0400 Subject: [PATCH 4/6] links and fly proxy terminology --- apps/concurrency.html.markerb | 8 ++++---- apps/fine-tune-apps.html.markerb | 4 ++-- apps/going-to-production.html.markerb | 2 +- blueprints/autoscale-machines.html.md | 6 +++--- blueprints/autostart-internal-apps.html.md | 4 ++-- blueprints/multi-region-fly-replay.html.md | 4 ++-- .../resilient-apps-multiple-machines.html.md | 14 ++++++------- launch/autostop-autostart.html.markerb | 20 +++++++++++-------- networking/flycast.html.markerb | 2 +- networking/private-networking.html.md | 2 +- rails/advanced-guides/machine.html.md | 2 +- reference/autoscaling.html.md | 6 +++--- .../fly-proxy-autostop-autostart.html.markerb | 12 ++++++----- reference/fly-proxy.html.markerb | 6 +++--- reference/index.html.md | 2 ++ 15 files changed, 51 insertions(+), 43 deletions(-) diff --git a/apps/concurrency.html.markerb b/apps/concurrency.html.markerb index 91b1baa597..163e19402a 100644 --- a/apps/concurrency.html.markerb +++ b/apps/concurrency.html.markerb @@ -5,7 +5,7 @@ nav: apps redirect_from: /docs/reference/concurrency/ --- -Concurrency settings are used by the Fly Proxy for important things like [load balancing](/docs/reference/load-balancing/#load-balancing-strategy) and [autostart/autostop](/docs/launch/autostop-autostart/) for Machines. +Concurrency settings are used by Fly Proxy for important things like [load balancing](/docs/reference/load-balancing/#load-balancing-strategy) and [autostop/autostart](/docs/launch/autostop-autostart/) for Machines. The following concurrency settings apply per Machine and per service in your app: @@ -23,9 +23,9 @@ The default concurrency settings when not specified are a `soft_limit` of 20 and `connections` is the default concurrency type. -## How the Fly Proxy handles concurrency +## How Fly Proxy handles concurrency -The Fly Proxy doesn't consider the exact number of concurrent connections or requests. From the proxy's point of view a Machine is handling between 0 and the soft limit, handling between the soft limit and the hard limit, or is at the hard limit. Along with region, this is how the proxy decides which Machine gets traffic. +Fly Proxy doesn't consider the exact number of concurrent connections or requests. From the proxy's point of view a Machine is handling between 0 and the soft limit, handling between the soft limit and the hard limit, or is at the hard limit. Along with region, this is how the proxy decides which Machine gets traffic. The proxy's view of a Machine's load is affected when the gap between the soft limit and the hard limit is small and/or the app's concurrent connections or requests oscillate between them more frequently. In that case, the proxy routes according to the settings, but the thresholds change too quickly and requests or connections end up being re-routed. @@ -45,4 +45,4 @@ The decision to use `connections` or `requests` for concurrency depends on the t ## Concurrency limit tuning tips -When tuning concurrency, try setting a relatively high `hard_limit`, or leave it unset to have no hard limit. If you do want to set a `hard_limit` to have more control over load balancing, then you might have to do an initial benchmark to estimate the maximum number of concurrent connections or requests that your app can handle. Then tune the `soft_limit` and [create more Machines](/docs/blueprints/resilient-apps-multiple-machines/) to optimize autostart/autostop and load balancing. Once your app is getting real-world traffic, you can continue to monitor your app and adjust the `soft_limit` further to suit your workload. +When tuning concurrency, try setting a relatively high `hard_limit`, or leave it unset to have no hard limit. If you do want to set a `hard_limit` to have more control over load balancing, then you might have to do an initial benchmark to estimate the maximum number of concurrent connections or requests that your app can handle. Then tune the `soft_limit` and [create more Machines](/docs/blueprints/resilient-apps-multiple-machines/) to optimize autostop/autostart and load balancing. Once your app is getting real-world traffic, you can continue to monitor your app and adjust the `soft_limit` further to suit your workload. diff --git a/apps/fine-tune-apps.html.markerb b/apps/fine-tune-apps.html.markerb index 8a7778d256..5e037ec6f7 100644 --- a/apps/fine-tune-apps.html.markerb +++ b/apps/fine-tune-apps.html.markerb @@ -11,7 +11,7 @@ That settled, you can still use the following guidelines to gather data to fine- ## Fly.io metrics -We have various ways to access metrics for the Fly Proxy, your app, and Fly Machines. Learn more about [Fly Metrics](/docs/reference/metrics/). +We have various ways to access metrics for Fly Proxy, your app, and Fly Machines. Learn more about [Fly Metrics](/docs/reference/metrics/). - Edge HTTP response time is the total time it takes to respond with the first HTTP response headers. - App HTTP response time is the same, but only considers your app’s response time, not the overhead of routing, load balancing, retries, and so on. @@ -78,7 +78,7 @@ Refer to our guidelines for [concurrency settings](/docs/reference/concurrency/) [Auto stop and start](/docs/launch/autostop-autostart/) can be turned off if you want to test the performance of your app without that variable. -If you don’t turn off auto stop and start, then you’ll likely have some pretty high p99 values due to the initial cost of the Fly Proxy queueing connections while waiting for the Machine to start. +If you don’t turn off auto stop and start, then you’ll likely have some pretty high p99 values due to the initial cost of Fly Proxy queueing connections while waiting for the Machine to start. ## Distributed systems diff --git a/apps/going-to-production.html.markerb b/apps/going-to-production.html.markerb index 3402413f48..c175cdfa5b 100644 --- a/apps/going-to-production.html.markerb +++ b/apps/going-to-production.html.markerb @@ -46,7 +46,7 @@ This document lists important topics to consider when you set up a production en - **Use multiple Machines for resiliency:** Make your app resilient to single-host failures with multiple Machines that stay stopped until you need them. See [Blueprint: Resilient apps use multiple Machines](/docs/blueprints/resilient-apps-multiple-machines/). -- **Set up autoscaling by load or metric:** Use Fly Proxy autostart/autostop or the metrics-based autoscaler app. See [Autoscaling](/docs/reference/autoscaling/). +- **Set up autoscaling by load or metric:** Use Fly Proxy autostop/autostart or the metrics-based autoscaler app. See [Autoscaling](/docs/reference/autoscaling/). ## CI/CD diff --git a/blueprints/autoscale-machines.html.md b/blueprints/autoscale-machines.html.md index cb0f1379bc..7fe1842c2b 100644 --- a/blueprints/autoscale-machines.html.md +++ b/blueprints/autoscale-machines.html.md @@ -12,7 +12,7 @@ Machines during the period of low traffic. This blueprint will guide you through the process of configuring the [`fly-autoscaler` app](/docs/launch/autoscale-by-metric/) in conjunction with -the [Fly Proxy autostart/autostop](/docs/launch/autostop-autostart/) feature to +[Fly Proxy autostop/autostart](/docs/launch/autostop-autostart/) to always keep a fixed number of stopped Machines ready to be quickly started by Fly Proxy. @@ -105,6 +105,6 @@ $ fly deploy --ha=false - [Autoscale based on metrics](/docs/launch/autoscale-by-metric/) -- [Autostart/autostop Machines](/docs/launch/autostop-autostart/) +- [Autostop/autostart Machines](/docs/launch/autostop-autostart/) -- [Autostart/autostop private apps](/docs/blueprints/autostart-internal-apps/) +- [Autostop/autostart private apps](/docs/blueprints/autostart-internal-apps/) diff --git a/blueprints/autostart-internal-apps.html.md b/blueprints/autostart-internal-apps.html.md index 5ed09537cd..60fc92947c 100644 --- a/blueprints/autostart-internal-apps.html.md +++ b/blueprints/autostart-internal-apps.html.md @@ -6,7 +6,7 @@ nav: firecracker You have a private, or internal, app that communicates only with other apps on your [private network](/docs/networking/private-networking/). This private app might be a database, authentication server, or any other "backend" app that you don't want exposed to the public Internet. You want the app's Machines to stop when they're not serving requests from your other apps, and start again automatically when needed. -To use the Fly Proxy autostart and autostop feature you need to configure services in `fly.toml`, like you would for a public app. But instead of using a public Anycast address, you assign a Flycast address to expose those services only on your private network. +To use Fly Proxy autostop/autostart you need to configure services in `fly.toml`, like you would for a public app. But instead of using a public Anycast address, you assign a Flycast address to expose those services only on your private network. This blueprint focuses on using autostart and autostop to control Machines based on incoming requests. But when you use Flycast for private apps you also get other Fly Proxy features like geographically aware load balancing. @@ -97,7 +97,7 @@ Here's an example `fly.toml` snippet: **Important:** Set `force_https = false` since Flycast only works over HTTP. HTTPS isn't necessary because all your private network traffic goes through encrypted WireGuard tunnels. -Learn more about [Fly Launch configuration](/docs/reference/configuration/) and the [autostart and autostop](/docs/apps/autostart-stop/) feature. +Learn more about [Fly Launch configuration](/docs/reference/configuration/) and [Fly Proxy autostop/autostart](/docs/launch/autostop-autostart/). ### Make sure your app binds to `0.0.0.0:` diff --git a/blueprints/multi-region-fly-replay.html.md b/blueprints/multi-region-fly-replay.html.md index 52aa06a096..2250814509 100644 --- a/blueprints/multi-region-fly-replay.html.md +++ b/blueprints/multi-region-fly-replay.html.md @@ -20,7 +20,7 @@ When your app receives write requests, you can use the `fly-replay` response hea ## How it works -The [`fly-replay` response header](/docs/networking/dynamic-request-routing/) instructs the Fly proxy to redeliver (replay) the original request to another region or Machine in your app, or even another app in your organization. In this case, you’ll be replaying write requests to the Machine in the primary region. Using `fly-replay` to replay write requests is a general pattern that can be applied in most languages and frameworks for databases with one primary and multiple read replicas. +The [`fly-replay` response header](/docs/networking/dynamic-request-routing/) instructs Fly proxy to redeliver (replay) the original request to another region or Machine in your app, or even another app in your organization. In this case, you’ll be replaying write requests to the Machine in the primary region. Using `fly-replay` to replay write requests is a general pattern that can be applied in most languages and frameworks for databases with one primary and multiple read replicas. In the following diagram, the app is running one Machine in each of three regions. The primary region is Chicago, and this is where the read/write primary database resides. There are Machines in two other regions, Rio de Janeiro and Amsterdam, each of which has a read replica. This example uses three regions for simplicity, but you could deploy in more than three regions and have more than one Machine per region connecting to the same read replica. @@ -51,7 +51,7 @@ Your app can check the `FLY_REGION` against the `PRIMARY_REGION`, and modify the ### Replay write requests to the primary region -Your app can detect write requests and send a response with the `fly-replay` header that tells the Fly Proxy to replay the whole request to the Fly Machine in the primary region. +Your app can detect write requests and send a response with the `fly-replay` header that tells Fly Proxy to replay the whole request to the Fly Machine in the primary region. #### Detect write requests diff --git a/blueprints/resilient-apps-multiple-machines.html.md b/blueprints/resilient-apps-multiple-machines.html.md index e3d5413a85..03a03b16e4 100644 --- a/blueprints/resilient-apps-multiple-machines.html.md +++ b/blueprints/resilient-apps-multiple-machines.html.md @@ -6,7 +6,7 @@ nav: firecracker Fly Machines are fast-launching VMs; they're the compute of the Fly.io platform. Every Machine runs on a single physical host. If that host fails, the Machine becomes unavailable; it does not automatically get rescheduled on another host. -To make your app resilient to single-host failure, create at least two Machines per app or process. The Fly Proxy autostart/autostop feature and standby Machines are built-in platform features that you can use to start extra Machines only when needed. +To make your app resilient to single-host failure, create at least two Machines per app or process. Fly Proxy autostop/autostart and standby Machines are built-in platform features that you can use to start extra Machines only when needed. ## Multiple Machines for apps with services @@ -14,7 +14,7 @@ You can add more Machines for Fly Proxy to start and stop as needed, which is gr ### You get two Machines on first deploy -When you deploy an app for the first time with the `fly launch` or `fly deploy` command, you automatically get two identical Machines for processes that have HTTP/TCP services configured in `fly.toml`. The Machines have autostart/autostop enabled so that Fly Proxy can start and stop them based on traffic to your app. You'll also get this default redundant Machine when you `fly deploy` after scaling to zero. +When you deploy an app for the first time with the `fly launch` or `fly deploy` command, you automatically get two identical Machines for processes that have HTTP/TCP services configured in `fly.toml`. The Machines have autostop/autostart enabled so that Fly Proxy can start and stop them based on traffic to your app. You'll also get this default redundant Machine when you `fly deploy` after scaling to zero.
**Volumes:** You'll only get one Machine with `fly launch` for processes or apps with volumes mounted. Volumes don't automatically replicate your data for you, so you'll need to set that up before intentionally creating more Machines with volumes. @@ -22,11 +22,11 @@ When you deploy an app for the first time with the `fly launch` or `fly deploy` ### Add more Machines yourself -If your app doesn't already have multiple Machines with autostart/autostop, then you can set it up yourself. You can create any number of Machines to both meet user demand and provide redundancy against host failures. +If your app doesn't already have multiple Machines with autostop/autostart configured, then you can set it up yourself. You can create any number of Machines to both meet user demand and provide redundancy against host failures. -#### 1. Set up autostart/autostop +#### 1. Set up autostop/autostart -Use autostart/autostop to tell the Fly Proxy to start and stop Machines based on traffic. Keep one or more Machines running in your primary region if you want to. Example from `fly.toml` config: +Use [Fly Proxy autostop/autostart](/docs/launch/autostop-autostart/#apps-that-shut-down-when-idle) to automatically stop and start Machines based on traffic. Keep one or more Machines running in your primary region if you want to. Example from `fly.toml` config: ```toml [http_service] @@ -42,7 +42,7 @@ Use autostart/autostop to tell the Fly Proxy to start and stop Machines based on Fly Proxy uses the concurrency `soft_limit` to determine if Machines have capacity. Learn more about [how Fly Proxy autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/). -**Using the Machines API:** To add or change the autostart/autostop settings with the Machines API, use the settings in the `services` object of the [Machine config](/docs/machines/api/machines-resource/#machine-config-object-properties) in your create or update calls. +**Using the Machines API:** To add or change the autostop/autostart settings with the Machines API, use the settings in the `services` object of the [Machine config](/docs/machines/api/machines-resource/#machine-config-object-properties) in your create or update calls. #### 2. Create more Machines @@ -66,7 +66,7 @@ Learn more about [scaling the number of Machines](/docs/apps/scale-count/). When apps or processes are running tools like cron that don't require local storage or accept external requests, it's common to run only one Machine. Since these Machines don't have services configured, they can't be automatically started and stopped by the Fly Proxy. To add redundancy against host failures for this kind of Machine, use a standby Machine; it stays stopped and ready to take over in case the original Machine becomes unavailable. -Unlike the autostart/autostop feature, which starts Machines based on app traffic, a standby Machine watches the Machine it's paired to and starts only if that Machine becomes unavailable. Learn more about [standby Machines](https://fly.io/docs/reference/app-availability/#standby-machines-for-process-groups-without-services). +Unlike Fly Proxy autostop/autostart, which starts Machines based on app traffic, a standby Machine watches the Machine it's paired to and starts only if that Machine becomes unavailable. Learn more about [standby Machines](https://fly.io/docs/reference/app-availability/#standby-machines-for-process-groups-without-services). ### You get a standby Machine on first deploy diff --git a/launch/autostop-autostart.html.markerb b/launch/autostop-autostart.html.markerb index 612af445c0..25acd4cb3b 100644 --- a/launch/autostop-autostart.html.markerb +++ b/launch/autostop-autostart.html.markerb @@ -5,15 +5,15 @@ nav: apps redirect_from: - /docs/apps/autostart-stop/ - /docs/launch/autostart-stop/ ---- +--- -For Fly Apps with a service configured, [Fly Proxy](/docs/reference/fly-proxy/) can automatically start and stop or suspend existing Machines based on incoming requests, so that your app can meet demand without keeping extra Machines running. Fly Machines are fast to start and stop, and you don't pay for their CPU and RAM when they're in a `stopped` or `suspended` state. +For Fly Apps with a service configured, [Fly Proxy](/docs/reference/fly-proxy/) can automatically start and stop or suspend existing Machines based on incoming requests, so that your app can meet demand without keeping extra Machines running. Fly Machines are fast to start and stop, and you [don't pay for their CPU and RAM](/docs/about/pricing/#stopped-fly-machines) when they're in a `stopped` or `suspended` state. -Learn more about how [Fly Proxy autostops and autostarts Machines](/docs/reference/fly-proxy-autostop-autostart/). +Learn the details of how [Fly Proxy autostops and autostarts Machines](/docs/reference/fly-proxy-autostop-autostart/). ## When to use autostop/autostart -You can reduce resource usage and costs by using autostop/autostart to manage your Fly Machines as demand decreases and increases. You'll never have to run excess Machines to handle peak load; you'll only run, and get charged for, the number of Machines that you need. Autostart/autostop works well for apps with highly variable workloads, for smaller apps with low or sporadic traffic, and for most apps that aren't receiving requests continuously. If you need to, you can keep one or more Machines running in your primary region. +Autostop/autostart works well for apps with highly variable workloads, for smaller apps with low or sporadic traffic, and for most apps that aren't receiving requests continuously. You can reduce resource usage and costs by using autostop/autostart to manage your Fly Machines as demand decreases and increases. You'll never have to run excess Machines to handle peak load; you'll only run, and get charged for, the number of Machines that you need. If you need to, you can keep one or more Machines running in your primary region. ## Configure autostop/autostart @@ -40,9 +40,9 @@ Example configuration: In the preceding example, Fly Proxy will automatically stop Machines when the app has excess capacity and start them again when needed based on traffic to the app. The app will eventually scale down to one running Machine in the primary region if there's no traffic. -Concurrency limits configured for services in the `fly.toml` file also affect [how autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/). +Concurrency limits configured for services in the `fly.toml` file also affect [how autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/): Fly Proxy uses the concurrency [`soft_limit` setting](/docs/reference/configuration/#http_service-concurrency) for each Machine to determine if there's excess capacity per region. -## Recommendations for autostop/autostart configuration +## Autostop/autostart configuration recommendations In general, unless your app shuts itself down when idle, we recommend setting `auto_stop_machines` and `auto_start_machines` so that they are both enabled or both disabled, to avoid having Machines that either never start or never stop. For example, if `auto_start_machines = false` and `auto_stop_machines = "stop"`, then Fly Proxy automatically stops your Machines when there's low traffic but doesn't start them again. When all or most of your Machines stop, requests to your app could start failing. @@ -54,7 +54,7 @@ To keep one or more Machines running all the time in your primary region, set `m There's no "maximum machines running" setting, because the maximum number of Machines is the total number of Machines in your app. -The autostop/autostart feature never creates or destroys Machines for you. The maximum number of running Machines is the number of Machines created for your app on launch, or using `fly scale count` or `fly machine clone`. For example, if you want to have a maximum of 10 Machines available to service requests, then you need to create 10 Machines for your app. Learn more about [scaling the number of Machines](/docs/apps/scale-count/). +Fly Proxy autostop/autostart never creates or destroys Machines for you. The maximum number of running Machines is the number of Machines created for your app on launch, or using `fly scale count` or `fly machine clone`. For example, if you want to have a maximum of 10 Machines available to service requests, then you need to create 10 Machines for your app. Learn more about [scaling the number of Machines](/docs/apps/scale-count/). ### Keep all Machines running continuously @@ -62,6 +62,10 @@ If you need all your app's Machines to run continuously, then you can set `auto_ If you only need a certain number of your app's Machines to run continuously, then you can set `auto_stop_machines` to `"suspend"` or `"stop"` and `min_machines_running` to `1` or higher. Note that `min_machines_running` only applies to your app's primary region. +### Private apps + +Requests to apps without services configured on your private network don't get routed through Fly Proxy and so Machines can't be automatically stopped or started by Fly Proxy. To use Fly Proxy autostart/autostart for private apps, set up services and use a Flycast private IPv6 address. See [Flycast - Private Fly Proxy Services](/docs/networking/flycast/) and [Autostop/autostart private apps](/docs/blueprints/autostart-internal-apps/). + ### Apps that shut down when idle Setting your app to automatically stop or suspend Machines when there's excess capacity using `auto_stop_machines` can be a substitute for when your app doesn't shut itself down automatically after a period of inactivity. If your app already shuts down when idle, then you can set `auto_start_machines = true` and `auto_stop_machines = "off"` to have Fly Proxy automatically restart the Machines stopped by your app. @@ -98,4 +102,4 @@ When you create a new app using the `fly launch` command, the default settings i ### Apps with no autostop/autostart settings -For apps that don't explicitly specify any autostop/autostart settings in `fly.toml`, the Fly Proxy will automatically start stopped Machines when needed, but won't automatically stop or suspend them. +For apps that don't explicitly specify any autostop/autostart settings in `fly.toml`, Fly Proxy will automatically start stopped Machines when needed, but won't automatically stop or suspend them. diff --git a/networking/flycast.html.markerb b/networking/flycast.html.markerb index e82db1f598..2ca60b11df 100644 --- a/networking/flycast.html.markerb +++ b/networking/flycast.html.markerb @@ -40,7 +40,7 @@ To use Flycast for your app, you need: ## Allocate a Flycast address -A Flycast address is an app-wide private IPv6 address that the Fly Proxy can route to over the private network. By default, the Flycast IP address is allocated on an app's default organization network. +A Flycast address is an app-wide private IPv6 address that Fly Proxy can route to over the private network. By default, the Flycast IP address is allocated on an app's default organization network. There are a few ways to add a Flycast address to an existing app. diff --git a/networking/private-networking.html.md b/networking/private-networking.html.md index 51cffc9b4a..f75e4a6083 100644 --- a/networking/private-networking.html.md +++ b/networking/private-networking.html.md @@ -22,7 +22,7 @@ A Fly Machine is configured to resolve domain names with a custom DNS server fro Underneath `.internal` there are second-level domains for every app in your Fly organization. For example, if your app is in an organization with another app called `my-app-name`, then there will be a AAAA record at `my-app-name.internal`. The AAAA record will contain all the 6PN addresses of the started Fly Machines that belong to the `my-app-name` Fly App. Note that different libraries and tools will use multi-address AAAA records differently; most will only use the first address that is returned, but others might round-robin between entries for every request -- if you'd like to know more, consult the documentation for the library or tool you are using for DNS lookup.
-**Important:** All queries to Fly.io `.internal` domains only return information for started (running) Machines. Any stopped Machines, including those autostopped by the Fly Proxy, are not included in the response to the DNS query. +**Important:** All queries to Fly.io `.internal` domains only return information for started (running) Machines. Any stopped Machines, including those autostopped by Fly Proxy, are not included in the response to the DNS query.
Each `.internal` domain has further subdomains which can be used to return a more precise subset of the started Machines in that app. For example, you can add a region name qualifier to return the 6PN addresses of an app's Machines in a specific region: `iad.my-app-name.internal`. Querying this domain returns the 6PN addresses of `my-app-name` Machines in the `iad` region. diff --git a/rails/advanced-guides/machine.html.md b/rails/advanced-guides/machine.html.md index ca9ad0519f..dcb7b36355 100644 --- a/rails/advanced-guides/machine.html.md +++ b/rails/advanced-guides/machine.html.md @@ -43,7 +43,7 @@ Optionally configure your project: bin/rails generate fly:app --passenger --serverless ``` -The above command will configure you application to scale to zero whenever it has been idle for 5 minutes. See [generator options](https://github.com/rubys/fly.io-rails#generator-options) for more details. +The above command will configure your application to scale to zero whenever it has been idle for 5 minutes. See [generator options](https://github.com/rubys/fly.io-rails#generator-options) for more details. Feel free to tailor the generated files further to suit your needs. If you don't run the `fly:app` generator, the files necessary to deploy your application will be generated on your first deploy using default options. diff --git a/reference/autoscaling.html.md b/reference/autoscaling.html.md index aa04e9bdbe..852888de0a 100644 --- a/reference/autoscaling.html.md +++ b/reference/autoscaling.html.md @@ -6,12 +6,12 @@ nav: firecracker Autoscaling adjusts the number of running or created Fly Machines dynamically. We support two forms of autoscaling: -- Autostart/autostop Machines +- Autostop/autostart Machines - Metrics-based autoscaling -## Autostart/autostop Machines +## Autostop/autostart Machines -The Fly Proxy autostart/autostop feature starts and stops Machines based on load; Machines are never created or deleted. You create a "pool" of Machines in one or more regions and the Fly Proxy starts and stops them as needed. +Fly Proxy autostop/autostart starts and stops Machines based on load; Machines are never created or deleted. You create a "pool" of Machines in one or more regions and Fly Proxy starts and stops them as needed. For details about how it works, see [Fly Proxy autostop/autostart](/docs/launch/autostop-autostart/). To configure autostop/autostart for your app, see [Autostop/autostart Machines](/docs/launch/autostop-autostart/). diff --git a/reference/fly-proxy-autostop-autostart.html.markerb b/reference/fly-proxy-autostop-autostart.html.markerb index d79c265172..0d80c70b74 100644 --- a/reference/fly-proxy-autostop-autostart.html.markerb +++ b/reference/fly-proxy-autostop-autostart.html.markerb @@ -8,18 +8,20 @@ nav: firecracker -For Fly Apps with a service configured, [Fly Proxy](/docs/reference/fly-proxy) can automatically stop (or suspend) and start existing Machines based on incoming requests, so that your app can meet demand without keeping extra Machines running. Fly Machines are fast to start and stop, and you don't pay for their CPU and RAM when they're in a `stopped` or `suspended` state. You can also set a minimum number of machines to keep running at all times in your primary region. +Fly Proxy autostop/autostart automatically stops (or suspends) and starts existing Machines based on incoming requests to your app. You can scale your app for peak loads, and save on resource costs by only running those Machines when needed, since you [don't pay for Machine CPU and RAM](/docs/about/pricing/#stopped-fly-machines) when they're in a `stopped` or `suspended` state. You can set a minimum number of machines to keep running at all times in your primary region. -You can also use the autostop/autostart with apps that [shut down automatically when idle](#stop-a-machine-by-terminating-its-main-process). The Fly Proxy can still restart your app when there's traffic. +Fly Proxy autostop/autostart works for Fly Apps with a service configured in the `fly.toml` file, which covers publicly available apps that accept requests from the internet. You can also use Fly Proxy autostart/autostart for private apps, by setting up services and using a Flycast private IPv6 address. See [Flycast - Private Fly Proxy Services](/docs/networking/flycast/) and [Autostop/autostart private apps](/docs/blueprints/autostart-internal-apps/). + +Autostop/autostart works well with apps that [shut down automatically when idle](/docs/launch/autostop-autostart/#apps-that-shut-down-when-idle). The Fly Proxy can still restart your app when there's traffic. Learn how to [configure autostop/autostart](/docs/launch/autostop-autostart/). -## How Fly Proxy autostops and autostarts Machines +## How Fly Proxy autostop/autostart works -The Fly Proxy runs a process to determine whether an app has excess capacity every few minutes. +Fly Proxy runs a process to determine whether an app has excess capacity every few minutes.
-The autostop/autostart feature only works on existing Machines and never creates or destroys Machines for you. The maximum number of running Machines is the number of Machines you've created for your app using `fly scale count` or `fly machine clone`. Learn more about [scaling the number of Machines](/docs/apps/scale-count/). +Autostop/autostart only works on existing Machines and never creates or destroys Machines for you. The maximum number of running Machines is the number of Machines you've created for your app using `fly scale count` or `fly machine clone`. Learn more about [scaling the number of Machines](/docs/apps/scale-count/).
### Fly Proxy process to stop or suspend Machines diff --git a/reference/fly-proxy.html.markerb b/reference/fly-proxy.html.markerb index 9d04f70ba9..d7eb3cb6a1 100644 --- a/reference/fly-proxy.html.markerb +++ b/reference/fly-proxy.html.markerb @@ -16,7 +16,7 @@ A summary of things Fly Proxy does: - terminate TLS by default for most public apps - terminate TLS for Postgres connections - write PROXY protocol headers on TCP connections -- autostart/autostop Machines based on traffic to your app +- autostop/autostart Machines based on traffic to your app - dynamic request routing with the `fly-replay` response header ## How Fly Proxy routes incoming public requests @@ -49,9 +49,9 @@ Learn more about [connection handlers](/docs/networking/services/#connection-han Fly Proxy can start and stop existing Machines based on incoming requests, so that your app can accommodate bursts in demand without keeping extra Machines running constantly. -Fly Proxy uses the same concurrency settings for autostart/autostop as for load balancing to determine when Machines have excess capacity and can be stopped. +Fly Proxy uses the same concurrency settings for autostop/autostart as for load balancing to determine when Machines have excess capacity and can be stopped. -Learn more about [how autostart/autostop works](/docs/reference/fly-proxy-autostop-autostart/) and [how to configure it](/docs/apps/autostop-autostart/). +Learn more about [how autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/) and [how to configure it](/docs/apps/autostop-autostart/). ## Dynamic request routing with `fly-replay` diff --git a/reference/index.html.md b/reference/index.html.md index 380c7172ca..6014cfa94f 100644 --- a/reference/index.html.md +++ b/reference/index.html.md @@ -26,6 +26,8 @@ Quick references for often-used resources like flyctl and `fly.toml`. Or dig a l * **[Fly Proxy](/docs/reference/fly-proxy/):** Learn about how Fly Proxy routes requests, handles connections, and does load balancing. +* **[Fly Proxy autostop/autostart](/docs/reference/fly-proxy-autostop-autostart/):** Learn how Fly Proxy determines excess capacity for an app to shut down or suspend Machines when they're not needed and start them back up when there's traffic. + --- ## Working with Fly.io From 47baec997c9bc50bb009c72c2d280612c39bc5ed Mon Sep 17 00:00:00 2001 From: andie787 <4andie@gmail.com> Date: Wed, 14 Aug 2024 14:34:50 -0400 Subject: [PATCH 5/6] related topics --- launch/autostop-autostart.html.markerb | 10 ++++++++-- reference/fly-proxy-autostop-autostart.html.markerb | 6 +++--- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/launch/autostop-autostart.html.markerb b/launch/autostop-autostart.html.markerb index 25acd4cb3b..53a3fa1162 100644 --- a/launch/autostop-autostart.html.markerb +++ b/launch/autostop-autostart.html.markerb @@ -9,11 +9,11 @@ redirect_from: For Fly Apps with a service configured, [Fly Proxy](/docs/reference/fly-proxy/) can automatically start and stop or suspend existing Machines based on incoming requests, so that your app can meet demand without keeping extra Machines running. Fly Machines are fast to start and stop, and you [don't pay for their CPU and RAM](/docs/about/pricing/#stopped-fly-machines) when they're in a `stopped` or `suspended` state. -Learn the details of how [Fly Proxy autostops and autostarts Machines](/docs/reference/fly-proxy-autostop-autostart/). +Learn the details of [how Fly Proxy autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/). ## When to use autostop/autostart -Autostop/autostart works well for apps with highly variable workloads, for smaller apps with low or sporadic traffic, and for most apps that aren't receiving requests continuously. You can reduce resource usage and costs by using autostop/autostart to manage your Fly Machines as demand decreases and increases. You'll never have to run excess Machines to handle peak load; you'll only run, and get charged for, the number of Machines that you need. If you need to, you can keep one or more Machines running in your primary region. +Autostop/autostart works well for apps with highly variable workloads, for smaller apps with low or sporadic traffic, and for most apps that aren't receiving requests continuously. You can reduce resource usage and costs by using autostop/autostart to manage your Fly Machines as demand decreases and increases. You'll never have to run excess Machines to handle peak load; you'll only run, and get charged for, the number of Machines that you need. You can choose to keep one or more Machines running in your primary region. ## Configure autostop/autostart @@ -103,3 +103,9 @@ When you create a new app using the `fly launch` command, the default settings i ### Apps with no autostop/autostart settings For apps that don't explicitly specify any autostop/autostart settings in `fly.toml`, Fly Proxy will automatically start stopped Machines when needed, but won't automatically stop or suspend them. + +## Related topics + +- [How Fly Proxy autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/) +- [Autostop/autostart private apps](/docs/blueprints/autostart-internal-apps/) +- [Autoscale based on metrics](/docs/launch/autoscale-by-metric/) diff --git a/reference/fly-proxy-autostop-autostart.html.markerb b/reference/fly-proxy-autostop-autostart.html.markerb index 0d80c70b74..6af82b0bfe 100644 --- a/reference/fly-proxy-autostop-autostart.html.markerb +++ b/reference/fly-proxy-autostop-autostart.html.markerb @@ -8,9 +8,9 @@ nav: firecracker -Fly Proxy autostop/autostart automatically stops (or suspends) and starts existing Machines based on incoming requests to your app. You can scale your app for peak loads, and save on resource costs by only running those Machines when needed, since you [don't pay for Machine CPU and RAM](/docs/about/pricing/#stopped-fly-machines) when they're in a `stopped` or `suspended` state. You can set a minimum number of machines to keep running at all times in your primary region. +Fly Proxy autostop/autostart automatically stops (or suspends) and starts existing Machines based on incoming requests to your app. You can scale your app for peak loads, and save on resource costs by running those Machines only when needed, since you [don't pay for Machine CPU and RAM](/docs/about/pricing/#stopped-fly-machines) when they're in a `stopped` or `suspended` state. You can set a minimum number of machines to keep running at all times in your primary region. -Fly Proxy autostop/autostart works for Fly Apps with a service configured in the `fly.toml` file, which covers publicly available apps that accept requests from the internet. You can also use Fly Proxy autostart/autostart for private apps, by setting up services and using a Flycast private IPv6 address. See [Flycast - Private Fly Proxy Services](/docs/networking/flycast/) and [Autostop/autostart private apps](/docs/blueprints/autostart-internal-apps/). +Fly Proxy autostop/autostart works for Fly Apps with a service configured in the `fly.toml` file, which covers publicly available apps that accept requests from the internet. But you can also use Fly Proxy autostart/autostart for private apps, by setting up services and using a Flycast private IPv6 address. See [Flycast - Private Fly Proxy Services](/docs/networking/flycast/) and [Autostop/autostart private apps](/docs/blueprints/autostart-internal-apps/). Autostop/autostart works well with apps that [shut down automatically when idle](/docs/launch/autostop-autostart/#apps-that-shut-down-when-idle). The Fly Proxy can still restart your app when there's traffic. @@ -51,6 +51,6 @@ Fly Proxy determines when to start a Machine as follows: ## Related topics -- [Fly Proxy features](/docs/reference/fly-proxy) - [Configure autostop/autostart for Fly Launch apps](/docs/launch/autostop-autostart/) - [Autostop/autostart private apps](/docs/blueprints/autostart-internal-apps/) +- [Fly Proxy features](/docs/reference/fly-proxy) From 0beaaf4082ad6391b7c904da1f3551ea7cf357ef Mon Sep 17 00:00:00 2001 From: andie787 <4andie@gmail.com> Date: Wed, 14 Aug 2024 16:07:07 -0400 Subject: [PATCH 6/6] feedback --- launch/autostop-autostart.html.markerb | 8 ++++---- reference/fly-proxy-autostop-autostart.html.markerb | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/launch/autostop-autostart.html.markerb b/launch/autostop-autostart.html.markerb index 53a3fa1162..3eff5dfe49 100644 --- a/launch/autostop-autostart.html.markerb +++ b/launch/autostop-autostart.html.markerb @@ -7,9 +7,9 @@ redirect_from: - /docs/launch/autostart-stop/ --- -For Fly Apps with a service configured, [Fly Proxy](/docs/reference/fly-proxy/) can automatically start and stop or suspend existing Machines based on incoming requests, so that your app can meet demand without keeping extra Machines running. Fly Machines are fast to start and stop, and you [don't pay for their CPU and RAM](/docs/about/pricing/#stopped-fly-machines) when they're in a `stopped` or `suspended` state. +Your app can meet peak demand without keeping extra Machines running. Use autostop/autostart to automatically start and stop or suspend existing Machines based on incoming requests. Fly Machines are fast to start and stop, and you [don't pay for their CPU and RAM](/docs/about/pricing/#stopped-fly-machines) when they're in a `stopped` or `suspended` state. -Learn the details of [how Fly Proxy autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/). +Get all the details of [how Fly Proxy autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/). ## When to use autostop/autostart @@ -17,11 +17,11 @@ Autostop/autostart works well for apps with highly variable workloads, for small ## Configure autostop/autostart -The autostop/autostart settings apply per service in an app's `fly.toml` file. See the [[[services]]](/docs/reference/configuration/#the-services-sections) or [[http_service]](/docs/reference/configuration/#the-http_service-section) docs for details about service configuration. +The autostop/autostart settings apply per service in an app's `fly.toml` file. See the [[[services]]](/docs/reference/configuration/#the-services-sections) or [[http_service]](/docs/reference/configuration/#the-http_service-section) docs for details about service configuration. You can also add services to [private apps](#private-apps). Autostop/autostart settings: -* **`auto_stop_machines`:** The action, if any, that Fly Proxy should take when the app is idle for several minutes. Options are `"off"`, `"stop"`, or `"suspend"`. If `"off"`, Fly Proxy will never stop Machines. If `"stop"`, Fly Proxy stops Machines when the app has excess capacity. If `"suspend"`, Fly Proxy suspends Machines (if possible) when the app has excess capacity. Starting a Machine from a `suspended` state is faster than starting a Machine from a `stopped` state, but there are [some caveats](https://community.fly.io/t/autosuspend-is-here-machine-suspension-is-enabled-everywhere/20942). +* **`auto_stop_machines`:** The action, if any, that Fly Proxy should take when the app is idle for several minutes. Options are `"off"`, `"stop"`, or `"suspend"`. If `"off"`, Fly Proxy will never stop Machines. If `"stop"`, Fly Proxy stops Machines when the app has excess capacity. If `"suspend"`, Fly Proxy suspends Machines (if possible) when the app has excess capacity. Starting a Machine from a `suspended` state is faster than starting a Machine from a `stopped` state, but there are [some caveats](https://community.fly.io/t/autosuspend-is-here-machine-suspension-is-enabled-everywhere/20942) that you should know about. * **`auto_start_machines`:** Whether Fly Proxy should automatically start Machines based on requests and capacity. * **`min_machines_running`:** The minimum number of Machines to keep running in the primary region when `auto_stop_machines` is set to `"stop"` or `"suspend"`. diff --git a/reference/fly-proxy-autostop-autostart.html.markerb b/reference/fly-proxy-autostop-autostart.html.markerb index 6af82b0bfe..5dd4930e97 100644 --- a/reference/fly-proxy-autostop-autostart.html.markerb +++ b/reference/fly-proxy-autostop-autostart.html.markerb @@ -8,9 +8,9 @@ nav: firecracker -Fly Proxy autostop/autostart automatically stops (or suspends) and starts existing Machines based on incoming requests to your app. You can scale your app for peak loads, and save on resource costs by running those Machines only when needed, since you [don't pay for Machine CPU and RAM](/docs/about/pricing/#stopped-fly-machines) when they're in a `stopped` or `suspended` state. You can set a minimum number of machines to keep running at all times in your primary region. +With Fly Proxy autostop/autostart, you can scale your app for peak loads and run those Machines only when needed to save on resource costs; you [don't pay for Machine CPU and RAM](/docs/about/pricing/#stopped-fly-machines) when they're in a `stopped` or `suspended` state. Fly Proxy automatically stops (or suspends) and starts existing Machines based on incoming requests to your app, and you have the option to keep a minimum number of machines running at all times in your primary region. -Fly Proxy autostop/autostart works for Fly Apps with a service configured in the `fly.toml` file, which covers publicly available apps that accept requests from the internet. But you can also use Fly Proxy autostart/autostart for private apps, by setting up services and using a Flycast private IPv6 address. See [Flycast - Private Fly Proxy Services](/docs/networking/flycast/) and [Autostop/autostart private apps](/docs/blueprints/autostart-internal-apps/). +Fly Proxy autostop/autostart works for Fly Apps with a service configured in the `fly.toml` file, which generally covers publicly available apps that accept requests from the internet. But you can also use Fly Proxy autostart/autostart for private apps, by setting up services and using a Flycast private IPv6 address. See [Flycast - Private Fly Proxy Services](/docs/networking/flycast/) and [Autostop/autostart private apps](/docs/blueprints/autostart-internal-apps/). Autostop/autostart works well with apps that [shut down automatically when idle](/docs/launch/autostop-autostart/#apps-that-shut-down-when-idle). The Fly Proxy can still restart your app when there's traffic.