Using the aarch64-apple-darwin binary from https://github.com/restatedev/restate/actions/runs/7465082170 or locally from source built using Rust 1.75.0-aarch64-apple-darwin, I get strange behavior when running a basic stateful doGreet keyed handler against it.
The client gets a successful response from the ingress endpoint:
❯ http :8080/hello/greet key=pavel
HTTP/1.1 200 OK
access-control-allow-credentials: true
content-length: 30
content-type: application/json
date: Wed, 10 Jan 2024 09:19:36 GMT
vary: origin
vary: access-control-request-method
vary: access-control-request-headers
{
"response": "Hello pavel :-)"
}
However the server seems stuck repeatedly trying to invoke the handler over and over, believing the invocation is not complete:
2024-01-10T09:19:36.564491Z INFO restate_ingress_grpc::handler
Processing ingress request
in restate_ingress_grpc::handler::ingress_invoke
otel.name: "ingress_invoke greet"
rpc.system: "grpc"
rpc.service: hello
rpc.method: greet
client.socket.address: 127.0.0.1
client.socket.port: 64729
2024-01-10T09:19:36.572689Z INFO restate_invoker_impl::invocation_task
Executing invocation at deployment
deployment.address: http://localhost:9080/
path: /invoke/hello/greet
2024-01-10T09:19:36.592269Z WARN restate_invoker_impl
Error when executing the invocation, retrying in 50ms.
error: [RT0007] [Unknown] unknown. For more details, look at the docs with https://docs.restate.dev/references/errors#RT0007
restate.error.code: RT0007
restate.invocation.id: NS-fNXAbfe0AYzyq4Z1fWS50UpqzdB3ag
RT0007
A retry-able error was received from the service deployment while processing the invocation.
Suggestions:
• Check the service/deployment logs to get more info about the error cause, like the stacktrace.
• Look at the https://docs.restate.dev/services/sdk/error-handling for more info about error handling in services.
On the worker-side, the handler keeps getting the request replayed:
[restate] [2024-01-10T09:17:35.878Z] INFO: Listening on 9080...
[restate] [2024-01-10T09:17:58.983Z] INFO: Answering discovery request. Registering these services: ["hello"]
[restate] [2024-01-10T09:18:03.478Z] INFO: Answering discovery request. Registering these services: ["hello"]
[restate] [2024-01-10T09:19:36.585Z] DEBUG: [hello/greet] [NS-fNXAbfe0AYzyq4Z1fWS50UpqzdB3ag] : Invoking function.
[restate] [2024-01-10T09:19:36.591Z] DEBUG: [hello/greet] [NS-fNXAbfe0AYzyq4Z1fWS50UpqzdB3ag] : Function completed successfully.
[restate] [2024-01-10T09:19:36.648Z] DEBUG: [hello/greet] [NS-fNXAbfe0AYzyq4Z1fWS50UpqzdB3ag] : Resuming (replaying) function.
[restate] [2024-01-10T09:19:36.649Z] DEBUG: [hello/greet] [NS-fNXAbfe0AYzyq4Z1fWS50UpqzdB3ag] : Function completed successfully.
[restate] [2024-01-10T09:19:36.756Z] DEBUG: [hello/greet] [NS-fNXAbfe0AYzyq4Z1fWS50UpqzdB3ag] : Resuming (replaying) function.
[restate] [2024-01-10T09:19:36.757Z] DEBUG: [hello/greet] [NS-fNXAbfe0AYzyq4Z1fWS50UpqzdB3ag] : Function completed successfully.
[restate] [2024-01-10T09:19:36.967Z] DEBUG: [hello/greet] [NS-fNXAbfe0AYzyq4Z1fWS50UpqzdB3ag] : Resuming (replaying) function.
...
This is the invocation status:
❯ [2024-01-10 11:19:36.568 +02:00] NS-fNXAbfe0AYzyq4Z1fWS50UpqzdB3ag [hello @ pavel]::greet
Status: backing-off (5 minutes, 29 seconds and 241 ms. Retried 40 time(s). Next retry in in 4
seconds and 19 ms))
Deployment: bG9jYWxob3N0OjkwODAv [required]
Trace ID: 00000000000000000000000000000000
Error: [2024-01-10 11:24:59.833 +02:00]
[Unknown] unknown
Using the aarch64-apple-darwin binary from https://github.com/restatedev/restate/actions/runs/7465082170 or locally from source built using Rust 1.75.0-aarch64-apple-darwin, I get strange behavior when running a basic stateful doGreet keyed handler against it.
The client gets a successful response from the ingress endpoint:
However the server seems stuck repeatedly trying to invoke the handler over and over, believing the invocation is not complete:
On the worker-side, the handler keeps getting the request replayed:
This is the invocation status: