feature suggestion: --folow argument for pueue add

[katrinafyi]

A detailed description of the feature you would like to see added.

I would like to have a simple --follow flag to add. Running pueue add --folow <CMD> should act just like:

pueue add <CMD>
pueue follow <TASK_ID_FROM_ADD>

Basically, it adds the task and follows its output. Just like the separate commands, Ctrl+C during a pueue add --folow should stop following but continue running the task.

I am not quite sure what the behaviour should be if the task cannot be immediately started. Running separate add/follow will just block while waiting for the command to start. Maybe it should print a warning then continue waiting.

Explain your usecase of the requested feature

This will help get instant feedback when there is an initialisation problem for long-running tasks. A lot of the time, I run pueue add but make simple mistakes in the command (e.g. typo, wrong working directory). The mistakes cause the task to fail immediately but this is not immediately visible.

At the moment, I have to remember to pueue status after every add to make sure that the command is running properly. It's easy to forget this step, go away, and come back to an error instead of the the results I waited for.

Alternatives

One could make an alias, however this makes it harder to add more options to add. I guess this bash would work:

pueue-add-follow() {
  task=$(pueue add --print-task-id "$@")
  sleep 1 # XXX: race condition! hopefully, if it is not queued, it will start after one second.
  pueue --color=always status | grep '^\s*'"$task"' '  # XXX: hack: status query does not support filtering by ID
  pueue follow $task
}

but it's a little janky.
The comments show how difficult it is to mimic the behaviour, especially if we want user-friendly information about whether the new task is running or in a queue.

Myself, I just try to remember pueue status after every pueue add.

Additional context

Thank you for the well-made project! :)

[Nukesor]

Hey @katrinafyi :)

Thanks for the kind words and your detailed feature request.

I've thought about your request for a bit and I feel like your proposed alternative bash function should actually be the way to go.

My reasoning

The normal usecase of how pueue is usually used (and what it's designed for) is to queue multiple long running commands that run sequentially to at least some degree. Having a --follow flag on add if the command might only be started hours in the future will be very confusing for some people (At least I find it confusing :D).

I see how that makes total sense if one has mostly short-lived commands or if the parallelization of a group is set to a very high number. However, that's not what I consider the 95% usecase.

Regarding the script:

The race condition issue in your script is fixed in v4.0.0, for which there's already a release candidate

To wait for the task to enter the running state, the pueue wait subcommand can be used.
The script would then look something like this:

pueue-add-follow() {
  task_id=$(pueue add --print-task-id "$@")
  pueue wait --status running $task_id
  pueue follow $task_id
}

It also prints out state transitions with time stamps (even though the original state isn't printed. But I would be up for a feature request that prints the original states of all tasks that're waited for)
Would that be a more reasonable solution for you?

[katrinafyi]

I guess I am the 5%, I use pueue as a more powerful & (indeed, with unbounded parallelism :)). It is pretty nice for this use case, as commands are detached from the parent shell and can accessed across shells.

I would still like to see this feature, as I feel like it's a small change that would patch up a big weakness in this use case.

That said, I see how it could have confusing interactions with queued tasks. Maybe --follow could be made to imply --immediate? When looking through help for this option, I actually assumed --immediate would act like my --follow, but it does not :)

But it is up to you, using the function alias will not be too bad.

[Nukesor]

Hmm. Let me think about the --follow as only applicable in conjunction with --immediate for a bit. Sounds reasonable :)

[Nukesor]

@katrinafyi Could you take a look at #595 and test it?

Right now the output looks something like this:

[nuke:../repos/tools/pueue] immediate-follow > pueue add -i --follow -- 'ls'
Cargo.lock
Cargo.toml
CHANGELOG.md
codecov.yml
Cross.toml
docs
Justfile
LICENSE.APACHE
LICENSE.MIT
pueue
pueue_lib
README.md
target
utils

To build from source:

git clone [email protected]:Nukesor/pueue
cd pueue
git switch immediate-follow
cargo build --release --locked --path ./pueue