Dependent Jobs

[bassrock]

Unless this exists and I can not find a cool new feature would be dependent jobs. Whereas a job will stay queued unless a job it depends on has finished executing.

[danhunsaker]

This can be tricky to implement. Useful, certainly, but tricky. What kind of dependency do you want, here? By job ID? By job class? Something else? If you're using the by-ID approach, it's not terribly hard to determine when the job is done, but if you want to support by-class, things get more complex. Would you want support for a job depending on multiple prior jobs, waiting for all of them to complete before running? That adds a bit more complexity as well.

You'd want to add dependent jobs to a separate Redis key (probably a set) until the job they depend on completes, using a hook or the job's teardown to move them onto the regular job queue when the dependency is met. The alternative is to have the job itself determine if its dependency is met, then requeue itself if not. This is a really inefficient approach, which makes it a bad option.

It can be done, but it'll take some thinking to do it right.

[bassrock]

I was thinking by job id much like the JobQueueBundle. We should probably decide on tactics before someone starts coding it though.

[bassrock]

Working on this and more management features with a db persistence here: https://github.com/DABSquared/BCCResqueBundle

[danhunsaker]

By-ID makes sense. I'd like to see multi-ID dependency support (job B needs job A to complete before it can start, but job D needs jobs B and C to both complete first), if possible.

Of course, I'd also like to see this upstream, in PHP-Resque proper. But getting code merged there is unlikely for now.