Async sync strategies

[ltouro]

Hello,

I have the following scenario: Several front-end web servers serving a web app, each one having a local filesystem, to where user uploads are placed temporarily. An external storage, in which is applied sync strategies from the different local filesystems, in order to ensure that external storage has all files uplodaded by users.

For instance, It is desirable that front-end servers write files on the local fileserver on synchronous fashion but writes the file to the external storage on asynchronous fashion.

I was wondering the best path to achieve this sync scenario: Eventable Filesystem allow me to hook events on after-write of files to the different local filesystems but i'm not sure how it would work. I'm afraid that these events are triggered in synchronous fashion and I dont wan't to make the user to wait for these sync events to complete.

I'm also have some application servers that run workers to do jobs throught supervisord like service.
The sync strategy could be a job itself but how workers (in app server) would know about each front-end filesystem access endpoint? The front-end webservers may have horizontal scalling configured. Thus, I dont know in advance how many front end server will be running at given time.

Running the worker/job on each front-end is not an option due to shell access restrictions which prevents running workers reliably.

Is the Eventable Filesystem suitable for this use-case? Does anyone have any tips to this kind of use-case? I'm doing something wrong (besides my poor english)? Any comments are welcome.

[basz]

I place uploaded files to an s3 bucket with an lifetime policy attached to à specific path. That means that when the upload is abanded the file is deleted automaticly. It also means a worker can have access to it no matter which webserver handled the upload.