Predictor.from_path using in memory file instead of disk

[smiles3983]

We use openshift, and our enterprise sets limits on ephemeral storage. We need to be able to load a 1.4GB Model and all I see available is Predictor.from_path, which requires the model to be on disk before we can use it. They wont let us use something like S3FS at the moment, so we cant "mount" a network storage location. Is there a way we can take the model that we have loaded into memory as a binary file, and load it to the Predictor directly? We apparently have plenty of RAM to work with.

[smiles3983]

We asked the same to Spacy, and they had a response for us. explosion/spaCy#3841

[epwalsh]

Hi @smiles3983, so if I'm understanding this correctly, you'd like to have something like a Predictor.from_bytes() method?

[smiles3983]

Hi @smiles3983, so if I'm understanding this correctly, you'd like to have something like a Predictor.from_bytes() method?

Yes that would be great! Does it exist and I’m just not seeing it?

[epwalsh]

It does not exist at the moment, but it's feasible to implement. If you (or someone else) is interested in making a PR to implement this, we'd love to see it. Otherwise I could take a look next week.

[smiles3983]

Yeah, not sure our team will be able to do it, unless they want to do it from home on their own time :). Dont think we can clone outside our Intranet. We use Nexus to bring down packages. Were open source leeches haha.

[smiles3983]

It looks like torch has the ability to load with io.bytesIO. We will be pulling our model from an S3 bucket using s3.get_object(). So if we can convert that to what torch needs without touching the file system.

# Load tensor from io.BytesIO object
>>> with open('tensor.pt', 'rb') as f:
...     buffer = io.BytesIO(f.read())
>>> torch.load(buffer)

[krishna-anumula]

Hi @epwalsh any update on this request... Even I would like to have this configuration.

[epwalsh]

Hi @krishna-anumula, I don't have the bandwidth to tackle this at the moment, but I'm happy to review a PR if you'd like to work on it.