Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blob Get By File Name #31

Open
andscoop opened this issue Jan 18, 2018 · 1 comment
Open

Blob Get By File Name #31

andscoop opened this issue Jan 18, 2018 · 1 comment

Comments

@andscoop
Copy link

My current understanding of the API is that in order to download a file (blobs_get) you must have a file key. In order to obtain a file key you must call files_list which requires commitId. In order to get a commitId you must run a file and check the status of that run.

Is there a better way to obtain the file key through the API? Currently downloading a file through the API is nearly pointless if I have to run a file first.

@InbarRose
Copy link

InbarRose commented Mar 20, 2018

I am having the same issue. I also want this solution.

It would be great to be able to download a file by simply providing the file-path (perhaps a different function). Even if I still have to provide the commitId. But I would also like to be able to leave commitId None and it simply take the commitId of the latest state of the project.

For anyone interested, here is the workaround I wrote using the logic you mentioned above:

def download_file(domino, file_path, local_path):
    # == start run : this can take a while ==
    # start a new run with a script that doesn't do anything
    run_results = domino.runs_start_blocking(command=['no-op-script.py'])
    # extract the run id
    run_id = run_results['runId']
    # fetch information about the run with the run_id
    run_info = domino.get_run_info(run_id)
    # find the commit_id of that run
    commit_id = run_info['commitId']  # maybe need to use 'outputCommitId'

    # == find file blob : this could fail, make sure you have the right path! ==
    # fetch all the files in this commit
    files_list = domino.files_list(commit_id)
    # make a list of files
    files = files_list['data']
    # iterate through all the files looking for file with matching path
    for f in files:
        if f['path'] == file_path:
            # save file key when we find the right file
            file_key = f['key']
            # stop looping over files
            break
    else:
        # if we didn't find the file, lets raise an exception
        raise RuntimeError('could not find file with path ({}) in commitId ({})'.format(
            file_path, commit_id))

    # == download file : saves the file to a local path ==
    # get the file object with the file key
    file_object = domino.blobs_get(file_key)
    # read the file object into a string
    file_content = file_object.read()
    # open a file locally to save file contents
    if not os.path.exists(os.path.dirname(local_path)):
        # if file save path dir does not exist we need to create it
        os.makedirs(os.path.dirname(local_path))
    # write file
    with open(local_path, 'wb') as f:
        f.write(file_content)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants