Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update generic_parser from_json return after ingestion #816

Merged
merged 2 commits into from
Jun 11, 2024

Conversation

EddieLF
Copy link
Contributor

@EddieLF EddieLF commented Jun 11, 2024

This change makes the generic_parser.from_json function return the full response it receives from Metamist.

When the parser is used to ingest from a manifest, the parser is instantiated:

parser = GenericParser(
    project=dataset,
    search_paths=search_paths,
)

We then call parser.from_manifest_path

response = await parser.from_manifest_path(manifest=manifest_path, dry_run=dry_run)

This returns parser.parse_manifest, which in turn returns parser.from_json. Currently, from_json has no returned value unless the --dry-run flag is set.
This means that nothing can be done to analyse the response beyond what gets logged by the generic parser.

response = await parser.from_manifest_path(manifest=manifest_path, dry_run=dry_run)
print(response)
>>> None

By adding a return result to from_json, the result/response can be printed or analysed by any custom parser implementation.

response = await parser.from_manifest_path(manifest=manifest_path, dry_run=dry_run)
print(json.dumps(response, indent=2))
>>> { Actual JSON response }

Copy link
Collaborator

@illusional illusional left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems sensible!

@EddieLF EddieLF merged commit f53f16d into dev Jun 11, 2024
5 checks passed
@EddieLF EddieLF deleted the generic_parser_return branch June 11, 2024 06:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants