Feature/pdct 1419 map family metadata to metadata key in family json structure #13

odrakes-cpr · 2024-09-05T10:59:46Z

What's changed?

Add functionality to the GCF data mapper tool; mapping family metadata from the source file to json output suitable for the bulk import tool

Proposed version

Please select the option below that is most relevant from the list below. This
will be used to generate the next tag version name during auto-tagging.

Skip auto-tagging
Patch
Minor version
Major version

Visit the Semver website to understand the
difference between MAJOR, MINOR, and PATCH versions.

Notes:

If none of these options are selected, auto-tagging will fail
Where multiple options are selected, the most senior option ticked will be
used -- e.g. Major > Minor > Patch
If you are selecting the version in the list above using the textbox, make
sure your selected option is marked [x] with no spaces in between the
brackets and the x

checks if row has any required columns with empty values, checks if required columns from a row are missing

add tests accordingly

will revisit to try and fix pyright typing errors

- add function to check for value and key existence in nested objects - add private methods - doc string tidy ups - tidy up function that gets budget information

it can be used more widely across the codebase

linear · 2024-09-05T10:59:48Z

PDCT-1419 Map family metadata to metadata key in family JSON structure

instead of raising and stopping the tool from running we can just stdout the error

- remove the function that checks for values in nested objects - we don't want the tool to stop running for malformed data, only when a key is missing. - so we have updated the helper function to check for false/empty values in a list and just echo it out and then take it to the client - We have gone back to pythons normal way of accessing keys in a dict, where a key error will be raised if we try access something that is not there

… legible

gcf_data_mapper/parsers/family.py

gcf_data_mapper/parsers/helpers.py

gcf_data_mapper/parsers/family.py

katybaulch

Great work! Just some small changes

katybaulch

nice work!!

Osneil Drakes and others added 11 commits September 3, 2024 18:40

WIP : implement mapping for family metadata

e6e17b3

feat: add helper functions

56c05f2

checks if row has any required columns with empty values, checks if required columns from a row are missing

refactor: update helper function to display columns with empty values

ad87fc3

add tests accordingly

refactor: revert code back to display generic value error message

30f931c

will revisit to try and fix pyright typing errors

feat: update family data mapper

aa1a2de

- add function to check for value and key existence in nested objects - add private methods - doc string tidy ups - tidy up function that gets budget information

refactor: move function into helper file

e9aea30

it can be used more widely across the codebase

refactor: update required columns to be a set instead of a list

b96dea8

test: add test for family mapping

b3fe618

merge main

4f92cce

Bump version to 0.1.9

3914c69

test: update family test data

97da22c

odrakes-cpr requested a review from a team as a code owner September 5, 2024 10:59

Osneil Drakes and others added 8 commits September 5, 2024 12:48

test: update conftest fixture to include fields that we interact with

8f58db3

merge main

d911b87

bump version to 0.1.10

4ac7a0b

refactor: tidy up mapper to limit value errors being raised

67d464f

instead of raising and stopping the tool from running we can just stdout the error

refactor: update naming to improve readability

7c17f52

refactor: renaming of variables and functions to make them a bit more…

5565e92

… legible

test: update tests

dc05686