Genes data pipeline: update field mapping for variant cooccurrence #1403
Conversation
There was a problem hiding this comment.
The code changes look good to me, nice work and thanks for diving into this pipeline and fixing the things that were breaking it since the data last changed!
My only comment is a bit nitpicky, and so it is not blocking and I'll approve the PR here and just mention my thought re: commit messages.
The commit messages are a bit vague and hard to parse what really they're doing at first glance if they get into main (i.e. update field mapping could refer to quite a few things), if the plan was to squash and merge this PR then that makes sense to me if the PR title includes something about the genes pipeline. In my humble opinion squash and merge seems like a good fit for this PR since one of the commits is one to appease the formatter, and then the individual commits will be saved in the PR history while being a single commit in the main branch history.
Nicely done going for it on the pipelines and getting the genes pipeline to succesfully run in dataproc.
nadeaujoshua
changed the title
While running the data-pipeline for
genes, there was a missing field error for theprepare_heterozygous_variant_cooccurrence_countstask.The source data readme doc shows that the field names have been updated, and so this PR updates the field mapping to use the new field names.
This update was tested locally by running the aforementioned pipeline task successfully.