Medication Source Code Verification: Student 2

[callahantiff]

Student (GitHub Username): @avnmatre
Verification Number: 2

Assigned Domain Expert: @trinklek

Verification Task Description: DropBox Paper Documentation

Phenotype Descriptions: project wiki

Verification Assignments:
UPDATE: counts below updated to reflect today's meeting.

Verification Assignments:

• ADHD (n=34,184 n=6,837) -- Student 2 Verification: ADHD (drugs) #41 (completed by AVM)
• Appendicitis (n=15,304 n=3,061) -- Student 2 Verification: Appendicitis (Drugs) #45
• Crohn’s Disease (n=17,887 n=3,577) -- Student 2 Verification: Crohn's Disease (drugs) #47
• Hypothyroidism (n=15,243 n=3,049) -- Student 2 Verification: Hypothyroidism (drugs) #48
• Peanut Allergy (n=57) -- Student 2 Verification: Peanut Allergy (drugs) #53
• Steroid-Induced Osteonecrosis (n=39,099 n=7,820) -- Student 2 Verification: Steroid-Induced Osteonecrosis (drugs) #54
• Systemic Lupus Erythematosus (n=23,816 n=4,763) -- Student 2 Verification: Systemic Lupus Erythematosus (drugs) #55

[avanmatre]

@trinklek Can you look at this range? I chose partial/appropriate + clinically relevant because the dosages of dex/amphetamine in the source_name column don't correspond to the true dose of the med (row 31 would correspond to a 10mg dose, yes?). https://docs.google.com/spreadsheets/d/1C9WOk9jKgYh7R50Rb8Ck3pV0sMikrbNrJDmGBnytOCE/edit#gid=0&range=31:34

[trinklek]

@avanmatre I agree that row 31 is referring to 10mg of Adderall, but I don't think there is any disconcordance. Are you comparing columns D and G and thinking column D is referring to 2.5mg? I think column D is just collapsing each of the salt forms of amphetamine and dextroamphetamine for simplicity. It looks like a number of columns do this and I don't think it is wrong. If the collapsing of the salt forms in column D is what has you stuck, I think we should explore different references to see if this is an appropriate collapsing/categorization - I am leaning towards it is appropriate.

[avanmatre]

@trinklek I am comparing columns D and G - I understand that the salt forms are collapsed but row #31 for instance is showing 2.5/2.5 in column D which doesn't clearly correlate with a 10mg dose whereas column G correlates with a 10mg dose. Columns D and G don't EXACTLY match so I wasn't sure about indicating exactly vs partial match. I agree it is appropriate, just unsure to what end "exactly" needs to be followed.

[avanmatre]

@trinklek Compare my previous question to https://docs.google.com/spreadsheets/d/1C9WOk9jKgYh7R50Rb8Ck3pV0sMikrbNrJDmGBnytOCE/edit#gid=0&range=126:126
where column D adds to 5mg dose

[trinklek]

@avanmatre Great comparison! Row 31 and row 126 definitely are inconsistent in content for column D.

@callahantiff Do you only want the source string (column B) compared to the source name (column D), or do you also want standard name (column G) compared/evaluated?

[callahantiff]

@avanmatre Great comparison! Row 31 and row 126 definitely are inconsistent in content for column D.

@callahantiff Do you only want the source string (column B) compared to the source name (column D), or do you also want standard name (column G) compared/evaluated?

Hey @avanmatre and @trinklek! Sorry for being late to the party! To answer your question, the initial plan was only to have you verify the string in column B to the result returned in column D. The mapping between column D and column G is done by the OMOP community. I think your point is very valid and definitely worth capturing in the comments column now that you identified it!

With that said, our primary focus is on verifying the mappings between column B and column D. If you happen to notice something worth mentioning in the other columns, note, but don't spend too much time on it. Does that sound OK? What do you think about that plan @trinklek?

[trinklek]

@avanmatre and @callahantiff I completely agree with the plan to focus on Columns B and D.

[callahantiff]

@avanmatre - Just a reminder, once you have finished verifying each file, can you please do the following:

• Check the box that corresponds to the file above?
• Close the specific issue (hyperlinked above) and include a brief comment to confirm its completion? This is also a great time to add any additional information that you think is important for me to know or keep in mind as I move onto the next phase of the analysis.
• Add the date of completion to the Dropbox Paper table (here).

Thanks and great work so far! 💪

[callahantiff]

@avanmatre

UPDATE: counts below updated to reflect today's meeting.

Verification Assignments:

• ADHD (n=34,184 n=6,837) -- Student 2 Verification: ADHD (drugs) #41
• Appendicitis (n=15,304 n=3,061) -- Student 2 Verification: Appendicitis (Drugs) #45
• Crohn’s Disease (n=17,887 n=3,577) -- Student 2 Verification: Crohn's Disease (drugs) #47
• Hypothyroidism (n=15,243 n=3,049) -- Student 2 Verification: Hypothyroidism (drugs) #48
• Peanut Allergy (n=57) -- Student 2 Verification: Peanut Allergy (drugs) #53
• Steroid-Induced Osteonecrosis (n=39,099 n=7,820) -- Student 2 Verification: Steroid-Induced Osteonecrosis (drugs) #54
• Systemic Lupus Erythematosus (n=23,816 n=4,763) -- Student 2 Verification: Systemic Lupus Erythematosus (drugs) #55

[avanmatre]

Hi Tiffany, I hope this finds you doing well! I’m sorry I haven’t been able to complete my verification responsibilities. The beginning of residency has been more intense than I intended and I don’t know that I can fulfill these responsibilities. I’m really sorry that I misjudged my ability to commit to this project with all these new things in my life. Please let me know if there is anything I can do to ease the transition process. Amanda From: "Tiffany J. Callahan" <[email protected]> Reply-To: callahantiff/PheKnowVec <[email protected]> Date: Tuesday, June 11, 2019 at 1:52 AM To: callahantiff/PheKnowVec <[email protected]> Cc: "Van Matre, Amanda" <[email protected]>, Mention <[email protected]> Subject: Re: [callahantiff/PheKnowVec] Medication Source Code Verification: Student 2 (#42) @avanmatre<https://github.com/avanmatre> UPDATE: counts below updated to reflect today's meeting<#56 (comment)>. Verification Assignments: * ADHD (n=34,184 n=6,837) -- #41<#41> * Appendicitis (n=15,304 n=3,061) -- #45<#45> * Crohn’s Disease (n=17,887 n=3,577) -- #47<#47> * Hypothyroidism (n=15,243 n=3,049) -- #48<#48> * Peanut Allergy (n=57) -- #53<#53> * Steroid-Induced Osteonecrosis (n=39,099 n=7,820) -- #54<#54> * Systemic Lupus Erythematosus (n=23,816 n=4,763) -- #55<#55> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#42?email_source=notifications&email_token=AMEFL4XZP72GJYQTBJYEGV3PZ44SVA5CNFSM4HM7J7UKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXMAP5I#issuecomment-500697077>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMEFL4SK67GRHNDOWBXWE33PZ44SVANCNFSM4HM7J7UA>.

[callahantiff]

@avanmatre - Thanks so much for your email. I completely understand, your residency is WAY more important. Please don't worry about it. Are there hours that we still need to pay you for? Aside from that, there is nothing further needed at this time. Thanks for being willing to help me with this work. If you are still interested in submitting something for your conference, I'm still very open to discussing that. Just let me know.

Once you confirm whether or not we have outstanding hours to pay you for, I will close out all issues assigned to you for mapping.

[callahantiff]

@trinklek - I spoke with @mgkahn today regarding this. It was brought up that there is a chance that not having two reviewers could bite us with reviewers, especially since we will have two medical students reviewing the non-medication domains. Do you buy the following argument -- we need two reviewers for the non-medication domain because the two reviewers are students. We don't need two reviewers for the medication domain because the first reviewer is not a student, but a licensed pharmacist with lots of relevant experience. Do you think that argument would hold or, should we be safe and find a second reviewer?

[avanmatre]

Thank you, Tiffany, for your understanding and offering to include me in the future. There are still outstanding hours – how should I submit them? My information should be updated as graduated at this point, but let me know if there is a problem. Including my hours below for your convenience (but I’m happy to submit an official timesheet). Amanda 5/22 1845-1945 5/28 0800-1330, 1500-1800 5/29 0745-1245, 1530-1630 6/10 1900-2000 From: "Tiffany J. Callahan" <[email protected]> Reply-To: callahantiff/PheKnowVec <[email protected]> Date: Wednesday, June 26, 2019 at 3:38 PM To: callahantiff/PheKnowVec <[email protected]> Cc: "Van Matre, Amanda" <[email protected]>, Mention <[email protected]> Subject: Re: [callahantiff/PheKnowVec] Medication Source Code Verification: Student 2 (#42) @avanmatre<https://github.com/avanmatre> - Thanks so much for your email. I completely understand, your residency is WAY more important. Please don't worry about it. Are there hours that we still need to pay you for? Aside from that, there is nothing further needed at this time. Thanks for being willing to help me with this work. If you are still interested in submitting something for your conference, I'm still very open to discussing that. Just let me know. Once you confirm whether or not we have outstanding hours to pay you for, I will close out all issues assigned to you for mapping. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#42?email_source=notifications&email_token=AMEFL4XSDFKK3ITICCZQYXDP4PAPLA5CNFSM4HM7J7UKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYUTA5Y#issuecomment-506015863>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMEFL4XLP57C3CSZ5UQRS3DP4PAPLANCNFSM4HM7J7UA>.

[trinklek]

@trinklek - I spoke with @mgkahn today regarding this. It was brought up that there is a chance that not having two reviewers could bite us with reviewers, especially since we will have two medical students reviewing the non-medication domains. Do you buy the following argument -- we need two reviewers for the non-medication domain because the two reviewers are students. We don't need two reviewers for the medication domain because the first reviewer is not a student, but a licensed pharmacist with lots of relevant experience. Do you think that argument would hold or, should we be safe and find a second reviewer?

I do think having a second reviewer would be ideal. KD is absolutely brilliant, but there are many nuances to this. I also think the perception of a pharmacist is not as strong as other clinician types given the huge spectrum of types of pharmacists adn variable training (e.g., walgreens versus clinical), thus your reviewer may wonder if the pharmacist was really credible - I absolutely would at least. I am confident a second reviewer will not be 100% consistent with what KD found, so I wonder if you are OK with someone doing a 10-20% sample or if you want a second person to review everything. If you are OK reporting an imperfect, but still positive interrater agreement, then I would go with a second reviewer looking at 10-20%, but if you want to come up with final mapping, then I would have a second person review all of it and I serve as the mediator for any disagreements so there...I am not 100% sure who the second reviewer should/could be, but my new fellow who started 7/1 is a possibility. I think you need a pharmacist (not student) for this type of work and not just any pharmacist - I am happy to reach out to folks if you want, or discuss further. Ideally we can find a trainee for cheap or a practicing clinical pharmacist who is willing to do at no cost, but I do want to find someone that I know is credible.

[callahantiff]

@trinklek - I spoke with @mgkahn today regarding this. It was brought up that there is a chance that not having two reviewers could bite us with reviewers, especially since we will have two medical students reviewing the non-medication domains. Do you buy the following argument -- we need two reviewers for the non-medication domain because the two reviewers are students. We don't need two reviewers for the medication domain because the first reviewer is not a student, but a licensed pharmacist with lots of relevant experience. Do you think that argument would hold or, should we be safe and find a second reviewer?

I do think having a second reviewer would be ideal. KD is absolutely brilliant, but there are many nuances to this. I also think the perception of a pharmacist is not as strong as other clinician types given the huge spectrum of types of pharmacists adn variable training (e.g., walgreens versus clinical), thus your reviewer may wonder if the pharmacist was really credible - I absolutely would at least. I am confident a second reviewer will not be 100% consistent with what KD found, so I wonder if you are OK with someone doing a 10-20% sample or if you want a second person to review everything. If you are OK reporting an imperfect, but still positive interrater agreement, then I would go with a second reviewer looking at 10-20%, but if you want to come up with final mapping, then I would have a second person review all of it and I serve as the mediator for any disagreements so there...I am not 100% sure who the second reviewer should/could be, but my new fellow who started 7/1 is a possibility. I think you need a pharmacist (not student) for this type of work and not just any pharmacist - I am happy to reach out to folks if you want, or discuss further. Ideally we can find a trainee for cheap or a practicing clinical pharmacist who is willing to do at no cost, but I do want to find someone that I know is credible.

@trinklek - Sorry I somehow missed this! Thank you for the great information! OK, let's move forward with finding someone credible and trustworthy. Perhaps we can reassess whether they do 10-20% or all terms depending on who shows interest? We would also be able to pay them for their time, same rate as before.

[trinklek]

@trinklek - I spoke with @mgkahn today regarding this. It was brought up that there is a chance that not having two reviewers could bite us with reviewers, especially since we will have two medical students reviewing the non-medication domains. Do you buy the following argument -- we need two reviewers for the non-medication domain because the two reviewers are students. We don't need two reviewers for the medication domain because the first reviewer is not a student, but a licensed pharmacist with lots of relevant experience. Do you think that argument would hold or, should we be safe and find a second reviewer?

I do think having a second reviewer would be ideal. KD is absolutely brilliant, but there are many nuances to this. I also think the perception of a pharmacist is not as strong as other clinician types given the huge spectrum of types of pharmacists adn variable training (e.g., walgreens versus clinical), thus your reviewer may wonder if the pharmacist was really credible - I absolutely would at least. I am confident a second reviewer will not be 100% consistent with what KD found, so I wonder if you are OK with someone doing a 10-20% sample or if you want a second person to review everything. If you are OK reporting an imperfect, but still positive interrater agreement, then I would go with a second reviewer looking at 10-20%, but if you want to come up with final mapping, then I would have a second person review all of it and I serve as the mediator for any disagreements so there...I am not 100% sure who the second reviewer should/could be, but my new fellow who started 7/1 is a possibility. I think you need a pharmacist (not student) for this type of work and not just any pharmacist - I am happy to reach out to folks if you want, or discuss further. Ideally we can find a trainee for cheap or a practicing clinical pharmacist who is willing to do at no cost, but I do want to find someone that I know is credible.

@trinklek - Sorry I somehow missed this! Thank you for the great information! OK, let's move forward with finding someone credible and trustworthy. Perhaps we can reassess whether they do 10-20% or all terms depending on who shows interest? We would also be able to pay them for their time, same rate as before.

@callahantiff I had an idea today of someone today. For this person, if they review a 10-20% sample, will participation in publication be an option? Your answer will help me pitch it correctly.

[callahantiff]

@trinklek - I spoke with @mgkahn today regarding this. It was brought up that there is a chance that not having two reviewers could bite us with reviewers, especially since we will have two medical students reviewing the non-medication domains. Do you buy the following argument -- we need two reviewers for the non-medication domain because the two reviewers are students. We don't need two reviewers for the medication domain because the first reviewer is not a student, but a licensed pharmacist with lots of relevant experience. Do you think that argument would hold or, should we be safe and find a second reviewer?
I do think having a second reviewer would be ideal. KD is absolutely brilliant, but there are many nuances to this. I also think the perception of a pharmacist is not as strong as other clinician types given the huge spectrum of types of pharmacists adn variable training (e.g., walgreens versus clinical), thus your reviewer may wonder if the pharmacist was really credible - I absolutely would at least. I am confident a second reviewer will not be 100% consistent with what KD found, so I wonder if you are OK with someone doing a 10-20% sample or if you want a second person to review everything. If you are OK reporting an imperfect, but still positive interrater agreement, then I would go with a second reviewer looking at 10-20%, but if you want to come up with final mapping, then I would have a second person review all of it and I serve as the mediator for any disagreements so there...I am not 100% sure who the second reviewer should/could be, but my new fellow who started 7/1 is a possibility. I think you need a pharmacist (not student) for this type of work and not just any pharmacist - I am happy to reach out to folks if you want, or discuss further. Ideally we can find a trainee for cheap or a practicing clinical pharmacist who is willing to do at no cost, but I do want to find someone that I know is credible.
@trinklek - Sorry I somehow missed this! Thank you for the great information! OK, let's move forward with finding someone credible and trustworthy. Perhaps we can reassess whether they do 10-20% or all terms depending on who shows interest? We would also be able to pay them for their time, same rate as before.

@callahantiff I had an idea today of someone today. For this person, if they review a 10-20% sample, will participation in publication be an option? Your answer will help me pitch it correctly.

---

@trinklek - Sorry I missed this, apparently these types of messages were getting sent to spam. Absolutely, reviewing a sample of 10-20% totally constitutes participation in publications that use this work. 😄

[callahantiff]

Closing and archiving this.