Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

first stab at a data backfill for StateFileArchivedIntake #5305

Draft
wants to merge 5 commits into
base: FYST-1466-prior-year-access-mvp
Choose a base branch
from

Conversation

jnf
Copy link
Contributor

@jnf jnf commented Dec 31, 2024

Link to pivotal/JIRA issue

Is PM acceptance required? (delete one)

  • Yes - don't merge until JIRA issue is accepted!
  • No - merge after code review approval

Reminder: merge main into this branch and get green tests before merging to main

What was done?

  • Explain the implementation goals being solved or the feature with the reviewer in mind
  • Mention any relevant issues or insights to be shared with the reviewer.
  • Alternatives considered

How to test?

  • Describe the testing approach taken to verify the changes, including:
    • Unit/integration/manual tests
    • Test data used
  • Specify any relevant testing environments used (e.g., development, staging, demo, Heroku).
  • Risk Assessment
    • Risks or side effects associated with the changes and how they were mitigated.
    • Highlight areas that may need extra attention during code review or testing.
    • Paste SQL queries or output where relevant

Screenshots (for visual changes)

  • Before
  • After

@jnf jnf added the wip denotes a work in progress that isn't ready for formal review label Dec 31, 2024
Copy link

Heroku app: https://gyr-review-app-5305-3f92ceae63ac.herokuapp.com/
View logs: heroku logs --app gyr-review-app-5305 (optionally add --tail)

jnf added 4 commits December 31, 2024 10:48
…off date configurable; this will help in testing where we won't reliably have accepted returns with transition dates before the 'real' ty23 cutoff of may-ish 2024
@@ -0,0 +1,74 @@
module StateFile
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at a high level, i think it would be helpful to write tests for this file.

next
end
Rails.logger.info("Archived #{archived_ids.count} #{data_source} intakes: [#{archived_ids.join(', ')}]")
@current_batch = nil # reset the batch
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is the @current_batch set to nil for this method?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, i think it is to exit the while loop in the rake task.

next
end
Rails.logger.info("Archived #{archived_ids.count} #{data_source} intakes: [#{archived_ids.join(', ')}]")
@current_batch = nil # reset the batch
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, i think it is to exit the while loop in the rake task.

archive = StateFileArchivedIntake.new(record.without('source_intake_id'))
archive.submission_pdf.attach(data_source.find(record['source_intake_id']).submission_pdf.blob)
archive.save!
archived_ids << record['source_intake_id']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 i am assuming that we don't want to archive and delete in the same rake task.

maybe it would be helpful to have a method that after the archive_batch is run, we verify that the archivable intakes now have an archivable record.

i don't if that makes sense or is easy.

Copy link
Contributor

@anisharamnani anisharamnani Jan 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also how were you thinking of recording the intake IDs for deleting? or were you thinking of running the query again and then deleting them?

@anisharamnani
Copy link
Contributor

also, the annotate error should be resolved if you merge in FYST-1466-prior-year-access-mvp 🎉

#{tax_year} AS tax_year, '#{state_code}' AS state_code,
email_address, hashed_ssn, id AS source_intake_id
FROM
state_file_az_intakes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 just realized this query is just for AZ. since this is a WIP PR, i'm sure you were gonna change this, but just pointing out for other reviewers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wip denotes a work in progress that isn't ready for formal review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants