Skip to content

Conversation

douten
Copy link

@douten douten commented Jul 13, 2025

Resolves #4987

Reviewer Notes

  • please test with various data, I've only used the seed data locally
  • if commits need to be squashed, lmk. I believe I need to push w/ force to do that, so I'm holding if off for now.

Description

  • new path to get bulk annual reports
    • /reports/annual_reports/range/{year_start}/{year_end}.csv
      ex: ../reports/annual_reports/range/2023/2024.csv
  • years in url must be 4 digits or Routing Error will be raised
  • the latest year of earliest_reporting_year and year_start will be used
  • the earliest year of the current year and year_end will be used
  • will use existing AnnualReport. If latest data is needed, the individual year's annual report will have to be recalculated

Type of change

  • New feature (non-breaking change which adds functionality)
  • Documentation update

How Has This Been Tested?

  • tested on page
  • made a small rspec test

Screenshots

Screen.Recording.2025-07-12.at.8.32.24.PM.mov

Copy link
Collaborator

@cielf cielf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @douten!
Thank you for this!

When doing the functional QA on this, I ran into an unanticipated situation.

On testing against production data, it looks like some of the reports have not been recalculated since new fields were added / changed. This results in the columns not lining up.

I want to discuss with the senior team and/or the stakeholder liaison what we should do in this situation, specifically whether we should just recalculate those reports -- which are currently only manually recalculated -- when we do the export.

So - this isn't a problem with what you've done, but we're going to have to hold moving forward on this PR while we decide what to do with this. The next meeting of the senior team is Sunday, so you won't have an answer until next week at least.

@douten
Copy link
Author

douten commented Jul 14, 2025

Thanks for the info @cielf
I'll check back when there's an update then.

Comment on lines +42 to +47
@reports.each do |report|
report_data = report.all_reports
year = report['year']
values = report_data.flat_map { |r| r['entries'].values }
csv_data << [year] + values
end
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi! Different reports end up having different columns. Here, instead of plucking out the values in the specific order, can you instead do a pass through all the reports and get all the headers and then build out a hash or something so that we can handle blank values and then go through and output the CSV based on that?

Something like this (note this is COMPLETELY UNTESTED SUGGESTION) :)

Suggested change
@reports.each do |report|
report_data = report.all_reports
year = report['year']
values = report_data.flat_map { |r| r['entries'].values }
csv_data << [year] + values
end
@reports.each do |report|
report_data = report.all_reports
year = report['year']
values = report_data.flat_map { |r|
headers.map { |field| r['entries'][field] }
}
csv_data << [year] + values
end

That way if year 1 has some headers that year 2 doesn't have (or vice-versa) then we will get all of the entries.

@cielf
Copy link
Collaborator

cielf commented Aug 12, 2025

Thanks for the info @cielf I'll check back when there's an update then.

I'm sorry -- I thought that we'd included the information that yes -- we can recalculate the reports. They do, however, still end up having different columns.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Annual survey export - make it work across years
3 participants