Parsing Senate Estimates?

[vteague]

Hey @KipCrossing did you ever look at trying to parse the hearings schedule for Senate Estimates? One crucial part of RightToAsk is to help citizens identify a good place to pose their question, and I think Senate Estimates would be a good starting point. So for example, if I have a question about election conduct and I'd like to suggest that a Senator put it to the AEC, it would help me to know when the AEC is appearing before Senate Estimates.

The trouble is (like many aph websites) the info is there but it is organised in a top-down way which makes it hard to answer questions of the form "when is [some agency] appearing at Estimates next?"

In some ways, it's quite well organised, with a sensible division into 8 sub-categories, etc - see here

OTOH, there's some serious parsing work to be done. What do you think?

[king-millez]

Have you looked through the consolidated program document? The PDF is easy to search and has the exact schedule for every department:

[vteague]

Thanks @king-millez ! I now realise I got you and Kip confused. Would you say parsing that pdf is an easy extension of the bill-reading code you wrote for Flux? Any advice most welcome.

[king-millez]

We just built this, which can scrape the upcoming additional hearings. Next time Senate Estimates is rolling around we can update it to get all the hearings, PDF scraping can be a bit inconsistent so this is probably the easiest way to do it at the moment...

[KipCrossing]

RE: the pdf part. There are packages that help with that. eg https://pypi.org/project/python-pdf/

It would just require some RnD to figure how how to make it semi-robust but it can be done!