Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modularisation of FMTM Splitter #62

Open
spwoodcock opened this issue Oct 28, 2024 · 0 comments
Open

Modularisation of FMTM Splitter #62

spwoodcock opened this issue Oct 28, 2024 · 0 comments
Labels
effort:medium Likely a day or two enhancement New feature or request priority:high Should be addressed as a priority

Comments

@spwoodcock
Copy link
Member

Is your feature request related to a problem? Please describe.

  • Ivan actually did this a while ago, but we pulled out the changes he did in modules and dumped them in a single file for simplicity.
  • This was done to reduce the overhead of multiple SQL queries on the databases - we simply have one connection, one cursor, one query.
  • However it has come to light it might be best to modularise this again.

Describe the solution you'd like

  • The fmtm_algorithm.sql file should be divided into modules, described in How The FMTM Splitter Algorithm Works #61
  • From Python each module can be called via a new cursor.
  • The output from one cursor will be used as the input for the next SQL statement.

Additional considerations

  • The main advantage here is that we can add parameters to each stage, allowing us to tweak the output as needed.
  • It is quite clear that splitting is unique in each new area we are presented with - we need flexibility to generate the optimal split for a project.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort:medium Likely a day or two enhancement New feature or request priority:high Should be addressed as a priority
Projects
Development

No branches or pull requests

1 participant