Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a pre-test-report hook #261

Open
4 tasks
alysbrooks opened this issue Nov 18, 2021 · 1 comment
Open
4 tasks

Add a pre-test-report hook #261

alysbrooks opened this issue Nov 18, 2021 · 1 comment
Labels
feature New functionality first issue only These tickets are only for first-time contributors.

Comments

@alysbrooks
Copy link
Member

alysbrooks commented Nov 18, 2021

In #260, a user was trying to extend Kaocha to include details about any Postgres errors that happened during a test. They used pre-test to log the test start time and attempted to use post-test to add additional metadata to identify the end of test (allowing them to search the logs for events that happened between the start and end times of the test), but this wasn't working because the reporter would handle the error result before the post-test hook was called.

How this would work

This hook would happen immediately after the test runs and before the reporter runs, allowing for last-minute alterations to the testable map. I did some looking into wrap-run, and I don't think that hook covers this situation, but that should be verified before implementing this hook. In theory, if you're implementing your own reporter, it could run its own logic before actually doing any error reporting; however, this would only work for custom reporters, so I think this hook is useful.

For future contributors

Two design decisions:

  • Whether we should call this hook for tests that fail to load or are pending. I'm inclined to say yes because the reporter runs in those cases.
  • Whether we should call this hook for successful tests.

Actually adding this hook wouldn't be too tricky:

  • Register a new hook in hierarchy.clj
  • Call the hook in the testable/run-testable function.
  • Add the hook to the documentation.
  • Add some tests along the lines of our existing hook tests.

All that being said, I consider this a low-priority feature since you can often hack around it using an entirely separate custom reporter or overriding the default error reporter method.

@alysbrooks alysbrooks added first issue only These tickets are only for first-time contributors. feature New functionality labels Nov 18, 2021
@alysbrooks
Copy link
Member Author

alysbrooks commented Nov 18, 2021

Any thoughts on this? @plexus

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New functionality first issue only These tickets are only for first-time contributors.
Projects
Status: 🙈Out of Scope
Development

No branches or pull requests

1 participant