updated sdic to not print anything and use the logging module, ran black

[dimitrylukashov]

Jira ticket: https://percolate.atlassian.net/browse/AI-432

[dimitrylukashov]

@lra how do we remove flake8 from here?

[lra]

We should not switch from printing to logging. We should add an option on the commandline to switch and change the default, the default should always be interactive mode (launched by user) so that it's easy for users to try things out.

Also, I'm not sure syslog is the right recipient for it, as it's usually a multiline output. Why not letting the user what to do with the output? With stdout, he can run it, email the output, syslog it, ... We could just document sample use case if you think users don't know this stuff.

This would be better than forcing all use cases to use syslog.

[lra]

Can you use a public image? We don't want to use private stuff in our public repo to prevent leaking (anyone can fork, open a PR, and dump some env vars)

[lra]

Papertrail is really specific to us, can you talk about syslog in general instead?

[lra]

When the PR is ready to be merged, I can set github to replace the flake8 test by black.

[dimitrylukashov]

@lra the primary use case of this tool is to check the integrity of bad data in a production database. Developers should not be manually using this tool at all, and that use case is not described in the README. The tool is also not usable for individual developer's use as you cannot isolate to a specific check, you have to run all the jobs per server at a time. It would be trivial, of course, to modify this, but developer's should not be encouraged to manually run scripts until we have kubernetes jobs or some other equivalent.

As is, sdic is neither usable or scalable. Emailing output from cron is also a bad pattern, insecure, and a ridiculous way of distributing results to individual teams. By setting up papertrail alerts each team can keep track of their own checkers.

I agree that multi-line output is a challenge but I would rather fitz with the format than deal with email. Feel free to propose any other alerting solution.

[lra]

Developers should not be manually using this tool at all, and that use case is not described in the README. The tool is also not usable for individual developer's use as you cannot isolate to a specific check, you have to run all the jobs per server at a time. It would be trivial, of course, to modify this, but developer's should not be encouraged to manually run scripts until we have kubernetes jobs or some other equivalent.

This is not only about devs, this is about anyone using the tool. When you write tools, you should always focus on making it easy to use with sane defaults. By keeping sdic easy to run on the command line, you make it easy to run on every env out there (be it a cron, a container or a k8s job) and easy to test and to learn.

When a tool requires outside state (e.g. syslog output), you lose users down the line as the tool is asking users to change their environment to be able to be used. Think unix commands.

As is, sdic is neither usable or scalable. Emailing output from cron is also a bad pattern, insecure, and a ridiculous way of distributing results to individual teams. By setting up papertrail alerts each team can keep track of their own checkers.

sdic is using stdout, which is the standard in unix: it can be used as is on the command line, or piped to a cron, to an email, to syslog, or even to pagerduty. By hardcoding the output to syslog, you rule out all the other possibilities.

I agree that multi-line output is a challenge but I would rather fitz with the format than deal with email. Feel free to propose any other alerting solution.

Whatever solution fits you, you can do everything by piping an stdout output.