add lots of debug logging to investigate flakiness #421
Conversation
Codecov Report
@@ Coverage Diff @@
## master #421 +/- ##
==========================================
- Coverage 56.58% 56.56% -0.02%
==========================================
Files 111 111
Lines 8280 8281 +1
==========================================
- Hits 4685 4684 -1
- Misses 3142 3144 +2
Partials 453 453
Continue to review full report at Codecov.
|
| "net" | ||
| "sync" | ||
| "time" | ||
|
|
There was a problem hiding this comment.
Could you keep this newline? we like to keep our imports separated by stdlib / 3rdparty / scoot stuff
| "net" | ||
| "sync" | ||
| "time" | ||
|
|
||
| "github.com/golang/protobuf/proto" | ||
| log "github.com/sirupsen/logrus" | ||
| remoteexecution "github.com/twitter/scoot/bazel/remoteexecution" |
There was a problem hiding this comment.
related to above comment, as long as you're in here, could you move this down to the scoot stuff below? thanks!
|
Don't logs already have timestamps in them from logrus, or do these not appear when running locally? If that's the case can we fix that at the logging layer instead of explicitly logging timestamps only in a single function of a single module? |
|
Closing to cleanup PRs, open & update per comments if you'd like to revisit |
I'm not sure if this change is something Scoot would want to do differently.
Problem
When running a local cluster with
go run ./binaries/setup-cloud-scoot/main.go --strategy local.localto run pants against, we uncovered some flakiness, which was resolved in pants with pantsbuild/pants#7422. We investigated that by pairing timestamped logs from pants (which are being done in pantsbuild/pants#7536) with timestamped logs from scoot when uploading files (from this change).Explain the context and why you're making that change. What is
the problem you're trying to solve? In some cases there is not a
problem and this can be thought of being the motivation for your change.
Solution
Describe the modifications you've done.
Result
CAS writes via Scoot and any flakiness around them are easier to debug!
What will change as a result of your pull request? Note that sometimes
this section is unnecessary because it is self-explanatory based on
the solution.