Lines are getting skipped when reading multiple lines within the same millisecond

[colblitz]

I'm using logstation on a file that I'm printing to from a Python script, and I'm noticing lines being skipped when the Python print call contains newlines (\n). Was able to reproduce with a small example script, the only config change was specifying the file.

with open('test.log', 'a') as f:
    f.write("test line\n")
    f.write("1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n")

When watching test.log I get

test line
1
2
4
6
8
10

[jdrews]

Hmm very interesting! Thanks for reporting it. I'll see if I can recreate it.

Is this on Ubuntu? That magenta terminal background looks familiar.

[colblitz]

yup you got it, ubuntu 22.04

[jdrews]

Quick update: I have been able to recreate this. Surprisingly, if I put in breakpoints at key points in the app, it behaves as expected. This must be a speed/performance problem. I'll keep digging.

I also thought switching to a polling tailing method (e.g. check the file every 500ms for new log lines) instead of a filesystem alert tailing method (e.g. wait for the filesystem to tell me the log file was written to) would fix it. But it didn't.

More to come.

[jdrews]

Seems to be only happening when lines are written out all at the same time or under a millisecond. I still want to fix this, but it shouldn't affect most people.