Sporadic test failures in BPD tests

[sampsyo]

Travis seems to fail surprisingly often in the BPD tests with this error:

[...]
  File "/home/travis/build/beetbox/beets/beets/util/bluelet.py", line 404, in __init__
    self.sock.bind((host, port))
OSError: [Errno 98] Address already in use

Here's an example job, for instance.

The problem is that these tests, since #3190, use a socket to set up communication between a server and a client. But if two of these run at the same time, then the server runs into an error when trying to listen on that socket. I tried to mitigate this in a hacky way in 99778d9 by randomizing the port over 10,000 options, but failures of this kind are still happening pretty often on Travis.

A potential solution would be to catch this error and re-randomize a few times before giving up.

[arcresu]

It seems really weird that there would still be collisions with the port randomised like that. It's not like any of the tests lasts for very long either, and the server should be killed reliably in the finally clause. Maybe there's something deterministic about the RNG in the CI environment? I'll take a look but I agree that catching the error is the most reliable way to deal with this flakiness.

[zsinskri]

I have not yet looked at this thoroughly, so do not believe anything I say!

According to this StackOverflow question we can just use port 0 to let the OS choose an available port. The assigned port can then be read as sock.getsockname()[1].

To get that port number from the server process to the main process in order to create the clients, we could use a multiprocessing.Value. Though to write to that Value after binding the socket and before waiting on it we probably have to mock something.

[sampsyo]

Wow, that’s very cool! I didn’t know about that Unix feature. It’s at least worth giving it a try!