Poll netlink socket using Erlang port

[msantos]

Replace the now broken (after R15) use of gen_udp for socket notification
with open_port/2.

All the tests passed for me except test_rt_delroute/1 but that seems to be a problem with netlink:nl_rt_enc/1 rather than with the socket.

[RoadRunnr]

Sorry for the late response, I simply didn't get the notification...

One problem with the use of gen_udp was, that it could cause the kernel internal socket buffers to overflow. We have added some code recently to counter that.

For the port approach, that raises the question if the port driver can drain the kernel input queue fast enough?
./erts/emulator/sys/unix/sys.c seems to read data in 64kbyte chunks, but it would be interesting to know how often
it will be called.

I'll try to run this on our test bed where it will get some load to see if it can keep up.

[msantos]

On Fri, Jun 15, 2012 at 01:30:41AM -0700, Andreas Schultz wrote:

Sorry for the late response, I simply didn't get the notification...

Hey Andreas, no prob, thanks for considering this patch!

One problem with the use of gen_udp was, that it could cause the kernel internal socket buffers to overflow. We have added some code recently to counter that.

Really?! This is the OS buffers?

For the port approach, that raises the question if the port driver can drain the kernel input queue fast enough?
./erts/emulator/sys/unix/sys.c seems to read data in 64kbyte chunks, but it would be interesting to know how often
it will be called.

Curious to see the results too. The other problem is there is no flow
control. It's on my todo list to see if it can be added that to the
ports interface.

I'll try to run this on our test bed where it will get some load to see if it can keep up.

Sounds great!

It's been a while since I looked at gen_netlink, but: the
routing/conntrack interfaces are really cool. I'm interested in talking
to other parts of the kernel too (for now, the audit system and the
openvswitch data path). What do you think of splitting out the
routing/conntrack into separate modules? Then gen_netlink could contain
just the generic netlink support.

Not sure if that makes sense for you and I'm happy to keep a fork, just
thought I'd ask.

[RoadRunnr]

On Fri, Jun 15, 2012 at 2:21 PM, Michael Santos
[email protected]
wrote:

On Fri, Jun 15, 2012 at 01:30:41AM -0700, Andreas Schultz wrote:

Sorry for the late response, I simply didn't get the notification...

Hey Andreas, no prob, thanks for considering this patch!

One problem with the use of gen_udp was, that it could cause the kernel internal socket buffers to overflow. We have added some code recently to counter that.

Really?! This is the OS buffers?

Yes, the socket buffer to be precise. I have some code to resize the
buffers even beyond the normal buffer limit.
If the buffer is exhausted, netlink will generate a enobufs error on
the socket and skip messages until enough
space becomes available again. The netlink sequence numbers will also
increase for the missing messages.

For the port approach, that raises the question if the port driver can drain the kernel input queue fast enough?
./erts/emulator/sys/unix/sys.c seems to read data in 64kbyte chunks, but it would be interesting to know how often
it will be called.

Curious to see the results too. The other problem is there is no flow
control. It's on my todo list to see if it can be added that to the
ports interface.

We have worked on extending the gen_socket module with that. The idea
is to have something
like gen_tcp/gen_udp that works with all kinds of sockets.

I'll try to run this on our test bed where it will get some load to see if it can keep up.

Sounds great!

It's been a while since I looked at gen_netlink, but: the
routing/conntrack interfaces are really cool. I'm interested in talking
to other parts of the kernel too (for now, the audit system and the
openvswitch data path). What do you think of splitting out the
routing/conntrack into separate modules? Then gen_netlink could contain
just the generic netlink support.

There is not much generic code in there, most of the stuff is concerned with
packet decoding/encoding. I was thinking about removing all 20 lines of socket
code completely from gen_netlink, but moving it into an extra module could
be a better idea.

Not sure if that makes sense for you and I'm happy to keep a fork, just
thought I'd ask.

I was thinking about talking to OpenVSwitch for a while as well, but
for now i'll talk to it through OpenFlow ;-)

---

Reply to this email directly or view it on GitHub:
#1 (comment)