Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

X710 10G interface produces "No buffer space available" error. #40

Open
1 of 3 tasks
jwbensley opened this issue Nov 13, 2017 · 1 comment
Open
1 of 3 tasks

X710 10G interface produces "No buffer space available" error. #40

jwbensley opened this issue Nov 13, 2017 · 1 comment
Labels
minor bug

Comments

@jwbensley
Copy link
Owner

ISSUE TYPE

  • Bug Report
  • Enhancement/Feature Request
  • Question

VERSION DETAILS

$ uname -a
Linux ucpe_002 4.13.12-1.el7.elrepo.x86_64 #1 SMP Wed Nov 8 13:01:08 EST 2017 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/redhat-release || lsb_release -a || cat /etc/issue
CentOS Linux release 7.4.1708 (Core)

$ lspci -nn | grep -i eth
02:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
02:00.1 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
06:00.0 Ethernet controller [0200]: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe [14e4:1657] (rev 01)
06:00.1 Ethernet controller [0200]: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe [14e4:1657] (rev 01)
06:00.2 Ethernet controller [0200]: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe [14e4:1657] (rev 01)
06:00.3 Ethernet controller [0200]: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe [14e4:1657] (rev 01)
09:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 01)
09:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 01)

$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens2f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq portid 1402ec6d9dd0 state UP mode DEFAULT qlen 1000
    link/ether 14:02:ec:6d:9d:d0 brd ff:ff:ff:ff:ff:ff
3: ens3f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:f8 brd ff:ff:ff:ff:ff:ff
4: ens3f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:f9 brd ff:ff:ff:ff:ff:ff
5: ens2f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq portid 1402ec6d9dd8 state UP mode DEFAULT qlen 10000
    link/ether 14:02:ec:6d:9d:d8 brd ff:ff:ff:ff:ff:ff
6: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 94:18:82:ab:ae:f3 brd ff:ff:ff:ff:ff:ff
7: ens3f2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:fa brd ff:ff:ff:ff:ff:ff
8: ens3f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:fb brd ff:ff:ff:ff:ff:ff
9: eno2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
    link/ether 94:18:82:ab:ae:f4 brd ff:ff:ff:ff:ff:ff
10: ens3f3.501@ens3f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br501 state UP mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:fb brd ff:ff:ff:ff:ff:ff
11: br501: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:fb brd ff:ff:ff:ff:ff:ff
12: ens3f3.502@ens3f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br502 state UP mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:fb brd ff:ff:ff:ff:ff:ff
13: br502: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:fb brd ff:ff:ff:ff:ff:ff
14: br0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
15: br1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
16: br2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1000
    link/ether 02:84:2b:68:80:af brd ff:ff:ff:ff:ff:ff
17: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT qlen 1000
    link/ether 52:54:00:41:c6:5e brd ff:ff:ff:ff:ff:ff
18: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 1000
    link/ether 52:54:00:41:c6:5e brd ff:ff:ff:ff:ff:ff

$ ./etherate -V
Etherate version 1.15 2017-10

$ sudo ./etherate -l
Device lo with address 00:00:00:00:00:00, has interface index 1
Device ens2f0 with address 14:02:ec:6d:9d:d0, has interface index 2
Device ens3f0 with address 3c:a8:2a:e7:05:f8, has interface index 3
Device ens3f1 with address 3c:a8:2a:e7:05:f9, has interface index 4
Device ens2f1 with address 14:02:ec:6d:9d:d8, has interface index 5
Device eno1 with address 94:18:82:ab:ae:f3, has interface index 6
Device ens3f2 with address 3c:a8:2a:e7:05:fa, has interface index 7
Device ens3f3 with address 3c:a8:2a:e7:05:fb, has interface index 8
Device eno2 with address 94:18:82:ab:ae:f4, has interface index 9
Device ens3f3.501 with address 3c:a8:2a:e7:05:fb, has interface index 10
Device br501 with address 3c:a8:2a:e7:05:fb, has interface index 11
Device ens3f3.502 with address 3c:a8:2a:e7:05:fb, has interface index 12
Device br502 with address 3c:a8:2a:e7:05:fb, has interface index 13
Device br0 with address 00:00:00:00:00:00, has interface index 14
Device br1 with address 00:00:00:00:00:00, has interface index 15
Device br2 with address 02:84:2b:68:80:af, has interface index 16
Device virbr0 with address 52:54:00:41:c6:5e, has interface index 17
Device virbr0-nic with address 52:54:00:41:c6:5e, has interface index 18

SUMMARY
Only on the two 10G interfaces is this issue experianced, not on the 1G interfaces. Does increasing the socket buffer help?

STEPS TO REPRODUCE
This is happeneing at all packet sizes, 256 is not specific to this issue:

$ sudo ./etherate -i ens2f1 -g -G -f 256
Running in TX mode
Using device ens2f1 with address 14:02:ec:6d:9d:d8, interface index 5
Entering promiscuous mode
Source MAC 00:00:5e:00:00:01
Destination MAC 00:00:5e:00:00:02
Sending gratuitous broadcasts...
Done.
Frame size is 270 bytes
Starting test on Mon Nov 13 17:03:13 2017

Seconds         Mbps Tx         MBs Tx          FrmTx/s         Frames Tx
1               914.41          109             423336          423336
2               1174.33         248             543672          967008
3               1174.70         389             543841          1510849
4               1174.62         529             543805          2054654
Speed test Tx error : No buffer space available
Ending test on Mon Nov 13 17:03:17 2017


Leaving promiscuous mode

This issue doesn't occur using the 1G interface, which was tested using various packet sizes. However it does seem to be happening on the 10G interfaces at any packet size I try.
@jwbensley
Copy link
Owner Author

Socket is hitting ENOBUFS, need to ignore this errno rather than dying. stall and carry on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
minor bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jwbensley