Skip to content

Conversation

@florianl
Copy link
Contributor

@florianl florianl commented Jun 25, 2024

Reduce memory footprint by starting with a smaller buffer and adjust the buffer size accordingly after peeking at the upcoming netlink message.

// on current master
$ go test -memprofile mem.orig.out .
// on this PR
$ go test  -memprofile mem.new.out .
$ go tool pprof -base=mem.orig.out mem.new.out
(pprof) top
Showing nodes accounting for -1081.85MB, 41.71% of 2593.47MB total
Dropped 34 nodes (cum <= 12.97MB)
Showing top 10 nodes out of 25
      flat  flat%   sum%        cum   cum%
-1226.36MB 47.29% 47.29% -1074.85MB 41.44%  github.com/mdlayher/netlink.(*conn).Receive
   92.51MB  3.57% 43.72%   144.01MB  5.55%  github.com/mdlayher/socket.rwT[go.shape.struct { github.com/mdlayher/socket.n int; github.com/mdlayher/socket.oobn int; github.com/mdlayher/socket.recvflags int; github.com/mdlayher/socket.from golang.org/x/sys/unix.Sockaddr }]
   46.50MB  1.79% 41.93%    51.51MB  1.99%  golang.org/x/sys/unix.Recvmsg
     -19MB  0.73% 42.66%      -19MB  0.73%  github.com/mdlayher/socket.rwT[go.shape.int]
   10.50MB   0.4% 42.25%   154.51MB  5.96%  github.com/mdlayher/socket.(*Conn).Recvmsg
       8MB  0.31% 41.95%      -11MB  0.42%  github.com/mdlayher/socket.(*Conn).Sendmsg
    3.50MB  0.13% 41.81% -1071.35MB 41.31%  github.com/mdlayher/netlink.(*Conn).receive
    2.50MB 0.096% 41.71%    -7.50MB  0.29%  github.com/mdlayher/netlink.(*conn).Send
         0     0% 41.71%  -516.45MB 19.91%  github.com/mdlayher/netlink.(*Conn).Execute
         0     0% 41.71%  -553.40MB 21.34%  github.com/mdlayher/netlink.(*Conn).Receive

flag_trunc

florianl added a commit to florianl/go-diag that referenced this pull request Jul 20, 2024
Until mdlayher/netlink#215 is merged use a replace
statement to reduce the memory footprint.

Signed-off-by: Florian Lehner <[email protected]>
florianl added a commit to florianl/go-diag that referenced this pull request Jun 7, 2025
Until mdlayher/netlink#215 is merged use a replace
statement to reduce the memory footprint.

Signed-off-by: Florian Lehner <[email protected]>
Reduce memory footprint by starting with a smaller buffer and adjust the
buffer size accordingly after peeking at the upcoming netlink message.

Signed-off-by: Florian Lehner <[email protected]>
// Receive receives one or more Messages from netlink.
func (c *conn) Receive() ([]Message, error) {
b := make([]byte, os.Getpagesize())
for {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder whether the reason for the loop here was to make sure we are not falling behind the kernel in an async scenario. Perhaps @mdlayher wanted to avoid setting the buffer to the exact number of available bytes.

Is it possible for more bytes come in before the next recvmsg call ? I think that shouldn't happen but have you confirmed that @florianl ?

If it's indeed safe then it'd be nice to have this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants