2

I have an application that sends lot of traffic over an UDP socket, every packet is sent on 2 interfaces: enp2s0 (1Gbit ethernet device) and enx00808a8eba78 (100Mbit usb-ethernet device).
The maximum socket send buffer is the default (212992 bytes), and it is full most of the time when the traffic is running:

root@punk:~# netstat -a Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State udp 0 211968 0.0.0.0:x11-2 0.0.0.0:*

Data in qdisc queue of the two interfaces is about 40k:

root@punk:~# tc -s qdisc show dev enp2s0 qdisc pfifo_fast 0: root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1 Sent 1697909529 bytes 1136534 pkt (dropped 0, overlimits 0 requeues 12) backlog 0b 0p requeues 12 root@punk:~# tc -s qdisc show dev enx00808a8eba78 qdisc pfifo_fast 0: root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1 Sent 1675952337 bytes 1121840 pkt (dropped 0, overlimits 0 requeues 55) backlog 43326b 29p requeues 55

Since 200k of data is pending in the socket but only 40k is queued in the second qdisc, I assume that the remaining 160k are pending inside the slow interface driver (enx00808a8eba78).

Is there a way to check how many packets (or data) is pending for transmission in a USB device or, more generically, in a network device?
Something like the number of DMA buffers ready for TX but not sent yet.

Improve this question
1
  • You would be far better off without the USB ethernet. Commented Jan 28, 2019 at 17:54

2 Answers 2

Reset to default
3

Seems there isn't a way to retrieve the device queue length from userspace.

BTW, some details if someone is interested:

usbnet devices keep track of the queued TX packets using the field txq.qlen of the struct usbnet. Maximum TX queue length is defined by field tx_qlen of struct usbnet.

In my example I have 60 (tx_qlen) packets queued in the USB driver and (more or less) 30 packets in the qdisc, each one carrying 1500 bytes of data. Since socket buffer is calculated considering the skb->truesize (i.e. skb data + skb structure size), each packet is 2.3k:

2.3k * (60 + 30) ~= 200k

This confirm that 138k of the socket buffer are consumed by packets queued in the network driver, while 69k of the socket buffer are in the qdisc queue: there aren't other packets queued somewhere else in the kernel.

Improve this answer
2

I know I am late to the party but I just submitted BQL support for the usbnet drivers, see here. I hope it gets accepted soon :)

Apart from reducing the queue size whenever possible, BQL also allows to check for inflight packets. This can be done with:

cat /sys/class/net/YOURINTERFACE/queues/tx-0/byte_queue_limits/inflight

There are also some other interesting things there like the limit, which sets the queue limit. With that buffer bloat can be reduced to a minimum. However, as described in my patch, TCP is still an issue as it produces packets up to 64K in size due to offloading. Disabling it did not work in my testing.

Cheers!

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.