4

I have that server running kernel 4.18, where some memory seems to have gone missing in action - with the common culprits (kernel slab memory, page cache) already ruled out.

Some 6 GB are being used (MemTotal - MemAvailable), but if I add up what is used by processes, kernel slab and shmem, that barely gets me to 1 GB. That is right after a cold boot. Where might the other 5 have gone?

Strangest thing is, I have an identical server running the exact same applications, and MemAvailable is, in fact, 5 GB higher there.

# ps fauxw | sed 1d | awk '{ x += $6 } END { print x }' 757368 # cat /proc/meminfo MemTotal: 32826040 kB MemFree: 26493788 kB MemAvailable: 26322932 kB Buffers: 11948 kB Cached: 72584 kB SwapCached: 0 kB Active: 247756 kB Inactive: 41348 kB Active(anon): 204952 kB Inactive(anon): 1012 kB Active(file): 42804 kB Inactive(file): 40336 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 4 kB Writeback: 0 kB AnonPages: 204632 kB Mapped: 33052 kB Shmem: 1396 kB Slab: 219376 kB SReclaimable: 30744 kB SUnreclaim: 188632 kB KernelStack: 18960 kB PageTables: 12548 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 16413020 kB Committed_AS: 4565744 kB VmallocTotal: 34359738367 kB VmallocUsed: 0 kB VmallocChunk: 0 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 186396 kB DirectMap2M: 9152512 kB DirectMap1G: 26214400 kB 

1 Answer 1

2

Answering my own question:

I eventually found that the memory usage was linked to a different setting for the network interfaces' ring size (ethtool -G). It was increased to 4096 for RX and TX on that host and, with six 10G NICs and many queues each (56 vcores), that ended up using gigabytes of memory.

Still an odd thing though, that that use of memory does not seem to be visible anywhere in the /proc/ information.

1
  • How did you find the root cause? Commented Mar 1, 2024 at 2:09

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.