2

I have a large slow HDD and a small fast SSD. This is about reads not RAID. My desktop grinds to a near-halt when switching back to Firefox or man pages after (re/un)-loading 12+ GiB of Linux kernel build trees and 39 GiB total of different LLMs on the SSD while I only have 31 GiB of RAM:

$ free -h total used free shared buff/cache available Mem: 31Gi 10Gi 2.4Gi 1.0Gi 19Gi 20Gi Swap: 0B 0B 0B $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 1.8T 0 disk ├─sda1 8:1 0 512M 0 part /boot/efi ├─sda2 8:2 0 1.7G 0 part /boot └─sda3 8:3 0 1.8T 0 part └─sda3_crypt 254:0 0 1.8T 0 crypt ├─vgubuntu-root 254:1 0 1.8T 0 lvm / └─vgubuntu-swap_1 254:2 0 1.9G 0 lvm nvme0n1 259:0 0 953.9G 0 disk └─nvme0n1p1 259:1 0 100G 0 part └─luks-... 254:3 0 100G 0 crypt /media/... $ sysctl vm.swappiness vm.swappiness = 60

The SSD is fast, so I'd rather Linux evict the SSD's page-cached files first. Its uncached read time is seconds anyway. What should stop is eviction of any file under /usr or /home. My man bash and dpkg -S bin/bash return instantly from the page cache, but uncached they take half a minute after exiting the LLMs. More severely, Firefox needs my ~/.mozilla folder for history and cache; with it uncached, waiting for the address bar to work takes minutes.

I am looking for an configuration option. systemd-run could set MemoryMax for ktorrent, but I frequently restart llama-server to switch between the ~6 GiB LLMs, and I don't want a separate daemon to keep the cgroup alive. The man and dpkg problems will be fixed when my / moves to the SSD once I sort out fscrypt fears; in the meantime, /usr on tmpfs would leave insufficient available RAM and overlayfs is too much complexity. The LLM workload could, but shouldn't, remount the SSD as a workaround. That leaves the niced kernel build workload still evicting my web browsing one's cache.

I looked in /sys/block but couldn't find the right config. Cgroups v2 has per-device options but only for parallel write workloads (io.max) not for controlling how sequential workloads affect the cache. A 2011 patch and a 2023 question don't see any userspace interface. Which setting can be used to force the SSD's page cache to be evicted before that of the HDD's?

Improve this question
2
  • It's unusual to boot off HDD when SSD is available. What's stopping you from relocating most essential boot, root, home files to SSD? For Firefox there are some scripts that relocate ~/.mozilla to tmpfs, but how about you just don't put it on HDD if that causes problems? Commented Apr 14 at 9:45
  • @frostschutz The blocker might be only my misreading of github.com/google/fscrypt/blob/… , so I might try soon. Moving / would solve half of the problem. The other half I know I can bind mount or symlink, but my browsing data might not be completely deletable on a SSD Commented Apr 14 at 18:23

1 Answer 1

Reset to default
3

Running the program through nocache might help with that. It doesn't work with all programs (due to having to hook into glibc internals) but it helps with some.

Alternatively use vmtouch to manually evict the previous model file after llama-server exits, before loading the new one.

Improve this answer
1
  • This worked! nocache reduced cachestats on the file from 100% to pages in cache: 72146/1201353 (6.0%), which was corroborated with free -h and htop. I thought it'd only work for the kernel build task, but it also worked for llama-server despite me enabling its mmap feature. I find this solution convenient as it's just one word to prefix to my commands. I'm accepting this answer assuming that no per-filesystem config feature exists yet Commented Apr 14 at 18:27

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.