Skip to main content
added 5296 characters in body
Source Link
teeeeee
  • 305
  • 1
  • 15

EDIT

After the answer from @grawity, I did a simpler check:

  • I removed the idle timeout (by setting x-systemd.idle-timeout=0)
  • I removed explicit bind-mounts for these drives from the docker run command

In this situation, I found the following:

  1. Immediately after boot, an automount is set up, but nothing triggered it yet, as expected:
root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.automount -b Jun 06 12:22:20 docker-debian systemd[1]: Set up automount mnt-nfs-SSD_240GB-backups-TIG_backups.automount. root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.mount -b -- No entries -- 
  1. I start a simple container up, with no explicit bind mounts for those drives (only the hostfs structure) :
docker run -d \ --name telegraf_container \ --mount type=bind,source=/,destination=/hostfs \ -e HOST_MOUNT_PREFIX=/hostfs \ -e HOST_PROC=/hostfs/proc \ telegraf:latest 

This still does not trigger any automounts on the host.

  1. Now I manually trigger an automount on the host by accessing the drive:
ls /mnt/nfs/SSD_240GB/backups/TIG_backups/ 

The automount is triggered and mounts the drive successfully:

root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.automount -b Jun 06 12:22:20 docker-debian systemd[1]: Set up automount mnt-nfs-SSD_240GB-backups-TIG_backups.automount. Jun 06 12:35:20 docker-debian systemd[1]: mnt-nfs-SSD_240GB-backups-TIG_backups.automount: Got automount request for /mnt/nfs/SSD_240GB/backups/TIG_backups, triggered by 936 (ls) root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.mount -b Jun 06 12:35:21 docker-debian systemd[1]: Mounting mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups... Jun 06 12:35:21 docker-debian systemd[1]: Mounted mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups. 

Interestingly, the mounted drive now automatically appears inside the container (even though no bind-mounts have been used), but it appears under /hostfs instead:

monitoring@docker-debian:~$ docker exec -it telegraf_container df Filesystem 1K-blocks Used Available Use% Mounted on overlay 15421320 4686888 9929264 33% / tmpfs 65536 0 65536 0% /dev shm 65536 0 65536 0% /dev/shm /dev/sda1 15421320 4686888 9929264 33% /hostfs udev 983908 0 983908 0% /hostfs/dev tmpfs 1007084 0 1007084 0% /hostfs/dev/shm tmpfs 201420 656 200764 1% /hostfs/run tmpfs 5120 0 5120 0% /hostfs/run/lock tmpfs 201416 0 201416 0% /hostfs/run/user/1001 tmpfs 1007084 0 1007084 0% /proc/acpi tmpfs 1007084 0 1007084 0% /sys/firmware 192.168.0.67:/mnt/SSD_240GB/backups/TIG_backups 16337920 5799936 9682944 38% /hostfs/mnt/nfs/SSD_240GB/backups/TIG_backups 

If I unmount the drive directly on the host (using umount), then it disappears from the container again.

  1. I repeated this but instead using an idle timeout of 2mins now. What I found was that having the docker container running prevents the autounmount after 2 mins from happening (even though the container does not explicitly bind-mount in the drive, but instead appears automatically in the container under /hostfs).

If I stop and remove the container, then the idle timeout unmounts the drive after the 2mins:

root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.mount -b Jun 06 12:49:40 docker-debian systemd[1]: Mounting mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups... Jun 06 12:49:41 docker-debian systemd[1]: Mounted mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups. Jun 06 13:10:28 docker-debian systemd[1]: Unmounting mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups... Jun 06 13:10:28 docker-debian systemd[1]: mnt-nfs-SSD_240GB-backups-TIG_backups.mount: Deactivated successfully. Jun 06 13:10:28 docker-debian systemd[1]: Unmounted mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups. 

This makes me think a couple of things:

  • If I want to use telegraf to monitor drives that are mounted on the host, I don't need to bind mount them in explicitly, because they are present already due to the /hostfs bind-mount.
  • I should never see what I was originally expecting - namely, a drive automatically unmounting due to the idle timeout, and then the container triggering a remount. Because I observed above that once a drive has been mounted in (in my case at /hostfs), the container actually prevents it from ever being auto-unmounted.

EDIT

After the answer from @grawity, I did a simpler check:

  • I removed the idle timeout (by setting x-systemd.idle-timeout=0)
  • I removed explicit bind-mounts for these drives from the docker run command

In this situation, I found the following:

  1. Immediately after boot, an automount is set up, but nothing triggered it yet, as expected:
root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.automount -b Jun 06 12:22:20 docker-debian systemd[1]: Set up automount mnt-nfs-SSD_240GB-backups-TIG_backups.automount. root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.mount -b -- No entries -- 
  1. I start a simple container up, with no explicit bind mounts for those drives (only the hostfs structure) :
docker run -d \ --name telegraf_container \ --mount type=bind,source=/,destination=/hostfs \ -e HOST_MOUNT_PREFIX=/hostfs \ -e HOST_PROC=/hostfs/proc \ telegraf:latest 

This still does not trigger any automounts on the host.

  1. Now I manually trigger an automount on the host by accessing the drive:
ls /mnt/nfs/SSD_240GB/backups/TIG_backups/ 

The automount is triggered and mounts the drive successfully:

root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.automount -b Jun 06 12:22:20 docker-debian systemd[1]: Set up automount mnt-nfs-SSD_240GB-backups-TIG_backups.automount. Jun 06 12:35:20 docker-debian systemd[1]: mnt-nfs-SSD_240GB-backups-TIG_backups.automount: Got automount request for /mnt/nfs/SSD_240GB/backups/TIG_backups, triggered by 936 (ls) root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.mount -b Jun 06 12:35:21 docker-debian systemd[1]: Mounting mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups... Jun 06 12:35:21 docker-debian systemd[1]: Mounted mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups. 

Interestingly, the mounted drive now automatically appears inside the container (even though no bind-mounts have been used), but it appears under /hostfs instead:

monitoring@docker-debian:~$ docker exec -it telegraf_container df Filesystem 1K-blocks Used Available Use% Mounted on overlay 15421320 4686888 9929264 33% / tmpfs 65536 0 65536 0% /dev shm 65536 0 65536 0% /dev/shm /dev/sda1 15421320 4686888 9929264 33% /hostfs udev 983908 0 983908 0% /hostfs/dev tmpfs 1007084 0 1007084 0% /hostfs/dev/shm tmpfs 201420 656 200764 1% /hostfs/run tmpfs 5120 0 5120 0% /hostfs/run/lock tmpfs 201416 0 201416 0% /hostfs/run/user/1001 tmpfs 1007084 0 1007084 0% /proc/acpi tmpfs 1007084 0 1007084 0% /sys/firmware 192.168.0.67:/mnt/SSD_240GB/backups/TIG_backups 16337920 5799936 9682944 38% /hostfs/mnt/nfs/SSD_240GB/backups/TIG_backups 

If I unmount the drive directly on the host (using umount), then it disappears from the container again.

  1. I repeated this but instead using an idle timeout of 2mins now. What I found was that having the docker container running prevents the autounmount after 2 mins from happening (even though the container does not explicitly bind-mount in the drive, but instead appears automatically in the container under /hostfs).

If I stop and remove the container, then the idle timeout unmounts the drive after the 2mins:

root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.mount -b Jun 06 12:49:40 docker-debian systemd[1]: Mounting mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups... Jun 06 12:49:41 docker-debian systemd[1]: Mounted mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups. Jun 06 13:10:28 docker-debian systemd[1]: Unmounting mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups... Jun 06 13:10:28 docker-debian systemd[1]: mnt-nfs-SSD_240GB-backups-TIG_backups.mount: Deactivated successfully. Jun 06 13:10:28 docker-debian systemd[1]: Unmounted mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups. 

This makes me think a couple of things:

  • If I want to use telegraf to monitor drives that are mounted on the host, I don't need to bind mount them in explicitly, because they are present already due to the /hostfs bind-mount.
  • I should never see what I was originally expecting - namely, a drive automatically unmounting due to the idle timeout, and then the container triggering a remount. Because I observed above that once a drive has been mounted in (in my case at /hostfs), the container actually prevents it from ever being auto-unmounted.
added 2133 characters in body
Source Link
teeeeee
  • 305
  • 1
  • 15

After the drive has auto-unmounted, it is missing from the host as expected:

monitoring@docker-debian:/$ df Filesystem 1K-blocks Used Available Use% Mounted on udev 983908 0 983908 0% /dev tmpfs 201420 816 200604 1% /run /dev/sda1 15421320 4779404 9836748 33% / tmpfs 1007084 0 1007084 0% /dev/shm tmpfs 5120 0 5120 0% /run/lock tmpfs 201416 0 201416 0% /run/user/1001 

But it is present in the container:

monitoring@docker-debian:/$ docker exec -it telegraf_container df Filesystem 1K-blocks Used Available Use% Mounted on overlay 15421320 4779404 9836748 33% / tmpfs 65536 0 65536 0% /dev shm 65536 0 65536 0% /dev/shm /dev/sda1 15421320 4779404 9836748 33% /hostfs udev 983908 0 983908 0% /hostfs/dev tmpfs 1007084 0 1007084 0% /hostfs/dev/shm tmpfs 201420 820 200600 1% /hostfs/run tmpfs 5120 0 5120 0% /hostfs/run/lock 192.168.0.67:/mnt/SSD_240GB/backups/TIG_backups/telegraf_backups 229608448 42336256 175535104 20% /mnt/nfs/SSD_240GB/backups/TIG_backups/telegraf_backups tmpfs 1007084 0 1007084 0% /proc/acpi tmpfs 1007084 0 1007084 0% /sys/firmware tmpfs 201416 0 201416 0% /hostfs/run/user/1001 

In case it's relevant, the Telegraf config is here:

In case it's relevant, the Telegraf config is here:

After the drive has auto-unmounted, it is missing from the host as expected:

monitoring@docker-debian:/$ df Filesystem 1K-blocks Used Available Use% Mounted on udev 983908 0 983908 0% /dev tmpfs 201420 816 200604 1% /run /dev/sda1 15421320 4779404 9836748 33% / tmpfs 1007084 0 1007084 0% /dev/shm tmpfs 5120 0 5120 0% /run/lock tmpfs 201416 0 201416 0% /run/user/1001 

But it is present in the container:

monitoring@docker-debian:/$ docker exec -it telegraf_container df Filesystem 1K-blocks Used Available Use% Mounted on overlay 15421320 4779404 9836748 33% / tmpfs 65536 0 65536 0% /dev shm 65536 0 65536 0% /dev/shm /dev/sda1 15421320 4779404 9836748 33% /hostfs udev 983908 0 983908 0% /hostfs/dev tmpfs 1007084 0 1007084 0% /hostfs/dev/shm tmpfs 201420 820 200600 1% /hostfs/run tmpfs 5120 0 5120 0% /hostfs/run/lock 192.168.0.67:/mnt/SSD_240GB/backups/TIG_backups/telegraf_backups 229608448 42336256 175535104 20% /mnt/nfs/SSD_240GB/backups/TIG_backups/telegraf_backups tmpfs 1007084 0 1007084 0% /proc/acpi tmpfs 1007084 0 1007084 0% /sys/firmware tmpfs 201416 0 201416 0% /hostfs/run/user/1001 

In case it's relevant, the Telegraf config is here:

added 399 characters in body
Source Link
teeeeee
  • 305
  • 1
  • 15

I would expect the telegraf container to trigger an automount every hour. The reason I expect this is because the container is essentially running a .go program (as seen here in the source code) which is querying the filesystem. I believe under the hood it is calling some .go libraries (here and here), which are essentially calling statfs(). I was under the impression that statfs() should trigger a systemd automount.

I would expect the telegraf container to trigger an automount every hour. The reason I expect this is because the container is essentially running a .go program (as seen here in the source code) which is querying the filesystem.

I would expect the telegraf container to trigger an automount every hour. The reason I expect this is because the container is essentially running a .go program (as seen here in the source code) which is querying the filesystem. I believe under the hood it is calling some .go libraries (here and here), which are essentially calling statfs(). I was under the impression that statfs() should trigger a systemd automount.

Source Link
teeeeee
  • 305
  • 1
  • 15
Loading