20

Yesterday, one of our computers dropped to grub shell or honestly, I am unsure what shell it was when we turned on the machine.

It showed that it can't mount the root filesystem or something in this sense, because of inconsistencies.

I ran, I believe:

fsck -fy /dev/sda2

Rebooted and the problem was gone.

Here comes the question part:

I already have in her root's crontab:

@reboot /home/ruzena/Development/bash/fs-check.sh

while the script contains:

#!/bin/bash touch /forcefsck

Thinking about it, I don't know, why I created script file for such a short command, but anyways...

Further, in the file:

/etc/default/rcS

I have defined:

FSCKFIX=yes

So I don't get it. How could the situation even arise?

What should I do to force the root filesystem check (and optionally a fix) at boot?

Or are these two things the maximum, that I can do?

OS: Linux Mint 18.x Cinnamon 64-bit.

fstab:

cat /etc/fstab | grep ext4

shows:

UUID=a121371e-eb12-43a0-a5ae-11af58ad09f4 / ext4 errors=remount-ro 0 1

grub:

fsck.mode=force

was already added to the grub configuration.

Improve this question
0

2 Answers 2

Reset to default
23

ext4 filesystem check during boot

Tested on OS: Linux Mint 18.x in a Virtual Machine

Basic information

/etc/fstab has the fsck order as the last (6th) column, for instance:

<file system> <mount point> <type> <options> <dump> <fsck> UUID=2fbcf5e7-1234-abcd-88e8-a72d15580c99 / ext4 errors=remount-ro 0 1

FSCKFIX=yes variable in /etc/default/rcS

This will change the fsck to auto fix, but not force a fsck check.

From man rcS:

FSCKFIX When the root and all other file systems are checked, fsck is invoked with the -a option which means "autorepair". If there are major inconsistencies then the fsck process will bail out. The system will print a message asking the administrator to repair the file system manually and will present a root shell prompt (actually a sulogin prompt) on the console. Setting this option to yes causes the fsck commands to be run with the -y option instead of the -a option. This will tell fsck always to repair the file systems without asking for permission.

From man tune2fs

If you are using journaling on your filesystem, your filesystem will never be marked dirty, so it will not normally be checked.

Start with

Setting the following

FSCKFIX=yes

in the file

/etc/default/rcS

Check and note last time fs was checked:

sudo tune2fs -l /dev/sda1 | grep "Last checked"

These two options did NOT work

  1. Passing -F (force fsck on reboot) argument to shutdown:

    shutdown -rF now

    Nope; see: man shutdown.

  2. Adding the /forcefsck empty file with:

    touch /forcefsck

    These scripts seem to use this:

    /etc/init.d/checkfs.sh /etc/init.d/checkroot.sh

    did NOT work on reboot, but the file was deleted.

    Verified by:

    sudo tune2fs -l /dev/sda1 | grep "Last checked" sudo less /var/log/fsck/checkfs sudo less /var/log/fsck/checkroot

    These seem to be the logs for the init scripts.

I repeat, these two options did NOT work!

Both of these methods DID work

  1. systemd-fsck kernel boot switches

    Editing the main grub configuration file:

    sudoedit /etc/default/grub 
    GRUB_CMDLINE_LINUX="fsck.mode=force" 
    sudo update-grub sudo reboot

    This did do a file system check as verified with:

    sudo tune2fs -l /dev/sda1 | grep "Last checked"

    Note: This DID a check, but to force a fix too, you need to specify fsck.repair="preen", or fsck.repair="yes".

  2. Using tune2fs to set the number of file system mounts before doing a fsck, man tune2fs

    tune2fs' info is kept in the file system superblock

    -c switch sets the number of times to mount the fs before checking the fs.

    sudo tune2fs -c 1 /dev/sda1

    Verify with:

    sudo tune2fs -l /dev/sda1

    This DID work as verified with:

    sudo tune2fs -l /dev/sda1 | grep "Last checked"

Summary

To force a fsck on every boot on Linux Mint 18.x, use either tune2fs, or fsck.mode=force, with optional fsck.repair=preen / fsck.repair=yes, the kernel command line switches.

Improve this answer
2
  • 1
    On a Grub-based Debian system, consider creating /etc/default/grub.d/local.cfg file with GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE_LINUX_DEFAULT fsck.mode=force". This will avoid need to merge /etc/default/grub config file during upgrades. Commented May 26, 2020 at 8:28
  • This is a fantastic answer - but I think that the command towards the end should be using -i not -c. -i sets the check interval - and -c sets the maximum mount count. Commented Nov 30, 2021 at 22:10
7

Further investigation and updates on the existing answer

I now just wanted to check, whether the above still works on Ubuntu 20.04/22.04-LTS-based systems (directly tested on Linux Mint 20/21 Cinnamon amd64 desktop and Ubuntu MATE 20.04/22.04 amd64 desktop), and I found out a few things, let's start with the file system check interval (I ran all commands as root (as you might notice ~# in front of commands):

File system check interval

~# LC_ALL=C tune2fs -l /dev/nvme0n1p2 | grep 'Check interval' Check interval: 0 (<none>)

Well, this was unexpected. I thought we took care of it, but can be fixed very easily. Take note, the number it takes as an argument is by default in days, so be sure to use 1s (1 seconds) instead of just 1 which would mean 1 day (86400 seconds):

~# LC_ALL=C tune2fs -i 1s /dev/nvme0n1p2 tune2fs 1.45.5 (07-Jan-2020) Setting interval between checks to 1 seconds

Now, if we repeat the above check, we get:

Check interval: 1 (0:00:01)

This does not mean the file system will be checked every second, of course. Rather, in effect it will force the file system check on every file system mount. (As there is no way of booting any system twice in one second 😀.)

File system maximum number of mounts before check

Default setting:

~# LC_ALL=C tune2fs -l /dev/nvme0n1p2 | grep 'Maximum mount count' Maximum mount count: -1

This setting adjusts how many mounts it takes till the file system gets checked. It's ok what is written in the original answer:

~# LC_ALL=C tune2fs -c 1 /dev/nvme0n1p2 tune2fs 1.45.5 (07-Jan-2020) Setting maximal mount count to 1

Just make sure you do not use 0 or -1 as it would become disregarded.

---

Combination of both - The recommended way

Here you only change your disk name (e.g. sda, vda, nvme0n1, etc.), and partition number:

First, we find out the current status with:

partition=/dev/nvme0n1p2; LC_ALL=C tune2fs -l $partition | grep -E 'Check\ interval|Maximum\ mount\ count'

=

Maximum mount count: -1 Check interval: 0 (<none>)

Afterward, we set these values so that the file system is checked on every boot with:

partition=/dev/nvme0n1p2; LC_ALL=C tune2fs -i 1s -c 1 $partition 2>&1 | grep Setting

=

Setting maximal mount count to 1 Setting interval between checks to 1 seconds

Finally, let's review the new status with the same first command:

partition=/dev/nvme0n1p2; LC_ALL=C tune2fs -l $partition | grep -E 'Check\ interval|Maximum\ mount\ count'

=

Maximum mount count: 1 Check interval: 1 (0:00:01)

Information sources:

Keep checking (pun intended)! 😎

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.