25

I have read some question, that ask advice how to rsync sparse files efficiently mentioning the files /var/log/lastlog and /var/log/faillog. Indeed I myself have stumpled over those files being an "issue" as their being backup via rsync turns them to become "unsparse".

What I hence wonder is, what is the need/backgrounding motivation to have those files as sparse, huge files (in my case it was 1.1TB)?

Also in relationship to this a follow up: Since I was assuming them to be logfiles I do not care about excesively I truncated those files, did I corrupt anything with truncating those files ?

Improve this question
3
  • configure logrotate Commented Jul 14, 2019 at 0:25
  • 6
    It's a sparse file since the user-id's are used for indexing into it, and those can be large values, e.g., when using Samba. Commented Jul 14, 2019 at 20:56
  • @ThomasDickey according to your comment, the answer by waitinator that originally hints that truncating the file is the original problem would not ring truth, right? as soon as there is a "strangly" large uid the file will become apparently huge (though taking few bytes being "sparse"), correct? Commented Jul 15, 2019 at 10:01

4 Answers 4

Reset to default
25

What I hence wonder is, what is the need/backgrounding motivation to have those files as sparse, huge files (in my case it was 1.1TB)?

This is how it's supposed to be.

/var/log/lastlog is not a log file like /var/log/syslog, and its name should be read as "last logins list" rather than "last logfile".

It's maintained by the pam_lastlog(8) module, and it's basically an array like this:

struct lastlog { time_t ll_time; // 4 char ll_line[UT_LINESIZE]; // 32 char ll_host[UT_HOSTSIZE]; // 256 } entry[UINT_MAX];

Sizes of the fields on a typical x86-64 machine are in comments; an entry should be 4 + 32 + 256 = 292 bytes.

Every time a program using the pam_lastlog(8) pam module is logging a user in, it will seek to uid * sizeof(struct lastlog) and overwrite the entry corresponding to that user.

did I corrupt anything with truncating those files ?

You did corrupt the output of the lastlog(1) command, which nobody is using anyway ;-)

Improve this answer
9
  • I am happy I refrained from accepting an earlier answer, that mentioned the reason for huge apparent file size being truncating of the file. Your answer seems to provide better, more correct advice, well done! Commented Jul 15, 2019 at 10:07
  • 2
    Couriousity has me here, this concept to use sparse files and an then using a arraykey /offset to map into the correct small peace of information (leading to sparse files) is something that I only ever encountered with /var/log/lastlog and /var/log/faillog, making me feel prompted to ask here. Is this a common technique, i.e. something other software in an normal unix/linux setup do also? Commented Jul 15, 2019 at 10:11
  • No, most common software don't do that, because that doesn't work on windows ;-) Commented Jul 15, 2019 at 10:16
  • 4
    @Edmund That's a loaded question. Let's have a different take: why would someone NOT use sparse files and instead of it re-invent the wheel (or use some heavy-weight general purpose database) if they're sure that their software will never have to work on broken/primitive OSs without support for sparse files? Commented Sep 11, 2020 at 16:52
  • 1
    @Feuermurmel - there is software which can backup a filesystem as a disk image, walking through the filesystem structure. It was the norm at one time in the 1970s. Also, sparse blocks read VERY fast, at bzero() speed. Commented Feb 24, 2023 at 21:02
4

I have seen this behaviour after we have introduced some (large) LDAP UID numbers co-existing with "normal" UNIX UID numbers in a linux (RHEL 7) machine.

Also, in the "CAVEATS" section of the manual page for "lastlog" it states that

 *"Large gaps in UID numbers will cause the lastlog program to run longer with no output to the screen"*

"lastlog" will appear to hang as it processes entries with the intermediate unused UIDs

Maybe this could be also related to the observed problem, as FreeIPA LDAP UID numbers assigned were much larger than the ones from Linux

Improve this answer
1
  • 3
    That's one of those things which hadn't aged well. The code was written for much smaller machines, many, many years ago. One of these days I should join the maintenance team. Or re-do a bunch of that code from scratch. Commented Sep 23, 2020 at 21:21
2

Those 2 files are "data" type files. If you check using ls -l or ls -lh command then you will see it's occupying very huge amount of disk space, but actually it's not.

ls -s is the command to check the actual size of these files.

[root@LinuxServer ~]# file /var/log/lastlog /var/log/faillog /var/log/lastlog: data /var/log/faillog: data [root@LinuxServer ~]# ls -l /var/log/lastlog /var/log/faillog -rw------- 1 root osg 3166464 Jul 8 21:20 /var/log/faillog -rw-r--r-- 1 root root 304854077980 Jul 14 21:38 /var/log/lastlog [root@LinuxServer ~]# ls -lh /var/log/lastlog /var/log/faillog -rw------- 1 root osg 3.1M Jul 8 21:20 /var/log/faillog -rw-r--r-- 1 root root 284G Jul 14 21:38 /var/log/lastlog [root@LinuxServer ~]#

The actual size is only few KB's, if you check with the ls -s command (output of -s is in disk blocks).

[root@LinuxServer ~]# ls -s /var/log/lastlog /var/log/faillog 3100 /var/log/faillog 748 /var/log/lastlog
Improve this answer
3
  • 4
    An apparent size larger than the amount of data stored is the very definition of a sparse file. Commented Jul 14, 2019 at 21:27
  • thank you! I understand you are quite new to this stackexchange community. Most likely you meant very well providing some insight to this issue, but as @ilkkachu suggest the very question mentions the fact that we deal with sparse files hence I knew that they actually use less space on disk. Do not be too discouraged by the downvotes. Commented Jul 15, 2019 at 10:05
  • 1
    this is the only answer that explains how to get the actual or IMO the "true" file size. for more information why see: stackoverflow.com/questions/43126760/… . Commented Jan 16, 2023 at 16:11
-4

By "truncating the file", without informing (or restarting) the programs that are writing to the log file, YOU have caused the sparse file.

The best fix is to find and fix the problem causing all those log messages.

Another way to avoid the problem is Stop the logging service, use lsof to ensure no other processes have the file open, then truncate the file, and restart logging.

As requested, here's a model of how it (doesn't) work:

A program opens the file for "append", is told that the file ends at block X, and writes block X+1. You "truncate the file", but the program "knows" the next block to write is X+2. Thus, the file will might gave data at the beginning, but definitely has data at block X+2. Ta-Da! Sparse file.

Improve this answer
7
  • So then I must have gotten myself into an endless loop, since my cure for when those files have grown excessively huge has since been to truncate them back to size (which as if I read correctly your answer is supposed to be the original cause of the huge size in and on itself?). I am puzzled how this however reaches sizes up to 1TB since, I would supposed imparative reboots for kernel updates would have interrupted any logging processes anyway. I want to believe you, but I am struggling to really grasp the particularities of this, i..e. how a file becomes sparse + huge, can you enlight me? Commented Jul 14, 2019 at 8:10
  • 1
    How do you know the files ended up being sparse because of truncating without restarting the programs? Give some reasoning for that claim. Commented Jul 14, 2019 at 21:30
  • 2
    Actually, writing in append mode doesn't create a sparse file like that, the writes go to the end of the file, where ever it is at that point. (A write in normal mode would work as you described though.) But anyway, you've described how a file becomes sparse if it's truncated while being open, but where does it say that they truncated the file without restarting the relevant processes? Also, given that the man page for lastlog explicitly says that the file is sparse, are you sure it's used like a regular log file to begin with? Commented Jul 14, 2019 at 22:03
  • 3
    lastlog has nothing to do with log files. Your answer is completely off. Try man lastlog and man pam_lastlog. Commented Jul 14, 2019 at 23:25
  • 1
    @ilkkachu lastlog is not a log file like /var/log/messages. It's normal to be a big sparse binary file, since it's basically an array of structs indexed by user id, and the user ids are not contiguous. Commented Jul 14, 2019 at 23:33

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.