1

I have a question I'm struggling to research an answer for. I am building a Ubuntu media server at home and decided to carve up my 2TB bulk storage disk into various partitions.

I created a 700GB partition for Music and formatted EXT4. Then migrating about 370GB from an NTFS external HD, using Beets to build my music library and copy the files. Problem is the total directory size is exceeding 730GB!

Why is this? Is there an error in the way Beets is processing the files? Or is this something to do with the filesystem choice?

Beets is fetching album art and some other metadata, but this definitely would not increase the overall footprint by a factor of two?

Any advice appreciated! I'm new to Linux filesystems so probably missing something elementary about inodes, journaling?

dumep2fs shows:

chris@MICROSERVER:~$ sudo dumpe2fs /dev/mapper/MICROSERVER--vg-music [sudo] password for chris: dumpe2fs 1.42.9 (4-Feb-2014) Filesystem volume name: <none> Last mounted on: /media/ubuntu/62c278e1-4c1e-470a-b68c-f0a5d777c165 Filesystem UUID: 62c278e1-4c1e-470a-b68c-f0a5d777c165 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) cFilesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file Filesystem flags: signed_directory_hash Default mount options: user_xattr acl Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 1966080 Block count: 7864320 Reserved block count: 393216 Free blocks: 7681951 Free inodes: 1966057 First block: 0 Block size: 4096 Fragment size: 4096 Reserved GDT blocks: 1022 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 8192 Inode blocks per group: 512 Filesystem created: Sat Jul 25 10:22:35 2015 Last mount time: Sun Aug 2 21:28:44 2015 Last write time: Sun Aug 2 21:28:44 2015 Mount count: 23 Maximum mount count: -1 Last checked: Sat Jul 25 10:22:35 2015 Check interval: 0 (<none>) Lifetime writes: 121 MB Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 256 Required extra isize: 28 Desired extra isize: 28 Journal inode: 8 Default directory hash: half_md4 Directory Hash Seed: f7ccf1c6-402b-4666-bdd1-2a10d7fa1948 Journal backup: inode blocks Journal features: journal_incompat_revoke Journal size: 128M Journal length: 32768 Journal sequence: 0x00000023 Journal start: 0

df -h

chris@MICROSERVER:~$ df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/MICROSERVER--vg-root 20G 12G 7.5G 60% / none 4.0K 0 4.0K 0% /sys/fs/cgroup udev 3.9G 4.0K 3.9G 1% /dev tmpfs 796M 1.2M 795M 1% /run none 5.0M 0 5.0M 0% /run/lock none 3.9G 0 3.9G 0% /run/shm none 100M 0 100M 0% /run/user /dev/sda1 236M 98M 126M 44% /boot /dev/mapper/MICROSERVER--vg-music 673G 641G 372K 100% /media/music /dev/mapper/MICROSERVER--vg-backup 50G 38G 9.3G 81% /media/backup /dev/sdb2 673G 33G 606G 6% /music

df -i

Filesystem Inodes IUsed IFree IUse% Mounted on /dev/mapper/MICROSERVER--vg-root 1310720 267682 1043038 21% / none 1017972 2 1017970 1% /sys/fs/cgroup udev 1006939 502 1006437 1% /dev tmpfs 1017972 514 1017458 1% /run none 1017972 5 1017967 1% /run/lock none 1017972 1 1017971 1% /run/shm none 1017972 2 1017970 1% /run/user /dev/sda1 62248 311 61937 1% /boot /dev/mapper/MICROSERVER--vg-music 44802048 97110 44704938 1% /media/music /dev/mapper/MICROSERVER--vg-backup 3276800 1434 3275366 1% /media/backup /dev/sdb2 44802048 11710 44790338 1% /music

lsblk -b

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 160041885696 0 disk ├─sda1 8:1 0 254803968 0 part /boot ├─sda2 8:2 0 1024 0 part └─sda5 8:5 0 159784108032 0 part ├─MICROSERVER--vg-root (dm-0) 252:0 0 21441282048 0 lvm / ├─MICROSERVER--vg-swap_1 (dm-1) 252:1 0 8552185856 0 lvm [SWAP] ├─MICROSERVER--vg-music (dm-2) 252:2 0 32212254720 0 lvm └─MICROSERVER--vg-backup (dm-3) 252:3 0 53687091200 0 lvm /media/backup sdb 8:16 0 2000398934016 0 disk ├─sdb1 8:17 0 734003200000 0 part /media/music └─sdb2 8:18 0 734003200000 0 part /music

mount

/dev/mapper/MICROSERVER--vg-root on / type ext4 (rw,errors=remount-ro) proc on /proc type proc (rw,noexec,nosuid,nodev) sysfs on /sys type sysfs (rw,noexec,nosuid,nodev) none on /sys/fs/cgroup type tmpfs (rw) none on /sys/fs/fuse/connections type fusectl (rw) none on /sys/kernel/debug type debugfs (rw) none on /sys/kernel/security type securityfs (rw) udev on /dev type devtmpfs (rw,mode=0755) devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620) tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755) none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880) none on /run/shm type tmpfs (rw,nosuid,nodev) none on /run/user type tmpfs (rw,noexec,nosuid,nodev,size=104857600,mode=0755) none on /sys/fs/pstore type pstore (rw) /dev/sda1 on /boot type ext2 (rw) /dev/mapper/MICROSERVER--vg-music on /media/music type ext3 (rw) /dev/mapper/MICROSERVER--vg-backup on /media/backup type ext3 (rw) /dev/sdb2 on /music type ext4 (rw) /dev/sdb1 on /media/music type ext4 (rw) systemd on /sys/fs/cgroup/systemd type cgroup (rw,noexec,nosuid,nodev,none,name=systemd)

sudo lvdisplay

--- Logical volume --- LV Path /dev/MICROSERVER-vg/root LV Name root VG Name MICROSERVER-vg LV UUID Lwx28M-m2k9-XIpB-fUEC-HiWa-5QsN-SlNOt3 LV Write Access read/write LV Creation host, time MICROSERVER, 2015-07-24 20:55:58 +0100 LV Status available # open 1 LV Size 19.97 GiB Current LE 5112 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 252:0 --- Logical volume --- LV Path /dev/MICROSERVER-vg/swap_1 LV Name swap_1 VG Name MICROSERVER-vg LV UUID vFQxWC-g7xw-Ilc0-Eixj-51Fv-HaXV-t3uAuq LV Write Access read/write LV Creation host, time MICROSERVER, 2015-07-24 20:55:58 +0100 LV Status available # open 2 LV Size 7.96 GiB Current LE 2039 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 252:1 --- Logical volume --- LV Path /dev/MICROSERVER-vg/music LV Name music VG Name MICROSERVER-vg LV UUID AA9x7O-ZBRd-uBr4-6hPO-jMeS-3ien-r6zvew LV Write Access read/write LV Creation host, time MICROSERVER, 2015-07-25 10:19:33 +0100 LV Status available # open 1 LV Size 30.00 GiB Current LE 7680 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 252:2 --- Logical volume --- LV Path /dev/MICROSERVER-vg/backup LV Name backup VG Name MICROSERVER-vg LV UUID o6tyxv-eTik-OcpX-KhdE-rI4h-z4Rq-tG2NUS LV Write Access read/write LV Creation host, time MICROSERVER, 2015-07-25 20:58:27 +0100 LV Status available # open 1 LV Size 50.00 GiB Current LE 12800 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 252:3

To add some more context, I am moving the data using beets from /music to /media/music. The original music library size was ~370GB.

Improve this question
5
  • What kinds of files are on the disk other than music files? find /media/ubuntu/62c278e1-4c1e-470a-b68c-f0a5d777c165 \! \( -name \*.mp3 -o \*.ogg <other music-file extension tests> \) Commented Aug 3, 2015 at 9:47
  • that does not look like 700 GB Commented Aug 3, 2015 at 9:49
  • I may be totally wrong, but that looks like 30G, Even if I am wrong it would be 300G. No matter how I do it I don't get 700G Commented Aug 3, 2015 at 9:53
  • Can you run dh -h and paste the output. I don't have dumep2fs installed and I can't figure out why your BS * BC doesn't equal 700Gigs. Commented Aug 3, 2015 at 9:56
  • Please post the output of lsblk -b, mount, and sudo lvdisplay. Commented Aug 3, 2015 at 11:17

1 Answer 1

Reset to default
1

While I have never used beets, I can give you some general info. First you are correct, in one regard at least. Generally speaking 300 GB of data is 300 GB of data. The file system should not matter too much. There are a few things you can check to make sure things are going smoothly.

First is free inodes. When you run df -h you should get free space in a nice format. If you use df -i you get inode stats. It is possible (but not likely) that you are running out of inodes and not actual space. This can happen with many small files.

Second, and this is more likely, embedded album art can really take up that much space. It has to be embedded in each MP3. So a album with 15 songs and album art with front, back, and insert images, each at 700k just added 31.5 Megs.

Third make sure beets is not copying, but moving. It beets tries to copy from /music to /music/processed and /music is the mount point, then you will easily more then double your space. This is true even if it moves from /old_music to /music then as part of processing copies from /music to /music/processed.

Improve this answer
1
  • Thanks for your insight, fair point about album art, I might have to consider that! Updated the question with the df outputs. Commented Aug 3, 2015 at 10:03

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.