Timeline for How to fix intermittant "No space left on device" errors during mv when device has plenty of space?
Current License: CC BY-SA 3.0
28 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 30, 2016 at 7:34 | answer | added | Barani r | timeline score: 2 | |
| S Oct 15, 2015 at 22:56 | history | suggested | Thomas Dickey | CC BY-SA 3.0 | fix a typo and grammar |
| Oct 15, 2015 at 22:38 | review | Suggested edits | |||
| S Oct 15, 2015 at 22:56 | |||||
| Aug 12, 2015 at 18:46 | vote | accept | Chris.Caldwell | ||
| Aug 11, 2015 at 23:21 | answer | added | Peter Cordes | timeline score: 9 | |
| Aug 10, 2015 at 15:19 | history | tweeted | twitter.com/#!/StackUnix/status/630760405221122049 | ||
| Aug 10, 2015 at 12:55 | history | edited | Chris.Caldwell | CC BY-SA 3.0 | edited body |
| Aug 10, 2015 at 12:54 | comment | added | Chris.Caldwell | yeah, i was anonymizing it on purpose and missed one. | |
| Aug 10, 2015 at 12:48 | history | edited | steve | added ext4 tag | |
| Aug 10, 2015 at 12:46 | answer | added | steve | timeline score: 28 | |
| Aug 10, 2015 at 12:44 | comment | added | Stéphane Chazelas | (compare the output of getfattr -dm '' on files that can be transferred and files that cannot | |
| Aug 10, 2015 at 12:38 | comment | added | Chris.Caldwell | No, I havent setup any ACLs or anything special. Its just a standard Ubuntu 14.04 fresh installation, and basically the raid / lvm is the only thing I setup on it since. Im not aware of any default quotas, I havent set any, but if that were the case wouldnt it be a full stop, not intermittent? Most files go just fine, even after some fail. but success or fail looks to be specific to a file since failed ones never succeed no matter how many times I retry. | |
| Aug 10, 2015 at 12:32 | comment | added | Andrew Henle | Assuming you're doing the mv as root, does the target file system have any user quotas, and if so would moving the failed files result in a quota violation? | |
| Aug 10, 2015 at 12:28 | comment | added | Stéphane Chazelas | Do the files have ACLs or extended attributes? Do the directories have default ACLs? | |
| Aug 10, 2015 at 12:22 | comment | added | Chris.Caldwell | @mjturner Thanks for the reply, im really getting clueless here. I think there's an issue with your linked article in that it is talking about a 2GB filesize limit in ext3, but im using ext4 for that exact reason. I hadnt thought of it, but you're right, its really odd that both are ext4 but the source had no problem holding it. If it matters, it was put ONTO the source drive via wget not mv. Anyway, outputs in original post as an edit. | |
| Aug 10, 2015 at 12:20 | history | edited | Chris.Caldwell | CC BY-SA 3.0 | added 4233 characters in body |
| Aug 10, 2015 at 8:25 | comment | added | mjturner | Can you paste the output of the following - ls -ld /mnt/archive/targetDir? That will let us know if you're getting close to the 2GB directory inode limit (as per this answer). Also, the output of tune2fs -l /dev/mapper/archive-lvarchive and tune2fs -l /dev/sda1so we can see block sizes, etc. It's still very odd, as the source filesystem is also ext4 and that's storing all of the files already... | |
| Aug 10, 2015 at 8:05 | comment | added | Chris.Caldwell | @Snoopy its a reasonable approach, but its a lot of work for a guess. Without knowing what is happening, how do we know this will fix it? In particular, if it were a "too many files in a folder" error I would expect to hit a wall. Instead its intermittent, and, stranger, files which dont work NEVER work. | |
| Aug 10, 2015 at 3:56 | comment | added | snoopy | @Chris other approach could be to break these up into multiple subdirectories somehow, e.g. /mnt/archive/targetDir/70909/709099107.pdf instead of /mnt/archive/targetDir/709099107.pdf | |
| Aug 10, 2015 at 3:53 | history | migrated | from stackoverflow.com (revisions) | ||
| Aug 10, 2015 at 1:52 | comment | added | Chris.Caldwell | @technosaurus thx, has been flagged for move. | |
| Aug 10, 2015 at 1:39 | comment | added | Chris.Caldwell | @dwarring, thanks for link. That does in fact look like it may be the same issue. Unfortunately he didnt really get an answer. inodes for ext4 is global, not per directory, and according to df -i the limit is 289,966,080. Perhaps I've misunderstood, but it seems I'm only ever going to be at about 25% capacity. In the linked to post, the answer was "use xfs", which, as OP noted, isnt an answer to "whats going on". Id be willing to use xfs if I could learn why this is insufficient and xfs is not, but given that it will take nearly a month to implement Im not anxious to just try stuff. | |
| Aug 10, 2015 at 1:30 | comment | added | snoopy | @Chris - seems similar to this issue on SF: serverfault.com/questions/384541/… | |
| Aug 10, 2015 at 1:30 | comment | added | technosaurus | The down vote was probably because your question is better suited to Unix.stackexchange or askubuntu since it is not programming related. If there isn't a programming language in your tags, it will likely get a down vote. | |
| Aug 10, 2015 at 1:21 | comment | added | Chris.Caldwell | oh look, the random downvote troll struck again. I dont guess you would care to mention WHY you downvote? | |
| Aug 10, 2015 at 1:20 | comment | added | Chris.Caldwell | not 1.5m, 15m, and yes, all to the same directory. In fact there is already over 40m there, and about 30m more to go total. | |
| Aug 10, 2015 at 1:11 | comment | added | snoopy | Are the files being coped to multiple directories, or are you attempting to write 1.5M files to a single target directory? | |
| Aug 10, 2015 at 0:48 | history | asked | Chris.Caldwell | CC BY-SA 3.0 |