Timeline for How to fix intermittant "No space left on device" errors during mv when device has plenty of space?
Current License: CC BY-SA 3.0
13 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 25, 2017 at 5:43 | comment | added | Anwar | The blog link is not accessible now (at least from my end) | |
| Aug 12, 2015 at 18:46 | vote | accept | Chris.Caldwell | ||
| Aug 11, 2015 at 23:21 | comment | added | Peter Cordes | turned this into an answer. You should accept @Steve's, since it's the answer to main question you asked, of what's up with the ENOSPC errors. | |
| Aug 11, 2015 at 23:04 | comment | added | Peter Cordes | @Chris.Caldwell: XFS is not ideal for using the filesystem as a database / object store. It doesn't have ENOSPC on hash collisions, but I think it will slow down some in this use-case. oss.sgi.com/archives/xfs/2014-08/msg00353.html might be useful. IDK if btrfs is good at this, but I think it may be. I know reiserfs was optimized for this case, but it's barely if at all maintained anymore. | |
| Aug 11, 2015 at 17:46 | comment | added | Chris.Caldwell | I think this is a great answer, and Id like to mark it as such, but I think it would be nice if we could come to a fix, not just a diagnosis. Does anyone know if xfs suffers from anything like this? Ive read mixed reviews that it scales fine, or not over 1m. | |
| Aug 10, 2015 at 18:35 | comment | added | Chris.Caldwell | Yeah. Im not needing peak performance, but a fs search for each file would be horrible. So now Im looking at xfs or an array of 10k or so subfolders. Subfolders is a reasonable solution, however with ext4 I still run the risk of collision. does xfs suffer from the same issue? I read it uses a B+ tree, but that doesnt mean so much to me as far as ensuring theres never a collision. There is a world of misinformation out there, and Ive heard claims that it slows considerably over a million files, and claims that it doesnt. | |
| Aug 10, 2015 at 13:11 | history | edited | steve | CC BY-SA 3.0 | backup backup backup |
| Aug 10, 2015 at 13:03 | comment | added | mjturner | Well spotted @steve. Unfortunately turning off dir_index will probably kill access performance with 70m files in one directory. | |
| Aug 10, 2015 at 13:02 | history | edited | steve | CC BY-SA 3.0 | added 27 characters in body |
| Aug 10, 2015 at 12:57 | comment | added | steve | Added the tune2fs command to disable the indexes, in case you want to try that. | |
| Aug 10, 2015 at 12:57 | history | edited | steve | CC BY-SA 3.0 | add tune2fs, caution |
| Aug 10, 2015 at 12:54 | comment | added | Chris.Caldwell | oh crap, that sounds exactly like it, and like a complete pain to fix. Its about a month to recopy. can this be done without losing the contents? Ill have to research dir_index etc more tomorrow. Wow, never would have thought of that. | |
| Aug 10, 2015 at 12:46 | history | answered | steve | CC BY-SA 3.0 |