I had a lost+found folder in all Unix file systems I used since the 80s. It's where fsck places files that it found during a scan and can't figure out to which directory they belong. Sometimes I found stuff in there.
From what I googled XFS, Btrfs and ZFS don't use lost+found. It's a thing of the old not journaled filesystems and of the ext family.
XFS does use /lost+found, it calls it the "orphanage directory" and xfs_repair reparents children of corrupt directories there.
Based on comments in the kernel source, it seems like the userspace fsck for JFS and F2FS will also sometimes create /lost+found. There might be more that do.
XFS filesystems do not have a "/lost+found" directory in their normal state.
In the very rare occasions when one has to run "xfs_repair", it will create a "/lost+found" directory, if it is required for recovered files.
After the repair and after investigating whether the recovered files contain useful data or not (and after moving the useful files elsewhere), one should normally delete the "/lost+found" directory, because it is no longer needed.
XFS as implemented in RHEL8+ (the only places i've used it in anger) tends to handle being full very badly, leading to system lockups and blocked tasks necessitating a hard reboot. Worse yet is when it's in this state the journal fills and nothing can be done with the volume.
To recover from this on a volume mounted at boot mandates going to either a live disk, or stopping boot in initramfs and running xfs_repair there, I've fruitlessly attempted to play back the journal on many separate occasions by attempting to mount the filesystem (again causing a lock up due to no space) in that state you have to drop the journal, run xfs_repair and then clean up the detritus from /lost+found (and then the location that caused the disk to fill altogether).
EXT4 has other issues certainly, but at least it reserves blocks for the root user explicitly so the system doesn't stop.
XFS tip:
If you truncate a file it doesn't update metadata. That's how you can get back space for the journal log to start cleaning up crap without rebooting the box and/or taking services down.
Even with journaling, you might need one. ZeroFS [0] almost had a lost+found directory (even with the WAL enabled), because you might have consistency issues between your in-memory state and what was flushed, and especially in what order.
ZeroFS ended up not needing recovery at all through atomic, strictly ordered commits [1], but it was far from trivial (and not just a matter of requiring a WAL).
I have a book on my bookshelf, Eric Foxley's Unix for Super-Users. It was published in 1985, and it answers this question on page 52, the first page listed for the entry 'lost+found' in its index.
This is surely not the earliest book mention, is it? (It'll be in earlier man pages, of course.) Google Books does not give me an earlier one, although it does yield another 1985 book.
Fun fact: Foxley cautioned that lost+found must be pre-sized ahead of time, because the fsck of the time did not grow the directory to fit found files.
How do you size a directory? Just by creating entries in it and then unlinking them?
Usually the filesystem driver provides an API call that sets the appropriate data structures so that the directory entry table is preallocated.
The ext-family filesystems provide the mklost+found command to tap into this call if you need to recreate the lost+found directory specifically.
In a couple of decades running Linux installations of all flavours, I have never seen anything in lost+found!
Yea, run an old kernel with ext2 on a busy system writing a bunch of small files and have a power supply fail and you'll end up with something there.
fsck on large hard drives was scary on how long it could take to finish.
The occassional "Drive has not been checked in <n> days, forcing check" message on bootup got annoying sometimes, yeah. It could easily take tens of minutes to finish, exactly when I wanted to use the computer!
(At least this is what my memory is telling me. I could be mistaken, but that's what I remember.)
"check forced", as though the machine is advising me to go look at some daemon named forced.
Forced sounds like a systemd service for Jedi stuff.
I had a largish raid back then, under ext2, may have been a massive 40G.
I recall going to sleep and it still not being done when I woke up. Bleh.
One of several reasons to partition a large volume into smaller ones. At least, back then.
You need to use worse hardware and bad power :)
I used to develop SSD firmware and one of things I worked on is making it robust to power failure. The power supplies have lots of capacitance so the voltage drop was slow so we would use a special test board that would disconnect from power and discharge fast to test it.
Thank you for your service!
My main experience is with pre-scsi/ide systems :)
ESDI?
Yeah, I think lack of any decent hardware RAID could be a prerequisite.
And more concurrent writes.
But I think ext4 will only let things appear there if you change some default flags.
Umm .. how about a Raspberry Pi Zero 2W powered by a 2000MaH " lipstick style" powerbank?
That's what the answers are missing, of course. In some filesystem formats, it's possible either to recover completely from a journal/intent log, or at least to recover everything to the point that recovered files can be placed into the correct directory.
Same here. And I had some pretty f**ed up file systems.
At one point, I had one where the directory structure was completely broken and had circles in it (broken SSD). To be fair, in that particular case, I did not look for lost+found and just wrote a tool to extract the data manually that I was looking for.
My SD cards have always had stuff in that folder. It scares me. I try not to look
Have to run fsck. This used to be forced about once a month but don’t remember it happening in the last decade or so.
[deleted][deleted]
We once deleted the lost+found folder on an old Unix system* by accident. Things went very badly the next time the system rebooted, fsck did not handle it at all well.
* Probably DEC Ultrix 2.2, a BSD 4.2 derivative.
Why can't a filesystem create the lost+found folder only when it needs to store files in it?
That would be a much cleaner approach, imho.
Added benefit is that you'd immediately see it if something is wrong with a disk.
As the submission explains, the lost+found folder has pre-allocated space for the directory entries. From the mklost+found man page [0]:
mklost+found pre-allocates disk blocks to the lost+found directory
so that when e2fsck(8) is being run to recover a file system, it
does not need to allocate blocks in the file system to store a
large number of unlinked files. This ensures that e2fsck will not
have to allocate data blocks in the file system during recovery.
Pre-allocating space without making the directory visible would require more arcane file system magic.
> Preallocating space without making the directory visible would require more arcane file system magic.
If those filesystem engineers had a manager that said: make this nice for the user, then it would have been done.
But these developers had no managers and were OK eating their own unpalatable dogfood.
[deleted][deleted]
Because you need to pre-allocate space and indies for it in case things go wrong and the disk is almost full
[deleted]
Back in the day I accidentally deleted all my stuff because I had it all in a special dir of this user in suse Linux. When I deleted the user, yast deleted everything.
Fortunately I was using ReiserFS at the time and something about its murderous tree data structure made it trivial to undelete.
Reiser_fsck found ALL my stuff, mostly with full dir tree structure in tact and put it all in lost+found
(2012), maybe?
It is StackExchange. So in theory someone could modernize it at any time.
Indeed and I just saw it was edited in 2012 - originally posed in 2011, answers last updated in 2014.
Yeah, it seems I improved a tiny tiny bit a couple of them.
glad to see that Stack Overflow (or stackexchange.com) is still a thing.
they offer private instances to school too, where moderation is left to school policy, and mine seems to be good enough to use whenever i have frontend questions (i'm 10 year into my career and still use my juniors to answer my frontend questions, i think i won't ever change)
I keep everything hidden there.
In reallife I would rename this to "trash".
lost+found is still used on OpenBSD, seems it is created when needed. Only /home has that directory on my system. IIRC, it was created when a kernel panic happened a few releases ago. Plus some files were placed in it when fsck executed on /home
Same, it's only in /home on my system also. Also /home is pretty much the only directory where I see fsck needing to do a lot of recovery after a power failure. Makes sense I guess, because that's where processes such as web browsers are likely to have lots of files open in RW mode.
[deleted]
lost+found is the Thumbs.db and .DS_Store of Linux
More like a FOUND.000 folder or a root directory filled with .CHK files
Not really, as it's only once per file system mount, whereas those Windows and MacOS files are sprinkled in most directories with images and almost every non-network drive directory respectively.
How do questions like this make it to the top? It is an obvious thing if you search for it or ask AI, but people seem to just ignore those in favor of generating new human responses.
Thing is, any time I try to replicate something like that, I basically get a flippant response saying to go look elsewhere.
Reaching HN or other sites allows for exposure to information that a person otherwise would not ask themselves.
I also respect human responses over AI ones every day that ends in Y.
I had a lost+found folder in all Unix file systems I used since the 80s. It's where fsck places files that it found during a scan and can't figure out to which directory they belong. Sometimes I found stuff in there.
From what I googled XFS, Btrfs and ZFS don't use lost+found. It's a thing of the old not journaled filesystems and of the ext family.
XFS does use /lost+found, it calls it the "orphanage directory" and xfs_repair reparents children of corrupt directories there.
Based on comments in the kernel source, it seems like the userspace fsck for JFS and F2FS will also sometimes create /lost+found. There might be more that do.
XFS filesystems do not have a "/lost+found" directory in their normal state.
In the very rare occasions when one has to run "xfs_repair", it will create a "/lost+found" directory, if it is required for recovered files.
After the repair and after investigating whether the recovered files contain useful data or not (and after moving the useful files elsewhere), one should normally delete the "/lost+found" directory, because it is no longer needed.
XFS as implemented in RHEL8+ (the only places i've used it in anger) tends to handle being full very badly, leading to system lockups and blocked tasks necessitating a hard reboot. Worse yet is when it's in this state the journal fills and nothing can be done with the volume.
To recover from this on a volume mounted at boot mandates going to either a live disk, or stopping boot in initramfs and running xfs_repair there, I've fruitlessly attempted to play back the journal on many separate occasions by attempting to mount the filesystem (again causing a lock up due to no space) in that state you have to drop the journal, run xfs_repair and then clean up the detritus from /lost+found (and then the location that caused the disk to fill altogether).
EXT4 has other issues certainly, but at least it reserves blocks for the root user explicitly so the system doesn't stop.
XFS tip:
If you truncate a file it doesn't update metadata. That's how you can get back space for the journal log to start cleaning up crap without rebooting the box and/or taking services down.
Even with journaling, you might need one. ZeroFS [0] almost had a lost+found directory (even with the WAL enabled), because you might have consistency issues between your in-memory state and what was flushed, and especially in what order.
ZeroFS ended up not needing recovery at all through atomic, strictly ordered commits [1], but it was far from trivial (and not just a matter of requiring a WAL).
[0] https://github.com/Barre/ZeroFS
[1] https://github.com/Barre/ZeroFS/blob/main/zerofs/src/fs/writ...
I have a book on my bookshelf, Eric Foxley's Unix for Super-Users. It was published in 1985, and it answers this question on page 52, the first page listed for the entry 'lost+found' in its index.
This is surely not the earliest book mention, is it? (It'll be in earlier man pages, of course.) Google Books does not give me an earlier one, although it does yield another 1985 book.
Fun fact: Foxley cautioned that lost+found must be pre-sized ahead of time, because the fsck of the time did not grow the directory to fit found files.
How do you size a directory? Just by creating entries in it and then unlinking them?
Usually the filesystem driver provides an API call that sets the appropriate data structures so that the directory entry table is preallocated.
The ext-family filesystems provide the mklost+found command to tap into this call if you need to recreate the lost+found directory specifically.
In a couple of decades running Linux installations of all flavours, I have never seen anything in lost+found!
Yea, run an old kernel with ext2 on a busy system writing a bunch of small files and have a power supply fail and you'll end up with something there.
fsck on large hard drives was scary on how long it could take to finish.
The occassional "Drive has not been checked in <n> days, forcing check" message on bootup got annoying sometimes, yeah. It could easily take tens of minutes to finish, exactly when I wanted to use the computer!
(At least this is what my memory is telling me. I could be mistaken, but that's what I remember.)
"check forced", as though the machine is advising me to go look at some daemon named forced.
Forced sounds like a systemd service for Jedi stuff.
I had a largish raid back then, under ext2, may have been a massive 40G.
I recall going to sleep and it still not being done when I woke up. Bleh.
One of several reasons to partition a large volume into smaller ones. At least, back then.
You need to use worse hardware and bad power :)
I used to develop SSD firmware and one of things I worked on is making it robust to power failure. The power supplies have lots of capacitance so the voltage drop was slow so we would use a special test board that would disconnect from power and discharge fast to test it.
Thank you for your service!
My main experience is with pre-scsi/ide systems :)
ESDI?
Yeah, I think lack of any decent hardware RAID could be a prerequisite.
And more concurrent writes.
But I think ext4 will only let things appear there if you change some default flags.
Umm .. how about a Raspberry Pi Zero 2W powered by a 2000MaH " lipstick style" powerbank?
That's what the answers are missing, of course. In some filesystem formats, it's possible either to recover completely from a journal/intent log, or at least to recover everything to the point that recovered files can be placed into the correct directory.
Same here. And I had some pretty f**ed up file systems.
At one point, I had one where the directory structure was completely broken and had circles in it (broken SSD). To be fair, in that particular case, I did not look for lost+found and just wrote a tool to extract the data manually that I was looking for.
My SD cards have always had stuff in that folder. It scares me. I try not to look
Have to run fsck. This used to be forced about once a month but don’t remember it happening in the last decade or so.
We once deleted the lost+found folder on an old Unix system* by accident. Things went very badly the next time the system rebooted, fsck did not handle it at all well.
* Probably DEC Ultrix 2.2, a BSD 4.2 derivative.
Why can't a filesystem create the lost+found folder only when it needs to store files in it?
That would be a much cleaner approach, imho.
Added benefit is that you'd immediately see it if something is wrong with a disk.
As the submission explains, the lost+found folder has pre-allocated space for the directory entries. From the mklost+found man page [0]:
Pre-allocating space without making the directory visible would require more arcane file system magic.[0] https://man7.org/linux/man-pages/man8/mklost+found.8.html
> Preallocating space without making the directory visible would require more arcane file system magic.
If those filesystem engineers had a manager that said: make this nice for the user, then it would have been done.
But these developers had no managers and were OK eating their own unpalatable dogfood.
Because you need to pre-allocate space and indies for it in case things go wrong and the disk is almost full
Back in the day I accidentally deleted all my stuff because I had it all in a special dir of this user in suse Linux. When I deleted the user, yast deleted everything.
Fortunately I was using ReiserFS at the time and something about its murderous tree data structure made it trivial to undelete.
Reiser_fsck found ALL my stuff, mostly with full dir tree structure in tact and put it all in lost+found
(2012), maybe?
It is StackExchange. So in theory someone could modernize it at any time.
Indeed and I just saw it was edited in 2012 - originally posed in 2011, answers last updated in 2014.
Yeah, it seems I improved a tiny tiny bit a couple of them.
glad to see that Stack Overflow (or stackexchange.com) is still a thing.
Won't be for long.
https://sherwood.news/tech/stack-overflow-forum-dead-thanks-...
https://news.ycombinator.com/item?id=48282709
It's a question from 2011...
they offer private instances to school too, where moderation is left to school policy, and mine seems to be good enough to use whenever i have frontend questions (i'm 10 year into my career and still use my juniors to answer my frontend questions, i think i won't ever change)
I keep everything hidden there.
In reallife I would rename this to "trash".
lost+found is still used on OpenBSD, seems it is created when needed. Only /home has that directory on my system. IIRC, it was created when a kernel panic happened a few releases ago. Plus some files were placed in it when fsck executed on /home
Same, it's only in /home on my system also. Also /home is pretty much the only directory where I see fsck needing to do a lot of recovery after a power failure. Makes sense I guess, because that's where processes such as web browsers are likely to have lots of files open in RW mode.
lost+found is the Thumbs.db and .DS_Store of Linux
More like a FOUND.000 folder or a root directory filled with .CHK files
Not really, as it's only once per file system mount, whereas those Windows and MacOS files are sprinkled in most directories with images and almost every non-network drive directory respectively.
How do questions like this make it to the top? It is an obvious thing if you search for it or ask AI, but people seem to just ignore those in favor of generating new human responses.
Thing is, any time I try to replicate something like that, I basically get a flippant response saying to go look elsewhere.
Reaching HN or other sites allows for exposure to information that a person otherwise would not ask themselves.
I also respect human responses over AI ones every day that ends in Y.
Why not on sundae?