Skip to main content
added 611 characters in body
Source Link
LSerni
  • 4.7k
  • 16
  • 20

For clarity's sake - the real mechanics is more complicated to give even better security - you can imagine the write-to-disk operation like this:

  • application writes bytes (1)
  • the kernel (and/or the file system IOSS) buffers them
  • once the buffer is full, it gets flushed to the file system:
    • the block is allocated (2)
    • the block is written (3)
    • the file and block information is updated (4)

If the process gets interrupted at (1), you don't get anything on the disk, the file is intact and truncated at the previous block. You sent 5000 bytes, only 4096 are on the disk, you restart transfer at offset 4096.

If at (2), nothing happens except in memory. Same as (1). If at (3), the data is written but nobody remembers about it. You sent 9000 bytes, 4096 got written, 4096 got written and lost, the rest just got lost. Transfer resumes at offset 4096.

If at (4), the data isshould now have been committed on disk. The next bytes in the stream may be lost. You sent 9000 bytes, 8192 get written, the rest is lost, transfer resumes at offset 8192.

This is a simplified take. For example, each "logical" write in stages 3-4 is not "atomic", but gives rise to another sequence (let's number it #5) whereby the block, subdivided into sub-blocks suitable for the destination device (e.g. hard disk) is sent to the device's host controller, which also has a caching mechanism, and finally stored on the magnetic platter. This sub-sequence is not always completely under the system's control, so having sent data to the hard disk is not a guarantee that it has been actually written and will be readable back.

Several file systems implement journaling, to make sure that the most vulnerable point, (4), is not actuallyactually vulnerable, by writing meta-data in, you guessed it, transactions that will work consistently whatever happens in stage (5). 

If the system gets reset in the middle of a transaction, it can resume its way to the earliernearest intact checkpointscheckpoint. Data written is still lost, same as case (1), but resumption will take care of that. No information actually gets lost.

For clarity's sake - the real mechanics is more complicated to give even better security - you can imagine the write-to-disk operation like this:

  • application writes bytes (1)
  • the kernel (and/or the file system IOSS) buffers them
  • once the buffer is full, it gets flushed to the file system:
    • the block is allocated (2)
    • the block is written (3)
    • the file and block information is updated (4)

If the process gets interrupted at (1), you don't get anything on the disk, the file is intact and truncated at the previous block. You sent 5000 bytes, only 4096 are on the disk, you restart transfer at offset 4096.

If at (2), nothing happens except in memory. Same as (1). If at (3), the data is written but nobody remembers about it. You sent 9000 bytes, 4096 got written, 4096 got written and lost, the rest just got lost. Transfer resumes at offset 4096.

If at (4), the data is now committed on disk. The next bytes in the stream may be lost. You sent 9000 bytes, 8192 get written, the rest is lost, transfer resumes at offset 8192.

This is a simplified take. Several file systems implement journaling to make sure that the most vulnerable point, (4), is not actually vulnerable, by writing meta-data in, you guessed it, transactions. If the system gets reset in the middle of a transaction, it can resume its way to the earlier intact checkpoints. Data written is still lost, same as case (1), but resumption will take care of that. No information actually gets lost.

For clarity's sake - the real mechanics is more complicated to give even better security - you can imagine the write-to-disk operation like this:

  • application writes bytes (1)
  • the kernel (and/or the file system IOSS) buffers them
  • once the buffer is full, it gets flushed to the file system:
    • the block is allocated (2)
    • the block is written (3)
    • the file and block information is updated (4)

If the process gets interrupted at (1), you don't get anything on the disk, the file is intact and truncated at the previous block. You sent 5000 bytes, only 4096 are on the disk, you restart transfer at offset 4096.

If at (2), nothing happens except in memory. Same as (1). If at (3), the data is written but nobody remembers about it. You sent 9000 bytes, 4096 got written, 4096 got written and lost, the rest just got lost. Transfer resumes at offset 4096.

If at (4), the data should now have been committed on disk. The next bytes in the stream may be lost. You sent 9000 bytes, 8192 get written, the rest is lost, transfer resumes at offset 8192.

This is a simplified take. For example, each "logical" write in stages 3-4 is not "atomic", but gives rise to another sequence (let's number it #5) whereby the block, subdivided into sub-blocks suitable for the destination device (e.g. hard disk) is sent to the device's host controller, which also has a caching mechanism, and finally stored on the magnetic platter. This sub-sequence is not always completely under the system's control, so having sent data to the hard disk is not a guarantee that it has been actually written and will be readable back.

Several file systems implement journaling, to make sure that the most vulnerable point, (4), is not actually vulnerable, by writing meta-data in, you guessed it, transactions that will work consistently whatever happens in stage (5). 

If the system gets reset in the middle of a transaction, it can resume its way to the nearest intact checkpoint. Data written is still lost, same as case (1), but resumption will take care of that. No information actually gets lost.

Source Link
LSerni
  • 4.7k
  • 16
  • 20

For clarity's sake - the real mechanics is more complicated to give even better security - you can imagine the write-to-disk operation like this:

  • application writes bytes (1)
  • the kernel (and/or the file system IOSS) buffers them
  • once the buffer is full, it gets flushed to the file system:
    • the block is allocated (2)
    • the block is written (3)
    • the file and block information is updated (4)

If the process gets interrupted at (1), you don't get anything on the disk, the file is intact and truncated at the previous block. You sent 5000 bytes, only 4096 are on the disk, you restart transfer at offset 4096.

If at (2), nothing happens except in memory. Same as (1). If at (3), the data is written but nobody remembers about it. You sent 9000 bytes, 4096 got written, 4096 got written and lost, the rest just got lost. Transfer resumes at offset 4096.

If at (4), the data is now committed on disk. The next bytes in the stream may be lost. You sent 9000 bytes, 8192 get written, the rest is lost, transfer resumes at offset 8192.

This is a simplified take. Several file systems implement journaling to make sure that the most vulnerable point, (4), is not actually vulnerable, by writing meta-data in, you guessed it, transactions. If the system gets reset in the middle of a transaction, it can resume its way to the earlier intact checkpoints. Data written is still lost, same as case (1), but resumption will take care of that. No information actually gets lost.