2

I am testing S3 command line tool performances at the moment and I am testing them using files I have created with fallocate. Im measuring the time different tools take to upload, say, 200 files of 1MB each to the S3 storage. Is there a difference between the files I have created with fallocate compared to a different file I have created myself, for example a .txt document of 1MB size? Or is a test.txt file created with fallocate the exact same as a test.txt created with notepad?

I want to make sure that the tests I make are realistic and not just fast because fallocate creates a different type of file.

Improve this question

1 Answer 1

Reset to default
2

Depending on the tools used, and the filesystem used, there may be differences.

From the application point of view, fallocate created files are usually full of zeros; your measuring times will be skewed and they will be even more affected if you are using either protocols or filesystems that have compression or deduplication built-in, as the amount of data transferred/written won't ever reflect the same operations with an equivalent file of the same size created by other means.

Also as Stéphane Chazelas correctly points out, the files appear as containing 0s because the extents are marked as "unwritten". When reading those, the system returns 0s without even reading the disk.

It would be preferred either to use .txt files, or better yet files generated with random binary data.

Improve this answer
1
  • 1
    Also, the files appear as containing 0s because the extents are marked as "unwritten". When reading those, the system returns 0s without even reading the disk. Commented May 3, 2017 at 9:49

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.