Fix corrupted block causing reads to fail#17564
Merged
alluxio-bot merged 8 commits intoAlluxio:master-2.xfrom Jun 12, 2023
Merged
Fix corrupted block causing reads to fail#17564alluxio-bot merged 8 commits intoAlluxio:master-2.xfrom
alluxio-bot merged 8 commits intoAlluxio:master-2.xfrom
Conversation
Contributor Author
| To detect corrupted blocks, two extra file system calls |
beinan approved these changes Jun 8, 2023
Contributor
beinan left a comment
beinan left a comment There was a problem hiding this comment.
looks good to me though there might be very minor performance impact on the read path
Contributor Author
| alluxio-bot, merge this please. |
Contributor
| merge failed: |
Contributor Author
| alluxio-bot, merge this please |
Contributor Author
| alluxio-bot, cherry-pick this branch 2.10 please |
Contributor Author
| alluxio-bot, cherry-pick this to branch-2.10 please |
Contributor
| Auto cherry-pick to branch |
alluxio-bot pushed a commit that referenced this pull request Jun 12, 2023
### What changes are proposed in this pull request? Fix read failure when a mismatch occurs between the block size recorded in memory by `BlockMeta` and the length of the actual physical block file. When such a mismatch is detected, the block is removed from worker storage, and worker falls back to reading from UFS. ### Why are the changes needed? This causes reading the block to fail. ### Does this PR introduce any user facing changes? No. pr-link: #17564 change-id: cid-2758ff97e5016c1aae7ededb244e919233ae6d3b
Contributor Author
| alluxio-bot, cherry-pick this to branch-2.9 please |
Contributor
| Auto cherry-pick unsuccessful Failed to setup local git for auto cherry-pick |
Contributor Author
| alluxio-bot, cherry-pick this to branch-2.9 please |
Contributor
| Auto cherry-pick unsuccessful Failed to setup local git for auto cherry-pick |
jiacheliu3 pushed a commit to jiacheliu3/alluxio that referenced this pull request Jun 30, 2023
…reads to fail Fix read failure when a mismatch occurs between the block size recorded in memory by `BlockMeta` and the length of the actual physical block file. When such a mismatch is detected, the block is removed from worker storage, and worker falls back to reading from UFS. This causes reading the block to fail. No. pr-link: Alluxio#17564 change-id: cid-2758ff97e5016c1aae7ededb244e919233ae6d3b
jiacheliu3 pushed a commit to jiacheliu3/alluxio that referenced this pull request Jul 11, 2023
…reads to fail Fix read failure when a mismatch occurs between the block size recorded in memory by `BlockMeta` and the length of the actual physical block file. When such a mismatch is detected, the block is removed from worker storage, and worker falls back to reading from UFS. This causes reading the block to fail. No. pr-link: Alluxio#17564 change-id: cid-2758ff97e5016c1aae7ededb244e919233ae6d3b
dbw9580 added a commit to jiacheliu3/alluxio that referenced this pull request Jul 12, 2023
…reads to fail Fix read failure when a mismatch occurs between the block size recorded in memory by `BlockMeta` and the length of the actual physical block file. When such a mismatch is detected, the block is removed from worker storage, and worker falls back to reading from UFS. This causes reading the block to fail. No. pr-link: Alluxio#17564 change-id: cid-2758ff97e5016c1aae7ededb244e919233ae6d3b
jiacheliu3 pushed a commit to jiacheliu3/alluxio that referenced this pull request Jul 13, 2023
…reads to fail Fix read failure when a mismatch occurs between the block size recorded in memory by `BlockMeta` and the length of the actual physical block file. When such a mismatch is detected, the block is removed from worker storage, and worker falls back to reading from UFS. This causes reading the block to fail. No. pr-link: Alluxio#17564 change-id: cid-2758ff97e5016c1aae7ededb244e919233ae6d3b
maobaolong pushed a commit to maobaolong/alluxio that referenced this pull request Jan 3, 2024
### What changes are proposed in this pull request? Fix read failure when a mismatch occurs between the block size recorded in memory by `BlockMeta` and the length of the actual physical block file. When such a mismatch is detected, the block is removed from worker storage, and worker falls back to reading from UFS. ### Why are the changes needed? This causes reading the block to fail. ### Does this PR introduce any user facing changes? No. pr-link: Alluxio#17564 change-id: cid-2758ff97e5016c1aae7ededb244e919233ae6d3b
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
Fix read failure when a mismatch occurs between the block size recorded in memory by
BlockMetaand the length of the actual physical block file. When such a mismatch is detected, the block is removed from worker storage, and worker falls back to reading from UFS.Why are the changes needed?
This causes reading the block to fail.
Does this PR introduce any user facing changes?
No.