Skip to content

Fix S3 UFS executor service thread leak #15748

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alluxio-bot merged 1 commit into from
Jun 16, 2023
Merged

Fix S3 UFS executor service thread leak #15748

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions underfs/s3a/src/main/java/alluxio/underfs/s3a/S3AUnderFileSystem.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -361,6 +361,11 @@ public void cleanup() {
mManager.abortMultipartUploads(mBucketName, cleanBefore);
}

@Override
public void close() {
mExecutor.shutdown();
Copy link
Copy Markdown
Contributor

@yuzhu yuzhu Jun 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just want to make sure this mExecutor is not passed in or shared right?
Copy link
Copy Markdown
Contributor Author

@tcrain tcrain Jun 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is used by the multipart upload. This is a safe shutdown so it will wait for any live tasks that have not finished, but any upload objects that have been created but not yet started will fail.

Actually it looks like the UFS class isn't actually be closed in all places where it is created either, so there might be other things to fix as well.
}

@Override
protected boolean copyObject(String src, String dst) {
LOG.debug("Copying {} to {}", src, dst);
Expand Down