Skip to content

Remove the usage of must_cache and async_cache#17963

Merged
alluxio-bot merged 15 commits intoAlluxio:mainfrom
Jackson-Wang-7:remove_must_cache
Aug 16, 2023
Merged

Remove the usage of must_cache and async_cache#17963
alluxio-bot merged 15 commits intoAlluxio:mainfrom
Jackson-Wang-7:remove_must_cache

Conversation

@Jackson-Wang-7
Copy link
Copy Markdown
Contributor

@Jackson-Wang-7 Jackson-Wang-7 commented Aug 10, 2023

What changes are proposed in this pull request?

current version didn't support must_cache and async_cache, so remove the usage of must_cache and async_cache usage in code and mark deprecated to prevent any use.

Why are the changes needed?

This change focuses on removing the write type of MUST_CACHE and ASYNC_CACHE, and #17980 would remove the master-side code about async persist.

Does this PR introduce any user facing changes?

No

@alluxio-bot
Copy link
Copy Markdown
Contributor

Automated checks report:

  • PR title follows the conventions: FAIL
    • The title of the PR does not pass all the checks. Please fix the following issues:
      • First word must be capitalized
  • Commits associated with Github account: PASS

Some checks failed. Please fix the reported issues and reply 'alluxio-bot, check this please' to re-run checks.

@Jackson-Wang-7 Jackson-Wang-7 changed the title remove must_cache and async_cache Remove must_cache and async_cache Aug 10, 2023
@alluxio-bot
Copy link
Copy Markdown
Contributor

Automated checks report:

  • PR title follows the conventions: PASS
  • Commits associated with Github account: PASS

All checks passed!

@Jackson-Wang-7 Jackson-Wang-7 changed the title Remove must_cache and async_cache Remove the usage of must_cache and async_cache Aug 10, 2023
@yyongycy
Copy link
Copy Markdown
Contributor

Please also have a simple test to make sure it works.

Copy link
Copy Markdown
Contributor

@yyongycy yyongycy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@Test
public void checksum() throws Exception {
FileSystemTestUtils.createByteFile(sFileSystem, "/testFile", WritePType.MUST_CACHE, 10);
FileSystemTestUtils.createByteFile(sFileSystem, "/testFile", WritePType.CACHE_THROUGH, 10);
Copy link
Copy Markdown
Contributor

@jiacheliu3 jiacheliu3 Aug 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this kind of manually set write types, what do you think about declaring a shared var?

class ChecksumCommandIntegrationTest { public static final WritePType DEFAULT_WRITE_TYPE = WritePType.CACHE_THROUGH; // keep using this constant throughout your test } 

You don't save any code in this PR, but you may save a lot in later PRs (if you tweak the default write type).
All these test class files may share the same DEFAULT_WRITE_TYPE if you find a good place to declare this constant like some existing TestUtils.java or XXXTestBase.java?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jiacheliu3 I'm not sure if we expect a default value for the different unit tests. For the general default value, we only need to set DEFAULT_WRITE_TYPE in the cluster resouce configuration in some based test class.
Except that, I saw the most cases is it wants to excute a specific opration with a specific write type. we can not easily replace them with a DEFAULT_WRITE_TYPE in general.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. In that case feel free to use a local DEFAULT_WRITE_TYPE if you feel we might wanna change the write type in tests again. Otherwise no need to bother :)

� Conflicts: �	dora/tests/src/test/java/alluxio/client/cli/fs/command/DuCommandIntegrationTest.java
� Conflicts: �	dora/tests/src/test/java/alluxio/client/cli/fs/command/PersistCommandTest.java �	dora/tests/src/test/java/alluxio/client/cli/fs/command/PinCommandIntegrationTest.java �	dora/tests/src/test/java/alluxio/client/cli/fs/command/SetFaclCommandIntegrationTest.java �	dora/tests/src/test/java/alluxio/client/cli/fs/command/SetTtlCommandIntegrationTest.java
� Conflicts: �	dora/stress/shell/src/main/java/alluxio/stress/cli/MaxFileBench.java �	dora/tests/src/test/java/alluxio/client/cli/fs/command/CountCommandTest.java
Copy link
Copy Markdown
Contributor

@jiacheliu3 jiacheliu3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with one minor comment, thanks for the work!

*/
public final class ReportCommandIntegrationTest extends AbstractFsAdminShellTest {
@Test
@Ignore
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add @DoraTestTodoItem?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@Jackson-Wang-7 Jackson-Wang-7 added type-feature This issue is a feature request type-debt This issue is about tech debt and removed type-feature This issue is a feature request labels Aug 15, 2023
� Conflicts: �	dora/tests/integration/src/main/java/alluxio/master/backcompat/ops/AsyncPersist.java �	dora/tests/integration/src/test/java/alluxio/client/fs/io/FileOutStreamAsyncWriteIntegrationTest.java �	dora/tests/integration/src/test/java/alluxio/client/fs/io/FileOutStreamAsyncWriteJobIntegrationTest.java
@Jackson-Wang-7
Copy link
Copy Markdown
Contributor Author

alluxio-bot, merge this please

@alluxio-bot alluxio-bot merged commit 6dc8ed2 into Alluxio:main Aug 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type-debt This issue is about tech debt

4 participants