This issue tracks the effort started in #17463

TODO list

• Incorporate CLI commands defined in https://github.com/Alluxio/alluxio/blob/main/dora/integration/fuse/bin/alluxio-fuse (from @LuQQiu in Build alluxio cli and start/stop scripts in golang #17463 (review))
• Handle expected user errors more gracefully, hiding the stacktrace by default (from @ssz1997 in Build alluxio cli and start/stop scripts in golang #17463 (comment))
• Define guidelines for the CLI command layout as inspired by https://primer.style/design/native/cli/foundations (from @jenoudet in Build alluxio cli and start/stop scripts in golang #17463 (comment))
• Use process registry to produce entrypoint for docker image, started in Implement entrypoint in go #17597
• Add tarball generation logic into build workflow
  • Handle changing the assembly jar path when the jars are reorganized within a tarball
  • Omit line in bin/cli.sh that builds the CLI
• Incorporate validation via struct field annotations as descsribed in Add quorum commands to CLI #17570 (comment)
• (nice to have) Mechanism to generate CLI documentation from golang code
• (nice to have) Generate alluxio-env.sh.template and validate in PR check that it is up to date

Background

The CLI in the bash format has grown to be a giant monolith of bash code. One of the goals in the new main branch is to properly modularize various portions of the codebase and the set of bash scripts within the bin/ folder is one obvious counterexample to this goal.

Advantages:

• Writing in a more structured language other than bash will prevent mistakes such as Fix Xmx check #17459
• Golang can be compiled into an executable binary such that there are no additional runtime requirements for Alluxio, as opposed to using an interpreted language such as python

Disadvantages

• Introduces golang as a new compilation/development requirement, which was previously a soft requirement as it was needed to run certain scripts such as building the tarball
• Compiled executables delivered as part of the tarball cannot be edited, unlike the bash scripts

Layout/Implementation

• All cli go code are in the cli/ directory.
  • If the -PgoCli flag is added when running maven, it will build the required go binary to execute.
  • If running the bin/cli.sh command and the go binary isn't built yet, it will build on demand
  • Otherwise, the go cli will not be built. This prevents golang to be a must-have requirement for building the project; will translate it to be a default after deprecating the existing bash cli.
• The user provided environment variables remains the same as before, via setting in conf/alluxio-env.sh.
• The environment variables are managed by viper
  • InitAlluxioEnv() in cli/env/env.go initializes the default values and reading from the above user config file, storing them in a global env.Env variable; effectively replacing libexec/alluxio-config.sh
  • Unlike in bash where variable substitution is rampant, all variables values are always stored in their fully expanded form
• The CLI command tree is managed by cobra
  • Logging with log levels is enabled by logrus
  • Adding --debugLog enables verbose logging to diagnose the logic flow and when appropriate, prints out the corresponding and fully expanded bash command
• The Process interface outlines the definition of an Alluxio process
  • The process defines its own environment variables and sets them in env.Env for subsequent logic to access
  • The process is added to the command tree under the process subcommand, where currently the only actionable command is Start()
  • The only current process defined is the Master process in cli/process/master.go
• The Command interface outlines the definition for running a java class
  • The command is added to the command tree under the root command, but could easily expand into its own subcommand tree following cobra's conventions
  • The BaseCommand struct is designated to be embedded in full implementations of the command interface
    • BaseCommand.RunJavaClassCmd() organizes the environment variables, java opts, and command line arguments, effectively replacing function runJavaClass in bin/alluxio
• Processes and commands are registered in cli/main.go, the entrypoint go file that is compiled to a binary.
  • Combined with the process and command interfaces, this code architecture promotes extensibility, allowing another go project to use this code as a dependency, picking which processes and commands to register as they choose.