CI interface

[eddiebergman]

Template:

neps run --config <path_to_run_args>

• Starts NEPS with an optional argument for the configuration file path if the user has not used the init command, which sets the default path.

---

neps init --path </path/to/project> --template <template_name> --interactive

• Initializes a new NEPS project, creating the necessary directories and yaml configuration files.
• To prevent accidentally overwriting an existing project, the command should check if the target directory already exists and prompt the user for confirmation before proceeding.
• Optionally, you could add template options or interactive questions to customize the initialization process.

---

neps info --config <config_id>

• Find out about the status of an individual configuration that is in NePS file-state.
  • [Kind of machine-readble]

status: finished configuration: batch_size: 64 epochs: 1 learning_rate: 0.050129220995556854 optimizer: Adam metadata: time_end: 1718721385.6890116 time_sampled: 1718721385.6864812 result: loss: -683.5864767923081 

---

neps status

Problem: Overlap with neps.status

• Find out about the status of what the file-state is like
  • [Human readable]

NEPS Project Status ==================== Overall Status: --------------- Project Name: my_project Total Configurations: 150 Completed Configurations: 120 Running Configurations: 25 Pending Configurations: 5 Failed Configuration: 0 Individual Configuration Status: -------------------------------- | Config ID | Status | Start Time | End Time | |-----------|----------|--------------------|--------------------| | 001 | Finished | 2023-07-08 08:30:00| 2023-07-08 09:00:00| | 002 | Running | 2023-07-08 09:10:00| - | | 003 | Pending | - | - | | ... | ... | ... | ... | Best Configurations: -------------------- | Loss | Config ID | Config | |----------------------|-----------|----------------------------------------------------| | 64.05012922099556 | 0_0 | {'batch_size': 64, 'epochs': 1, 'learning_rate': 0.050129220995556854, 'optimizer': 'a'} | | 65.23039842094534 | 1_2 | {'batch_size': 32, 'epochs': 2, 'learning_rate': 0.03019872084521212, 'optimizer': 'b'} | | 66.09832857328394 | 3_5 | {'batch_size': 16, 'epochs': 3, 'learning_rate': 0.02543827348217383, 'optimizer': 'c'

---

neps help

• Overview of all commands and options

Usage: neps [COMMAND] [OPTIONS] Available Commands: ------------------- neps init --path </path/to/project> --template <template_name> --interactive Initializes a new NEPS project, creating the necessary directories and YAML configuration files. To prevent accidentally overwriting an existing project, the command checks if the target directory already exists and prompts the user for confirmation before proceeding. Optionally, you can add template options or interactive questions to customize the initialization process. Example: ```bash neps init --path /my_project --interactive ``` neps run --config <path_to_run_args> Starts NEPS with an optional argument for the configuration file path if the user has not used the init command, which sets the default path. Example: ```bash neps run --config /path/to/config.yaml ``` neps status Find out about the status of what the file-state is like. Example: ```bash neps status ``` 

---

neps sample --config <path_to_yaml>

• add a specific config that you want to be evaluated next

---

neps monitor --path <path/to/root_directory>

• could embed tensorboard --logdir path/to/root_directory
• implement custom monitoring features, such as plotting the trajectory of incumbents like neps.plot
• provided an overview of the covered search areas

---

neps summary

• Calls post-summary if not already done

---

neps add_evaluations <int>

• Adds a specified number of evaluations to the running NEPS job.

---

neps config-check --config <path_to_run_args.yaml>

• validates the run_args file

---