[PoC] Disintermediate running end-to-end tests

[achamayou]

Headline: no ./tests.sh, no ctest for end to end tests. ctest stays for unit tests. More amenable to things like #7709. Also incorporates ideas from #7713.

# uv run commit_latency.py 18:43:53.082 | INFO | infra.e2e_args:cli_args:403 - Loaded default args from ../build/e2e_config.json 18:43:53.083 | INFO | __main__:run:119 - Test: Measure commit latency {'sig_interval': 1} 18:43:53.084 | INFO | infra.network:_setup_common_folder:549 - Creating common folder: /workspaces/CCF/tests/workspace/commit_latency_common 18:43:53.084 | INFO | infra.proc:ccall:11 - rm -rf /workspaces/CCF/tests/workspace/commit_latency_common 18:43:53.087 | INFO | infra.proc:ccall:11 - mkdir -p /workspaces/CCF/tests/workspace/commit_latency_common 18:43:53.088 | INFO | infra.proc:ccall:11 - cp /workspaces/CCF/samples/constitutions/default/actions.js /workspaces/CCF/tests/workspace/commit_latency_common 18:43:53.092 | INFO | infra.proc:ccall:11 - cp /workspaces/CCF/samples/constitutions/default/validate.js /workspaces/CCF/tests/workspace/commit_latency_common 18:43:53.093 | INFO | infra.proc:ccall:11 - cp /workspaces/CCF/samples/constitutions/default/resolve.js /workspaces/CCF/tests/workspace/commit_latency_common 18:43:53.095 | INFO | infra.proc:ccall:11 - cp /workspaces/CCF/samples/constitutions/default/apply.js /workspaces/CCF/tests/workspace/commit_latency_common ... 

[Copilot]

Pull request overview

This PoC adjusts the Python e2e test infrastructure to run more directly (eg via hatch run ...) without relying on ./tests.sh/ctest, primarily by making binary path resolution work when the binaries live outside the current working directory.

Changes:

• Resolve the “combined binary” entry-point via binary_dir in CCFRemote and avoid copying the same file twice for 7.x.
• Thread binary_dir through initial node creation in Network.
• Change the default --binary-dir CLI argument to ../build.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
tests/infra/remote.py	Adjusts how the executed binary path is derived for 7.x and how startup files are packaged for remotes.
tests/infra/network.py	Ensures binary_dir is explicitly forwarded when creating nodes.
tests/infra/e2e_args.py	Updates the default binary directory to match a “run from tests/” workflow.

---

You can also share your feedback on Copilot code review. Take the survey.

[Copilot]

The decision to override self.BIN based solely on major_version >= 7 conflicts with the later version-based CLI logic which still adds --enclave-file for versions <= 7.0.0-dev1. For tags like ccf-7.0.0-dev0/dev1 (see CHANGELOG), this would make the node try to execute enclave_file as the host binary even though these versions still require the cchost entry-point plus --enclave-file. Consider selecting the entry-point based on the parsed full version (the same threshold used for the --enclave-file flag) rather than major_version alone, so 7.0.0-dev0/dev1 keep using cchost.

Suggested change

	# 7.x releases combined binaries and removed the separate cchost entry-point
	if major_version is None or major_version >= 7:
	# 7.x releases combined binaries and removed the separate cchost entry-point.
	# Use the same full-version threshold as the CLI logic which drops --enclave-file:
	# only versions strictly greater than 7.0.0-dev1 use the combined binary as entry-point.
	if "version" in locals() and version is not None:
	if version > Version("7.0.0-dev1"):
	self.BIN = infra.path.build_bin_path(
	enclave_file, binary_dir=binary_dir
	)
	elif major_version is None or major_version >= 7:
	# Fallback for callers which only provide major_version

[Copilot]

Changing the default --binary-dir to ../build makes the CLI default dependent on the caller's current working directory being tests/ (or similar). This is likely to break existing workflows that invoke these scripts from the build directory (where -b . used to work without specifying it) or from an installed tree. Consider deriving the default from the script location and/or probing common candidate directories (e.g. prefer . if it contains keygenerator.sh, else fall back), rather than hard-coding a relative path.

[cjen1-msft]

What does Hatch give us more than a wrapper around pytest?

[achamayou]

What does Hatch give us more than a wrapper around pytest?

For our purposes, it would replace tests.sh, it's creating the venv if needed and setting it up with the dependencies. It looks like it also does exception pretty printing as well.