Skip to content

Conversation

@VietND96
Copy link
Member

@VietND96 VietND96 commented Oct 31, 2025

User description

Thanks for contributing to the Docker-Selenium project!
A PR well described will help maintainers to quickly review and merge it

Before submitting your PR, please check our contributing guidelines, applied for this repository.
Avoid large PRs, help reviewers by making them as simple and short as possible.

Description

Fixes #3013


Chrome for Testing Support

Overview

The chrome-for-testing branch extends the Docker Selenium images to ship Google Chrome for Testing (CFT) alongside the existing stable Chrome builds. CFT provides reproducible binaries that align with ChromeDriver versions, and the branch introduces everything required to build, tag, test, and release those images through the existing automation.

Image Build Changes

  • NodeChrome/Dockerfile now accepts INSTALL_CFT and CFT_VERSION build arguments and emits a CHROME_FOR_TESTING flag so runtime consumers can detect when the CFT binary is present.
  • A new installer (NodeChrome/install-chrome-for-testing.sh) pulls the requested version from chrome-for-testing-public, installs its dependencies from the bundled deb.deps manifest, and symlinks the binary to /usr/bin/google-chrome.
  • The Makefile gains targets to build, push, and test node and standalone variants (chrome-for-testing, standalone_chrome-for-testing, plus dev, beta, and canary channels). The targets are guarded so they only run on linux/amd64, which is the sole platform CFT currently supports.

Release and Tagging Flow

  • tag_and_push_browser_images.sh understands a new chrome-for-testing browser type and produces tags based on both the long and short Chrome/ChromeDriver version pairs as well as the build date and Grid revision.
  • generate_release_notes.sh now records the installed CFT version next to the other browser artifacts when preparing release notes.
  • Makefile release recipes push the new images for every tagging cadence (latest, semver, and legacy tags).

Version Management Automation

  • tests/build-backward-compatible/browser-matrix.yml carries CFT_VERSION entries per Chrome major version so the automation can map Grid releases to the correct browser payload.
  • tests/build-backward-compatible/fetch_chrome_for_testing_version.py populates those entries from known-good-versions.json, filtered to stable line releases.
  • tests/build-backward-compatible/update_workflow_versions.py now collects CFT majors, writes them into workflow matrices, and keeps the most recent versions to ensure CFT builds are kept up to date.
  • GitHub Actions gained a dedicated workflow (.github/workflows/release-chrome-for-testing-versions.yml) to build arbitrary CFT majors, while the existing Chrome/Edge/Firefox workflows were extended with the latest major defaults. The dev/beta updater workflow also schedules CFT dev, beta, and canary channels.

Test Coverage

  • tests/test.py maps the new image names so existing Selenium smoke suites exercise the CFT variants.
  • Makefile includes test_chrome-for-testing* targets that reuse the grid bootstrap harness for node, standalone, and Java bindings.
  • tests/SeleniumTests/__init__.py now guards against an empty download list before asserting on managed downloads, preventing flakiness introduced by CFT runs.

Usage Notes

  1. Build node and standalone images with make chrome-for-testing or make standalone_chrome-for-testing; append _dev, _beta, or _canary for pre-release channels.
  2. Run make test_chrome-for-testing (and the standalone variants) to validate the images locally.
  3. Use the Deploy specific Chrome for Testing version workflow when releasing historical majors; it bootstraps the build, runs the Selenium test suites, and prepares changelog artifacts.
  4. The standard release scripts (make update_browser_versions_matrix, make tag_and_push_browser_images, and generate_release_notes.sh) now cover Chrome for Testing automatically, so downstream processes pick up the new artifacts without further changes.

Motivation and Context

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • I have read the contributing document.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

PR Type

Enhancement


Description

  • Add Chrome for Testing (CFT) Docker image support

  • Create installation script for Chrome for Testing browser

  • Integrate CFT into build, test, and release workflows

  • Update browser matrix with CFT versions for all supported Chrome versions


Diagram Walkthrough

flowchart LR CFT["Chrome for Testing<br/>Installation"] --> NodeCFT["Node CFT<br/>Docker Image"] CFT --> StandaloneCFT["Standalone CFT<br/>Docker Image"] NodeCFT --> BuildWorkflow["Build Workflow<br/>release-chrome-for-testing-versions.yml"] StandaloneCFT --> BuildWorkflow BuildWorkflow --> TagPush["Tag and Push<br/>Images"] BrowserMatrix["Browser Matrix<br/>CFT Versions"] --> BuildWorkflow TestSuite["Test Suite<br/>Node/Standalone CFT"] --> BuildWorkflow 
Loading

File Walkthrough

Relevant files
Enhancement
11 files
install-chrome-for-testing.sh
New Chrome for Testing installation script                             
+92/-0   
Dockerfile
Add CFT build arguments and conditional installation         
+22/-5   
Makefile
Add CFT build targets and test commands                                   
+89/-0   
tag_and_push_browser_images.sh
Add CFT image tagging and push logic                                         
+50/-0   
generate_release_notes.sh
Include CFT version in release notes                                         
+2/-0     
release-chrome-for-testing-versions.yml
New workflow for CFT version releases                                       
+168/-0 
update-dev-beta-browser-images.yml
Add CFT dev/beta/canary channel builds                                     
+19/-2   
fetch_chrome_for_testing_version.py
New script to fetch CFT versions from API                               
+70/-0   
update_workflow_versions.py
Add CFT version filtering and workflow updates                     
+33/-9   
bootstrap.sh
Add CFT build and tag logic to bootstrap                                 
+21/-0   
test.py
Add CFT image mappings to test configuration                         
+4/-0     
Configuration changes
4 files
release-chrome-versions.yml
Update Chrome version range to 142                                             
+1/-1     
release-firefox-versions.yml
Update Firefox version range to 144                                           
+1/-1     
release-edge-versions.yml
Update Edge version range to 141                                                 
+1/-1     
browser-matrix.yml
Add CFT versions for all Chrome major versions                     
+30/-0   
Bug fix
1 files
__init__.py
Fix download file test race condition                                       
+1/-1     

@qodo-merge-pro
Copy link
Contributor

qodo-merge-pro bot commented Oct 31, 2025

PR Compliance Guide 🔍

(Compliance updated until commit 0f25a58)

Below is a summary of compliance checks for this PR:

Security Compliance
🟢
No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
🎫 No ticket provided
  • Create ticket/issue
Codebase Duplication Compliance
Codebase context is not defined

Follow the guide to enable codebase context checks.

Custom Compliance
🟢
Generic: Meaningful Naming and Self-Documenting Code

Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting

Status: Passed

Generic: Secure Error Handling

Objective: To prevent the leakage of sensitive system information through error messages while
providing sufficient detail for internal debugging.

Status: Passed

Generic: Secure Logging Practices

Objective: To ensure logs are useful for debugging and auditing without exposing sensitive
information like PII, PHI, or cardholder data.

Status: Passed

Generic: Comprehensive Audit Trails

Objective: To create a detailed and reliable record of critical system actions for security analysis
and compliance.

Status:
Action Logging: The installation and tagging scripts echo progress but do not produce structured audit
logs for critical actions like downloads, installs, and pushes, which may be acceptable
for build scripts but lacks explicit audit trail guarantees.

Referred Code
echo "Installing Chrome for Testing: ${CFT_VERSION} (${CFT_PLATFORM})" # Create temporary directory TEMP_DIR=$(mktemp -d) cd "${TEMP_DIR}" # Download Chrome for Testing DOWNLOAD_URL="${CFT_BASE_URL}/${CFT_VERSION}/${CFT_PLATFORM}/chrome-${CFT_PLATFORM}.zip" echo "Downloading from: ${DOWNLOAD_URL}" if ! wget -q --spider "${DOWNLOAD_URL}"; then echo "Error: Chrome for Testing version ${CFT_VERSION} not found for platform ${CFT_PLATFORM}" echo "Please check available versions at: https://googlechromelabs.github.io/chrome-for-testing/" rm -rf "${TEMP_DIR}" exit 1 fi wget -q -O chrome.zip "${DOWNLOAD_URL}" # Extract Chrome echo "Extracting Chrome for Testing..." ... (clipped 43 lines)
Generic: Robust Error Handling and Edge Case Management

Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation

Status:
Edge Cases: While the script validates channel resolution and download existence, dependency parsing
and installation from deb.deps lack explicit error handling for apt failures or empty
dependency lists which could lead to partial installs without clear recovery.

Referred Code
# Install Chrome dependencies from deb.deps file echo "Installing Chrome dependencies..." apt-get update -qqy if [ -f "${INSTALL_DIR}/deb.deps" ]; then echo "Found deb.deps file, parsing dependencies..." # Read dependencies from deb.deps file # Format: package-name (>= version) or package1 | package2 | package3 # We need to: # 1. Remove version constraints in parentheses # 2. Handle alternative packages (take the first one) # 3. Remove empty lines and comments DEPS=$(cat "${INSTALL_DIR}/deb.deps" |  grep -v '^#' |  grep -v '^$' |  sed 's/ *([^)]*)//g' |  sed 's/ *|.*//' |  tr '\n' ' ' |  sed 's/ */ /g' |  sed 's/^ *//;s/ *$//' |  sed 's/libasound2\b/libasound2t64/g') ... (clipped 3 lines)
Generic: Security-First Input Validation and Data Handling

Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent
vulnerabilities

Status:
Input Validation: The script fetches remote JSON and writes to YAML without explicit timeouts, retries, or
schema validation which could introduce instability or malformed data handling though it
does check HTTP status codes.

Referred Code
stable_url = 'https://googlechromelabs.github.io/chrome-for-testing/LATEST_RELEASE_STABLE' stable_resp = requests.get(stable_url) stable_resp.raise_for_status() max_stable_version = stable_resp.text.strip() print(f"Latest stable version: {max_stable_version}") # Parse max stable version for comparison max_stable_parts = list(map(int, max_stable_version.split('.'))) url = 'https://googlechromelabs.github.io/chrome-for-testing/known-good-versions.json' resp = requests.get(url) resp.raise_for_status() data = resp.json() # Extract versions from the JSON, filtering out versions higher than stable versions = [] for item in data.get('versions', []): version = item.get('version') if version: version_parts = list(map(int, version.split('.'))) # Only include versions <= max stable version (avoid dev/beta/canary) ... (clipped 35 lines)
Compliance status legend 🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

Previous compliance checks

Compliance check up to commit a97b3b7
Security Compliance
Unpinned package install

Description: The script installs APT packages derived from a downloaded deb.deps file without pinning
or verification, enabling potential supply-chain compromise if the upstream artifact is
tampered with.
install-chrome-for-testing.sh [63-84]

Referred Code
apt-get update -qqy if [ -f "${INSTALL_DIR}/deb.deps" ]; then echo "Found deb.deps file, parsing dependencies..." # Read dependencies from deb.deps file # Format: package-name (>= version) or package1 | package2 | package3 # We need to: # 1. Remove version constraints in parentheses # 2. Handle alternative packages (take the first one) # 3. Remove empty lines and comments DEPS=$(cat "${INSTALL_DIR}/deb.deps" |  grep -v '^#' |  grep -v '^$' |  sed 's/ *([^)]*)//g' |  sed 's/ *|.*//' |  tr '\n' ' ' |  sed 's/ */ /g' |  sed 's/^ *//;s/ *$//' |  sed 's/libasound2\b/libasound2t64/g') echo "Dependencies: ${DEPS}" apt-get install -qqy --no-install-recommends ${DEPS} ... (clipped 1 lines)
Unvalidated external fetch

Description: The script fetches JSON and text over HTTPS from third-party endpoints without timeouts or
response validation and writes directly to a local YAML file, which could be abused in CI
by network manipulation to inject unexpected matrix values.
fetch_chrome_for_testing_version.py [12-67]

Referred Code
stable_url = 'https://googlechromelabs.github.io/chrome-for-testing/LATEST_RELEASE_STABLE' stable_resp = requests.get(stable_url) stable_resp.raise_for_status() max_stable_version = stable_resp.text.strip() print(f"Latest stable version: {max_stable_version}") # Parse max stable version for comparison max_stable_parts = list(map(int, max_stable_version.split('.'))) url = 'https://googlechromelabs.github.io/chrome-for-testing/known-good-versions.json' resp = requests.get(url) resp.raise_for_status() data = resp.json() # Extract versions from the JSON, filtering out versions higher than stable versions = [] for item in data.get('versions', []): version = item.get('version') if version: version_parts = list(map(int, version.split('.'))) # Only include versions <= max stable version (avoid dev/beta/canary) ... (clipped 35 lines)
Ticket Compliance
🎫 No ticket provided
  • Create ticket/issue
Codebase Duplication Compliance
Codebase context is not defined

Follow the guide to enable codebase context checks.

Custom Compliance
🟢
Generic: Meaningful Naming and Self-Documenting Code

Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting

Status: Passed

Generic: Secure Error Handling

Objective: To prevent the leakage of sensitive system information through error messages while
providing sufficient detail for internal debugging.

Status: Passed

Generic: Secure Logging Practices

Objective: To ensure logs are useful for debugging and auditing without exposing sensitive
information like PII, PHI, or cardholder data.

Status: Passed

Generic: Comprehensive Audit Trails

Objective: To create a detailed and reliable record of critical system actions for security analysis
and compliance.

Status:
Missing Audit Logs: The installation script performs network downloads, installs packages, and modifies system
paths without emitting structured audit logs of actions, user context, or outcomes.

Referred Code
echo "Installing Chrome for Testing: ${CFT_VERSION} (${CFT_PLATFORM})" # Create temporary directory TEMP_DIR=$(mktemp -d) cd "${TEMP_DIR}" # Download Chrome for Testing DOWNLOAD_URL="${CFT_BASE_URL}/${CFT_VERSION}/${CFT_PLATFORM}/chrome-${CFT_PLATFORM}.zip" echo "Downloading from: ${DOWNLOAD_URL}" if ! wget -q --spider "${DOWNLOAD_URL}"; then echo "Error: Chrome for Testing version ${CFT_VERSION} not found for platform ${CFT_PLATFORM}" echo "Please check available versions at: https://googlechromelabs.github.io/chrome-for-testing/" rm -rf "${TEMP_DIR}" exit 1 fi wget -q -O chrome.zip "${DOWNLOAD_URL}" # Extract Chrome echo "Extracting Chrome for Testing..." ... (clipped 43 lines)
Generic: Robust Error Handling and Edge Case Management

Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation

Status:
Network Errors: External HTTP requests and file writes lack retry/fallback and broader exception handling
beyond raise_for_status, which may cause brittle failures without contextual logging.

Referred Code
def fetch_chrome_for_testing_versions(): # Fetch latest stable version to use as maximum version filter stable_url = 'https://googlechromelabs.github.io/chrome-for-testing/LATEST_RELEASE_STABLE' stable_resp = requests.get(stable_url) stable_resp.raise_for_status() max_stable_version = stable_resp.text.strip() print(f"Latest stable version: {max_stable_version}") # Parse max stable version for comparison max_stable_parts = list(map(int, max_stable_version.split('.'))) url = 'https://googlechromelabs.github.io/chrome-for-testing/known-good-versions.json' resp = requests.get(url) resp.raise_for_status() data = resp.json() # Extract versions from the JSON, filtering out versions higher than stable versions = [] for item in data.get('versions', []): version = item.get('version') if version: ... (clipped 40 lines)
Generic: Security-First Input Validation and Data Handling

Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent
vulnerabilities

Status:
Input Validation: Environment-derived inputs like CFT_VERSION and CFT_PLATFORM are used to construct URLs
and package lists without explicit validation or sanitization in the script.

Referred Code
# Default Chrome for Testing version CFT_VERSION="${CFT_VERSION}" CFT_PLATFORM="${CFT_PLATFORM:-linux64}" CFT_BASE_URL="https://storage.googleapis.com/chrome-for-testing-public" CFT_API_BASE="https://googlechromelabs.github.io/chrome-for-testing" # Resolve channel names to version numbers if [[ "${CFT_VERSION}" =~ ^(STABLE|BETA|DEV|CANARY)$ ]]; then CHANNEL="${CFT_VERSION}" echo "Fetching latest ${CHANNEL} version..." CFT_VERSION=$(wget -qO- "${CFT_API_BASE}/LATEST_RELEASE_${CHANNEL}" | sed 's/\r$//') echo "Resolved ${CHANNEL} to version: ${CFT_VERSION}" fi echo "Installing Chrome for Testing: ${CFT_VERSION} (${CFT_PLATFORM})" # Create temporary directory TEMP_DIR=$(mktemp -d) cd "${TEMP_DIR}" # Download Chrome for Testing ... (clipped 9 lines)
@qodo-merge-pro
Copy link
Contributor

qodo-merge-pro bot commented Oct 31, 2025

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
High-level
Refactor build logic to reduce duplication

Refactor the build system to eliminate duplicated logic for "Chrome for
Testing". Introduce parameterized build targets and reusable GitHub workflows to
support different browsers generically, improving maintainability.

Examples:

Makefile [178-213]
chromeForTesting_only: set -e; case "$(PLATFORMS)" in \ *linux/amd64*) \ echo "Chrome for Testing is only supported on linux/amd64" \ && cd ./NodeChrome && docker buildx build --platform linux/amd64 $(BUILD_ARGS) $(FROM_IMAGE_ARGS) --build-arg INSTALL_CFT=true -t $(NAME)/node-chrome-for-testing:$(TAG_VERSION) . \ ;; \ *) \ echo "Chrome for Testing doesn't support platform $(PLATFORMS)" ; \ ;; \ esac ... (clipped 26 lines)
.github/workflows/release-chrome-for-testing-versions.yml [1-168]
name: Deploy specific Chrome for Testing version on: workflow_dispatch: inputs: stable: description: 'Use upstream stable build' required: true type: string default: 'true' ... (clipped 158 lines)

Solution Walkthrough:

Before:

# .github/workflows/release-chrome-versions.yml name: Deploy specific Chrome version ... # .github/workflows/release-chrome-for-testing-versions.yml name: Deploy specific Chrome for Testing version ... # Makefile chrome_only: ... build node-chrome ... chrome: node_base chrome_only chromeForTesting_only: ... build node-chrome-for-testing ... chromeForTesting: node_base chromeForTesting_only 

After:

# .github/workflows/release-browser.yml (reusable workflow) name: Deploy Browser Version on: workflow_call: inputs: browser-name: required: true type: string jobs: deploy: ... run: make build BROWSER=${{ inputs.browser-name }} # Makefile # Use variables to avoid duplication build-browser: @echo "Building for $(BROWSER)" ... build logic for $(NAME)/node-$(BROWSER) ... 
Suggestion importance[1-10]: 9

__

Why: The suggestion correctly identifies a major architectural issue of widespread code duplication across the Makefile, GitHub Actions, and shell scripts, which significantly impacts maintainability.

High
Possible issue
Fix incorrect use of expressions

Remove the incorrect || operators from environment variable assignments that use
github.event.inputs. Instead, rely on the default values defined in the workflow
inputs.

.github/workflows/release-chrome-for-testing-versions.yml [46-55]

 env: GRID_VERSION: ${{ github.event.inputs.grid-version }} BROWSER_NAME: ${{ github.event.inputs.browser-name }} - REUSE_BASE: ${{ github.event.inputs.reuse-base || true }} - BUILD_DATE: ${{ github.event.inputs.build-date || '' }} + REUSE_BASE: ${{ github.event.inputs.reuse-base }} + BUILD_DATE: ${{ github.event.inputs.build-date }} NAMESPACE: ${{ vars.DOCKER_NAMESPACE || 'selenium' }} AUTHORS: ${{ vars.AUTHORS || 'SeleniumHQ' }} - PUSH_IMAGE: ${{ github.event.inputs.push-image || false }} - PR_CHANGELOG: ${{ github.event.inputs.pr-changelog || true }} + PUSH_IMAGE: ${{ github.event.inputs.push-image }} + PR_CHANGELOG: ${{ github.event.inputs.pr-changelog }} RUN_ID: ${{ github.run_id }}
  • Apply / Chat
Suggestion importance[1-10]: 8

__

Why: The suggestion correctly identifies a bug where using the || operator with boolean workflow inputs would cause them to be ignored, fixing the workflow's logic and preventing unintended behavior.

Medium
General
Fail build on unsupported platforms

In the chromeForTesting_only Makefile target, add exit 1 to the case for
unsupported platforms to ensure the build fails explicitly.

Makefile [178-187]

 chromeForTesting_only:	set -e; case "$(PLATFORMS)" in \	*linux/amd64*) \	echo "Chrome for Testing is only supported on linux/amd64" \	&& cd ./NodeChrome && docker buildx build --platform linux/amd64 $(BUILD_ARGS) $(FROM_IMAGE_ARGS) --build-arg INSTALL_CFT=true -t $(NAME)/node-chrome-for-testing:$(TAG_VERSION) . \	;; \	*) \	echo "Chrome for Testing doesn't support platform $(PLATFORMS)" ; \ +exit 1; \	;; \	esac
  • Apply / Chat
Suggestion importance[1-10]: 7

__

Why: The suggestion correctly points out that the build should fail explicitly for unsupported platforms instead of silently succeeding, which improves the robustness and clarity of the build process.

Medium
Learned
best practice
Verify archive integrity with checksums

Download the checksum alongside the archive and verify it before extracting;
also pin the URL by version and fail if checksum mismatch.

NodeChrome/install-chrome-for-testing.sh [36-50]

 DOWNLOAD_URL="${CFT_BASE_URL}/${CFT_VERSION}/${CFT_PLATFORM}/chrome-${CFT_PLATFORM}.zip" +CHECKSUM_URL="${DOWNLOAD_URL}.sha256" echo "Downloading from: ${DOWNLOAD_URL}" -if ! wget -q --spider "${DOWNLOAD_URL}"; then +if ! wget -q --spider "${DOWNLOAD_URL}" || ! wget -q --spider "${CHECKSUM_URL}"; then echo "Error: Chrome for Testing version ${CFT_VERSION} not found for platform ${CFT_PLATFORM}" - echo "Please check available versions at: https://googlechromelabs.github.io/chrome-for-testing/" rm -rf "${TEMP_DIR}" exit 1 fi wget -q -O chrome.zip "${DOWNLOAD_URL}" +wget -q -O chrome.zip.sha256 "${CHECKSUM_URL}" +echo "$(cat chrome.zip.sha256) chrome.zip" | sha256sum --check --status -# Extract Chrome echo "Extracting Chrome for Testing..." unzip -q chrome.zip
  • Apply / Chat
Suggestion importance[1-10]: 6

__

Why:
Relevant best practice - Pin external tool versions and verify downloads with checksums when fetching binaries to ensure reproducible, secure builds.

Low
  • Update
@VietND96 VietND96 force-pushed the chrome-for-testing branch 2 times, most recently from 15ad3b0 to 66f8e63 Compare October 31, 2025 08:21
Signed-off-by: Viet Nguyen Duc <nguyenducviet4496@gmail.com>
@VietND96 VietND96 enabled auto-merge (squash) October 31, 2025 12:04
@VietND96 VietND96 merged commit 1866f17 into trunk Oct 31, 2025
39 checks passed
@VietND96 VietND96 deleted the chrome-for-testing branch October 31, 2025 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

2 participants