Collect BanyanDB data files on e2e test failure#13765
Open
Collect BanyanDB data files on e2e test failure#13765
Conversation
- Update infra-e2e to 36221fa (adds collect-on-failure feature) - Add cleanup.collect to all 18 BanyanDB e2e YAMLs - Compose tests: collect by service name - Kind tests: collect by label-selector app=banyandb - Data dirs: trace, stream, measure, property, schema-property - Output to $SW_INFRA_E2E_LOG_DIR/banyandb-data (included in existing upload) TESTING: intentional failure in storage/banyandb to verify data collection
wu-sheng added the test Artifact confirmed: banyandb-data/ directory uploaded with trace (796K), property (1.1M), schema-property (14M), 234 files.
wu-sheng changed the title - Compose: storage/banyandb — failure added as LAST verify case (after all 122+ real cases run, ensuring all 5 data dirs populated) - Kind: profiling/ebpf/oncpu/banyandb — failure after profiling cases Both tests will fail late, with full data in trace, stream, measure, property, and schema-property directories.
Kind BanyanDB pods are in istio-system namespace (not default). Label selector updated to app.kubernetes.io/name=banyandb to match the SkyWalking Helm chart convention.
Use a real query (service layer) with wrong expected file instead of echo. This ensures real verify cases pass first (populating all 5 BanyanDB data dirs), then the intentional mismatch triggers failure after data is fully flushed across all retries.
Compose (storage/banyandb): trace 3.1M/926 files, property 1.1M, schema-property 14M Kind (ebpf/oncpu/banyandb): stream 256K/75 files, measure 20K/8 files, property 1.1M, schema-property 14M Both paths confirmed: data collected before cleanup and included in existing log artifact upload.
Member Author
Verification ResultsBoth Docker Compose and Kind (K8s) data collection verified with intentional test failures. Docker Compose ( |
Base docker-compose overrides root paths: --stream-root-path /tmp/stream-data --measure-root-path /tmp/measure-data Collect from /tmp/stream-data/ and /tmp/measure-data/ instead of /tmp/stream/ and /tmp/measure/ which are empty.
Remove --stream-root-path and --measure-root-path from base-compose, storage, tls, and data-generate docker-compose files. All modules now use default /tmp root: /tmp/trace/, /tmp/stream/, /tmp/measure/, /tmp/property/, /tmp/schema-property/ Updated stages data-generate workflow and volume mounts to match. Collect paths in e2e YAMLs now align with actual data locations.
There was a problem hiding this comment.
Pull request overview
This PR enhances BanyanDB-related E2E diagnostics by collecting BanyanDB on-disk data directories as artifacts on test failures, and aligns Compose/K8s setups to use BanyanDB’s default /tmp/* root layout (removing custom stream/measure root-path overrides).
Changes:
- Bump
apache/skywalking-infra-e2eaction SHA and addcleanup.collectblocks to BanyanDB E2E configs to gather/tmp/{trace,stream,measure,property,schema-property}on failure. - Remove
--stream-root-path/--measure-root-pathoverrides from BanyanDB Compose definitions. - Update the BanyanDB “stages” data-generate workflow + volume mounts to match the default
/tmp/measureand/tmp/streampaths.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| test/e2e-v2/script/docker-compose/base-compose.yml | Removes BanyanDB root-path overrides to rely on default /tmp layout. |
| test/e2e-v2/cases/zipkin/banyandb/e2e.yaml | Adds failure-only collection of BanyanDB data dirs. |
| test/e2e-v2/cases/storage/banyandb/e2e.yaml | Adds failure-only collection of BanyanDB data dirs. |
| test/e2e-v2/cases/storage/banyandb/tls/e2e.yaml | Adds failure-only collection of BanyanDB data dirs. |
| test/e2e-v2/cases/storage/banyandb/stages/e2e.yaml | Adds failure-only collection of BanyanDB data dirs (compose stages topology). |
| test/e2e-v2/cases/profiling/trace/banyandb/e2e.yaml | Adds failure-only collection of BanyanDB data dirs. |
| test/e2e-v2/cases/profiling/pprof/banyandb/e2e.yaml | Adds failure-only collection of BanyanDB data dirs. |
| test/e2e-v2/cases/profiling/async-profiler/banyandb/e2e.yaml | Adds failure-only collection of BanyanDB data dirs. |
| test/e2e-v2/cases/profiling/ebpf/oncpu/banyandb/e2e.yaml | Adds failure-only collection from K8s (namespace + label selector). |
| test/e2e-v2/cases/profiling/ebpf/network/banyandb/e2e.yaml | Adds failure-only collection from K8s (namespace + label selector). |
| test/e2e-v2/cases/profiling/ebpf/continuous/banyandb/e2e.yaml | Adds failure-only collection from K8s (namespace + label selector). |
| test/e2e-v2/cases/profiling/ebpf/access_log/banyandb/e2e.yaml | Adds failure-only collection from K8s (namespace + label selector). |
| test/e2e-v2/cases/menu/banyandb/e2e.yaml | Adds failure-only collection of BanyanDB data dirs. |
| test/e2e-v2/cases/log/banyandb/e2e.yaml | Adds failure-only collection of BanyanDB data dirs. |
| test/e2e-v2/cases/event/banyandb/e2e.yaml | Adds failure-only collection of BanyanDB data dirs. |
| test/e2e-v2/cases/cluster/zk/banyandb/e2e.yaml | Adds failure-only collection of BanyanDB data dirs. |
| test/e2e-v2/cases/baseline/banyandb/e2e.yaml | Adds failure-only collection of BanyanDB data dirs. |
| test/e2e-v2/cases/banyandb/e2e.yaml | Adds failure-only collection of BanyanDB data dirs. |
| test/e2e-v2/cases/alarm/banyandb/e2e.yaml | Adds failure-only collection of BanyanDB data dirs. |
| test/e2e-v2/cases/storage/banyandb/docker-compose.yml | Removes BanyanDB root-path overrides from auth-enabled compose test. |
| test/e2e-v2/cases/storage/banyandb/tls/docker-compose.yml | Removes BanyanDB root-path overrides from TLS compose test. |
| test/e2e-v2/cases/storage/banyandb/data-generate/docker-compose.yml | Removes BanyanDB root-path overrides from cold-data generator compose. |
| test/e2e-v2/cases/storage/banyandb/stages/docker-compose.yml | Updates mounted paths to the default /tmp/measure and /tmp/stream layout. |
| .github/workflows/skywalking.yaml | Updates infra-e2e action SHA and adjusts cold-data generation paths to default /tmp/measure/.... |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
kezhenxu94 previously approved these changes Mar 27, 2026
The stages docker-compose has no 'banyandb' service. It uses data-hot1, data-warm1, data-cold1 as data nodes. Collect from all three to capture data across hot/warm/cold stages.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Collect BanyanDB data files on e2e test failure
Problem: When BanyanDB-related e2e tests fail, only container logs are uploaded. The actual
BanyanDB data files (trace, stream, measure, property, schema-property) are lost with container
cleanup, making it hard to diagnose storage-level issues.
Changes:
1. Collect BanyanDB data on failure
skywalking-infra-e2eto36221fa(addscollect-on-failure)cleanup.collectto all 18 BanyanDB e2e YAMLs:service: banyandbnamespace: istio-system+label-selector: app.kubernetes.io/name=banyandb$SW_INFRA_E2E_LOG_DIR/banyandb-data— included in existing artifact upload2. Remove custom root-path overrides from BanyanDB base compose
--stream-root-path /tmp/stream-data --measure-root-path /tmp/measure-datafrom:test/e2e-v2/script/docker-compose/base-compose.yml(affects all Compose BanyanDB tests)test/e2e-v2/cases/storage/banyandb/docker-compose.ymltest/e2e-v2/cases/storage/banyandb/tls/docker-compose.ymltest/e2e-v2/cases/storage/banyandb/data-generate/docker-compose.yml/tmproot path consistentlyCollected directories (all under default
/tmproot):/tmp/trace//tmp/stream//tmp/measure//tmp/property//tmp/schema-property/Verified results:
Docker Compose (
test-logs-BanyanDB— 5.0MB):Kind / K8s (
test-logs-eBPF Profiling On CPU BanyanDB— 4.6MB):CHANGESlog.