Skip to content

Make Crashtracking smoke test more resilient #8685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 9 commits into from
Apr 18, 2025
Merged

Make Crashtracking smoke test more resilient #8685

merged 9 commits into from
Apr 18, 2025

Conversation

@randomanderson
Copy link
Contributor

@randomanderson randomanderson commented Apr 9, 2025

What Does This Do

Improves the resiliency of the Crashtracking Smoke Test. Previously, the smoke test relied on parsing output logs which in inherently flaky. With this PR, the smoke test instead listens for the crash telemetry submitted to mock servers.

The changes are:

  • Creation of a test UDP server
  • Check the telemetry data submitted to the MockHttpServer rather than log output of the crashed process
  • Check the telemetry event submitted to the UDP server for OOM events rather than log output of the crashed process
  • Extract the output log processing from ProcessManager into its own class. This gets rid of a bunch of code duplication.

Motivation

When migrating to Gitlab, this smoke test was extra flaky.

Additional Notes

TestUDPServer could probably go in a more generic location in the codebase.

Contributor Checklist

Jira ticket: [PROJ-IDENT]
@randomanderson randomanderson added tag: no release notes Changes to exclude from release notes type: refactoring comp: crash tracking Crash Tracking labels Apr 9, 2025
@randomanderson randomanderson requested a review from a team as a code owner April 9, 2025 16:33
@randomanderson randomanderson force-pushed the landerson/fix-crashtrackingsmoke branch from 1b8480c to 561f192 Compare April 9, 2025 16:35
@randomanderson randomanderson marked this pull request as draft April 9, 2025 17:05
@randomanderson
Copy link
Contributor Author

randomanderson commented Apr 9, 2025

Back to draft. Found some issues
@pr-commenter
Copy link

pr-commenter bot commented Apr 9, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master landerson/fix-crashtrackingsmoke
git_commit_date 1744296074 1744390695
git_commit_sha b304f4b ac863f0
release_version 1.49.0-SNAPSHOT~b304f4bd62 1.49.0-SNAPSHOT~ac863f0f44
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1744395302 1744395302
ci_job_id 893396591 893396591
ci_pipeline_id 61928858 61928858
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-u8dbh5tw-project-304-concurrent-0-s2fufd85 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-u8dbh5tw-project-304-concurrent-0-s2fufd85 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 64 metrics, 7 unstable metrics.

Startup time reports for insecure-bank 
gantt title insecure-bank - global startup overhead: candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62 dateFormat X axisFormat %s section tracing Agent [baseline] (1.062 s) : 0, 1062012 Total [baseline] (8.717 s) : 0, 8716625 Agent [candidate] (1.064 s) : 0, 1063824 Total [candidate] (8.713 s) : 0, 8713020 section iast Agent [baseline] (1.201 s) : 0, 1200903 Total [baseline] (9.304 s) : 0, 9303619 Agent [candidate] (1.187 s) : 0, 1186675 Total [candidate] (9.224 s) : 0, 9223811 section iast_HARDCODED_SECRET_DISABLED Agent [baseline] (1.193 s) : 0, 1193147 Total [baseline] (9.242 s) : 0, 9241804 Agent [candidate] (1.183 s) : 0, 1182865 Total [candidate] (9.2 s) : 0, 9199886 section iast_TELEMETRY_OFF Agent [baseline] (1.189 s) : 0, 1188675 Total [baseline] (9.288 s) : 0, 9288220 Agent [candidate] (1.182 s) : 0, 1181885 Total [candidate] (9.265 s) : 0, 9264841
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent iast 1.201 s 138.891 ms (13.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.193 s 131.135 ms (12.3%)
Agent iast_TELEMETRY_OFF 1.189 s 126.663 ms (11.9%)
Total tracing 8.717 s -
Total iast 9.304 s 586.995 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.242 s 525.179 ms (6.0%)
Total iast_TELEMETRY_OFF 9.288 s 571.595 ms (6.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent iast 1.187 s 122.851 ms (11.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.183 s 119.041 ms (11.2%)
Agent iast_TELEMETRY_OFF 1.182 s 118.061 ms (11.1%)
Total tracing 8.713 s -
Total iast 9.224 s 510.791 ms (5.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.2 s 486.865 ms (5.6%)
Total iast_TELEMETRY_OFF 9.265 s 551.821 ms (6.3%) 
gantt title insecure-bank - break down per module: candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62 dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (725.336 ms) : 0, 725336 BytebuddyAgent [candidate] (725.271 ms) : 0, 725271 GlobalTracer [baseline] (239.945 ms) : 0, 239945 GlobalTracer [candidate] (240.559 ms) : 0, 240559 AppSec [baseline] (54.948 ms) : 0, 54948 AppSec [candidate] (54.782 ms) : 0, 54782 Debugger [baseline] (4.429 ms) : 0, 4429 Debugger [candidate] (4.404 ms) : 0, 4404 Remote Config [baseline] (4.349 ms) : 0, 4349 Remote Config [candidate] (2.976 ms) : 0, 2976 Telemetry [baseline] (9.465 ms) : 0, 9465 Telemetry [candidate] (12.295 ms) : 0, 12295 section iast BytebuddyAgent [baseline] (853.157 ms) : 0, 853157 BytebuddyAgent [candidate] (840.463 ms) : 0, 840463 GlobalTracer [baseline] (231.004 ms) : 0, 231004 GlobalTracer [candidate] (230.054 ms) : 0, 230054 IAST [baseline] (23.105 ms) : 0, 23105 IAST [candidate] (24.735 ms) : 0, 24735 AppSec [baseline] (56.192 ms) : 0, 56192 AppSec [candidate] (54.497 ms) : 0, 54497 Debugger [baseline] (4.332 ms) : 0, 4332 Debugger [candidate] (4.263 ms) : 0, 4263 Remote Config [baseline] (638.81 µs) : 0, 639 Remote Config [candidate] (604.119 µs) : 0, 604 Telemetry [baseline] (8.792 ms) : 0, 8792 Telemetry [candidate] (8.733 ms) : 0, 8733 section iast_HARDCODED_SECRET_DISABLED BytebuddyAgent [baseline] (846.336 ms) : 0, 846336 BytebuddyAgent [candidate] (838.468 ms) : 0, 838468 GlobalTracer [baseline] (230.64 ms) : 0, 230640 GlobalTracer [candidate] (229.199 ms) : 0, 229199 IAST [baseline] (24.507 ms) : 0, 24507 IAST [candidate] (22.719 ms) : 0, 22719 AppSec [baseline] (54.458 ms) : 0, 54458 AppSec [candidate] (55.61 ms) : 0, 55610 Debugger [baseline] (4.259 ms) : 0, 4259 Debugger [candidate] (4.204 ms) : 0, 4204 Remote Config [baseline] (625.875 µs) : 0, 626 Remote Config [candidate] (605.928 µs) : 0, 606 Telemetry [baseline] (8.732 ms) : 0, 8732 Telemetry [candidate] (8.645 ms) : 0, 8645 section iast_TELEMETRY_OFF BytebuddyAgent [baseline] (842.895 ms) : 0, 842895 BytebuddyAgent [candidate] (837.177 ms) : 0, 837177 GlobalTracer [baseline] (230.756 ms) : 0, 230756 GlobalTracer [candidate] (229.495 ms) : 0, 229495 IAST [baseline] (22.287 ms) : 0, 22287 IAST [candidate] (22.465 ms) : 0, 22465 AppSec [baseline] (55.866 ms) : 0, 55866 AppSec [candidate] (55.894 ms) : 0, 55894 Debugger [baseline] (4.152 ms) : 0, 4152 Debugger [candidate] (4.125 ms) : 0, 4125 Remote Config [baseline] (619.212 µs) : 0, 619 Remote Config [candidate] (641.48 µs) : 0, 641 Telemetry [baseline] (8.651 ms) : 0, 8651 Telemetry [candidate] (8.691 ms) : 0, 8691
Loading
Startup time reports for petclinic 
gantt title petclinic - global startup overhead: candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62 dateFormat X axisFormat %s section tracing Agent [baseline] (1.061 s) : 0, 1060656 Total [baseline] (10.511 s) : 0, 10510685 Agent [candidate] (1.061 s) : 0, 1060776 Total [candidate] (10.492 s) : 0, 10491807 section appsec Agent [baseline] (1.195 s) : 0, 1194691 Total [baseline] (10.808 s) : 0, 10807874 Agent [candidate] (1.2 s) : 0, 1200224 Total [candidate] (10.772 s) : 0, 10771932 section iast Agent [baseline] (1.186 s) : 0, 1185843 Total [baseline] (11.051 s) : 0, 11051335 Agent [candidate] (1.185 s) : 0, 1184896 Total [candidate] (11.073 s) : 0, 11073494 section profiling Agent [baseline] (1.292 s) : 0, 1291855 Total [baseline] (10.929 s) : 0, 10928720 Agent [candidate] (1.299 s) : 0, 1299350 Total [candidate] (10.912 s) : 0, 10911969
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent appsec 1.195 s 134.035 ms (12.6%)
Agent iast 1.186 s 125.187 ms (11.8%)
Agent profiling 1.292 s 231.199 ms (21.8%)
Total tracing 10.511 s -
Total appsec 10.808 s 297.189 ms (2.8%)
Total iast 11.051 s 540.65 ms (5.1%)
Total profiling 10.929 s 418.036 ms (4.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent appsec 1.2 s 139.448 ms (13.1%)
Agent iast 1.185 s 124.12 ms (11.7%)
Agent profiling 1.299 s 238.575 ms (22.5%)
Total tracing 10.492 s -
Total appsec 10.772 s 280.125 ms (2.7%)
Total iast 11.073 s 581.686 ms (5.5%)
Total profiling 10.912 s 420.162 ms (4.0%) 
gantt title petclinic - break down per module: candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62 dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (725.055 ms) : 0, 725055 BytebuddyAgent [candidate] (721.941 ms) : 0, 721941 GlobalTracer [baseline] (239.434 ms) : 0, 239434 GlobalTracer [candidate] (239.385 ms) : 0, 239385 AppSec [baseline] (54.759 ms) : 0, 54759 AppSec [candidate] (55.224 ms) : 0, 55224 Debugger [baseline] (4.374 ms) : 0, 4374 Debugger [candidate] (4.388 ms) : 0, 4388 Remote Config [baseline] (709.158 µs) : 0, 709 Remote Config [candidate] (692.622 µs) : 0, 693 Telemetry [baseline] (12.902 ms) : 0, 12902 Telemetry [candidate] (15.763 ms) : 0, 15763 section appsec BytebuddyAgent [baseline] (736.461 ms) : 0, 736461 BytebuddyAgent [candidate] (740.09 ms) : 0, 740090 GlobalTracer [baseline] (235.163 ms) : 0, 235163 GlobalTracer [candidate] (236.201 ms) : 0, 236201 IAST [baseline] (21.723 ms) : 0, 21723 IAST [candidate] (21.688 ms) : 0, 21688 AppSec [baseline] (175.226 ms) : 0, 175226 AppSec [candidate] (176.256 ms) : 0, 176256 Debugger [baseline] (4.944 ms) : 0, 4944 Debugger [candidate] (4.598 ms) : 0, 4598 Remote Config [baseline] (637.549 µs) : 0, 638 Remote Config [candidate] (640.386 µs) : 0, 640 Telemetry [baseline] (8.126 ms) : 0, 8126 Telemetry [candidate] (8.238 ms) : 0, 8238 section iast BytebuddyAgent [baseline] (839.862 ms) : 0, 839862 BytebuddyAgent [candidate] (838.844 ms) : 0, 838844 GlobalTracer [baseline] (230.223 ms) : 0, 230223 GlobalTracer [candidate] (230.013 ms) : 0, 230013 IAST [baseline] (23.804 ms) : 0, 23804 IAST [candidate] (22.899 ms) : 0, 22899 AppSec [baseline] (54.951 ms) : 0, 54951 AppSec [candidate] (56.222 ms) : 0, 56222 Debugger [baseline] (4.226 ms) : 0, 4226 Debugger [candidate] (4.269 ms) : 0, 4269 Remote Config [baseline] (606.134 µs) : 0, 606 Remote Config [candidate] (622.49 µs) : 0, 622 Telemetry [baseline] (8.736 ms) : 0, 8736 Telemetry [candidate] (8.666 ms) : 0, 8666 section profiling BytebuddyAgent [baseline] (711.442 ms) : 0, 711442 BytebuddyAgent [candidate] (716.53 ms) : 0, 716530 GlobalTracer [baseline] (359.98 ms) : 0, 359980 GlobalTracer [candidate] (361.811 ms) : 0, 361811 AppSec [baseline] (53.965 ms) : 0, 53965 AppSec [candidate] (53.819 ms) : 0, 53819 Debugger [baseline] (4.398 ms) : 0, 4398 Debugger [candidate] (4.419 ms) : 0, 4419 Remote Config [baseline] (680.578 µs) : 0, 681 Remote Config [candidate] (661.614 µs) : 0, 662 Telemetry [baseline] (9.017 ms) : 0, 9017 Telemetry [candidate] (9.047 ms) : 0, 9047 ProfilingAgent [baseline] (102.166 ms) : 0, 102166 ProfilingAgent [candidate] (102.663 ms) : 0, 102663 Profiling [baseline] (102.193 ms) : 0, 102193 Profiling [candidate] (102.69 ms) : 0, 102690
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-04-11T17:45:12 2025-04-11T17:53:00
git_branch master landerson/fix-crashtrackingsmoke
git_commit_date 1744296074 1744390695
git_commit_sha b304f4b ac863f0
release_version 1.49.0-SNAPSHOT~b304f4bd62 1.49.0-SNAPSHOT~ac863f0f44
start_time 2025-04-11T17:44:58 2025-04-11T17:52:46
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1744394380 1744394380
ci_job_id 893396592 893396592
ci_pipeline_id 61928858 61928858
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-fbx1gztc-project-304-concurrent-0-9wwzf5w9 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-fbx1gztc-project-304-concurrent-0-9wwzf5w9 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt title insecure-bank - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62 dateFormat X axisFormat %s section baseline no_agent (388.391 µs) : 368, 409 . : milestone, 388, iast (523.685 µs) : 501, 547 . : milestone, 524, iast_FULL (738.922 µs) : 717, 761 . : milestone, 739, iast_GLOBAL (562.875 µs) : 541, 585 . : milestone, 563, iast_HARDCODED_SECRET_DISABLED (516.263 µs) : 495, 538 . : milestone, 516, iast_INACTIVE (476.324 µs) : 455, 498 . : milestone, 476, iast_TELEMETRY_OFF (511.96 µs) : 490, 534 . : milestone, 512, tracing (469.309 µs) : 448, 491 . : milestone, 469, section candidate no_agent (384.765 µs) : 364, 406 . : milestone, 385, iast (517.306 µs) : 496, 539 . : milestone, 517, iast_FULL (737.188 µs) : 715, 759 . : milestone, 737, iast_GLOBAL (569.584 µs) : 547, 593 . : milestone, 570, iast_HARDCODED_SECRET_DISABLED (521.371 µs) : 499, 543 . : milestone, 521, iast_INACTIVE (471.942 µs) : 450, 494 . : milestone, 472, iast_TELEMETRY_OFF (510.031 µs) : 488, 532 . : milestone, 510, tracing (472.035 µs) : 451, 493 . : milestone, 472,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 388.391 µs [367.956 µs, 408.826 µs] -
iast 523.685 µs [500.785 µs, 546.584 µs] 135.294 µs (34.8%)
iast_FULL 738.922 µs [716.849 µs, 760.994 µs] 350.531 µs (90.3%)
iast_GLOBAL 562.875 µs [540.791 µs, 584.959 µs] 174.484 µs (44.9%)
iast_HARDCODED_SECRET_DISABLED 516.263 µs [494.678 µs, 537.848 µs] 127.872 µs (32.9%)
iast_INACTIVE 476.324 µs [454.532 µs, 498.116 µs] 87.933 µs (22.6%)
iast_TELEMETRY_OFF 511.96 µs [489.98 µs, 533.941 µs] 123.57 µs (31.8%)
tracing 469.309 µs [448.009 µs, 490.609 µs] 80.918 µs (20.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 384.765 µs [363.812 µs, 405.719 µs] -
iast 517.306 µs [495.585 µs, 539.027 µs] 132.541 µs (34.4%)
iast_FULL 737.188 µs [715.177 µs, 759.2 µs] 352.423 µs (91.6%)
iast_GLOBAL 569.584 µs [546.631 µs, 592.537 µs] 184.819 µs (48.0%)
iast_HARDCODED_SECRET_DISABLED 521.371 µs [499.405 µs, 543.337 µs] 136.606 µs (35.5%)
iast_INACTIVE 471.942 µs [450.264 µs, 493.621 µs] 87.177 µs (22.7%)
iast_TELEMETRY_OFF 510.031 µs [488.109 µs, 531.952 µs] 125.265 µs (32.6%)
tracing 472.035 µs [450.613 µs, 493.457 µs] 87.27 µs (22.7%)
Request duration reports for petclinic 
gantt title petclinic - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62 dateFormat X axisFormat %s section baseline no_agent (1.364 ms) : 1344, 1384 . : milestone, 1364, appsec (1.754 ms) : 1730, 1777 . : milestone, 1754, appsec_no_iast (1.754 ms) : 1731, 1778 . : milestone, 1754, code_origins (1.701 ms) : 1674, 1728 . : milestone, 1701, iast (1.537 ms) : 1513, 1560 . : milestone, 1537, profiling (1.561 ms) : 1535, 1587 . : milestone, 1561, tracing (1.491 ms) : 1466, 1517 . : milestone, 1491, section candidate no_agent (1.343 ms) : 1324, 1363 . : milestone, 1343, appsec (1.738 ms) : 1714, 1762 . : milestone, 1738, appsec_no_iast (1.745 ms) : 1721, 1769 . : milestone, 1745, code_origins (1.698 ms) : 1670, 1725 . : milestone, 1698, iast (1.51 ms) : 1486, 1534 . : milestone, 1510, profiling (1.583 ms) : 1559, 1607 . : milestone, 1583, tracing (1.511 ms) : 1486, 1536 . : milestone, 1511,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.364 ms [1.344 ms, 1.384 ms] -
appsec 1.754 ms [1.73 ms, 1.777 ms] 389.294 µs (28.5%)
appsec_no_iast 1.754 ms [1.731 ms, 1.778 ms] 390.148 µs (28.6%)
code_origins 1.701 ms [1.674 ms, 1.728 ms] 336.585 µs (24.7%)
iast 1.537 ms [1.513 ms, 1.56 ms] 172.216 µs (12.6%)
profiling 1.561 ms [1.535 ms, 1.587 ms] 196.359 µs (14.4%)
tracing 1.491 ms [1.466 ms, 1.517 ms] 126.907 µs (9.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.343 ms [1.324 ms, 1.363 ms] -
appsec 1.738 ms [1.714 ms, 1.762 ms] 394.837 µs (29.4%)
appsec_no_iast 1.745 ms [1.721 ms, 1.769 ms] 401.539 µs (29.9%)
code_origins 1.698 ms [1.67 ms, 1.725 ms] 354.24 µs (26.4%)
iast 1.51 ms [1.486 ms, 1.534 ms] 166.183 µs (12.4%)
profiling 1.583 ms [1.559 ms, 1.607 ms] 239.529 µs (17.8%)
tracing 1.511 ms [1.486 ms, 1.536 ms] 167.539 µs (12.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master landerson/fix-crashtrackingsmoke
git_commit_date 1744296074 1744390695
git_commit_sha b304f4b ac863f0
release_version 1.49.0-SNAPSHOT~b304f4bd62 1.49.0-SNAPSHOT~ac863f0f44
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1744394802 1744394802
ci_job_id 893396593 893396593
ci_pipeline_id 61928858 61928858
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-ubwazs2a-project-304-concurrent-2-sifzleix 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-ubwazs2a-project-304-concurrent-2-sifzleix 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt title biojava - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62 dateFormat X axisFormat %s section baseline no_agent (15.542 s) : 15542000, 15542000 . : milestone, 15542000, appsec (15.146 s) : 15146000, 15146000 . : milestone, 15146000, iast (18.454 s) : 18454000, 18454000 . : milestone, 18454000, iast_GLOBAL (17.952 s) : 17952000, 17952000 . : milestone, 17952000, profiling (15.209 s) : 15209000, 15209000 . : milestone, 15209000, tracing (15.026 s) : 15026000, 15026000 . : milestone, 15026000, section candidate no_agent (14.922 s) : 14922000, 14922000 . : milestone, 14922000, appsec (14.806 s) : 14806000, 14806000 . : milestone, 14806000, iast (18.127 s) : 18127000, 18127000 . : milestone, 18127000, iast_GLOBAL (18.167 s) : 18167000, 18167000 . : milestone, 18167000, profiling (15.2 s) : 15200000, 15200000 . : milestone, 15200000, tracing (15.074 s) : 15074000, 15074000 . : milestone, 15074000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.542 s [15.542 s, 15.542 s] -
appsec 15.146 s [15.146 s, 15.146 s] -396.0 ms (-2.5%)
iast 18.454 s [18.454 s, 18.454 s] 2.912 s (18.7%)
iast_GLOBAL 17.952 s [17.952 s, 17.952 s] 2.41 s (15.5%)
profiling 15.209 s [15.209 s, 15.209 s] -333.0 ms (-2.1%)
tracing 15.026 s [15.026 s, 15.026 s] -516.0 ms (-3.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.922 s [14.922 s, 14.922 s] -
appsec 14.806 s [14.806 s, 14.806 s] -116.0 ms (-0.8%)
iast 18.127 s [18.127 s, 18.127 s] 3.205 s (21.5%)
iast_GLOBAL 18.167 s [18.167 s, 18.167 s] 3.245 s (21.7%)
profiling 15.2 s [15.2 s, 15.2 s] 278.0 ms (1.9%)
tracing 15.074 s [15.074 s, 15.074 s] 152.0 ms (1.0%)
Execution time for tomcat 
gantt title tomcat - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62 dateFormat X axisFormat %s section baseline no_agent (1.475 ms) : 1464, 1487 . : milestone, 1475, appsec (2.375 ms) : 2328, 2421 . : milestone, 2375, iast (2.162 ms) : 2103, 2220 . : milestone, 2162, iast_GLOBAL (2.202 ms) : 2143, 2261 . : milestone, 2202, profiling (2.484 ms) : 2303, 2665 . : milestone, 2484, tracing (1.981 ms) : 1936, 2026 . : milestone, 1981, section candidate no_agent (1.48 ms) : 1469, 1492 . : milestone, 1480, appsec (2.375 ms) : 2328, 2422 . : milestone, 2375, iast (2.152 ms) : 2094, 2211 . : milestone, 2152, iast_GLOBAL (2.201 ms) : 2142, 2260 . : milestone, 2201, profiling (1.999 ms) : 1952, 2045 . : milestone, 1999, tracing (1.98 ms) : 1936, 2025 . : milestone, 1980,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.464 ms, 1.487 ms] -
appsec 2.375 ms [2.328 ms, 2.421 ms] 899.306 µs (61.0%)
iast 2.162 ms [2.103 ms, 2.22 ms] 686.334 µs (46.5%)
iast_GLOBAL 2.202 ms [2.143 ms, 2.261 ms] 726.397 µs (49.2%)
profiling 2.484 ms [2.303 ms, 2.665 ms] 1.009 ms (68.4%)
tracing 1.981 ms [1.936 ms, 2.026 ms] 505.676 µs (34.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.48 ms [1.469 ms, 1.492 ms] -
appsec 2.375 ms [2.328 ms, 2.422 ms] 894.645 µs (60.4%)
iast 2.152 ms [2.094 ms, 2.211 ms] 671.769 µs (45.4%)
iast_GLOBAL 2.201 ms [2.142 ms, 2.26 ms] 720.59 µs (48.7%)
profiling 1.999 ms [1.952 ms, 2.045 ms] 518.314 µs (35.0%)
tracing 1.98 ms [1.936 ms, 2.025 ms] 500.055 µs (33.8%)
@randomanderson randomanderson force-pushed the landerson/fix-crashtrackingsmoke branch from c2e0084 to b5a1003 Compare April 10, 2025 20:34
@randomanderson randomanderson mentioned this pull request Apr 11, 2025
Merged
@randomanderson randomanderson changed the base branch from master to landerson/dd-dogstatsd-port April 11, 2025 16:49
@randomanderson randomanderson force-pushed the landerson/fix-crashtrackingsmoke branch from a64de40 to ac863f0 Compare April 11, 2025 17:13
Base automatically changed from landerson/dd-dogstatsd-port to master April 14, 2025 06:31
@randomanderson randomanderson marked this pull request as ready for review April 14, 2025 14:33
@randomanderson randomanderson enabled auto-merge (squash) April 14, 2025 14:33
@randomanderson randomanderson merged commit 0382750 into master Apr 18, 2025
250 of 252 checks passed
@randomanderson randomanderson deleted the landerson/fix-crashtrackingsmoke branch April 18, 2025 07:08
@github-actions github-actions bot added this to the 1.49.0 milestone Apr 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: crash tracking Crash Tracking tag: no release notes Changes to exclude from release notes type: refactoring

3 participants

@randomanderson @jbachorik