-
Notifications
You must be signed in to change notification settings - Fork 477
test(profiling): try to unflake test_resource_not_collected
#16280
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
test(profiling): try to unflake test_resource_not_collected
#16280
Conversation
Codeowners resolved as |
a2dca31 to
fac291e
Compare
Performance SLOsComparing candidate kowalski/test-profiling-try-to-unflake-test_resource_not_collected (fac291e) with baseline main (1e7e0c4) 📈 Performance Regressions (2 suites)📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 498.586µs (SLO: <700.000µs 📉 -28.8%) vs baseline: 📈 +17.7% Memory: ✅ 42.841MB (SLO: <46.000MB -6.9%) vs baseline: +5.4% ✅ ospathbasename_noaspectTime: ✅ 433.039µs (SLO: <700.000µs 📉 -38.1%) vs baseline: +0.4% Memory: ✅ 42.723MB (SLO: <46.000MB -7.1%) vs baseline: +5.2% ✅ ospathjoin_aspectTime: ✅ 632.099µs (SLO: <700.000µs -9.7%) vs baseline: +0.3% Memory: ✅ 42.684MB (SLO: <46.000MB -7.2%) vs baseline: +5.3% ✅ ospathjoin_noaspectTime: ✅ 640.596µs (SLO: <700.000µs -8.5%) vs baseline: +0.5% Memory: ✅ 42.861MB (SLO: <46.000MB -6.8%) vs baseline: +5.5% ✅ ospathnormcase_aspectTime: ✅ 352.130µs (SLO: <700.000µs 📉 -49.7%) vs baseline: -0.2% Memory: ✅ 42.841MB (SLO: <46.000MB -6.9%) vs baseline: +5.4% ✅ ospathnormcase_noaspectTime: ✅ 359.021µs (SLO: <700.000µs 📉 -48.7%) vs baseline: +0.7% Memory: ✅ 42.802MB (SLO: <46.000MB -7.0%) vs baseline: +5.2% ✅ ospathsplit_aspectTime: ✅ 494.005µs (SLO: <700.000µs 📉 -29.4%) vs baseline: +0.5% Memory: ✅ 42.664MB (SLO: <46.000MB -7.3%) vs baseline: +4.9% ✅ ospathsplit_noaspectTime: ✅ 506.908µs (SLO: <700.000µs 📉 -27.6%) vs baseline: +0.5% Memory: ✅ 42.723MB (SLO: <46.000MB -7.1%) vs baseline: +5.0% ✅ ospathsplitdrive_aspectTime: ✅ 377.026µs (SLO: <700.000µs 📉 -46.1%) vs baseline: +0.5% Memory: ✅ 42.861MB (SLO: <46.000MB -6.8%) vs baseline: +4.9% ✅ ospathsplitdrive_noaspectTime: ✅ 73.140µs (SLO: <700.000µs 📉 -89.6%) vs baseline: +0.6% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +5.7% ✅ ospathsplitext_aspectTime: ✅ 459.707µs (SLO: <700.000µs 📉 -34.3%) vs baseline: +0.9% Memory: ✅ 42.762MB (SLO: <46.000MB -7.0%) vs baseline: +5.4% ✅ ospathsplitext_noaspectTime: ✅ 467.159µs (SLO: <700.000µs 📉 -33.3%) vs baseline: +0.6% Memory: ✅ 42.861MB (SLO: <46.000MB -6.8%) vs baseline: +5.2% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.423µs (SLO: <20.000µs 📉 -82.9%) vs baseline: 📈 +17.6% Memory: ✅ 35.547MB (SLO: <38.000MB -6.5%) vs baseline: +4.9% ✅ 1-count-metrics-100-timesTime: ✅ 207.842µs (SLO: <220.000µs -5.5%) vs baseline: +0.2% Memory: ✅ 35.527MB (SLO: <38.000MB -6.5%) vs baseline: +4.8% ✅ 1-distribution-metric-1-timesTime: ✅ 3.290µs (SLO: <20.000µs 📉 -83.5%) vs baseline: +0.6% Memory: ✅ 35.507MB (SLO: <38.000MB -6.6%) vs baseline: +4.5% ✅ 1-distribution-metrics-100-timesTime: ✅ 216.311µs (SLO: <230.000µs -6.0%) vs baseline: -0.4% Memory: ✅ 35.488MB (SLO: <38.000MB -6.6%) vs baseline: +4.5% ✅ 1-gauge-metric-1-timesTime: ✅ 2.158µs (SLO: <20.000µs 📉 -89.2%) vs baseline: +0.1% Memory: ✅ 35.547MB (SLO: <38.000MB -6.5%) vs baseline: +5.2% ✅ 1-gauge-metrics-100-timesTime: ✅ 136.989µs (SLO: <150.000µs -8.7%) vs baseline: +0.4% Memory: ✅ 35.507MB (SLO: <38.000MB -6.6%) vs baseline: +4.7% ✅ 1-rate-metric-1-timesTime: ✅ 3.062µs (SLO: <20.000µs 📉 -84.7%) vs baseline: ~same Memory: ✅ 35.527MB (SLO: <38.000MB -6.5%) vs baseline: +4.9% ✅ 1-rate-metrics-100-timesTime: ✅ 222.738µs (SLO: <250.000µs 📉 -10.9%) vs baseline: +1.2% Memory: ✅ 35.507MB (SLO: <38.000MB -6.6%) vs baseline: +5.0% ✅ 100-count-metrics-100-timesTime: ✅ 20.723ms (SLO: <22.000ms -5.8%) vs baseline: -1.4% Memory: ✅ 35.527MB (SLO: <38.000MB -6.5%) vs baseline: +4.9% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.254ms (SLO: <2.550ms 📉 -11.6%) vs baseline: +0.7% Memory: ✅ 35.507MB (SLO: <38.000MB -6.6%) vs baseline: +5.1% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.415ms (SLO: <1.550ms -8.7%) vs baseline: -0.1% Memory: ✅ 35.566MB (SLO: <38.000MB -6.4%) vs baseline: +5.0% ✅ 100-rate-metrics-100-timesTime: ✅ 2.263ms (SLO: <2.550ms 📉 -11.3%) vs baseline: ~same Memory: ✅ 35.507MB (SLO: <38.000MB -6.6%) vs baseline: +5.0% ✅ flush-1-metricTime: ✅ 4.502µs (SLO: <20.000µs 📉 -77.5%) vs baseline: -0.5% Memory: ✅ 35.566MB (SLO: <38.000MB -6.4%) vs baseline: +4.9% ✅ flush-100-metricsTime: ✅ 175.730µs (SLO: <250.000µs 📉 -29.7%) vs baseline: +0.5% Memory: ✅ 35.566MB (SLO: <38.000MB -6.4%) vs baseline: +4.7% ✅ flush-1000-metricsTime: ✅ 2.181ms (SLO: <2.500ms 📉 -12.8%) vs baseline: -0.3% Memory: ✅ 36.313MB (SLO: <38.750MB -6.3%) vs baseline: +4.3% 🟡 Near SLO Breach (1 suite)🟡 tracer - 6/6✅ largeTime: ✅ 31.634ms (SLO: <32.950ms -4.0%) vs baseline: -0.1% Memory: ✅ 36.707MB (SLO: <39.250MB -6.5%) vs baseline: +4.6% ✅ mediumTime: ✅ 3.131ms (SLO: <3.200ms -2.1%) vs baseline: -0.3% Memory: ✅ 35.330MB (SLO: <38.750MB -8.8%) vs baseline: +5.3% ✅ smallTime: ✅ 365.361µs (SLO: <370.000µs 🟡 -1.3%) vs baseline: +4.3% Memory: ✅ 35.193MB (SLO: <38.750MB -9.2%) vs baseline: +4.6%
|
Description
This is an attempt at making a test less flaky (not sure why it's flaky to be honest, so hard to fix for real).
Also, drive-by change to make a test print samples on failure for future debugging/unflaking.