cartesian: [CI Failure | UTests] Caching test using build time check is not stable

[FlorianDeconinck]

A recent Daily CI failure as shown that test test_stencil_object_cache relies on timing rather checking the underlying changes.

The core test looks like this

gt4py/tests/cartesian_tests/integration_tests/feature_tests/test_stencil_object.py

Lines 44 to 61 in 3807bca

	def runit(*args, **kwargs) -> float:
	exec_info: Dict[str, Any] = {}
	stencil(*args, **kwargs, exec_info=exec_info)
	run_time: float = exec_info["run_end_time"] - exec_info["run_start_time"]
	call_time: float = exec_info["call_run_end_time"] - exec_info["call_run_start_time"]
	return call_time - run_time
	base_time = runit(in_storage, out_storage, offset=1.0)
	fast_time = runit(in_storage, out_storage, offset=1.0)
	assert fast_time < base_time
	# When an origin changes, it needs to recompute more, so the time should increase
	other_out_storage = OriginWrapper(
	array=gt_storage.ones(backend=backend, aligned_index=(1, 0, 0), shape=shape, dtype=float),
	origin=(1, 0, 0),
	)
	other_origin_time = runit(in_storage, other_out_storage, offset=1.0)
	assert other_origin_time > fast_time

Though unlikely this can be made to fail if anything happens on the machine at the time of tests since it's based on a single timing.

Couple of solutions to implement:

• Run a few times and take median to avoid machine variation (still potentially unstable)
• Actually test the underlying cache changes in a programmatic way (best)