A python library containing a reader and data class for vernier output

[EdHone]

Description

This adds a python library with two classes which enable a user to read vernier output into a VernierData class. This class has a simple filter routine which allows a user to extract the calipers that they are interested in, as well as a simple routine to print the averaged output to a timer.txt-like table.
I've also added an example script which demonstrates the usage of these stuctures, along with some example vernier output data.

Linked issues

Closes # (issue)

Type of change

• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How has this been tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• New tests have been added
• Tests have been modified to accommodate this change

Checklist:

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes, for both debug and optimised builds

[mo-marqh]

if we're already using numpy, then why are these list[float] rather than np.array?

[EdHone]

Numpy append creates a new array each time rather than using an dynamically allocated block of memory, so in short we use list[float] because it's a lot faster to build our dataset this way. We could allocate our data at the beginning but this was still more expensive and a bit clunky so I went down the route of using basic lists.
Performing the single allocation when doing something like np.mean(timers.data['timestep_alg'].total_time) is really cheap, but we do miss out on the nice functionality of being able to just perform timers.data['timestep_alg'].total_time.mean(). We could probably implement a solution that captures the best of both worlds in the future.

[mo-marqh]

coding standards; prefer newline at end of each file

[EdHone]

coding standards; prefer newline at end of each file

Fixed in commit 68a901

[EdHone]

Open questions:

• How should the data be presented in the txt output?
  • Are we missing max/min?
• Should aggregate be exposed to users of the library?
• Should there be a different aggregator implementation?
• Do we want to (are we able to) remove the post-process.py script?

[EdHone]

@mo-marqh - I think this is now ready for review and merge

[mo-marqh]

I'd like
https://github.com/MetOffice/Vernier/blob/main/post-processing/post-process.py
removed as part of this change please, so that the commit is replacing that code with this (not having them side by side).

i'd also prefer to see a flat layout,
https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/
the lib/ directory seems superfluous.
prefer
post-processing/vernier
post-processing/tests
post-processing/tools

contents of the post-processing/example should move to post-processing/tools so they can easily be provided as CLI runnables
Additionally tests should include a test that runs the CLI tools (i didn't think this was tested here currently)

[mo-marqh]

#!/usr/bin/env python3
please

[EdHone]

Thanks Mark - I believe I've addressed all your review comments and fixed a bit of non-deterministic behaviour when reading on multiple processes. I'll pass the PR back to you