Subproject ngff-spec under specfications directory

[jo-mueller]

Fixes #399

This is a proposed solution on how a joint layout of the ngff page and the future home of the spec, ome/ngff-spec could look like.

Key changes:

• Deprecated bikeshed: The specs of .5 and 0.6.devX were converted into markdown/jupyter-book and are added as submodules under the specifications folder here in ngff
• Moved submodules: As mentioned before, I removed the current submodule structure with the versions on the repo root and the spec versions living on different branches of this repo. Instead, the submodules point to different branches of the ngff-spec repo
• Consistent build: Since the spec is now written in markdown, the respective md-pages can simply be added to a sphinx toctree directive over here on the ngff page - and as such be part of a uniform page layout. The URLs from which the spec documents were previously served (i.e. https://ngff.openmicroscopy.org/0.5/index.html now redirect to the respective location in the submodule:
  /0.5/index.html --> /specifications/0.5/index.html
  /0.5 --> /specifications/0.5
• Schemas still served at same location: As right now, you can still access and download schemas from, say, https://ngff.openmicroscopy.org/0.5/schemas/image.schema

TODO

• Include changelog, version history into specifications section?
• Add the examples/schemas subsections for the older versions, too?
• Fix providing html-rendered documents from json-schema-for-humans

[github-actions]

Automated Review URLs

• Readthedocs
• render latest/index.bs
• diff latest modified

[lubianat]

should we move with

• change layout and add navbar #408
  before this one?

preview on your link looks good, though

[jo-mueller]

@lubianat #408 makes total sense to go in first. On second and third thought, I think I may also undo the RTD deprecation in here. My original reason for doing this in the first place was the lack of customization options on how to integrate ngff-spec pages in the ngff page.

But since the placing of the spec submodules under the /specifications directory and the direct inclusion into the ngff page's toctree kind of solve that, maybe that's not an urgent need.

[lubianat]

I know this is WIP, but I don't really see 0.1, 0.2 etc there. Am I doing something wrong?

[jo-mueller]

@lubianat not at all :) We only merged the backport for 0.1...0.4 last week and I haven't updated the submodule configuration here yet. Will do asap!

[lubianat]

@jo-mueller can you build this locally?

I was hitting a submodule blocker

git submodule update --init --recursive

fatal: Erro remoto: upload-pack: not our ref 5c9f279a1ab48030ed94dea40ba4a45c4e3cb1bf
fatal: Fetched in submodule path 'specifications/0.3', but it did not contain 5c9f279a1ab48030ed94dea40ba4a45c4e3cb1bf. Direct fetching of that commit failed.

then I restarted it from scratch and it worked:

gh repo clone ome/ngff
cd ngff
gh pr checkout 404
git submodule update --init --recursive
// No errors
uv venv -p 310
source .venv/bin/activate
uv pip install -r requirements.txt
make install 

But I hit another error:

(ngff) ➜  ngff git:(subproject-ngff-spec) make html
Executando Sphinx v8.1.3
✅ Copied schemas for version 0.3
Traceback (most recent call last):
  File "/home/lubianat/Documents/gerbi/ngff/specifications/0.3/pre_build.py", line 5, in <module>
    import jsonc as json
ModuleNotFoundError: No module named 'jsonc'

I am trying to debug my way, but it could be good for us to add developer instructions for local builds

[lubianat]

Trying to see the blockers here from last review round:

• 0.2 instead of 0.1 issue
• "Dev" uppercase: https://ngff--404.org.readthedocs.build/specifications/Dev/index.html
• latest missing: https://ngff--404.org.readthedocs.build/specifications/latest --> 404 error, while https://ngff.openmicroscopy.org/latest/ works
• file extension for schemas (maybe separate?)
• copyright OME statement
• header info rendering weirdly (now it does not render at all)

my 2c is that we could add a statement for the "copyright OME" clause and a warning pointing to the Zenodo DOI for the metadata. Perhaps also fixing the "Dev" and "latest" URLs and try and get this merged. What do you think?

Dev docs, schema file extensions, header rendering, author lists from CITATION.CFF etc could better fit smaller, more reviewable PRs

[jo-mueller]

@lubianat thanks for the review

• Adding latest as its own submodule is going to be tricky here because the entry in the table of contents is generated by the headings in the actual doc. So even adding 0.5 as a latest submodule doesn't make a difference because it carries the same header (i.e., 0.5) as the 0.5 submodule. I added a redirect to the confic though so that https://ngff--404.org.readthedocs.build/specifications/latest resolves correctly to https://ngff--404.org.readthedocs.build/specifications/0.5
• header info rendering weirdly (now it does not render at all): I added the metadata that was just dumped into the document into the document's frontmatter section:

  ---
  metadata is here now
  ---
  

  apparently sphinx is able to parse the abstract field from that section, but not the rest 🙄. I can also remove the frontmatter section entirely from there 🤷‍♂️
• copyright OME statement: What do you mean with that, exactly?
• Dev uppercase: Needs to be fixed over at ngff-spec

[lubianat]

on the copyright statement:

on the current website, bikeshed docs have this:

maybe I am overthinking it, but it seems important

[jo-mueller]

I see! On the ngff-spec repo, the footer exists on the built pages:

Where do you think would be the correct place to put it? Would it collide with the copyright statement that's already on the page?

[lubianat]

Oh, I had not seen that. I think this may be more "political" than I can decide on, but perhaps the NGFF credit is enough?

Maybe that is a @joshmoore question, maybe for the PR review...

[lubianat]

I think it is in a safe shape to merge. Not perfect, perhaps, but good enough.

[jo-mueller]

I think this is ready to pull the trigger so we can go forward with other things :)

[imagesc-bot]

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/ngff-weekly-dev-update-thread/110810/56