Using as a template for new plugins: How to install this plugin or derivatives locally for testing?

[redhog]

I'm trying to make a new plugin, and I'm basing it on this one as per OpenCPN developer docs.

I have previously been playing with the radar_pi plugin, which seems to use a very similar build system.
To install that one locally, you'd do mkdir build; cd build; cmake ..; make tarball and end up with a tar file you can import in the plugin manager of opencpn. It has a metadata.xml inside at the toplevel.

This repo has a make package, but no make tarball. And make package generates a tar file and an xml file, but the xml file is outside the tarball. When I try to import it in the plugin manager, it reports "Er ror extracting metadata from tarball". How is this supposed to work / how do you install locally?

[jongough]

The testplugin build system is designed to allow both developer builds and automated builds when pushing to git repositories that are linked to both circleci and cloudsmith, The automated build process allows users to install the plugins from the plugin catalog.
I presume you are still in the development stage so there are a couple of ways of building and testing the plugin.

• You can use your IDE to build using cmake and you will end up with a plugin executable which can be hand copied to the location that OCPN uses for plugins.
• You can use 'circleci local' for some environments which circleci supports for local builds.
• You can use a command prompt, or script/shell/command file, to build the packages based on the circleci/config.yml file for your platform

I think you are using the third option which does create the installable files. If you look in the tar file you should see an xml file in the root directory which is the same as the external xml file. The internal file is used when importing the tar file to install it, the external xml file is used by plugin manager to present the plugin information in the catalog. The xml files should be exactly the same.

To install the locally build file use plugin manager in OCPN settings and use the 'Import plugin...' button. If this is not available then set 'CatalogExpert=1' in the opencpn.ini/conf in the [PlugIns] section. This will use the internal version of the xml file for plugin manager.

What platform are you building on and what platform are you trying to build for?

[redhog]

You guessed right, I'm just starting to build the plugin, so running it on the same laptop where I build it is my usecase. I'm on debian, and trying to build for debian. I'd indeed like to install it using the 'Import plugin...' button.

What I tried to do is what I thought was the third option:

mkdir build; cd build; cmake ..; make package

which is everything except the upload from build-local-package-example.sh

This produced

testplugin_pi-0.1.0.0-debian-x86_64-12-bookworm.tar.gz
testplugin_pi-0.1.0.0-debian-x86_64-12-bookworm.xml

The tgz did not include a metadata.xml anywhere, which is what opencpn seems to expect?

(As for IDE, I just use emacs and the command line... :P)

[jongough]

There are two example scripts in the root directory for local builds, one issues the commands to build and the other uses the 'circleci local' build process. I suggest that you look at build-local-package-example.sh and it will generate, on your machine, the installable package. This is done in the cloudsmith-upload.sh with the LOCAL_DEPLOY set to true.

I just created a build-local directory in my testplugin directory, then issued bash ../build-local-package-example.sh which built testplugin and created testplugin_pi-1.0.316.0-ubuntu-x86_64-22.04-jammy.tar.gz in the build directory, I then imported it and it worked fine. I did modify the default in the script file, I changed bookworm to jammy as that is the version of ubuntu I was running in a virtual machine. The file contents are:

#!/bin/bash
# This file should be run from the local build directory you want to use
# This file is a modification of the 'circleci local' build process, so it will complain the circleci-output directory is not found.
# If you want to do a 'circleci local' build use the 'run-circleci-local.txt' file as the example to follow.
#
# Change the exports to meet the build requirements, i.e. the values in the config.yml file relevant to the particular job
#export OCPN_TARGET=bookworm
export OCPN_TARGET=jammy
export BUILD_GTK3=true
export WX_VER=32
export LOCAL_DEPLOY=true
# this removes old xml files from the build directory
rm *.xml
# the actual configuration, build and installable package creation
cmake -DCMAKE_BUILD_TYPE=Debug ..
make -j2
make package
chmod a+x cloudsmith-upload.sh
./cloudsmith-upload.sh

[jongough]

If you are using Linux Mint then you should add: export BUILD_ENV=debian to the list to override what is found by lsb_release.

Basically you can build for environments supported by your base level operating system. If you need to build for others you will need to use circleci local, but there are still a few environments that use ORB environments in circleci that cannot be built locally without have the enderlaying environment available.

For IDE's I use kdevelop (on ubuntu) and VScodium, it just makes life easier as they allow easily finding where things are defined and used, as well as assisting in debugging. If your plugin is going to be more complex then an IDE can really help. I would find ocpn_draw very difficult using a text editor, and most IDE's have an easier interface to gdb for better debugging.

Whatever you use if you can get testplugin built and installed in OCPN that that is a good start. This plugin was what I used when building the OD API sets, but is now used as the base for the automated build process. If you decide to use this as a base it has all that is needed, you will just need to change the file names and probably the method names and possibly some variable names, to make the plugin make more sense, but......

[redhog]

Ah. So when I followed the steps of build-local-package-example.sh exactly, it worked.

So, what was the super unobvious step? That ./cloudsmith-upload.sh doesn't just upload the .tar.gz, but modifies it in place, adding the metadata.xml. I wrongly assumed it would only upload files, made by make package. Why isn't this part of make package? And why modify files in place (as opposed to making a new file next to it)?

It would be helpful to at least add a note about this in the README of the repo or something...

[jongough]

I added the example.sh file to show how to build using the 'automated' build process on the local machine. Basically the config.yml sets up the environment to allow the platform specific script to run. The cmake files have had changes made to allow this to happen whilst still supporting an IDE/cmd prompt to issue cmake/make commands.

cloudsmith-upload.sh is dynamically created in the cmake process from {project}/cmake/in-files/cloudsmith-upload.sh.in, as it is only at the build time is the required information known and what stage (alpha, beta, prod/master) the application is particularly when building via github/circleci.

The build process creates the executable, the upload process creates the installable, by plugin manager, files including the xml file. Seems a bit strange at first, but it does make it easier to make changes to the plugin then put it through whatever process you want for release to 'prod'. My github generates alpha versions when pushing code, beta versions when tagged in the non-master stream and beta when updates are accepted into the master version. It only generates prod versions when a new release number/tag is generated in github. For me this makes it easier to test and release code in a controlled fashion.

It is only recently that I have added the build scripts as examples to help developers. I normally don't use that as I build with an IDE, copy the plugin executable to the correct location in OCPN then run the debugger which lets me single step through the code if needed. Doing it the with the script will make debugging potentially harder, but.....

I am quite happy for you to update the README and provide a pull request. You, being new to this process, will know what you found unclear and can help clarify things. I already know how to use it and do miss things that new starters find confusing. I can have a go at it, but I may not express myself clearly enough.

[redhog]

I made this PR: #412 I also included a small script to do the "refactoring" of renaming the plugin.

See if this makes sense to you! I've added an empty section for you to fill in about the IDE setup / workflow you use.

[redhog]

Would you be open to forking out a "EmptyPlugin" out of this repo? The same build structure etc, but where the plugin doesn't do anything (so w/o all the JSON parsing, icons etc)?

Finding out exactly what is required minimal functionality to not get an error when importing the plugin, and what it optional, is a bit of work for someone new to the codebase, so it would be good to have an "empty slate" to start with.

(For example, the plugin I'm gonna build eventually only really needs SetSentence() and a preference dialog...)

I've made a PR for this. It's not intended to be merged, but so you can see what is being deleted, and what would be the basis for a new template repo: #414

[jongough]

The point of this plugin is that it is fully functional and uses most of the facilities available in OCPN. It is also used to test the OD API, both binary and JSON. It also includes JSON schema parsing for robustness. If a developer does not want that then just remove the classes involved. Each plugin has different requirements, some only need to be started, others have complex interactions. It can be difficult creating a new plugin from scratch, but at least with the current testplugin it is a complete working set including dialogs (and formbuilder data for easier modifications), control, installation, interactions, etc.

It may be a better idea to identify which bits you consider 'not needed' and then put comments in the class files and mainline code to express this. I do this for the JSON processing in the cmake files. All the icons provided are those that are needed to allow the plugin to work. For a new developer they could use these then, when ready, change them to match their plugin.

I have looked at your changes and I think they make the plugin too trivial. It took quite a bit of research and testing to come up with what is there. It is easy to delete, but can be really difficult to identify and create some of the stuff in testplugin. The automated build process is also not easy to build from scratch, hence using this plugin as a base. As it uses 'unusual' facilities it tests out some of the more unusual aspects on some platforms. It does work on all the OCPN target platforms, but on some I am not really sure it is usable due to the interface not being rich enough to easily support some functions.