RYA-70 web.rya spring xml files don't pull in environment properties …

[brushworth]

…consistently to a well-defined standard. This commit begins to tidy the Accumulo config (MongoDB to come) but more work is required.

Description

What Changed?

The current structure and documentation concerning the environment.properties and spring xml configuration seems to be lacking. Trying to get advanced features working in Tomcat, for example the AccumuloSelectivityEvalDAO or the various indexing strategies, is a very hard slog, particularly for new comers to the project.

I've started to tidy up the accumulo and extension spring xml files. I've tested the accumulo and extensions files on a test cluster.

In future, I propose we add default properties files to the project that work out of the box with Fluo Muchos (or similar) to allow for easy spin up of a development cluster to give Rya a whirl, for example on AWS or Azure. People can then easily edit them to their environment, rather than having to reverse engineer what parameters are available and what default values are in use.

I'm happy to do MongoDB configuration too but I don't have a test cluster running at present.

I'm after feedback from more experienced developers of Rya about whether these changes are heading in the correct direction. For example, I've replaced some of the configuration xml calling setter methods like this:

    <bean id="conf" class="org.apache.rya.accumulo.AccumuloRdfConfiguration">
        <!-- Calls setter method name -->
        <property name="tablePrefix" value="${rya.tableprefix}"/>
        <property name="displayQueryPlan" value="${rya.displayqueryplan}"/>
        <property name="useStats" value="false"/>
        <property name="useStats" value="${rya.usestats}"/>
        <property name="useSelectivity" value="${rya.useselectivity}"/>
        <property name="useStatementMetadata" value="${rya.usestatementmetadata}"/>
        <property name="numThreads" value="${rya.querythreads}"/>
        <property name="batchSize" value="${rya.batchsize}"/>
        <property name="dataWaveEdge" value="${rya.datawaveedge}"/>
        <property name="dataType" value="org.eclipse.rdf4j.model.Statement"/>
        <!--
        <property name="useEntity" value="${sc.use_entity}"/>
        <property name="useGeo" value="${sc.use_geo}"/>
        <property name="useFreeText" value="${sc.use_freetext}"/>
        <property name="useTemporal" value="${sc.use_temporal}"/>
        -->
    </bean>

with

    <bean id="conf" class="org.apache.rya.accumulo.AccumuloRdfConfiguration" factory-method="fromProperties">
        <constructor-arg ref="properties"/>
    </bean>

Additionally, there is a lot of duplication in the properties space. I'm trying understand why. I'm also trying to understand the differences.

For example, there are environment properties accumulo.instance, instance.name and sc.cloudbase.instancename in different places, all obviously referring to the same type of thing? Is this redundancy deliberate, or can I start consolidating it down? Can we get down to a single Rya configuration properties file for an entire Rya installation (e.g. ingest jobs, Tomcat, Fluo, etc)?

I'm still a little lost in the details here, and any background or advice would be much appreciated. It feels like a bit of a mess at the moment.

This pull request is ready to go and makes things a fair bit tidier, but I'm happy to put up a subsequent one taking these ideas further as the community provides feedback,

Tests

Coverage?

N/A

Links

Jira RYA-70

Checklist

• Code Review
• Squash Commits

People To Reivew

[Add those who should review this]

[brushworth]

This change does nothing other than tidy up the properties to make it easier for new users. There is also a name change of the default testing namespace to prevent any accidents from rya_ to ryatest_.

[DLotts]

You said it was a draft. If there is no further comment from anyone, you would call this complete? david.

…

On Sun, Nov 15, 2020 at 10:11 PM Brad Rushworth ***@***.***> wrote: This change does nothing other than tidy up the properties to make it easier for new users. There is also a name change of the default testing namespace to prevent any accidents from rya_ to ryatest_. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#317 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABRUH56JQ423K4S632OTO2TSQCJ45ANCNFSM4PLHKF2Q> .