Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
sub_stores/SubStorageAdapterconcept) to the Java SDK.metadata(add) /filters(search).ConfigLoaderto parse sub-store configuration from.env/ env vars (format inspired by Python.env.examplestyle).Motivation
Python PowerMem supports
sub_storesto partition data across multiple physical collections/tables and/or multiple embedding models, routing by metadata filters. This is useful for:tenant=A→ store A)This PR brings the same capability to the Java SDK.
What’s included
1) Storage adapter routing (
SubStorageAdapter)SubStorageAdapternow implements real routing instead of being a stub.(key,value)pairs inroutingFiltermust match the incomingmetadata/filters.listSubStores()getTargetStoreName(filtersOrMetadata)isSubStoreReady(storeName)/setSubStoreReady(storeName, ready)Files:
src/main/java/com/oceanbase/powermem/sdk/storage/adapter/SubStorageAdapter.javasrc/main/java/com/oceanbase/powermem/sdk/storage/adapter/StorageAdapter.java(addsembed(text, action, ctx)hook)2) Memory wiring (use routed embedder + store)
Memorynow:SubStorageAdapterwhenMemoryConfig.subStoresis configuredstorage.embed(...)so the right embedder is used per storeget()throughstorage.getMemory(...)so records can be found in sub storesFile:
src/main/java/com/oceanbase/powermem/sdk/core/Memory.java3) Configuration model
SubStoreConfigfor a sub store definition.MemoryConfig.subStores.SubStoreConfig.readyto emulate Python’s “migration readiness” gating (see parity notes below).Files:
src/main/java/com/oceanbase/powermem/sdk/config/SubStoreConfig.javasrc/main/java/com/oceanbase/powermem/sdk/config/MemoryConfig.java4) ConfigLoader:
.env/ env var parsingConfigLoadernow supports:SUB_STORES_COUNT=...(optional, indices can also be auto-detected)SUB_STORE_0_COLLECTION=...SUB_STORE_0_ROUTE_<KEY>=<VALUE>(keys are normalized to lower-case)SUB_STORE_0_...SUB_STORES_JSON=[{...}, {...}](best-effort)Files:
src/main/java/com/oceanbase/powermem/sdk/config/ConfigLoader.java.env.examplesrc/main/resources/.env.exampleUsage example (dotenv)
Python parity notes (differences and alignment)
Aligned / implemented
listSubStores/getTargetStoreName/isSubStoreReadyequivalents exist in Java.Intentional differences / current gaps
sub_storesfor OceanBase (non-OB configs are ignored with a warning).VectorStore(including SQLite), which is useful for local/offline testing.sub_store_migration_statusto persist readiness and supportsmigrate_to_sub_store(...).SubStoreConfig.ready+setSubStoreReady(...)provide a lightweight readiness gate.migrate_to_sub_store(...)parity is a follow-up.Test plan
mvn testSubStorageAdapterRoutingTest(routing + CRUD/search end-to-end on sqlite)SubStorageAdapterReadyTest(ready=false fallback behavior)SubStoreConfigLoaderTest(dotenv/env map parsing)Follow-ups (optional)
migrate_to_sub_storeto fully match Python behavior.