## Root cause
A findList() triggers a secondary @OneToMany query that populates from the immutable cache. ImmutableBeanCaches.QueryLoader runs find(type).setUnmodifiable(true)...findMap(), which hits the L2 bean cache (cacheIdLookup → loadBeanDirect). Two coupled bugs in the unmodifiable path:
## 2 Bugs:
1. Null pc (the NPE). In BeanDescriptorCacheHelp.loadBeanDirect, the if (context == null) context = new DefaultPersistenceContext() fallback was nested inside if (!unmodifiable). So an unmodifiable load passed a null context to CachedBeanDataToBean.load; converting a @ManyToOne (refBean → contextGet → pc.get) NPE'd — exactly Eddie's trace.
2. Mutable reference can't freeze. Once a context was provided, refBean created the @ManyToOne ref via createRef → a mutable InterceptReadWrite bean. The subsequent unmodifiableFreeze then threw UnsupportedOperationException("never expected") (only InterceptReadOnly is freezable).
## Fix (2 files)
- BeanDescriptorCacheHelp.loadBeanDirect — always ensure a non-null context before CachedBeanDataToBean.load (hoisted out of !unmodifiable).
- BeanPropertyAssocOne.setCacheDataValue/refBean — when the owning bean is unmodifiable (derived via !(intercept instanceof InterceptReadWrite), the same idiom CachedBeanDataToBean already uses), create the ref with createReference(unmodifiable, false, id, pc) → a freezable InterceptReadOnly reference. Also guards contextGet against null. Modifiable path is unchanged.
## Notes
The "fetch(path) without a FetchGroup" clue
That's the trigger, not a misuse. A restricting FetchGroup can exclude the @ManyToOne, so no refBean runs and the bug stays hidden. The default/fetch("path") select includes the FK, so the cached-bean conversion creates the assoc-one reference and hits the bug. His usage was fine — this was an Ebean gap (no immutable-cached entity with a @ManyToOne was test-covered).
## Problem
SequenceIdGenerator captured a single DataSource at deploy time and held one shared pre-fetch buffer. Under multi-tenancy this is wrong:
- TenantMode.DB/DB_WITH_MASTER — there is no bootstrap DataSource, so BeanDescriptorManager passed null; sequence allocation couldn't resolve the current tenant's database.
- TenantMode.SCHEMA/CATALOG — a single shared buffer let one tenant's pre-fetched ids be handed to another (cross-tenant bleed), and pre-fetch used a connection that wasn't scoped to the requesting tenant.
(Supersedes the per-datasource delegator approach in #2305, which leaked via a WeakHashMap whose values strongly referenced the keys, only handled DB mode, and re-resolved the tenant on the background thread.)
## Fix:
Make SequenceIdGenerator itself tenant aware, keeping all platform modules untouched.
- New TenantConnectionSource (ebean-api, additive): optional interface a DataSource may implement — currentTenantId() + connectionForTenant(tenantId).
- SequenceIdGenerator: the shared idList/lock/loading flag become a per-tenant TenantBuffer keyed by tenantId in a ConcurrentHashMap. Connections are obtained per tenant. Background pre-fetch captures the tenant at submit time (the executor thread has no tenant in scope) and fetches by explicit tenantId — fixing a latent ThreadLocal-propagation bug.
- SequenceDataSource (ebean-core): adapts DataSourceSupplier to TenantConnectionSource; routes to the tenant DB (DB mode) or sets schema/catalog (SCHEMA/CATALOG).
- Wiring: InternalConfiguration exposes the DataSourceSupplier; BeanDescriptorManager wraps it only for dynamic-datasource tenant modes.
## Performance (single-tenant unaffected)
- A cached single buffer field short-circuits the ConcurrentHashMap for the non-tenant key.
- NONE/PARTITION pass the plain DataSource (not wrapped), so tenantSource == null and the hot path is just a couple of cheap branches — equivalent to the original.
## Compatibility
- Platform constructor signature (be, ds, seqName, allocationSize) unchanged — no changes to the 9 platform modules.
- One protected-method signature changed: getMoreIds(int) → getMoreIds(Object tenantKey, int). Rarely overridden (subclasses override getSql/readIds), but a source-incompat for any external custom platform that did.
## Tests
- TenantSequenceTest — DB-per-tenant: tenant 1 → 1,2,3; tenant 2 independently → 1.
- SequenceBatchIdGeneratorTest adapted to the per-tenant buffer.
- Existing sequence + multitenancy suites pass.
## Potential Follow-ups (not in this PR)
- Add a SCHEMA-mode test.
- Optional removeTenant(tenantId) hook if unbounded tenant churn is a concern (buffers hold only Longs + a lock, no DataSource, so no real leak).
- SimpleSequenceIdGenerator (non-batching) left as-is — already uses the txn connection.
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
Bug (#3083): findVersions() on a @History root that joins to another @History entity bound the joined table's effective-date predicate with a null as-of timestamp → systime <@ null (Postgres) / sys_period_start <= null (H2) → zero rows.
Fix (CQueryPredicates.bind(), +6 lines): when the as-of is null but there are history-view join predicates to bind (asOfTableCount > 0 — only ever true for findVersions/findVersionsBetween), default to the current timestamp. Semantics: all versions of the root, joined to related @History entities as they are now. The root table is untouched (still returns all versions).
Test ported from #3082 :
- New HistoryManyToOne entity (@History + @SoftDelete) — kept lean, dropped the PR's unneeded @OneToMany
- HistorylessOneToOne gains @ManyToOne HistoryManyToOne
- New testVersionsWithHistoryOverHistoryless() — replaced the PR's always-failing placeholder (assertSql(...).contains("this does not exist")) with real assertions: count == 1 and SQL contains no asOf null bind
- Left the existing @OneToOne unchanged (the PR's optional = false tweak to the shared model wasn't needed)
* Fix NPE resolving generic types across multi-level mapped superclass hierarchies
The fix introduced in ebc90e0e resolved TypeVariables only one level at a time:
mapGenerics(beanType) read only the direct generic superclass, so for a chain like
A extends B<String> / B<T> extends C<T>, processing C's fields produced an empty
map, genericTypeMap.get(TypeVariable) returned null, propertyType became null, and
AnnotationFields.readField threw an NPE calling prop.getPropertyType().isEnum().
Fix: build the full type-variable map once for the concrete bean type using
TypeResolver.getTypeVariableMap, which walks the entire superclass/interface
hierarchy and composes TypeVariable bindings transitively. The same map is passed
at every level of the recursive createProperties walk, so any TypeVariable at any
depth resolves correctly.
Type-resolution helpers (resolveType, resolveToClass, resolveCollectionTarget,
ResolvedParameterizedType) are consolidated in TypeReflectHelper so they are
shared across callers and independently testable.
Tests: TypeReflectHelperTest covers single and multi-level TypeVariable resolution
and collection-element resolution. QProductWithGenericTest adds an integration
regression test using a two-level generic chain
(ProductWithGenericMiddle extends GenericMiddleModel<Long> extends GenericBaseModel<Long>).
* Restore prior format only on DeployCreateProperties
---------
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
Otherwise, can produce NullPointerException:
``
Caused by: java.lang.NullPointerException
at io.ebeaninternal.server.type.DefaultTypeManager.keepSource(DefaultTypeManager.java:340)
at io.ebeaninternal.server.type.DefaultTypeManager.dbJsonType(DefaultTypeManager.java:327)
at io.ebeaninternal.server.deploy.parse.DeployUtil.setDbJsonType(DeployUtil.java:207)
at io.ebeaninternal.server.deploy.parse.DeployUtil.setDbJsonBType(DeployUtil.java:201)
at io.ebeaninternal.server.deploy.parse.AnnotationFields.initDbJson(AnnotationFields.java:227)
at io.ebeaninternal.server.deploy.parse.AnnotationFields.readField(AnnotationFields.java:133)
at io.ebeaninternal.server.deploy.parse.AnnotationFields.parse(AnnotationFields.java:62)
at io.ebeaninternal.server.deploy.parse.ReadAnnotations.readInitial(ReadAnnotations.java:29)
...
``
Adds transaction-level control over whether Ebean auto-generates values for @WhenCreated, @WhenModified, @WhoCreated and @WhoModified properties.
Motivation
Backup/restore scenarios need to preserve the original audit timestamps and user values when re-inserting exported data. Without this, every save overwrites those fields with the current time/user.
Usage
try (Transaction txn = DB.beginTransaction()) {
txn.setGeneratedPropertiesEnabled(false);
bean.setWhenCreated(originalTimestamp);
bean.setWhenModified(originalTimestamp);
DB.save(bean);
txn.commit();
}
Behaviour
- Disabled (false): generated property values are only written if the property currently has a null value. Any non-null value set on the bean is preserved.
- @Version is unaffected: the version property always auto-increments regardless of this setting, preserving optimistic locking integrity.
- Default is true: all existing behaviour is unchanged.
Files changed
- Transaction — new setGeneratedPropertiesEnabled(boolean) with javadoc
- SpiTransaction — new isGeneratedPropertiesEnabled()
- SpiTransactionProxy — delegates both methods
- JdbcTransaction — field + implementation (default true)
- NoTransaction, ImplicitReadOnlyTransaction — no-op setter, true getter
- PersistRequestBean — onInsertGeneratedProperties, onUpdateGeneratedProperties, onFailedUpdateUndoGeneratedProperties all gate on isGeneratedPropertiesEnabled()
- TestGeneratedProperties — 3 new tests covering insert-preserves, insert-null-still-filled, and update-preserves
Supersedes
PR #2943 — same feature, renamed from setOverwriteGeneratedProperties to setGeneratedPropertiesEnabled for a clearer, positive-sense API.
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
Extends @Formula2 so it can be placed on a @ManyToOne field. Instead of embedding physical SQL aliases (as @Formula(join=...) requires), a @Formula2 expression uses logical bean paths — the required joins are derived automatically.
Example
``
// Before — physical SQL, brittle alias wiring:
@Formula(select = "coalesce(${ta}.some_bean_id, j1.some_bean_id)", join = PARENTS_JOIN)
@ManyToOne EBasic effectiveBean;
```
```
// After — logical paths, joins resolved automatically:
@Formula2("coalesce(someBean.id, parent.someBean.id)")
@ManyToOne EBasic effectiveBean;
``
The generated SQL is identical; the annotation is far more readable and maintainable.
What changed
Annotation parsing (AnnotationAssocOnes) — @Formula2 on a @ManyToOne is now recognised and the expression parsed into a select fragment and a set of dependency join paths.
Query tree building (SqlTreeBuilder) — three scenarios all handled correctly:
- Fetched as a tree node — dependency joins are inserted before the formula2 join using addChildFirst
- Partial parent fetch — dependency joins are registered even when the parent chunk only selects its ID column
- Predicate-only (where clause, no fetch) — addFormula2JoinsFromPredicates runs before buildSelectChain so dependency join paths are populated before buildExtraJoins constructs the extra-join tree; IncludesDistiller uses addChildFirst to preserve ordering within the extra-join tree
Init ordering (BeanDescriptorManager) — initFormula2Properties() moved to a dedicated pass 5, after all descriptors are fully initialised, so cross-descriptor path resolution (e.g. parent.parent.someBean.id) is always safe.
SqlTreeNodeExtraJoin — gained addChildFirst() to match SqlTreeNodeBean, enabling formula2 dependency joins to be prepended ahead of the formula2 property join in the extra-join tree.
Fixes
Supersedes and resolves#2773 — the reported bug (wrong join ordering when combining fetch and where on a formula-joined field) is eliminated for ChildPerson.effectiveBean and ParentPerson.effectiveBean, which are now expressed as @Formula2.
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
The test-java16 module needs to run AFTER a mvn install now due
to the SequencedSet/SequencedMap MR-JAR setup (as without the mvn
install it picks up BeanSet/BeanMap from target/classes and that's
the Java 11 version of BeanSet/BeanMap.
- DtoMetaDeployProperty: remove unused Method parameter from constructor
- DtoMetaProperty.findField: use loop variable 'type' (not outer 'dtoType') so
superclass fields are correctly found
- DtoMetaProperty constructor: call findMetaAnnotations() which merges both
field and setter annotations, rather than reading field annotations only
- DefaultTypeManager: keep both MethodType import and Annotation import
- Fix missing imports in EbeanServerFactory_ServerConfigStart_Test
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Root cause: SaveManyBeans.removeAssocManyOrphans() only called setModifyListening when insertedParent=true. A first save with null O2M skipped it; the lazily-initialized collection on subsequent saves had no listen mode → .clear() untracked → orphan not deleted.
- Fix: Added else { setListenMode(c, many); } to set the listen mode for existing collections that were never initialized (uses the existing null-guard helper).
- Tests: testModifyListenModeSet2 now passes; all 26 cascade tests green.
-------------
Original #3656 description:
We found an issue, when O2M relations are not correctly persisted to the DB.
This happens, when
a bean is saved and the O2M property is empty
something is added and cleared again in two subsequent saves.
the same master-bean object has to be used
The issue here is, that the BeanCollection is lazily initialized with no modifyListenMode set after the first save.
This happens only for O2M relations with no order column. (Others work fine) See: https://github.com/ebean-orm/ebean-agent/blob/d4c40f1ce85c58f99cb0a85152aaa0a0075a9c01/ebean-agent/src/main/java/io/ebean/enhance/entity/FieldMeta.java#L469
And we need also a save, where the bean is saved with an empty/null value in the O2M property.
Subsequent saves will not update the modifyListenMode. See SaveManyBeans
if (insertedParent) {
// after insert set the modify listening mode for private owned etc
c.setModifyListening(many.modifyListenMode());
}
We found this in one of our unit-tests, where we've configured a bean for different states. It is probably something, that should not be too critical in real code, as you normally save a bean only once (When the bean was retrieved from DB, it should not occur)
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
This is a replacement for #3150
**1. Read `dependentTables` from the query plan (single source of truth)**
`OrmQueryRequest` accumulated query-cache `dependentTables` into a field via
`addDependentTables(...)`, fed from per-CQuery `dependentTables()` helpers. The
`CQueryPlan` already owns this set, so the copying is redundant.
- `putToQueryCache(...)` now reads `dependentTables` directly from the request's
`CQueryPlan` at cache-put time.
- Removed the `OrmQueryRequest.dependentTables` field and `addDependentTables(Set)`.
- Removed the now-unused `dependentTables()` helpers from `CQuery`,
`CQueryRowCount` and `CQueryFetchSingleAttribute` (and their `Set` imports).
This is behaviour-preserving: each CQuery is built with exactly the plan stored
under `request.queryPlanKey`, so `request.queryPlan().dependentTables()` is the
same set the per-CQuery helpers returned.
**2. Fix a query-plan trim race that could null the plan at put time**
A freshly built `CQueryPlan` was stored in the plan cache with
`lastQueryTime == 0`, making it immediately eligible for `trimQueryPlans`
(runs every 60s; TTL default 300s) during its *first* execution. If the trim
fired mid-execution, `request.queryPlan()` could return `null` at put time — and
a cache entry with `null` dependentTables is never invalidated by table
modifications (`TableModState.isValid`), i.e. latent stale data.
- `CQueryPlanStats` now initialises `lastQueryTime` to construction time, so a
new plan is only trim-eligible after a genuine TTL idle. `trimQueryPlans` is
the sole consumer of `lastQueryTime()`, so this is safe.
- `putToQueryCache(...)` skips the put when the plan is `null` (fail safe rather
than caching an un-invalidatable entry) for the residual pathological case
(a single query running past the TTL on first execution).
### Tests
- Added `testFindSingleAttributeOnDependent` and `testFindListOnDependent` to
`TestQueryCacheTableDependency`, asserting cache-hit then dependent-table
invalidation for the single-attribute and findMany paths (the count path was
already covered).
- All cache/query tests pass: 69 in `ebean-test` `org.tests.cache.**`, 929 in
`ebean-core` cache + query packages.
* Add support for Java 21 SequencedSet and SequencedMap
Such that these can be used in place of Set and Map if desired.
* Refactor BeanSet, BeanList, BeanMap replacing setActualSet|List|Map
Replace with collectionAdd() and refresh() methods.
* Refactor rename method getBeanCollectionAdd() -> collectionAdd()
* Build needs to use Java 21 to support the multi-release jar
* Update SequencedSet etc from recent changes
* Update build, needs package to use MR-JAR
---------
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
@Formula2 is a logical-path alternative to @Formula and a full replacement for it.
Where @Formula requires physical SQL (${ta} placeholders and hand-written joins),
@Formula2 takes a property-path expression and resolves the required joins
automatically:
@Formula2("coalesce(familyName, parent.familyName)")
String derivedFamilyName;
The path prefixes (parent, parent.parent, ...) define the joins needed to
satisfy the formula. Supported everywhere @Formula is:
- select() — root and nested-fetch paths, auto-joined
- where() — predicate paths trigger the extra joins
- orderBy() — order-by paths trigger the extra joins
- having()
- DDL — excluded from generated columns (read-only, like @Formula)
Default behaviour matches @Formula: included in the default select unless marked
@Transient, in which case it is opt-in but still auto-joins when selected.
Implementation:
- Parse @Formula2 after associations are wired, into a logical select fragment
plus the set of join paths (BeanDescriptor/BeanProperty).
- DeployPropertyParser registers formula2 join paths into query includes so the
existing predicate-include -> extra-join machinery builds the LEFT JOINs.
- ElPropertyChain prefixes all ${}/${path} placeholders for nested-path use.
- SqlTreeBuilder accumulates formula2 joins for select paths.
Replace isMapValueTypeObject + isMapStringString with isBuiltinJsonMap:
the built-in JSON map handles a String or enum key with a String, Object
or wildcard value. This also routes Map<Enum,String> to the built-in type
and fixes non-String/enum keys with Object value (e.g. Map<Integer,Object>)
being mis-routed to the built-in type (ClassCastException) instead of the
object mapper.
Addresses #3735. Enum-keyed JSON maps previously failed (enum key cast to
String). Add ScalarTypeJsonMapEnum which converts enum keys via their
ScalarType (honouring @DbEnumValue) and reuses ScalarTypeJsonMap for all
storage/platform handling, so it works for VARCHAR/CLOB/BLOB and Postgres
JSON/JSONB with no per-storage variants.
- JsonStorage now reports jdbcType(), enabling a shared storageFor(...) reused
by the plain and enum-key Map types
- DefaultTypeManager routes Map<Enum,Object> to the new type and calls
setAccessible on the @DbEnumValue method (supports nested/non-public enums)
- add TypeReflectHelper.getMapKeyTypeRaw and TestEnumKeyMap
Collapse the per-storage and per-platform ScalarType subclass explosion for the built-in JSON value types into two orthogonal strategies.
- Add JsonStorage strategy (VARCHAR / CLOB / BLOB / Postgres) encapsulating
how the raw JSON string is read from / bound to JDBC. Postgres vs
non-Postgres is now a single reusable strategy rather than a subclass per
value type.
- Add ScalarTypeJsonValue<T> base holding the shared read / bind / L2-cache /
json plumbing once, plus ScalarTypeJsonCollectionValue<T> adding the
ScalarTypeArray (DB array column definition) aspect for List/Set.
- Rewrite ScalarTypeJsonMap, ScalarTypeJsonList and ScalarTypeJsonSet as thin
types: a typeFor factory selecting a JsonStorage + value marshalling that
delegates to the avaje-JsonMapper-backed EJson facade.
- Remove ScalarTypeJsonMapPostgres and ~16 nested storage/platform classes.
Reworks the DJsonService (the SpiJsonService SPI behind io.ebean.text.json.EJson) to use avaje JsonMapper instead of the bespoke EJsonReader/EJsonWriter, consolidating all read/write logic into a single JsonAdapter.
Changes
- New EbeanJsonAdapter — a JsonAdapter<Object> that materializes JSON into plain or modify-aware Map/List/Set, with two shared singletons (PLAIN, MODIFY_AWARE). Preserves existing EJson semantics: - Integral numbers → Long, decimals → BigDecimal
- Write coverage for String/Integer/Long/Double/Float/BigDecimal/Boolean/Map/Collection with a toString() fallback for other types (UUID, enum, etc.)
- Modify-aware loads share a single ModifyAwareFlag owner per root, reset to non-dirty once after the load completes
- DJsonService now builds one JsonMapper + two JsonMapper.Type<Object> and routes parse/write through them. Null serialization is retained via a serializeNulls(true) writer; null/blank-input guards, token-honoring entry points, and parseSet (modify-aware asSet()) are preserved.
- EJsonReader/EJsonWriter are now unused and should be deleted (couldn't remove them in this environment).
Why
Simplifies Ebean's JSON handling by reusing avaje-json-core's JsonMapper rather than maintaining a parallel reader/writer, while keeping behavior identical.
So when we used to get an inline comment like:
```sql
select /* Customer.hiLabel */ ...
```
We started to instead have (missing bean type):
```sql
select /* hiLabel */ ...
```
This fixes that regression that was introduced in #3779
robin.bygrave