1338 Commits
Author SHA1 Message Date
robin.bygrave 14e6e3f824 With InsertOnConflict fallback to PK constraint when no Unique constraint and id value provided
This is for the case where the primary key is a supplied value (like a string, iso code, serial
number, bar code etc)
2026-07-16 23:51:29 +12:00
robin.bygrave 857e92583d enh: Add findOneOrThrow() with decent derived message for EntityNotFoundException
Adding this as I am seeing a LOT of cases where ebean can derive a good enough
error message (id case, plus single unique key case) that it is worth adding
this syntax sugar.

Yes we have discussed this a well it's not so useful for the multi-lingual /
 non-english case and also marginal when the error message needs to be built
 case.
2026-07-16 23:15:15 +12:00
robin.bygrave 149ed8318d Fold id-only *ToOne fetches into parent select, avoiding unneeded join (#3643)
When a *ToOne association is fetched with only its id property, the
foreign key column already exists on the owning table so no join is
required. Previously this always generated an unnecessary join.

- OrmQueryProperties: add includesExactly() and withAddedInclude() to
  support adding an include without mutating a shared/cached instance
  (the included set and its precomputed hash prefix are immutable
  since #3761/#3763, so an "add" must produce a new instance).

- OrmQueryDetail: add convertIdFetches(), a bottom-up pass over
  fetchPaths that replaces an id-only *ToOne fetch with its FK
  property folded into the parent's select, removing the now-unneeded
  fetch path. Skips exported (mappedBy) one-to-one associations, which
  have no local FK column and always require a join. Tracks paths
  still depended on by a surviving child fetch so their join isn't
  incorrectly folded away.

- Add TestFetchIdOnly covering PathProperties, select(), and fetch()
  usages, plus a control case confirming the join is retained when
  more than the id is fetched.

- Update EqlParserTest, TestSubQuery and TestMergeCustomer assertions
  to reflect the new join-free SQL.
2026-07-16 22:28:42 +12:00
robin.bygrave f38f0ffa25 InsertOnConflict - change to exclude non-updatable and generated on insert only (e.g. @WhenCreated) 2026-07-16 17:12:46 +12:00
robin.bygrave d083b27e91 Version 18.3.0 2026-07-16 16:57:52 +12:00
Rob Bygraveandrobin.bygrave 636bb28df3 #2540 Add DTO mapping from Entity beans via generated mapper code (#3852)
* #2540 Add DTO mapping from Entity beans via generated mapper code

A large feature, refer to the docs / guides etc. This is a bit like MapStruct in that it generates mapping code via annotation processing but it ALSO creates an optimised FetchGroup for the destination DTO graph and it also implements a generic interface such that it can be fluently used via query.mapTo().

The value is that it automates the mapping code from entity -> dto plus optimises the query used to support building the DTO graph.

Expected to be better to use than MapStruct for ebean orm users.

* Bump ebean-annotation to 8.6 with the dto annotations

* Adjust the build order to via test dependency on ebean-api

---------

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-16 08:40:41 +12:00
Rob Bygraveandrobin.bygrave 84a311e521 Bug: filterMany() expressions when additional nested fetch has predicates in wrong place (#3851)
* Bug: filterMany() expressions when additional nested fetch has predicates in wrong place

When a filterMany() expression only references the many-root's own properties (e.g. .eq("status", ...)),
but the query also has an additional nested fetch beneath that many-path (e.g. .fetch("orders.customer")
or .fetch("contacts.group")), the filter predicate was incorrectly attached to the last/deepest join's
ON clause instead of the many-root's own join — silently misapplying the filter.

Fix: A hybrid approach:

- If the filterMany expression references only root-level properties → attach the predicate directly on the many-root's own join (correct).

 - If it references deeper/nested properties requiring "extra joins" (a separate existing mechanism) → fall back to the original end-of-subtree behavior, since no SqlTreeNode exists to attach to.

 - includeFilterMany() made idempotent so both mechanisms coexist safely.

* Bug: filterMany() with nested-property predicates silently ineffective
filterMany() expressions referencing a nested/dotted property (e.g.
.eq("group.name", "x"), alone or mixed with root properties) were
attached to a LEFT JOIN's ON clause via the "extra join" mechanism.

That only nulls the extra join's own columns on non-match - it does
not exclude the parent/many-side row, so the predicate had no actual
filtering effect (confirmed by data, not just SQL shape).

## Fix:

force any many-property fetch whose filterMany expression
references a nested property onto a query-join (secondary select
restoring filterMany's original always-fetchQuery behaviour for this
with a genuine WHERE clause) instead of leaving it as an inline JOIN, case.

- SpiExpressionValidation: track all visited property names (allProperties()), not just unknown ones.
- OrmQueryProperties: filterManyHasNestedProperty() walks the filterMany expression for any dotted property reference.
- OrmQueryDetail.markQueryJoins(): route nested-property filterMany chunks to markForQueryJoin() instead of the inline fetch-join slot, without consuming that slot for other many-paths.

Also simplifies the earlier root-only-predicate join-placement fix
(CQueryPredicates/DefaultDbSqlContext/SqlTreeNodeManyRoot) now that
a nested-property filterMany can never reach the JOIN-based code
path: removed the now-unreachable "deepest path" fallback branch,
the includeFilterMany() idempotency guard, and the redundant fallback
call in SqlTreeNodeManyRoot - isFilterManyAttachPoint() remains as
the single, always-used attach mechanism.

Updated TestQueryFilterMany/TestQueryFilterManySimple assertions to
expect the corrected query-join (2 statement) behaviour.

* Simplify SpiExpressionValidation with boolean nestedProperty flag

* Tidy up comments

* Fix OrmQueryProperties.filterManyHasNestedProperty() with real nested property check

---------

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-15 23:16:18 +12:00
Rob Bygraveandrobin.bygrave d54a24b27d #3848 Fix regression for Sql Server and Oracle with exists() introduced in 18.2.0 (#3849)
* #3848 Fix regression for Sql Server and Oracle with exists() introduced in 18.2.0

Fix #3848: exists() generates invalid scalar SQL on SQL Server and Oracle

Query.exists() generated `select exists(<subquery>)`, which is valid on
Postgres/H2/MySQL but rejected by SQL Server and Oracle since both only
support EXISTS as a predicate, not as a directly selectable scalar boolean
expression.                                                                                                                                                                              ┃
                                                                                                                                                                                            ┃
Add DatabasePlatform.existsWithCaseWhen/existsFromClause capability flags
and use them in CQueryBuilder.wrapSelectExists() to generate
`select case when exists(<subquery>) then 1 else 0 end` on platforms that                                                                                                                ┃
need it, with an additional ` from dual` suffix for Oracle (which requires                                                                                                               ┃
a FROM clause on every select). CQueryExists reads the boolean result via                                                                                                                ┃
ResultSet.getBoolean(1), which correctly interprets the resulting 0/1 int.

* Fix tests TestInsertCheckUnique for Oracle and SQL Server

---------

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-14 10:13:16 +12:00
Rob Bygraveandrobin.bygrave 73c9ff8d80 Throw exception on duplicate Database registration (#3838) (#3847)
* Throw exception on duplicate Database registration (#3838)

DatabaseFactory.create() with register(true) previously returned the
existing Database instance when another instance was already registered
under the same name (added in #3759). This silently caused two
independently created Database instances (e.g. with different
DataSourceConfig.url values) to collide/share state, since the
second "instance" was actually the first one in disguise.

Change this to throw an IllegalStateException instead, forcing
callers to either use a unique DatabaseConfig name or setRegister(false)
when the Database is not intended to be registered/looked up by name.

* Also handle deregistration on shutdown

---------

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-14 00:34:49 +12:00
Rob Bygraveandrobin.bygrave bdbe743f56 #3275: Fix NPE when update/delete with @IdClass mapping (#3846)
## Root cause:

For @IdClass composite keys, the "id" is cached only in EntityBeanIntercept.ownerId(), populated only when reading from DB or during insert derivation. A fresh bean instance passed straight to DB.update()/DB.delete() had this null, causing an NPE in BindableIdEmbedded.dmlBind().

## Fix:

dmlBind() now defensively derives the id (reusing the existing insert-time derivation logic, extracted into a shared deriveId() helper) when it's null but derivable — covering both update and delete, since they share the same DML bind path.

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-13 23:10:54 +12:00
Rob Bygraveandrobin.bygrave ed2026b858 #3110: Fix cache key with enum type (#3845)
## Root cause:

Finder.byId() on a @Cache entity converts the id to a String cache key via IdBinder.cacheKey() = scalarType.format(value). For enums, format() returns .name() (e.g. "APPROVED"), not the db-mapped value. On a  cache hit, BeanDescriptorCacheHelp.loadBeanDirect() converts that cache-key string back via desc.convertId() → IdBinderSimple.convertId() → scalarType.toBeanType(), which expects a db value (e.g. "2" for @EnumValue("2")),  not the enum's Java name — causing Integer.parseInt("APPROVED") to crash in EnumToDbIntegerMap.getBeanValue().

## Fix:

IdBinderSimple.convertId() now special-cases String input to use scalarType.parse() (the true inverse of  format()) instead of toBeanType() (the inverse of toJdbcType()/db-value). This mirrors an existing, already-correct pattern in IdBinderEmbedded.convertId(), which already handles String cache-key round-tripping via parse(). Verified  this doesn't affect other id types (Long/Integer/etc.) since parse() and toBeanType() are equivalent for them — the mismatch only exists for Enum-mapped types.

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-13 22:42:16 +12:00
Rob Bygraveandrobin.bygrave 8d21f91f62 #2477: Support @DbArray inside an @Embeddable (#3844)
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-13 21:45:11 +12:00
Rob Bygraveandrobin.bygrave 6d1d58b8c2 #1989: Add findPagedList() support to DtoQuery (when based on orm query) (#3843)
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-13 20:34:25 +12:00
Rob Bygraveandrobin.bygrave 94f252f423 #1700: Add support for @OrderColumn on @ManyToMany (#3841)
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-13 15:55:03 +12:00
Rob Bygrave 275f8ad9f9 #2393: @OrderColumn was not maintained for @ElementCollection — no order column populated on insert and no order by on fetch. (#3840)
## Root cause:
AnnotationAssocManys only parsed @OrderColumn inside the @OneToMany branch, and even if parsed, BeanDescriptorManager.checkMappedByOneToMany() returned early for element collections before reaching the order-column setup. The element collection's synthetic descriptor also isn't registered in deployInfoMap, so it couldn't reuse the existing @OneToMany lookup path.

## Fix (in ebean-core):

- BeanDescriptorManager: extracted makeOrderColumn(DeployBeanPropertyAssocMany, DeployBeanDescriptor) so the synthetic order property can be built against an explicit target descriptor, not just one looked up via deployInfoMap.

- AnnotationAssocManys.readElementCollection(): now reads @OrderColumn, sets fetchOrderBy, and builds the order property directly on the element descriptor before it's converted to a runtime BeanDescriptor.

- SaveManyElementCollection: binds a sequential 0-based index as the order column value on insert (element collections delete-then-reinsert the whole collection in list order, so this is a natural fit).

DDL generation and the fetch order by needed no changes - both already generalize over any BeanDescriptor's order column property once it exists.

Added EcolPerson / TestElementCollectionOrderColumn covering insert population, order preserved on reload, and order by present on fetch-join SQL.

Known limitation: @OrderColumn on a Map-based element collection (SaveManyElementCollectionMap) is not wired up - not part of this issue and not a standard JPA combination (Maps use @MapKeyColumn).
2026-07-13 15:11:49 +12:00
Rob Bygrave 9ac3bd71f6 Issue #2840: @DbJson/@DbJsonB Map<String,Object> fields ignored mutationDetection and always used ModifyAware-wrapper dirty checking — so NONE (and HASH) had no effect, unlike plain Object-typed @DbJson fields which correctly honored these modes. (#3839)
Root cause: The built-in ScalarTypeJsonMap/List/Set types (used for Map<String,Object>, simple List/Set) hard-coded mutable()=true and isDirty() to always do a ModifyAware check, regardless of the configured MutationDetection. Only SOURCE accidentally worked because it happened to route through BeanPropertyJsonMapper.

 Fix (in ebean-core, io.ebeaninternal.server.type):

 - ScalarTypeJsonValue now derives mutable (false only for NONE) and keepSource/jsonMapper() (true for HASH/SOURCE) from the property's MutationDetection, instead of a single hard-coded keepSource boolean.

- ScalarTypeJsonMap, ScalarTypeJsonMapEnum, ScalarTypeJsonList, ScalarTypeJsonSet, and ScalarTypeJsonCollectionValue now accept/pass through MutationDetection instead of a raw boolean.

- DefaultTypeManager.dbJsonType() passes the property's own mutationDetection() straight through for these collection types (per your choice: DEFAULT still always means ModifyAware for collections — only an explicit NONE/HASH/SOURCE annotation changes behavior; the DatabaseConfig-wide default is not applied to these types).

- @DbArray fallback call sites (PlatformArrayTypeJsonList/Set) updated to pass MutationDetection.DEFAULT, preserving unchanged behavior.
2026-07-13 10:24:25 +12:00
Rob Bygraveandrobin.bygrave 28e6108315 Bump mysql driver to mysql-connector-j (#3836)
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-07 10:17:09 +12:00
robin.bygrave 007409d21c Bump postgresql to 42.7.11 2026-07-07 09:21:14 +12:00
Rob Bygrave 8b61069b3f #3441 Handle type coercion in hits against bean cache (#3835) 2026-07-06 22:04:09 +12:00
Rob BygraveandRoland Praml a82dcb2509 Fix #3041 check uniqueness cache skipclean (#3834)
* checkUniqueness supports queryCache and skipClean

* Keep the DB.checkUniqueness() as per original (not overload there)

---------

Co-authored-by: Roland Praml <roland.praml@foconis.de>
2026-07-06 22:03:52 +12:00
Rob BygraveandRoland Praml 08bb170cb2 Fix #3156 inherited property access (#3833)
* Properties of inherited models can be used on different child models

* Adjust typos only

---------

Co-authored-by: Roland Praml <roland.praml@foconis.de>
2026-07-06 21:08:35 +12:00
Rob Bygraveandrobin.bygrave a2f954a60e #3529 - Fix for M2M property is empty in preDelete of BeanPersistAdapter (#3830)
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-06 19:47:37 +12:00
robin.bygrave 6dd1763e54 Fix test TestRawSqlWithPlaceholders for Postgres HAVING clause limitation
Postgres having can't use column alias from select clause
2026-07-06 16:42:51 +12:00
Rob Bygrave e525d4e159 Version 18.2.0 2026-07-03 19:38:05 +12:00
Rob Bygraveandrobin.bygrave 4e639cb88a Add deletePermanent() - hard delete for soft delete capable beans (#3828)
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-03 18:23:18 +12:00
Rob Bygraveandrobin.bygrave fc78b2af80 Add RawSqlBuilder.withPlaceholders() for CTEs and other complex SQL (#3649) (#3827)
* Add RawSqlBuilder.withPlaceholders() for CTEs and other complex SQL (#3649)

RawSqlBuilder.parse() uses keyword-based scanning to locate SELECT
columns and the WHERE/HAVING injection points. This fails for CTEs,
window functions and other complex SQL where "select"/"from" keywords
appear in places the scanner doesn't expect.

withPlaceholders(sql) skips SELECT/FROM column parsing entirely and
only locates the ${where}, ${andWhere}, ${having} and ${andHaving}
placeholder positions, requiring explicit columnMapping() calls (as
with unparsed()). This lets dynamic where()/having() expressions be
injected into otherwise unparseable SQL.

Also fixes two bugs in the underlying placeholder-position splitting:
- static SQL following a ${having}/${andHaving} placeholder (e.g. a
  trailing ORDER BY) was silently dropped when both a where and a
  having placeholder were present
- using only ${having}/${andHaving} (no where placeholder) caused a
  dynamically added HAVING clause to be appended after trailing
  static SQL, producing invalid SQL

Changes:
- RawSqlBuilder.withPlaceholders(sql) + SpiRawSqlService.withPlaceholders()
- DRawSqlParser.parseAsTemplate() / parseTemplate() - placeholder-position
  only parsing, correctly splitting preWhere/preHaving/trailing SQL
- CQueryBuilderRawSql - skip column-list and "select" prefix handling
  in template mode (signalled by an empty preFrom)
- Unit tests in DRawSqlServiceTest covering where/andWhere/having/andHaving
  placeholder combinations, including the two fixed edge cases
- Integration tests in TestRawSqlWithPlaceholders (ebean-test) covering
  CTE queries with dynamic where/having and verifying generated SQL

* Add examples test using query beans

* Add docs guides for RawSql

* Add ${orderBy} ${andOrderBy}

---------

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-03 18:22:42 +12:00
Rob Bygraveandrobin.bygrave 29647ebe1a Fire BeanPersistController when only ManyToMany collection changes (#3652) (#3826)
When only a @ManyToMany collection is modified and saved, the parent bean has no dirty scalar properties so BeanPersistController preUpdate/postUpdate were never invoked — only the junction table rows were written.

Fix mirrors the existing preElementCollectionUpdate() mechanism: add
preManyToManyUpdate() on PersistRequestBean and call it from
SaveManyBeans.saveAssocManyIntersection() when the intersection actually
changes (vanillaCollection, forcedUpdate, or a tracked BeanCollection
with non-empty additions or removals). The !dirty guard in
preManyToManyUpdate() prevents double-firing when the bean itself is
also dirty.

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-03 11:43:16 +12:00
Rob Bygraveandrobin.bygrave 38146fdce5 Support option to have always non-null @Embeddable(s) refereneces (#3825)
(#3702)

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-02 22:02:15 +12:00
Rob Bygraveandrobin.bygrave 864aff6c3b Implement query.exists() via select exists(...) (#3824)
* Implement query.exists() via select exists(...)

Previously query.exists() was implemented by running a findIds() limited to 1 row and checking whether the list was empty. This PR replaces that with a dedicated select exists(select 1 ...) execution path.

## Changes:

• CQueryExists — new query executor, analogous to CQueryCount, that runs select exists(...) and returns the boolean result directly from the JDBC ResultSet
• CQueryBuilder.buildExistsQuery() — builds select exists(select 1 ...) SQL, reusing the query plan cache on repeat calls
• CQueryEngine.findExists() / DefaultOrmQueryEngine / OrmQueryEngine interface — wire the new executor through the standard query engine stack, including SQL logging, summary logging, and query cache put
• DefaultServer.exists() — delegate to request.findExists() instead of findIds()

## Tests added to TestQueryExists:

• testExistsBoolean_returnsFalse — verifies false is returned when no rows match
• testExistsBoolean_withJoin — exercises the join SQL path in buildExistsQuery; also checks false when the join yields no match
• testExistsBoolean_queryPlanReuse — runs the same query twice and asserts the generated SQL is identical, confirming the plan cache is hit on the second call

* Fix TestQueryExists only

---------

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-02 18:32:50 +12:00
Rob Bygraveandrobin.bygrave 24da374aa8 Fix #3821 fetchQuery secondary tables not included in query cache dep… (#3823)
* Fix #3821 fetchQuery secondary tables not included in query cache dependent tables

When a query uses fetchQuery("path"), the secondary table(s) must be registered as dependent tables in the query cache entry so that updates to those tables properly invalidate the cache.

## Fix:
In CQueryPlan, after building the SQL-tree dependent tables, walk the query's OrmQueryDetail for any fetchQuery paths and resolve each path through the root BeanDescriptor to its target entity's baseTable(). These tables are merged into dependentTables at plan construction time (once, then cached in the plan).

This means the fix applies uniformly whether paths were specified via fetchQuery("path") calls or via query.select(fetchGroup) — both populate the same OrmQueryDetail.

### Tests added to TestQueryCacheTableDependency:

• fetchQuery_oneToMany_invalidatesQueryCacheOnSecondaryTableUpdate — updating a contact invalidates a Customer query cache that includes fetchQuery("contacts"); asserts the refreshed result contains the updated phone number

• fetchQuery_manyToOne_invalidatesQueryCacheOnSecondaryTableUpdate — updating a root record invalidates an ECacheChild query cache that includes fetchQuery("root")

* Ensure tests cleanup their test data

* query.secondaryQuery() (called during prepareQuery()) removes fetchQuery paths from OrmQueryDetail before CQueryPlan is built. The existing buildDependentTables iterated detail.entries() looking for isQueryFetch() entries — but they were already gone.

Changes:

1. OrmQueryRequest.java — added secondaryQueries() getter to expose the already-stored SpiQuerySecondary field.
2. CQueryPlan.java — rewrote buildDependentTables to accept SpiQuerySecondary instead of OrmQueryDetail. It now iterates secondary.queryJoins() (the paths already removed from detail) and adds each path's base table to the dependent tables set.

---------

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-02 17:44:10 +12:00
Rob Bygraveandrobin.bygrave 688eee532c Fix #3817 incorrect type deserialization for @DbJson field in generic superclass (#3819)
In createJsonObjectMapperType(), remove the instanceof DeployBeanProperty
branch that overrode ownerType() with getField().getDeclaringClass().

For a generic superclass B<T>, getDeclaringClass() returns B with T
unresolved, so Jackson treated the field as Object and deserialized to
LinkedHashMap. DeployBeanProperty.ownerType() already returns
desc.getBeanType() (the concrete subclass), which gives Jackson the full
supertype context needed to resolve T correctly.

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-02 00:08:47 +12:00
Rob BygraveandIliya Ivanov 88d7b5e85a Support @ManyToOne inside @Embeddable for query predicates and joins (#3816)
* Make embeddable diggable

* WIP

* WIP

* WIP

* WIP

* WIP

* Support @ManyToOne inside @Embeddable for query predicates and joins

 Fix path resolution for @ManyToOne associations nested inside @Embeddable
 classes. Previously, a query like .where().eq("address.country.name", "NZ")
 on an entity with an @Embedded EAddr (containing @ManyToOne Country country)
 would throw PersistenceException: Embedded Property country.name not found.

 Changes:
 - BeanPropertyAssocOne.buildElPropertyValue: correctly resolve @ManyToOne
   paths inside embedded by extracting first segment, validating via
   embeddedPropsMap, then delegating deeper traversal to the overridden
   property; restore embedded flag on ElPropertyChain for scalar leaves
 - BeanPropertyAssoc.targetDescriptor(): lazy-init fix for override copies
   created by BeanEmbeddedMetaFactory (initialise() is not called on them)
 - BeanDescriptor.extraJoin(): restore !assocProp.isEmbedded() guard to
   prevent null-table ExtraJoin for composite keys and other embeddables
 - ElPropertyChainBuilder/ElPropertyChain: restore embedded field and prefix
   computation so alias resolution works correctly for paths through embedded
   (e.g. outer.datePeriod.date1 → prefix "outer" not "outer.datePeriod")
 - SqlTreeBuilder.buildExtraJoins: remove erroneous removeAll on
   orderByIncludes that broke Formula2 placeholder resolution and distinct
   on aggregation queries with many-side order-by joins
 - SqlTreeAlias.addJoin: fix indentation
 - Tests: add three test cases to TestEmbeddedManyToOne covering WHERE
   predicate through embedded FK column, through association property
   (requiring a JOIN), and combined with fetch

* Throw PersistenceException with unknown path

---------

Co-authored-by: Iliya Ivanov <i.ivanov@proforge.org>
2026-07-01 23:33:40 +12:00
Andrey Glushkov 942bab5efb Fix: DuplicateKeyException when cascade-saving through a @ManyToOne to an unmodifiable bean (address #3813) (#3814)
* Fix DuplicateKeyException with unmodifiable cascaded bean

* Fix broken test TestDeleteByQuery.maxDelete_when_beanCaching_expect_selectThenDelete
2026-07-01 22:43:33 +12:00
robin.bygrave 9acc555e8e Sqlite: Add Insert On Conflict support (#3770) 2026-07-01 21:58:16 +12:00
Rob Bygrave 73a646e7dc Merge pull request #3812 from dragkes/feature/invalidate-one-to-one-and-collection-cache
Invalidate one to one and collection cache (address #3811)
2026-07-01 21:21:19 +12:00
Andrey Glushkov a59c70054b Tests 2026-07-01 11:48:28 +03:00
Rob Bygrave e372579b15 Merge branch 'master' into tenant-support-for-query-plans 2026-07-01 20:33:31 +12:00
robin.bygrave 3d2cc2d4af Version 18.1.0 2026-07-01 13:28:23 +12:00
Rob Bygrave 291966cea9 Test and Fix for #3806 - @OneToMany query that populates from the immutable cache gets NPE (#3807)
## Root cause

  A findList() triggers a secondary @OneToMany query that populates from the immutable cache. ImmutableBeanCaches.QueryLoader runs find(type).setUnmodifiable(true)...findMap(), which hits the L2 bean cache (cacheIdLookup → loadBeanDirect). Two coupled bugs in the unmodifiable path:

## 2 Bugs:
   1. Null pc (the NPE). In BeanDescriptorCacheHelp.loadBeanDirect, the if (context == null) context = new DefaultPersistenceContext() fallback was nested inside if (!unmodifiable). So an unmodifiable load passed a null context to CachedBeanDataToBean.load; converting a @ManyToOne (refBean → contextGet → pc.get) NPE'd — exactly Eddie's trace.
   2. Mutable reference can't freeze. Once a context was provided, refBean created the @ManyToOne ref via createRef → a mutable InterceptReadWrite bean. The subsequent unmodifiableFreeze then threw UnsupportedOperationException("never expected") (only InterceptReadOnly is freezable).

  ## Fix (2 files)

   - BeanDescriptorCacheHelp.loadBeanDirect — always ensure a non-null context before CachedBeanDataToBean.load (hoisted out of !unmodifiable).
   - BeanPropertyAssocOne.setCacheDataValue/refBean — when the owning bean is unmodifiable (derived via !(intercept instanceof InterceptReadWrite), the same idiom CachedBeanDataToBean already uses), create the ref with createReference(unmodifiable, false, id, pc) → a freezable InterceptReadOnly reference. Also guards contextGet against null. Modifiable path is unchanged.

## Notes
  The "fetch(path) without a FetchGroup" clue

  That's the trigger, not a misuse. A restricting FetchGroup can exclude the @ManyToOne, so no refBean runs and the bug stays hidden. The default/fetch("path") select includes the FK, so the cached-bean conversion creates the assoc-one reference and hits the bug. His usage was fine — this was an Ebean gap (no immutable-cached entity with a @ManyToOne was test-covered).
2026-07-01 12:54:12 +12:00
robin.bygrave ad05ed051b Modify SequenceIdGenerator to internally use ArrayDeque 2026-06-30 22:49:19 +12:00
Rob Bygraveandrobin.bygrave 01b8c3dbcb Multi-tenant aware DB sequence id generation (replaces #2305) (#3805)
## Problem

SequenceIdGenerator captured a single DataSource at deploy time and held one shared pre-fetch buffer. Under multi-tenancy this is wrong:

 - TenantMode.DB/DB_WITH_MASTER — there is no bootstrap DataSource, so BeanDescriptorManager passed null; sequence allocation couldn't resolve the current tenant's database.
 - TenantMode.SCHEMA/CATALOG — a single shared buffer let one tenant's pre-fetched ids be handed to another (cross-tenant bleed), and pre-fetch used a connection that wasn't scoped to the requesting tenant.

(Supersedes the per-datasource delegator approach in #2305, which leaked via a WeakHashMap whose values strongly referenced the keys, only handled DB mode, and re-resolved the tenant on the background thread.)

## Fix:

Make SequenceIdGenerator itself tenant aware, keeping all platform modules untouched.

 - New TenantConnectionSource (ebean-api, additive): optional interface a DataSource may implement — currentTenantId() + connectionForTenant(tenantId).
 - SequenceIdGenerator: the shared idList/lock/loading flag become a per-tenant TenantBuffer keyed by tenantId in a ConcurrentHashMap. Connections are obtained per tenant. Background pre-fetch captures the tenant at submit time (the executor thread has no tenant in scope) and fetches by explicit tenantId — fixing a latent ThreadLocal-propagation bug.
 - SequenceDataSource (ebean-core): adapts DataSourceSupplier to TenantConnectionSource; routes to the tenant DB (DB mode) or sets schema/catalog (SCHEMA/CATALOG).
 - Wiring: InternalConfiguration exposes the DataSourceSupplier; BeanDescriptorManager wraps it only for dynamic-datasource tenant modes.

## Performance (single-tenant unaffected)

 - A cached single buffer field short-circuits the ConcurrentHashMap for the non-tenant key.
 - NONE/PARTITION pass the plain DataSource (not wrapped), so tenantSource == null and the hot path is just a couple of cheap branches — equivalent to the original.

## Compatibility

 - Platform constructor signature (be, ds, seqName, allocationSize) unchanged — no changes to the 9 platform modules.
 - One protected-method signature changed: getMoreIds(int) → getMoreIds(Object tenantKey, int). Rarely overridden (subclasses override getSql/readIds), but a source-incompat for any external custom platform that did.

## Tests

 - TenantSequenceTest — DB-per-tenant: tenant 1 → 1,2,3; tenant 2 independently → 1.
 - SequenceBatchIdGeneratorTest adapted to the per-tenant buffer.
 - Existing sequence + multitenancy suites pass.

## Potential Follow-ups (not in this PR)

 - Add a SCHEMA-mode test.
 - Optional removeTenant(tenantId) hook if unbounded tenant churn is a concern (buffers hold only Longs + a lock, no DataSource, so no real leak).
 - SimpleSequenceIdGenerator (non-batching) left as-is — already uses the txn connection.

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-06-30 22:28:28 +12:00
robin.bygrave 443b68a3b0 For #3082 - Fix and test for History findVersions() that joins to other history table
Bug (#3083): findVersions() on a @History root that joins to another @History entity bound the joined table's effective-date predicate with a null as-of timestamp → systime <@ null (Postgres) / sys_period_start <= null (H2) → zero rows.

  Fix (CQueryPredicates.bind(), +6 lines): when the as-of is null but there are history-view join predicates to bind (asOfTableCount > 0 — only ever true for findVersions/findVersionsBetween), default to the current timestamp. Semantics: all versions of the root, joined to related @History entities as they are now. The root table is untouched (still returns all versions).

  Test ported from #3082 :
   - New HistoryManyToOne entity (@History + @SoftDelete) — kept lean, dropped the PR's unneeded @OneToMany
   - HistorylessOneToOne gains @ManyToOne HistoryManyToOne
   - New testVersionsWithHistoryOverHistoryless() — replaced the PR's always-failing placeholder (assertSql(...).contains("this does not exist")) with real assertions: count == 1 and SQL contains no asOf null bind
   - Left the existing @OneToOne unchanged (the PR's optional = false tweak to the shared model wasn't needed)
2026-06-30 21:08:30 +12:00
Andrey Glushkov 98f0d42b7e Fix: defer savepoint cache changes to parent transaction commit (#3804) 2026-06-30 18:47:24 +12:00
Rob Bygraveandrobin.bygrave 97bd0e1bb8 Add Transaction.setGeneratedPropertiesEnabled(boolean) (#3799)
Adds transaction-level control over whether Ebean auto-generates values for @WhenCreated, @WhenModified, @WhoCreated and @WhoModified properties.

Motivation

Backup/restore scenarios need to preserve the original audit timestamps and user values when re-inserting exported data. Without this, every save overwrites those fields with the current time/user.

Usage

 try (Transaction txn = DB.beginTransaction()) {
   txn.setGeneratedPropertiesEnabled(false);
   bean.setWhenCreated(originalTimestamp);
   bean.setWhenModified(originalTimestamp);
   DB.save(bean);
   txn.commit();
 }

Behaviour

 - Disabled (false): generated property values are only written if the property currently has a null value. Any non-null value set on the bean is preserved.
 - @Version is unaffected: the version property always auto-increments regardless of this setting, preserving optimistic locking integrity.
 - Default is true: all existing behaviour is unchanged.

Files changed

 - Transaction — new setGeneratedPropertiesEnabled(boolean) with javadoc
 - SpiTransaction — new isGeneratedPropertiesEnabled()
 - SpiTransactionProxy — delegates both methods
 - JdbcTransaction — field + implementation (default true)
 - NoTransaction, ImplicitReadOnlyTransaction — no-op setter, true getter
 - PersistRequestBean — onInsertGeneratedProperties, onUpdateGeneratedProperties, onFailedUpdateUndoGeneratedProperties all gate on isGeneratedPropertiesEnabled()
 - TestGeneratedProperties — 3 new tests covering insert-preserves, insert-null-still-filled, and update-preserves

Supersedes

PR #2943 — same feature, renamed from setOverwriteGeneratedProperties to setGeneratedPropertiesEnabled for a clearer, positive-sense API.

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-06-26 18:53:29 +12:00
Rob Bygraveandrobin.bygrave d7d040d031 Add @Formula2 support on @ManyToOne association properties (#3798)
Extends @Formula2 so it can be placed on a @ManyToOne field. Instead of embedding physical SQL aliases (as @Formula(join=...) requires), a @Formula2 expression uses logical bean paths — the required joins are derived automatically.

  Example
``
   // Before — physical SQL, brittle alias wiring:
   @Formula(select = "coalesce(${ta}.some_bean_id, j1.some_bean_id)", join = PARENTS_JOIN)
   @ManyToOne EBasic effectiveBean;
```
```
   // After — logical paths, joins resolved automatically:
   @Formula2("coalesce(someBean.id, parent.someBean.id)")
   @ManyToOne EBasic effectiveBean;
``
  The generated SQL is identical; the annotation is far more readable and maintainable.

  What changed

  Annotation parsing (AnnotationAssocOnes) — @Formula2 on a @ManyToOne is now recognised and the expression parsed into a select fragment and a set of dependency join paths.

  Query tree building (SqlTreeBuilder) — three scenarios all handled correctly:

   - Fetched as a tree node — dependency joins are inserted before the formula2 join using addChildFirst
   - Partial parent fetch — dependency joins are registered even when the parent chunk only selects its ID column
   - Predicate-only (where clause, no fetch) — addFormula2JoinsFromPredicates runs before buildSelectChain so dependency join paths are populated before buildExtraJoins constructs the extra-join tree; IncludesDistiller uses addChildFirst to preserve ordering within the extra-join tree

  Init ordering (BeanDescriptorManager) — initFormula2Properties() moved to a dedicated pass 5, after all descriptors are fully initialised, so cross-descriptor path resolution (e.g. parent.parent.someBean.id) is always safe.

  SqlTreeNodeExtraJoin — gained addChildFirst() to match SqlTreeNodeBean, enabling formula2 dependency joins to be prepended ahead of the formula2 property join in the extra-join tree.

  Fixes

  Supersedes and resolves #2773 — the reported bug (wrong join ordering when combining fetch and where on a formula-joined field) is eliminated for ChildPerson.effectiveBean and ParentPerson.effectiveBean, which are now expressed as @Formula2.

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-06-25 22:18:10 +12:00
Rob Bygraveandrobin.bygrave d5f32690ea Support aggregation functions like sum on Formula2 (#3796)
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-06-24 21:44:40 +12:00
robin.bygraveandCopilot 691f153d89 Fix DtoMetaDeployProperty: remove unused Method param; fix findField superclass traversal; wire up findMetaAnnotations for setter annotation support
- DtoMetaDeployProperty: remove unused Method parameter from constructor
- DtoMetaProperty.findField: use loop variable 'type' (not outer 'dtoType') so
  superclass fields are correctly found
- DtoMetaProperty constructor: call findMetaAnnotations() which merges both
  field and setter annotations, rather than reading field annotations only
- DefaultTypeManager: keep both MethodType import and Annotation import
- Fix missing imports in EbeanServerFactory_ServerConfigStart_Test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-24 18:16:41 +12:00
Roland Praml dd1acf58a0 DbJson Support for Dto-Queries 2026-06-24 18:14:45 +12:00
Rob Bygrave 7bf8f7fbce With Database register(true) check for existing registered Database by name (#3759)
And return the existing Database instance if present.

In theory this should not be needed. There are some test setups that hit this situation.
2026-06-24 18:02:45 +12:00
Rob Bygraveandrobin.bygrave 490c70a7b7 PR #3656 fix - lazy initialised O2M relation are not correctly persisted on subsequent saves (#3793)
- Root cause: SaveManyBeans.removeAssocManyOrphans() only called setModifyListening when insertedParent=true. A first save with null O2M skipped it; the lazily-initialized collection on subsequent saves had no listen mode → .clear() untracked → orphan not deleted.
 - Fix: Added else { setListenMode(c, many); } to set the listen mode for existing collections that were never initialized (uses the existing null-guard helper).
 - Tests: testModifyListenModeSet2 now passes; all 26 cascade tests green.

-------------
Original #3656 description:

We found an issue, when O2M relations are not correctly persisted to the DB.

This happens, when

a bean is saved and the O2M property is empty
something is added and cleared again in two subsequent saves.
the same master-bean object has to be used
The issue here is, that the BeanCollection is lazily initialized with no modifyListenMode set after the first save.
This happens only for O2M relations with no order column. (Others work fine) See: https://github.com/ebean-orm/ebean-agent/blob/d4c40f1ce85c58f99cb0a85152aaa0a0075a9c01/ebean-agent/src/main/java/io/ebean/enhance/entity/FieldMeta.java#L469

And we need also a save, where the bean is saved with an empty/null value in the O2M property.
Subsequent saves will not update the modifyListenMode. See SaveManyBeans

      if (insertedParent) {
        // after insert set the modify listening mode for private owned etc
        c.setModifyListening(many.modifyListenMode());
      }

We found this in one of our unit-tests, where we've configured a bean for different states. It is probably something, that should not be too critical in real code, as you normally save a bean only once (When the bean was retrieved from DB, it should not occur)

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-06-24 17:05:11 +12:00