* Metrics - support both CUMULATIVE and DELTA metrics collection concurrently
ebean insight [and StatsD] work off DELTA metrics, where as OTEL
and Prometheus want CUMULATIVE. With this change we can have a metrics
collection for ebean insight using DELTA mode and have a second collection
use CUMULATIVE for reporting to OTEL - for the case of fan-out metrics
going to 2 places.
This isn't strictly needed when only one collection mode is used.
* Metrics - Add Mode with RESET, DELTA and CUMULATIVE
Previously we were overloading RESET and DELTA but we really need
these to be 2 separate modes for the existing tests and the
get(reset) api
* Fix test TestNatKeyCacheWithForeignKey with cache stats reset
---------
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
* Add FinableQuery, StreamableQuery common interfaces for Query, DtoQuery, MappedQuery, SqlQuery
* QueryBean.cancel() added to support the new common FindableQuery interface
---------
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
Adding this as I am seeing a LOT of cases where ebean can derive a good enough
error message (id case, plus single unique key case) that it is worth adding
this syntax sugar.
Yes we have discussed this a well it's not so useful for the multi-lingual /
non-english case and also marginal when the error message needs to be built
case.
* #2540 Add DTO mapping from Entity beans via generated mapper code
A large feature, refer to the docs / guides etc. This is a bit like MapStruct in that it generates mapping code via annotation processing but it ALSO creates an optimised FetchGroup for the destination DTO graph and it also implements a generic interface such that it can be fluently used via query.mapTo().
The value is that it automates the mapping code from entity -> dto plus optimises the query used to support building the DTO graph.
Expected to be better to use than MapStruct for ebean orm users.
* Bump ebean-annotation to 8.6 with the dto annotations
* Adjust the build order to via test dependency on ebean-api
---------
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
* #3848 Fix regression for Sql Server and Oracle with exists() introduced in 18.2.0
Fix#3848: exists() generates invalid scalar SQL on SQL Server and Oracle
Query.exists() generated `select exists(<subquery>)`, which is valid on
Postgres/H2/MySQL but rejected by SQL Server and Oracle since both only
support EXISTS as a predicate, not as a directly selectable scalar boolean
expression. ┃
┃
Add DatabasePlatform.existsWithCaseWhen/existsFromClause capability flags
and use them in CQueryBuilder.wrapSelectExists() to generate
`select case when exists(<subquery>) then 1 else 0 end` on platforms that ┃
need it, with an additional ` from dual` suffix for Oracle (which requires ┃
a FROM clause on every select). CQueryExists reads the boolean result via ┃
ResultSet.getBoolean(1), which correctly interprets the resulting 0/1 int.
* Fix tests TestInsertCheckUnique for Oracle and SQL Server
---------
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
* Throw exception on duplicate Database registration (#3838)
DatabaseFactory.create() with register(true) previously returned the
existing Database instance when another instance was already registered
under the same name (added in #3759). This silently caused two
independently created Database instances (e.g. with different
DataSourceConfig.url values) to collide/share state, since the
second "instance" was actually the first one in disguise.
Change this to throw an IllegalStateException instead, forcing
callers to either use a unique DatabaseConfig name or setRegister(false)
when the Database is not intended to be registered/looked up by name.
* Also handle deregistration on shutdown
---------
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
* checkUniqueness supports queryCache and skipClean
* Keep the DB.checkUniqueness() as per original (not overload there)
---------
Co-authored-by: Roland Praml <roland.praml@foconis.de>
* Add RawSqlBuilder.withPlaceholders() for CTEs and other complex SQL (#3649)
RawSqlBuilder.parse() uses keyword-based scanning to locate SELECT
columns and the WHERE/HAVING injection points. This fails for CTEs,
window functions and other complex SQL where "select"/"from" keywords
appear in places the scanner doesn't expect.
withPlaceholders(sql) skips SELECT/FROM column parsing entirely and
only locates the ${where}, ${andWhere}, ${having} and ${andHaving}
placeholder positions, requiring explicit columnMapping() calls (as
with unparsed()). This lets dynamic where()/having() expressions be
injected into otherwise unparseable SQL.
Also fixes two bugs in the underlying placeholder-position splitting:
- static SQL following a ${having}/${andHaving} placeholder (e.g. a
trailing ORDER BY) was silently dropped when both a where and a
having placeholder were present
- using only ${having}/${andHaving} (no where placeholder) caused a
dynamically added HAVING clause to be appended after trailing
static SQL, producing invalid SQL
Changes:
- RawSqlBuilder.withPlaceholders(sql) + SpiRawSqlService.withPlaceholders()
- DRawSqlParser.parseAsTemplate() / parseTemplate() - placeholder-position
only parsing, correctly splitting preWhere/preHaving/trailing SQL
- CQueryBuilderRawSql - skip column-list and "select" prefix handling
in template mode (signalled by an empty preFrom)
- Unit tests in DRawSqlServiceTest covering where/andWhere/having/andHaving
placeholder combinations, including the two fixed edge cases
- Integration tests in TestRawSqlWithPlaceholders (ebean-test) covering
CTE queries with dynamic where/having and verifying generated SQL
* Add examples test using query beans
* Add docs guides for RawSql
* Add ${orderBy} ${andOrderBy}
---------
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
* Add tenant-partitioned L2 caches (#2956)
When `tenantPartitionedCache` is enabled, each tenant gets its own
cache namespace (keys include the tenant id), improving cache-hit
ratio by preventing cross-tenant key collisions.
Refactor BeanDescriptorCacheHelp to abstract base class with two
concrete subclasses:
- BeanDescriptorCacheHelpFixed - static cache refs (original behaviour)
- BeanDescriptorCacheHelpPartitioned - per-request Supplier<> lookup
so the correct tenant-scoped cache is resolved on each access
Fix background-thread invalidation: clear(name) in partitioned mode
now scans all matching cache entries by key prefix rather than calling
tenantProvider.currentId() which is null/wrong on background executor
threads.
Add SpiCacheManager.clearTenant(tenantId) to allow removal of all
cache entries for a deactivated tenant, preventing unbounded memory
growth in long-running multi-tenant deployments.
Also load cache settings (cacheMaxSize, cacheMaxIdleTime, etc.) from
application properties - these were previously missing from loadSettings().
* Fix versions in test-java16
---------
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
## Problem
SequenceIdGenerator captured a single DataSource at deploy time and held one shared pre-fetch buffer. Under multi-tenancy this is wrong:
- TenantMode.DB/DB_WITH_MASTER — there is no bootstrap DataSource, so BeanDescriptorManager passed null; sequence allocation couldn't resolve the current tenant's database.
- TenantMode.SCHEMA/CATALOG — a single shared buffer let one tenant's pre-fetched ids be handed to another (cross-tenant bleed), and pre-fetch used a connection that wasn't scoped to the requesting tenant.
(Supersedes the per-datasource delegator approach in #2305, which leaked via a WeakHashMap whose values strongly referenced the keys, only handled DB mode, and re-resolved the tenant on the background thread.)
## Fix:
Make SequenceIdGenerator itself tenant aware, keeping all platform modules untouched.
- New TenantConnectionSource (ebean-api, additive): optional interface a DataSource may implement — currentTenantId() + connectionForTenant(tenantId).
- SequenceIdGenerator: the shared idList/lock/loading flag become a per-tenant TenantBuffer keyed by tenantId in a ConcurrentHashMap. Connections are obtained per tenant. Background pre-fetch captures the tenant at submit time (the executor thread has no tenant in scope) and fetches by explicit tenantId — fixing a latent ThreadLocal-propagation bug.
- SequenceDataSource (ebean-core): adapts DataSourceSupplier to TenantConnectionSource; routes to the tenant DB (DB mode) or sets schema/catalog (SCHEMA/CATALOG).
- Wiring: InternalConfiguration exposes the DataSourceSupplier; BeanDescriptorManager wraps it only for dynamic-datasource tenant modes.
## Performance (single-tenant unaffected)
- A cached single buffer field short-circuits the ConcurrentHashMap for the non-tenant key.
- NONE/PARTITION pass the plain DataSource (not wrapped), so tenantSource == null and the hot path is just a couple of cheap branches — equivalent to the original.
## Compatibility
- Platform constructor signature (be, ds, seqName, allocationSize) unchanged — no changes to the 9 platform modules.
- One protected-method signature changed: getMoreIds(int) → getMoreIds(Object tenantKey, int). Rarely overridden (subclasses override getSql/readIds), but a source-incompat for any external custom platform that did.
## Tests
- TenantSequenceTest — DB-per-tenant: tenant 1 → 1,2,3; tenant 2 independently → 1.
- SequenceBatchIdGeneratorTest adapted to the per-tenant buffer.
- Existing sequence + multitenancy suites pass.
## Potential Follow-ups (not in this PR)
- Add a SCHEMA-mode test.
- Optional removeTenant(tenantId) hook if unbounded tenant churn is a concern (buffers hold only Longs + a lock, no DataSource, so no real leak).
- SimpleSequenceIdGenerator (non-batching) left as-is — already uses the txn connection.
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
Adds transaction-level control over whether Ebean auto-generates values for @WhenCreated, @WhenModified, @WhoCreated and @WhoModified properties.
Motivation
Backup/restore scenarios need to preserve the original audit timestamps and user values when re-inserting exported data. Without this, every save overwrites those fields with the current time/user.
Usage
try (Transaction txn = DB.beginTransaction()) {
txn.setGeneratedPropertiesEnabled(false);
bean.setWhenCreated(originalTimestamp);
bean.setWhenModified(originalTimestamp);
DB.save(bean);
txn.commit();
}
Behaviour
- Disabled (false): generated property values are only written if the property currently has a null value. Any non-null value set on the bean is preserved.
- @Version is unaffected: the version property always auto-increments regardless of this setting, preserving optimistic locking integrity.
- Default is true: all existing behaviour is unchanged.
Files changed
- Transaction — new setGeneratedPropertiesEnabled(boolean) with javadoc
- SpiTransaction — new isGeneratedPropertiesEnabled()
- SpiTransactionProxy — delegates both methods
- JdbcTransaction — field + implementation (default true)
- NoTransaction, ImplicitReadOnlyTransaction — no-op setter, true getter
- PersistRequestBean — onInsertGeneratedProperties, onUpdateGeneratedProperties, onFailedUpdateUndoGeneratedProperties all gate on isGeneratedPropertiesEnabled()
- TestGeneratedProperties — 3 new tests covering insert-preserves, insert-null-still-filled, and update-preserves
Supersedes
PR #2943 — same feature, renamed from setOverwriteGeneratedProperties to setGeneratedPropertiesEnabled for a clearer, positive-sense API.
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
* Add support for Java 21 SequencedSet and SequencedMap
Such that these can be used in place of Set and Map if desired.
* Refactor BeanSet, BeanList, BeanMap replacing setActualSet|List|Map
Replace with collectionAdd() and refresh() methods.
* Refactor rename method getBeanCollectionAdd() -> collectionAdd()
* Build needs to use Java 21 to support the multi-release jar
* Update SequencedSet etc from recent changes
* Update build, needs package to use MR-JAR
---------
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
So the desire here is to better support sending metrics to Prometheus style metrics collectors that prefer CUMULATIVE metrics rather than DELTA based metrics.
To do this, expose additional MetaInfoManager.collectMetrics(reset) method.
In supporting this, the MAX value really does need to reset even with CUMULATIVE metrics and act more like a gauge as otherwise it becomes almost useless as the max value over the lifetime. So we need to adjust MAX to always reset and act like a gauge to be useful in this CUMULATIVE metrics reporting mode.
* Deprecate DatabaseConfig and DatabaseFactory, prefer Database.builder()
Deprecate the DatabaseConfig way of creating Database instance.
Migrate to use Database.builder().
* Deprecate DatabaseConfig and DatabaseFactory, migrate test code
* Use @Deprecated(forRemoval = true) on DatabaseConfig and DatabaseFactory
* Tidy up Database javadoc for deprecation
* Change [implicit] Lazy loading to be by TYPE rather than by PATH
We have some graph models where a common type (in the tests it is Label)
is used in many different *paths* of the graph. When we are loading via
*path* then all the loading of the Labels isn't in a single batch / load context
but instead split into different load contexts PER PATH.
This change, means that lazy loading operates by TYPE instead of by PATH.
In the tests, all the Labels are read via a single lazy loading query rather
that one lazy loading query per PATH, and this is more efficient.
* ImmutableBeanCache part 1
* ImmutableBeanCache part 2
* ImmutableBeanCache part 3 - add ImmutableBeanCaches
* ImmutableBeanCache - add propagation of caches to secondary queries
* ImmutableBeanCache - use and improve BeanDescriptor.merge() to positional
Using positional avoids the lookup by property name.
* ImmutableBeanCache - add unit testing with BeanDescriptorMergeTest
* ImmutableBeanCache - add testing for the further lazy loading on mutable beans
Also add doc / guides
* Add ImmutableCacheBuilder with underlying DefaultServerCache implementation
This provides a cache with maxSize, maxIdleSeconds, maxSecondsToLive
* Improve javadoc and doc / guides for ImmutableBeanCache
* Add cache invalidation for ImmutableBeanCache
* Use immutable cache direct hits for unmodifiable assoc-one refs
- add ImmutableBeanCache.getIfPresent() as a non-loading probe
- use immutable cache hits in AssocOneHelp / AssocOneHelpRefInherit
- avoid creating ref beans and merge/copy for unmodifiable cache hits
- keep existing mutable-query and miss/backfill behavior unchanged
Add this as a helper for when using alsoIf(BooleanPredicate) seems overkill
and there isn't a built in IfPresent option for the desired expression and
wanting to keep the fluid style.
As per https://github.com/ebean-orm/ebean/discussions/3720
```
@Column
ZonedDateTime zonedDateTime2;
```
... resulting in DDL generated as `timestamp(255)`. This is
occurring when the JPA dependency is used like:
jakarta.persistence:jakarta.persistence-api:3.2.0
rather than using the transitive dependency that ebean includes.
Workaround:
Remove the jakarta.persistence:jakarta.persistence-api:3.2.0 dependency.
Fix:
The fix here is to use a timestamp type that has a maximum precision.
When a precision/length is specified greater than the maximum precision
then the fallback type is used which is `timestamp` without any precision.
This is to support the use case where other logic is used to determine if a query
should be "forced" to use the master data source or not, and so it's easier to
pass that as a boolean to the usingMaster() method.
- Add AggregateFormulaContext with ability to override the default via DatabaseBuilder
- Add group_concat, string_agg, listagg to set of known aggregation functions
* Version 16.1.0
* Improve javadoc for Query setUseQueryCache() and unmodifiable
Since 16.x a query using the query cache returns unmodifiable object graphs.
* Use autocommit false with findIterate for Postgres
* Use autocommit false with findIterate for Postgres
* Use autocommit false with FindDto queries for Postgres
* Use autocommit false with SqlQuery findIterate queries for Postgres
* Ensure that TransactionFactory.createReadOnlyTransaction() returns an ImplicitReadOnlyTransaction
This change changes TransactionFactoryTenant.createReadOnlyTransaction() to return a ImplicitReadOnlyTransaction, as that is now used to support Postgres use of cursors with findIterate style queries.
e.g. DatabaseBuilder.queryPlanExplain("explain (verbose)")
Also changes Postgres default explain to be:
explain (analyze, costs, verbose, buffers, format json)
