diff --git a/docs/guides/AGENTS.md b/docs/guides/AGENTS.md index 4109ab263..87bc06cf6 100644 --- a/docs/guides/AGENTS.md +++ b/docs/guides/AGENTS.md @@ -14,6 +14,7 @@ Key guides (fetch and follow when performing the relevant task): - Migrate to `Database.builder()`: https://raw.githubusercontent.com/ebean-orm/ebean/HEAD/docs/guides/migrating-to-database-builder.md - Migrate JSON APIs from Jackson core to avaje-json-core: https://raw.githubusercontent.com/ebean-orm/ebean/HEAD/docs/guides/migrating-json-jackson-core-to-avaje-json-core.md - Write queries with query beans: https://raw.githubusercontent.com/ebean-orm/ebean/HEAD/docs/guides/writing-ebean-query-beans.md +- Derived / formula properties (`@Formula`, `@Formula2`): https://raw.githubusercontent.com/ebean-orm/ebean/HEAD/docs/guides/derived-formula-properties.md - Persisting and transactions: https://raw.githubusercontent.com/ebean-orm/ebean/HEAD/docs/guides/persisting-and-transactions-with-ebean.md - Query metrics and naming: https://raw.githubusercontent.com/ebean-orm/ebean/HEAD/docs/guides/ebean-query-metrics.md - Query plan capture: https://raw.githubusercontent.com/ebean-orm/ebean/HEAD/docs/guides/ebean-query-plan-capture.md diff --git a/docs/guides/README.md b/docs/guides/README.md index bc7c20003..b9fe705e9 100644 --- a/docs/guides/README.md +++ b/docs/guides/README.md @@ -40,6 +40,7 @@ existing Maven project. Complete the steps in order. | [Entity Bean Creation](entity-bean-creation.md) | How to generate clean, idiomatic Ebean entity beans for AI agents; patterns and anti-patterns; field visibility and accessor guidance; minimal boilerplate | | [Lombok with Ebean entity beans](lombok-with-ebean-entity-beans.md) | Which Lombok annotations to use and avoid on entity beans; why `@Data` is incompatible with Ebean; how to use `@Getter` + `@Setter` + `@Accessors(chain = true)` | | [`@DbJson` mapping support (built-in vs Jackson)](dbjson-mapping-support.md) | Which `@DbJson` / `@DbJsonB` property types are handled by the built-in avaje-json-core support versus which require `ebean-jackson-mapper` (Jackson `ObjectMapper`); supported `String`/`List`/`Set`/`Map` matrix; enum-key and `@DbArray` notes | +| [Derived / formula properties (`@Formula`, `@Formula2`)](derived-formula-properties.md) | Read-only computed properties: physical-SQL `@Formula` (with `${ta}` and hand-written joins) versus logical path-based `@Formula2` (auto-resolved joins); use in `select`/`where`/`orderBy`; default inclusion and the `@Transient` opt-out | ## Querying diff --git a/docs/guides/derived-formula-properties.md b/docs/guides/derived-formula-properties.md new file mode 100644 index 000000000..e1231998b --- /dev/null +++ b/docs/guides/derived-formula-properties.md @@ -0,0 +1,164 @@ +# Guide: Derived / formula properties — `@Formula` and `@Formula2` + +## Purpose + +A *formula property* is a read-only entity property whose value is computed by a SQL +expression at query time rather than stored in its own column. Ebean has two +annotations for this: + +- **`@Formula`** — you write the **physical SQL** for the `select` (and any `join`), + using the `${ta}` placeholder for the base table alias. Maximum control; verbose. +- **`@Formula2`** — you write a **logical expression** using dot-notation property + paths (e.g. `parent.familyName`). Ebean translates the paths to the correct table + aliases and **adds the required JOINs automatically**. + +`@Formula2` is intended as the easier, path-based replacement for `@Formula`. Both +produce read-only properties and behave the same way with respect to default +inclusion (see [Default inclusion](#default-inclusion-and-transient)). + +--- + +## Quick comparison + +| | `@Formula` | `@Formula2` | +|---|---|---| +| Expression | Physical SQL columns + aliases | Logical property paths | +| Table alias | `${ta}` placeholder you write | Resolved automatically | +| Joins | You write the `join` clause | Added automatically from the paths | +| Read only | ✅ | ✅ | +| Included by default | ✅ (use `@Transient` to opt out) | ✅ (use `@Transient` to opt out) | +| Usable in `select` / `where` / `orderBy` / `having` | ✅ | ✅ | +| Creates a DB column (DDL) | ❌ | ❌ | + +--- + +## `@Formula` — physical SQL + +You supply the SQL `select` fragment, and an optional `join`. Use `${ta}` wherever you +need the base table alias of the entity. + +```java +@Entity +public class ParentPerson { + + // aggregation via a derived join; ${ta} is the base table alias + @Formula(select = "coalesce(f2.child_count, 0)", + join = "left join (select parent_id, count(*) as child_count" + + " from child group by parent_id) f2 on f2.parent_id = ${ta}.id") + Integer childCount; + + // coalesce across a joined table using an explicit join alias (j1) + @Formula(select = "coalesce(${ta}.family_name, j1.family_name)", + join = "join parent_person j1 on j1.id = ${ta}.parent_id") + String effectiveFamilyName; +} +``` + +Notes: +- The `join` string must start with `join` or `left join`. +- You manage the join aliases (`j1`, `f2`, …) yourself and reference them in `select`. +- `@Formula` is `@Repeatable` and supports a `platforms()` restriction. + +--- + +## `@Formula2` — logical property paths + +Write the expression using property paths. Ebean resolves each path to the right table +alias and adds the joins it needs. + +```java +@Entity +public class ParentPerson { + + @ManyToOne + GrandParentPerson parent; + + String familyName; + + // Ebean automatically left joins 'parent' and resolves the aliases + @Formula2("coalesce(familyName, parent.familyName)") + String derivedFamilyName; +} +``` + +A query selecting `derivedFamilyName` produces (roughly): + +```sql +select t0.id, coalesce(t0.family_name, t1.family_name) +from parent_person t0 +left join grand_parent_person t1 on t1.id = t0.parent_id +``` + +Multi-level paths join through each step: + +```java +// joins parent and parent.parent automatically +@Formula2("coalesce(familyName, parent.familyName, parent.parent.familyName)") +String deepFamilyName; +``` + +`@Formula2` works wherever a normal property does — the required joins are added +automatically in each case: + +```java +// selected explicitly +DB.find(ParentPerson.class).select("derivedFamilyName").findList(); + +// used in where (auto-joins even when not selected) +DB.find(ParentPerson.class).where().eq("derivedFamilyName", "Smith").findList(); + +// used in order by +DB.find(ParentPerson.class).orderBy("derivedFamilyName").findList(); + +// referenced via a path from another bean +DB.find(ChildPerson.class).where().eq("parent.derivedFamilyName", "Smith").findList(); +``` + +It also resolves correctly inside nested `fetch()` joins, so a `@Formula2` on a fetched +association is computed with its own joins relative to that association. + +Notes: +- The expression supports any SQL function whose arguments are logical property paths. +- `@Formula2` supports a `platforms()` restriction. +- No `${ta}` and no hand-written join — that is the point of `@Formula2`. + +--- + +## Default inclusion and `@Transient` + +Both annotations are **included in queries by default** (just like a normal mapped +property). When no explicit `select()`/`fetch()` is given, the formula — and for +`@Formula2` the joins it requires — are added to the query. + +Add `@Transient` to make the formula **opt-in**: it is then **not** selected by default +and must be requested explicitly via `select()` or `fetch()`. Do this when the formula +(or the joins it needs) is relatively expensive. + +```java +// not selected by default; must be requested explicitly +@Transient +@Formula2("coalesce(familyName, parent.familyName)") +String lazyDerivedFamilyName; +``` + +```java +DB.find(ParentPerson.class) + .select("lazyDerivedFamilyName") // explicitly included, join auto-added + .findList(); +``` + +This is the same `@Transient` opt-out mechanism used by `@Formula`. + +--- + +## Which should I use? + +- Prefer **`@Formula2`** for expressions over property paths (coalesce/case/functions + across associations). It is shorter, refactor-friendly, and the joins stay correct as + the model changes. +- Use **`@Formula`** when you need raw SQL that does not map cleanly to property paths — + for example a derived aggregate sub-select / dynamic view, or vendor-specific SQL. + +For read models that exist only to carry computed values, also consider projecting to a +DTO instead of mapping the formula onto the entity — see +[writing-ebean-query-beans.md](writing-ebean-query-beans.md). diff --git a/ebean-core/src/main/java/io/ebeaninternal/server/deploy/BeanDescriptor.java b/ebean-core/src/main/java/io/ebeaninternal/server/deploy/BeanDescriptor.java index 16da6b307..8fc19c828 100644 --- a/ebean-core/src/main/java/io/ebeaninternal/server/deploy/BeanDescriptor.java +++ b/ebean-core/src/main/java/io/ebeaninternal/server/deploy/BeanDescriptor.java @@ -527,6 +527,7 @@ public class BeanDescriptor implements BeanType, STreeType, SpiBeanType { if (unidirectional != null) { unidirectional.initialise(initContext); } + initFormula2Properties(); idBinder.initialise(); idBinderInLHSSql = idBinder.bindInSql(baseTableAlias); idBinderIdSql = idBinder.bindEqSql(baseTableAlias); @@ -556,6 +557,21 @@ public class BeanDescriptor implements BeanType, STreeType, SpiBeanType { } } + /** + * Parse @Formula2 logical expressions into placeholder-form SQL after all + * relationships have been wired up and elPropertyDeploy() is fully functional. + */ + private void initFormula2Properties() { + for (BeanProperty prop : propertiesAll()) { + String rawExpr = prop.formula2RawExpression(); + if (rawExpr != null) { + DeployPropertyParser parser = parser().setCatchFirst(true); + String parsed = parser.parse(rawExpr); + prop.initFormula2(parsed, parser.includes()); + } + } + } + private boolean hasCircularImportedId() { for (BeanPropertyAssocOne assocOne : propertiesOneImportedSave) { if (assocOne.hasCircularImportedId(this)) { diff --git a/ebean-core/src/main/java/io/ebeaninternal/server/deploy/BeanProperty.java b/ebean-core/src/main/java/io/ebeaninternal/server/deploy/BeanProperty.java index 9e9579637..98bc05445 100644 --- a/ebean-core/src/main/java/io/ebeaninternal/server/deploy/BeanProperty.java +++ b/ebean-core/src/main/java/io/ebeaninternal/server/deploy/BeanProperty.java @@ -125,6 +125,9 @@ public class BeanProperty implements ElPropertyValue, Property, STreeProperty { final String elPlaceHolderEncrypted; private final String sqlFormulaSelect; final String sqlFormulaJoin; + private String formula2Select; + private Set formula2Includes; + private final String formula2RawExpression; private final String aggregation; private final boolean formula; private final boolean dbEncrypted; @@ -242,6 +245,7 @@ public class BeanProperty implements ElPropertyValue, Property, STreeProperty { this.sqlFormulaJoin = InternString.intern(deploy.getSqlFormulaJoin()); this.sqlFormulaSelect = InternString.intern(deploy.getSqlFormulaSelect()); this.formula = sqlFormulaSelect != null; + this.formula2RawExpression = deploy.getFormula2Expression(); this.dbType = deploy.getDbType(); this.scalarType = deploy.getScalarType(); this.lob = isLobType(dbType); @@ -298,6 +302,7 @@ public class BeanProperty implements ElPropertyValue, Property, STreeProperty { this.sqlFormulaJoin = null; this.sqlFormulaSelect = null; this.formula = false; + this.formula2RawExpression = null; this.aggregation = null; this.excludedFromHistory = source.excludedFromHistory; this.tenantId = source.tenantId; @@ -401,7 +406,7 @@ public class BeanProperty implements ElPropertyValue, Property, STreeProperty { * Return true if this property should have a DB Column created in DDL. */ public boolean isDDLColumn() { - return !formula && !secondaryTable && (aggregation == null); + return !formula && formula2RawExpression == null && !secondaryTable && (aggregation == null); } /** @@ -487,6 +492,8 @@ public class BeanProperty implements ElPropertyValue, Property, STreeProperty { public void appendSelect(DbSqlContext ctx, boolean subQuery) { if (aggregation != null) { ctx.appendFormulaSelect(aggregation); + } else if (formula2Select != null) { + ctx.appendFormula2Select(formula2Select); } else if (formula) { ctx.appendFormulaSelect(sqlFormulaSelect); } else if (!isTransient && !ignoreDraftOnlyProperty(ctx.isDraftQuery())) { @@ -855,6 +862,30 @@ public class BeanProperty implements ElPropertyValue, Property, STreeProperty { return formula && sqlFormulaJoin != null; } + /** + * Initialise this property's formula2 parsed expression and required join paths. + * Called by BeanDescriptor.initialise() after all relationships are wired. + */ + public void initFormula2(String parsedSelect, Set includes) { + this.formula2Select = parsedSelect; + this.formula2Includes = includes.isEmpty() ? null : java.util.Collections.unmodifiableSet(includes); + } + + /** + * Return the raw @Formula2 expression (before parsing), or null. + */ + public String formula2RawExpression() { + return formula2RawExpression; + } + + /** + * Return the join paths required by this @Formula2 property, or null if not a formula2. + */ + @Override + public Set formula2Joins() { + return formula2Includes; + } + @Override public boolean containsManySince(String sinceProperty) { return containsMany(); @@ -922,6 +953,10 @@ public class BeanProperty implements ElPropertyValue, Property, STreeProperty { @Override public String elPlaceholder(boolean encrypted) { + if (formula2Select != null) { + // resolve to the parsed @Formula2 expression (with ${} / ${path} placeholders) + return formula2Select; + } return encrypted ? elPlaceHolderEncrypted : elPlaceHolder; } diff --git a/ebean-core/src/main/java/io/ebeaninternal/server/deploy/DbSqlContext.java b/ebean-core/src/main/java/io/ebeaninternal/server/deploy/DbSqlContext.java index b249cfbbb..40d43cd3d 100644 --- a/ebean-core/src/main/java/io/ebeaninternal/server/deploy/DbSqlContext.java +++ b/ebean-core/src/main/java/io/ebeaninternal/server/deploy/DbSqlContext.java @@ -63,6 +63,12 @@ public interface DbSqlContext { */ void appendParseSelect(String parseSelect, String alias); + /** + * Parse and add a @Formula2 path based formula resolving the path placeholders + * (e.g. ${} or ${parent}) relative to the current node prefix. + */ + void appendFormula2Select(String parseSelect); + /** * Append a Sql Formula select. This converts the "${ta}" keyword to the * current table alias. diff --git a/ebean-core/src/main/java/io/ebeaninternal/server/deploy/DeployPropertyParser.java b/ebean-core/src/main/java/io/ebeaninternal/server/deploy/DeployPropertyParser.java index 2cdb5d5d2..aac9fc442 100644 --- a/ebean-core/src/main/java/io/ebeaninternal/server/deploy/DeployPropertyParser.java +++ b/ebean-core/src/main/java/io/ebeaninternal/server/deploy/DeployPropertyParser.java @@ -1,5 +1,6 @@ package io.ebeaninternal.server.deploy; +import io.ebean.util.SplitName; import io.ebeaninternal.server.el.ElPropertyDeploy; import java.util.HashSet; @@ -64,6 +65,7 @@ public final class DeployPropertyParser extends DeployParser { firstProp = elProp; } addIncludes(elProp.elPrefix()); + addFormula2Includes(elProp); return elProp.elPlaceholder(encrypted); } } @@ -79,4 +81,22 @@ public final class DeployPropertyParser extends DeployParser { includes.add(prefix); } } + + /** + * Add the join paths required by a @Formula2 property so the joins it + * references (e.g. parent, parent.parent) are included to support the + * formula when used in where / order by / select clauses. + */ + private void addFormula2Includes(ElPropertyDeploy elProp) { + BeanProperty beanProperty = elProp.beanProperty(); + if (beanProperty != null) { + Set formula2Joins = beanProperty.formula2Joins(); + if (formula2Joins != null) { + String prefix = elProp.elPrefix(); + for (String join : formula2Joins) { + includes.add(prefix == null ? join : SplitName.add(prefix, join)); + } + } + } + } } diff --git a/ebean-core/src/main/java/io/ebeaninternal/server/deploy/meta/DeployBeanProperty.java b/ebean-core/src/main/java/io/ebeaninternal/server/deploy/meta/DeployBeanProperty.java index 6797158b6..1245296bc 100644 --- a/ebean-core/src/main/java/io/ebeaninternal/server/deploy/meta/DeployBeanProperty.java +++ b/ebean-core/src/main/java/io/ebeaninternal/server/deploy/meta/DeployBeanProperty.java @@ -144,6 +144,7 @@ public class DeployBeanProperty { private String aggregationParsed; private String sqlFormulaSelect; private String sqlFormulaJoin; + private String formula2Expression; /** * The jdbc data type this maps to. */ @@ -556,6 +557,23 @@ public class DeployBeanProperty { this.dbUpdateable = false; } + /** + * Return the raw logical expression set by {@code @Formula2}. + */ + public String getFormula2Expression() { + return formula2Expression; + } + + /** + * Set the raw logical expression for a {@code @Formula2} property. + */ + public void setFormula2Expression(String formula2Expression) { + this.formula2Expression = formula2Expression; + this.dbRead = true; + this.dbInsertable = false; + this.dbUpdateable = false; + } + public void setImportedPrimaryKey() { this.importedPrimaryKey = true; } @@ -1095,6 +1113,22 @@ public class DeployBeanProperty { return fallback; } + public Formula2 getMetaAnnotationFormula2(Platform platform) { + Formula2 fallback = null; + for (Annotation ann : metaAnnotations) { + if (ann.annotationType() == Formula2.class) { + Formula2 formula2 = (Formula2) ann; + final Platform[] platforms = formula2.platforms(); + if (platforms.length == 0) { + fallback = formula2; + } else if (matchPlatform(platforms, platform)) { + return formula2; + } + } + } + return fallback; + } + public Where getMetaAnnotationWhere(Platform platform) { Where fallback = null; for (Annotation ann : metaAnnotations) { diff --git a/ebean-core/src/main/java/io/ebeaninternal/server/deploy/parse/AnnotationFields.java b/ebean-core/src/main/java/io/ebeaninternal/server/deploy/parse/AnnotationFields.java index 489f8fb97..dc76fe72c 100644 --- a/ebean-core/src/main/java/io/ebeaninternal/server/deploy/parse/AnnotationFields.java +++ b/ebean-core/src/main/java/io/ebeaninternal/server/deploy/parse/AnnotationFields.java @@ -270,6 +270,10 @@ final class AnnotationFields extends AnnotationParser { if (formula != null) { prop.setSqlFormula(processFormula(formula.select()), processFormula(formula.join())); } + Formula2 formula2 = prop.getMetaAnnotationFormula2(platform); + if (formula2 != null) { + prop.setFormula2Expression(formula2.value()); + } final Aggregation aggregation = prop.getMetaAnnotation(Aggregation.class); if (aggregation != null) { prop.setAggregation(aggregation.value().replace("$1", prop.getName())); diff --git a/ebean-core/src/main/java/io/ebeaninternal/server/deploy/parse/ReadAnnotationConfig.java b/ebean-core/src/main/java/io/ebeaninternal/server/deploy/parse/ReadAnnotationConfig.java index 74268616b..b4c3ab5d5 100644 --- a/ebean-core/src/main/java/io/ebeaninternal/server/deploy/parse/ReadAnnotationConfig.java +++ b/ebean-core/src/main/java/io/ebeaninternal/server/deploy/parse/ReadAnnotationConfig.java @@ -2,6 +2,7 @@ package io.ebeaninternal.server.deploy.parse; import io.ebean.annotation.Aggregation; import io.ebean.annotation.Formula; +import io.ebean.annotation.Formula2; import io.ebean.annotation.Where; import io.ebean.config.ClassLoadConfig; import io.ebean.DatabaseBuilder; @@ -46,6 +47,7 @@ final class ReadAnnotationConfig { this.metaAnnotations.add(Column.class); this.metaAnnotations.add(Formula.class); this.metaAnnotations.add(Formula.List.class); + this.metaAnnotations.add(Formula2.class); this.metaAnnotations.add(Where.class); this.metaAnnotations.add(Where.List.class); this.metaAnnotations.add(Aggregation.class); diff --git a/ebean-core/src/main/java/io/ebeaninternal/server/el/ElPropertyChain.java b/ebean-core/src/main/java/io/ebeaninternal/server/el/ElPropertyChain.java index 12d376bed..6278bec1c 100644 --- a/ebean-core/src/main/java/io/ebeaninternal/server/el/ElPropertyChain.java +++ b/ebean-core/src/main/java/io/ebeaninternal/server/el/ElPropertyChain.java @@ -9,6 +9,8 @@ import io.ebeaninternal.api.SpiExpressionRequest; import io.ebeaninternal.server.deploy.BeanProperty; import java.util.Arrays; +import java.util.regex.Matcher; +import java.util.regex.Pattern; /** @@ -24,6 +26,9 @@ import java.util.Arrays; */ public final class ElPropertyChain implements ElPropertyValue { + /** Matches placeholders of the form ${} or ${path} (e.g. ${parent}) used by @Formula2. */ + private static final Pattern PLACEHOLDER = Pattern.compile("\\$\\{([^}]*)}"); + private final String prefix; private final String placeHolder; private final String placeHolderEncrypted; @@ -94,9 +99,18 @@ public final class ElPropertyChain implements ElPropertyValue { if (!el.contains("${}")) { // typically a secondary table property return el.replace("${", "${" + prefix + "."); - } else { - return el.replace(ROOT_ELPREFIX, "${" + prefix + "}"); } + // prefix the root placeholder ${} as well as any path placeholders ${path} + // (e.g. ${parent} used by a @Formula2 property referenced via a path) + Matcher matcher = PLACEHOLDER.matcher(el); + StringBuilder sb = new StringBuilder(); + while (matcher.find()) { + String path = matcher.group(1); + String replacement = path.isEmpty() ? "${" + prefix + "}" : "${" + prefix + "." + path + "}"; + matcher.appendReplacement(sb, Matcher.quoteReplacement(replacement)); + } + matcher.appendTail(sb); + return sb.toString(); } /** diff --git a/ebean-core/src/main/java/io/ebeaninternal/server/query/DefaultDbSqlContext.java b/ebean-core/src/main/java/io/ebeaninternal/server/query/DefaultDbSqlContext.java index 139ee6ca8..8c9690543 100644 --- a/ebean-core/src/main/java/io/ebeaninternal/server/query/DefaultDbSqlContext.java +++ b/ebean-core/src/main/java/io/ebeaninternal/server/query/DefaultDbSqlContext.java @@ -281,6 +281,14 @@ final class DefaultDbSqlContext implements DbSqlContext { } } + @Override + public void appendFormula2Select(String parseSelect) { + // resolve the path based placeholders relative to the current node prefix + sb.append(COMMA); + sb.append(alias.parseFormula2(parseSelect, currentPrefix)); + appendColumnAlias(); + } + @Override public void appendFormulaSelect(String sqlFormulaSelect) { String tableAlias = tableAliasStack.peek(); diff --git a/ebean-core/src/main/java/io/ebeaninternal/server/query/STreeProperty.java b/ebean-core/src/main/java/io/ebeaninternal/server/query/STreeProperty.java index ae8cf1b72..56b7e01f8 100644 --- a/ebean-core/src/main/java/io/ebeaninternal/server/query/STreeProperty.java +++ b/ebean-core/src/main/java/io/ebeaninternal/server/query/STreeProperty.java @@ -107,4 +107,12 @@ public interface STreeProperty extends ScalarDataReader { default void extraIncludes(Set predicateIncludes) { // do nothing } + + /** + * Return the association join paths required by a {@code @Formula2} property, or null. + * These paths are automatically added as joins when this property is selected. + */ + default Set formula2Joins() { + return null; + } } diff --git a/ebean-core/src/main/java/io/ebeaninternal/server/query/SqlTreeAlias.java b/ebean-core/src/main/java/io/ebeaninternal/server/query/SqlTreeAlias.java index 3a5e7bdc7..7cfe4529a 100644 --- a/ebean-core/src/main/java/io/ebeaninternal/server/query/SqlTreeAlias.java +++ b/ebean-core/src/main/java/io/ebeaninternal/server/query/SqlTreeAlias.java @@ -8,6 +8,8 @@ import java.util.HashSet; import java.util.Map; import java.util.Set; import java.util.TreeSet; +import java.util.regex.Matcher; +import java.util.regex.Pattern; /** * Special Map of the logical property joins to table alias. @@ -152,6 +154,33 @@ final class SqlTreeAlias { return parseAliasMap(clause, aliasMap); } + /** + * Placeholder pattern for @Formula2 path based formulas e.g. ${} or ${parent}. + */ + private static final Pattern FORMULA2_PLACEHOLDER = Pattern.compile("\\$\\{([^}]*)}"); + + /** + * Parse a @Formula2 select/predicate replacing the path based placeholders. + *

+ * The placeholder paths are relative to the bean the formula is declared on. They are combined + * with the current node prefix to give the root descriptor relative path and then mapped to the + * appropriate table alias. e.g. with currentPrefix "children.children" the placeholder ${parent} + * resolves via the path "children.children.parent". + */ + String parseFormula2(String clause, String currentPrefix) { + Matcher matcher = FORMULA2_PLACEHOLDER.matcher(clause); + StringBuilder sb = new StringBuilder(clause.length() + 16); + while (matcher.find()) { + String relativePath = matcher.group(1); + String fullPath = relativePath.isEmpty() ? currentPrefix : SplitName.add(currentPrefix, relativePath); + String tableAlias = tableAlias(fullPath); + String replacement = tableAlias == null ? "" : tableAlias + "."; + matcher.appendReplacement(sb, Matcher.quoteReplacement(replacement)); + } + matcher.appendTail(sb); + return sb.toString(); + } + /** * Parse the clause replacing the table alias place holders. */ diff --git a/ebean-core/src/main/java/io/ebeaninternal/server/query/SqlTreeBuilder.java b/ebean-core/src/main/java/io/ebeaninternal/server/query/SqlTreeBuilder.java index d3ce75c25..b48585d46 100644 --- a/ebean-core/src/main/java/io/ebeaninternal/server/query/SqlTreeBuilder.java +++ b/ebean-core/src/main/java/io/ebeaninternal/server/query/SqlTreeBuilder.java @@ -37,6 +37,7 @@ public final class SqlTreeBuilder { private final SqlTreeAlias alias; private final DefaultDbSqlContext ctx; private final HashSet selectIncludes = new HashSet<>(); + private final HashSet formula2JoinIncludes = new HashSet<>(); private final ManyWhereJoins manyWhereJoins; private final TableJoin includeJoin; private final boolean rawSql; @@ -220,6 +221,7 @@ public final class SqlTreeBuilder { if (!rawSql) { alias.addJoin(queryDetail.getFetchPaths(), desc); alias.addJoin(predicates.predicateIncludes(), desc); + alias.addJoin(formula2JoinIncludes, desc); alias.addManyWhereJoins(manyWhereJoins.propertyNames()); // build set of table alias alias.buildAlias(); @@ -255,7 +257,7 @@ public final class SqlTreeBuilder { } OrmQueryProperties queryProps = queryDetail.getChunk(prefix, false); - SqlTreeProperties props = getBaseSelect(desc, queryProps); + SqlTreeProperties props = getBaseSelect(desc, queryProps, prefix); if (prefix == null && !rawSql) { if (props.requireSqlDistinct(manyWhereJoins)) { @@ -354,7 +356,14 @@ public final class SqlTreeBuilder { } Set predicateIncludes = predicates.predicateIncludes(); if (predicateIncludes == null) { - return; + if (formula2JoinIncludes.isEmpty()) { + return; + } + // no predicate includes but we have formula2 joins - use formula2 set directly + predicateIncludes = new HashSet<>(formula2JoinIncludes); + } else { + // add formula2 required joins alongside predicate includes + predicateIncludes.addAll(formula2JoinIncludes); } // Note includes - basically means joins. @@ -482,6 +491,23 @@ public final class SqlTreeBuilder { if (p.isAggregationManyToOne()) { p.extraIncludes(predicates.predicateIncludes()); } + addFormula2Joins(p, queryProps.getPath()); + } + } + } + + /** + * Accumulate the auto-join paths required by a @Formula2 property. + *

+ * The property's formula2Joins() are relative to the property's own bean, so they are + * prefixed with the node path to give root-descriptor-relative join paths. These are later + * turned into extra joins (see buildExtraJoins) and registered with the SqlTreeAlias. + */ + private void addFormula2Joins(STreeProperty p, String prefix) { + Set f2Joins = p.formula2Joins(); + if (f2Joins != null) { + for (String join : f2Joins) { + formula2JoinIncludes.add(SplitName.add(prefix, join)); } } } @@ -516,7 +542,7 @@ public final class SqlTreeBuilder { return selectProps; } - private SqlTreeProperties getBaseSelect(STreeType desc, OrmQueryProperties queryProps) { + private SqlTreeProperties getBaseSelect(STreeType desc, OrmQueryProperties queryProps, String prefix) { boolean partial = queryProps != null && !queryProps.allProperties(); if (partial) { return getBaseSelectPartial(desc, queryProps); @@ -526,7 +552,12 @@ public final class SqlTreeBuilder { selectProps.setAllProperties(); // normal simple properties of the bean - selectProps.add(desc.propsBaseScalar()); + STreeProperty[] baseScalar = desc.propsBaseScalar(); + selectProps.add(baseScalar); + for (STreeProperty p : baseScalar) { + // @Formula2 auto-join: prefix the (bean relative) join paths with this node path + addFormula2Joins(p, prefix); + } selectProps.add(desc.propsEmbedded()); for (STreePropertyAssocOne propertyAssocOne : desc.propsOne()) { diff --git a/ebean-test/src/test/java/org/tests/family/TestFormula2NestedFetch.java b/ebean-test/src/test/java/org/tests/family/TestFormula2NestedFetch.java new file mode 100644 index 000000000..6685a100c --- /dev/null +++ b/ebean-test/src/test/java/org/tests/family/TestFormula2NestedFetch.java @@ -0,0 +1,146 @@ +package org.tests.family; + +import io.ebean.DB; +import io.ebean.test.LoggedSql; +import io.ebean.xtest.BaseTestCase; +import org.junit.jupiter.api.Test; +import org.tests.model.family.ChildPerson; +import org.tests.model.family.GrandParentPerson; +import org.tests.model.family.ParentPerson; + +import java.util.List; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * Verifies that the path based @Formula2 auto-join works not only on the root entity + * but also when the entity holding the @Formula2 property is loaded as a nested fetch + * (either a nested *ToMany included in the main query, or a nested *ToOne join). + */ +public class TestFormula2NestedFetch extends BaseTestCase { + + private GrandParentPerson setupFamily() { + ChildPerson fred = new ChildPerson(); + fred.setName("Fred"); // familyName null -> inherits from parent / grandparent + + ChildPerson julia = new ChildPerson(); + julia.setName("Julia"); + julia.setFamilyName("Baz"); + + ChildPerson roland = new ChildPerson(); + roland.setName("Roland"); // familyName null + + ParentPerson maria = new ParentPerson(); + maria.setName("Maria"); + maria.setFamilyName("Bar"); + maria.getChildren().add(fred); + maria.getChildren().add(julia); + + ParentPerson sandra = new ParentPerson(); + sandra.setName("Sandra"); // familyName null -> inherits "Foo" + sandra.getChildren().add(roland); + + GrandParentPerson josef = new GrandParentPerson(); + josef.setName("Josef"); + josef.setFamilyName("Foo"); + josef.getChildren().add(maria); + josef.getChildren().add(sandra); + + DB.save(josef); + return josef; + } + + private void cleanup() { + DB.find(ChildPerson.class).delete(); + DB.find(GrandParentPerson.class).delete(); + } + + @Test + void formula2_nestedMany_autoJoins() { + GrandParentPerson josef = setupFamily(); + try { + LoggedSql.start(); + + GrandParentPerson found = DB.find(GrandParentPerson.class) + .fetch("children", "name, derivedFamilyName") + .fetch("children.children", "name, derivedFamilyName") + .setId(josef.getIdentifier()) + .findOne(); + + List sql = LoggedSql.stop(); + + assertThat(found).isNotNull(); + + // ParentPerson (path "children") derivedFamilyName = coalesce(familyName, parent.familyName) + ParentPerson maria = byName(found.getChildren(), "Maria"); + ParentPerson sandra = byName(found.getChildren(), "Sandra"); + assertThat(maria.getDerivedFamilyName()).isEqualTo("Bar"); // own familyName + assertThat(sandra.getDerivedFamilyName()).isEqualTo("Foo"); // inherited from grandparent + + // ChildPerson (path "children.children") derivedFamilyName = + // coalesce(familyName, parent.familyName, parent.parent.familyName) + ChildPerson fred = childByName(maria.getChildren(), "Fred"); + ChildPerson julia = childByName(maria.getChildren(), "Julia"); + ChildPerson roland = childByName(sandra.getChildren(), "Roland"); + assertThat(fred.getDerivedFamilyName()).isEqualTo("Bar"); // inherited from parent Maria + assertThat(julia.getDerivedFamilyName()).isEqualTo("Baz"); // own familyName + assertThat(roland.getDerivedFamilyName()).isEqualTo("Foo"); // inherited from grandparent + + // the nested ParentPerson formula2 is part of the main grand_parent_person query and + // its auto-join (children.parent -> grand_parent_person t2) is added to that same sql + String main = sql.stream().filter(s -> s.contains("from grand_parent_person")).findFirst().orElseThrow(); + assertThat(main).contains("coalesce(t1.family_name, t2.family_name)"); + assertThat(main).contains("left join grand_parent_person t2 on t2.identifier = t1.parent_identifier"); + + // the ChildPerson many is loaded as its own query - formula2 auto-joins parent and parent.parent + String childSql = sql.stream().filter(s -> s.contains("from child_person")).findFirst().orElseThrow(); + assertThat(childSql).contains("coalesce(t0.family_name, t1.family_name, t2.family_name)"); + assertThat(childSql).contains("left join parent_person t1 on t1.identifier = t0.parent_identifier"); + assertThat(childSql).contains("left join grand_parent_person t2 on t2.identifier = t1.parent_identifier"); + } finally { + cleanup(); + } + } + + @Test + void formula2_nestedToOne_autoJoins() { + setupFamily(); + try { + LoggedSql.start(); + + // ChildPerson root, fetch the parent (ParentPerson) which has a @Formula2 requiring + // parent.parent -> the nested *ToOne formula2 auto-join must be added to the same sql + List children = DB.find(ChildPerson.class) + .select("name, derivedFamilyName") + .fetch("parent", "name, derivedFamilyName") + .where().eq("name", "Fred") + .findList(); + + List sql = LoggedSql.stop(); + + assertThat(children).hasSize(1); + ChildPerson fred = children.get(0); + assertThat(fred.getDerivedFamilyName()).isEqualTo("Bar"); + // the fetched parent (Maria) derivedFamilyName resolves via its own auto-joined parent + assertThat(fred.getParent().getDerivedFamilyName()).isEqualTo("Bar"); + + String main = sql.get(0); + // ChildPerson (t0) formula2 -> parent t1, parent.parent t2 + assertThat(main).contains("coalesce(t0.family_name, t1.family_name, t2.family_name)"); + // nested ParentPerson (t1) formula2 -> parent.parent t2 (reused grand_parent_person alias) + assertThat(main).contains("coalesce(t1.family_name, t2.family_name)"); + assertThat(main).contains("left join parent_person t1 on t1.identifier = t0.parent_identifier"); + assertThat(main).contains("left join grand_parent_person t2 on t2.identifier = t1.parent_identifier"); + } finally { + cleanup(); + } + } + + private static ParentPerson byName(List list, String name) { + return list.stream().filter(p -> name.equals(p.getName())).findFirst().orElseThrow(); + } + + private static ChildPerson childByName(List list, String name) { + return list.stream().filter(p -> name.equals(p.getName())).findFirst().orElseThrow(); + } +} diff --git a/ebean-test/src/test/java/org/tests/model/family/ChildPerson.java b/ebean-test/src/test/java/org/tests/model/family/ChildPerson.java index 2f9f718d4..866f49590 100644 --- a/ebean-test/src/test/java/org/tests/model/family/ChildPerson.java +++ b/ebean-test/src/test/java/org/tests/model/family/ChildPerson.java @@ -1,6 +1,7 @@ package org.tests.model.family; import io.ebean.annotation.Formula; +import io.ebean.annotation.Formula2; import org.tests.model.basic.EBasic; import jakarta.persistence.CascadeType; @@ -21,10 +22,13 @@ public class ChildPerson extends InheritablePerson { private String address; - //@Coalesce({ "familyName", "parent.familyName", "parent.parent.familyName" }) + //@Formula2("coalesce(familyName, parent.familyName, parent.parent.familyName)") @Formula(select = "coalesce(${ta}.family_name, j1.family_name, j2.family_name)", join = PARENTS_JOIN) private String effectiveFamilyName; + @Formula2("coalesce(familyName, parent.familyName, parent.parent.familyName)") + private String derivedFamilyName; + //@Coalesce({ "address", "parent.address", "parent.parent.address" }) @Formula(select = "coalesce(${ta}.address, j1.address, j2.address)", join = PARENTS_JOIN) private String effectiveAddress; @@ -61,6 +65,10 @@ public class ChildPerson extends InheritablePerson { return effectiveFamilyName; } + public String getDerivedFamilyName() { + return derivedFamilyName; + } + public String getEffectiveAddress() { return effectiveAddress; } diff --git a/ebean-test/src/test/java/org/tests/model/family/ParentPerson.java b/ebean-test/src/test/java/org/tests/model/family/ParentPerson.java index 8955af81d..4b8e95a7e 100644 --- a/ebean-test/src/test/java/org/tests/model/family/ParentPerson.java +++ b/ebean-test/src/test/java/org/tests/model/family/ParentPerson.java @@ -1,6 +1,7 @@ package org.tests.model.family; import io.ebean.annotation.Formula; +import io.ebean.annotation.Formula2; import org.tests.model.basic.EBasic; import jakarta.persistence.*; @@ -42,6 +43,15 @@ public class ParentPerson extends InheritablePerson { @Formula(select = "coalesce(${ta}.family_name, j1.family_name)", join = GRAND_PARENT_PERSON_JOIN) private String effectiveFamilyName; + //@Formula2 equivalent - logical path-based, joins are auto-detected + @Formula2("coalesce(familyName, parent.familyName)") + private String derivedFamilyName; + + // @Transient makes the @Formula2 property opt-in (excluded from queries by default) + @Transient + @Formula2("coalesce(familyName, parent.familyName)") + private String lazyDerivedFamilyName; + //@Coalesce({ "address", "parent.address" }) @Formula(select = "coalesce(${ta}.address, j1.address)", join = GRAND_PARENT_PERSON_JOIN) private String effectiveAddress; @@ -90,6 +100,14 @@ public class ParentPerson extends InheritablePerson { return effectiveFamilyName; } + public String getDerivedFamilyName() { + return derivedFamilyName; + } + + public String getLazyDerivedFamilyName() { + return lazyDerivedFamilyName; + } + public String getEffectiveAddress() { return effectiveAddress; } diff --git a/ebean-test/src/test/java/org/tests/query/joins/TestQueryJoinOnFormula.java b/ebean-test/src/test/java/org/tests/query/joins/TestQueryJoinOnFormula.java index e95ba72b3..a2b0803c8 100644 --- a/ebean-test/src/test/java/org/tests/query/joins/TestQueryJoinOnFormula.java +++ b/ebean-test/src/test/java/org/tests/query/joins/TestQueryJoinOnFormula.java @@ -219,6 +219,196 @@ public class TestQueryJoinOnFormula extends BaseTestCase { assertSql(sql.get(0)).contains("select t0.order_date from o_order t0"); } + @Test + void select_formala2_coalesceWithJoin() { + LoggedSql.start(); + + List names = DB.find(ParentPerson.class) + .select("coalesce(familyName, parent.familyName) as family_name") + .fetch("parent.familyName") + .where().eq("identifier", 1) + .findSingleAttributeList(); + + List sql = LoggedSql.stop(); + assertEquals(1, sql.size()); + + String querySql = sql.get(0); + assertThat(querySql).contains("select coalesce(t0.family_name, t1.family_name) family_name from parent_person t0 "); + assertThat(querySql).contains("from parent_person t0 left join grand_parent_person t1 on t1.identifier = t0.parent_identifier where t0.identifier = ?"); + } + + @Test + void formula2_autoJoin_parentPerson() { + LoggedSql.start(); + + // Select the @Formula2 property - the join to 'parent' should be auto-added without fetch() + List list = DB.find(ParentPerson.class) + .select("derivedFamilyName") + .where().eq("identifier", 1) + .findList(); + + List sql = LoggedSql.stop(); + assertEquals(1, sql.size()); + + String querySql = sql.get(0); + // @Formula2("coalesce(familyName, parent.familyName)") should auto-join parent + assertThat(querySql).contains("coalesce(t0.family_name, t1.family_name)"); + assertThat(querySql).contains("left join grand_parent_person t1 on t1.identifier = t0.parent_identifier"); + } + + @Test + void formula2_autoJoin_multiLevel_childPerson() { + LoggedSql.start(); + + // @Formula2("coalesce(familyName, parent.familyName, parent.parent.familyName)") + // should auto-join both parent and parent.parent + List list = DB.find(ChildPerson.class) + .select("derivedFamilyName") + .findList(); + + List sql = LoggedSql.stop(); + assertEquals(1, sql.size()); + + String querySql = sql.get(0); + assertThat(querySql).contains("coalesce(t0.family_name, t1.family_name, t2.family_name)"); + assertThat(querySql).contains("left join parent_person t1 on t1.identifier = t0.parent_identifier"); + assertThat(querySql).contains("left join grand_parent_person t2 on t2.identifier = t1.parent_identifier"); + } + + @Test + void formula2_where_autoJoin_parentPerson() { + LoggedSql.start(); + + // @Formula2 property referenced in where (and NOT selected) should + // resolve to the formula and auto-add the join to 'parent'. + DB.find(ParentPerson.class) + .where().eq("derivedFamilyName", "Smith") + .findList(); + + List sql = LoggedSql.stop(); + assertEquals(1, sql.size()); + + String querySql = sql.get(0); + assertThat(querySql).contains("left join grand_parent_person t1 on t1.identifier = t0.parent_identifier"); + assertThat(querySql).contains("where coalesce(t0.family_name, t1.family_name) = ?"); + } + + @Test + void formula2_orderBy_autoJoin_parentPerson() { + LoggedSql.start(); + + // @Formula2 property referenced in order by (and NOT selected) should + // resolve to the formula and auto-add the join to 'parent'. + DB.find(ParentPerson.class) + .orderBy("derivedFamilyName") + .findList(); + + List sql = LoggedSql.stop(); + assertEquals(1, sql.size()); + + String querySql = sql.get(0); + assertThat(querySql).contains("left join grand_parent_person t1 on t1.identifier = t0.parent_identifier"); + assertThat(querySql).contains("order by coalesce(t0.family_name, t1.family_name)"); + } + + @Test + void formula2_where_autoJoin_multiLevel_childPerson() { + LoggedSql.start(); + + // @Formula2("coalesce(familyName, parent.familyName, parent.parent.familyName)") + // referenced in where should auto-join both parent and parent.parent. + DB.find(ChildPerson.class) + .where().eq("derivedFamilyName", "Smith") + .findList(); + + List sql = LoggedSql.stop(); + assertEquals(1, sql.size()); + + String querySql = sql.get(0); + assertThat(querySql).contains("left join parent_person t1 on t1.identifier = t0.parent_identifier"); + assertThat(querySql).contains("left join grand_parent_person t2 on t2.identifier = t1.parent_identifier"); + assertThat(querySql).contains("where coalesce(t0.family_name, t1.family_name, t2.family_name) = ?"); + } + + @Test + void formula2_where_viaPath_nestedFormula2_childPerson() { + LoggedSql.start(); + + // Reference a @Formula2 property of an associated bean via a path: + // ChildPerson.parent -> ParentPerson.derivedFamilyName = coalesce(familyName, parent.familyName) + // The path placeholders must be resolved relative to 'parent', auto-joining + // parent (ParentPerson) and parent.parent (GrandParentPerson). + DB.find(ChildPerson.class) + .where().eq("parent.derivedFamilyName", "Smith") + .findList(); + + List sql = LoggedSql.stop(); + assertEquals(1, sql.size()); + + String querySql = sql.get(0); + assertThat(querySql).contains("left join parent_person t1 on t1.identifier = t0.parent_identifier"); + assertThat(querySql).contains("left join grand_parent_person t2 on t2.identifier = t1.parent_identifier"); + assertThat(querySql).contains("where coalesce(t1.family_name, t2.family_name) = ?"); + } + + @Test + void formula2_includedByDefault() { + LoggedSql.start(); + + // A non-@Transient @Formula2 property is included by default (no explicit select), + // consistent with @Formula. Its join is auto-added. + DB.find(ParentPerson.class) + .where().eq("identifier", -1) + .findList(); + + List sql = LoggedSql.stop(); + assertEquals(1, sql.size()); + + String querySql = sql.get(0); + // derivedFamilyName (@Formula2, not @Transient) present in the default select + assertThat(querySql).contains("coalesce(t0.family_name, t1.family_name)"); + assertThat(querySql).contains("left join grand_parent_person t1 on t1.identifier = t0.parent_identifier"); + } + + @Test + void formula2_transient_excludedByDefault() { + LoggedSql.start(); + + // lazyDerivedFamilyName is @Formula2 + @Transient -> NOT selected by default + DB.find(ParentPerson.class) + .where().eq("identifier", -1) + .findList(); + + List sql = LoggedSql.stop(); + assertEquals(1, sql.size()); + + String querySql = sql.get(0); + // only one coalesce(family_name,...) -> the non-transient derivedFamilyName. + // If the transient property leaked into the default select there would be two. + int occurrences = querySql.split("coalesce\\(t0.family_name, t1.family_name\\)", -1).length - 1; + assertThat(occurrences) + .as("transient @Formula2 must not be selected by default") + .isEqualTo(1); + } + + @Test + void formula2_transient_explicitlySelected() { + LoggedSql.start(); + + // A @Transient @Formula2 property can still be explicitly selected, with its join auto-added. + DB.find(ParentPerson.class) + .select("lazyDerivedFamilyName") + .where().eq("identifier", -1) + .findList(); + + List sql = LoggedSql.stop(); + assertEquals(1, sql.size()); + + String querySql = sql.get(0); + assertThat(querySql).contains("coalesce(t0.family_name, t1.family_name)"); + assertThat(querySql).contains("left join grand_parent_person t1 on t1.identifier = t0.parent_identifier"); + } + @Test public void test_OrderFindOne() { diff --git a/pom.xml b/pom.xml index fceab4790..057c90afb 100644 --- a/pom.xml +++ b/pom.xml @@ -45,7 +45,7 @@ 3.1 3.0 - 8.4 + 8.5 2.3 1.2 14.3.0