Enhancement : Expand and Harden the Safe Expression Engine

[lattln]

Summary
Improve eSheet’s safe expression engine so it can handle more real-world formula, conditional, and display scenarios without relying on gated JavaScript execution. The goal is to grow the custom tokenizer/AST-based evaluator into a more capable, safer default expression system for guided UI authoring and complex form logic.

Background
eSheet currently has two expression paths:

• Safe expressions: evaluated by a custom tokenizer, parser, and AST evaluator
• Dangerous JS: evaluated with new Function behind explicit host/schema opt-in

The safe path is the preferred default and is already used in:

• conditional rules
• display field interpolation
• expression validation

Today, the safe engine already supports:

• field references like {fieldId}
• property accessors like .length and .count
• arithmetic operators like +, -, *, /, %
• comparison and boolean operators
• if(...)
• date helpers like addDays, subDays, diffDays, today, year
• numeric helpers like min, max, round, floor, ceil, abs

It does not currently include helpers like sum() or avg() in the safe evaluator. Also, {a} + {b} already covers basic summation, so sum() should be treated as optional convenience rather than a required gap.

Goal
Expand the safe expression system with a curated set of helpers and syntax improvements that make common calculations and conditional logic easier to author without enabling dangerous JS.

What We Want to Enable

• More expressive safe formulas
• Cleaner authoring for non-technical users in the builder
• Better support for IDE-style guided editing, validation, and autocomplete
• Fewer cases where authors need to fall back to dangerous JS

Proposed Scope

1. Safe helper expansion
   Add a small, high-value set of safe functions for common expression patterns.

2. AST/parser robustness
   Improve support for more complex but still bounded expressions, while keeping evaluation non-eval and non-new Function.

3. Editor and UX support
   Make the expression authoring experience better in Monaco/builder flows with validation, discovery, and suggestions.

Candidate Safe Helpers

Priority 1, high value:

• avg(...) for averages
• coalesce(a, b, c) or default(a, b) for fallback values
• clamp(value, min, max)
• roundTo(value, digits) or similar precision helper

Priority 2, useful but secondary:

• trim(value)
• lower(value)
• upper(value)
• startsWith(value, prefix)
• endsWith(value, suffix)
• replace(value, search, replacement)
• substring(value, start, end)

Priority 3, if useful and safe to keep bounded:

• addMonths(date, n)
• subMonths(date, n)
• startOfMonth(date)
• endOfMonth(date)
• startOfWeek(date)
• endOfWeek(date)

Notes on sum()

• sum() is not strictly necessary for basic arithmetic, because {a} + {b} + {c} already works.
• If we add it, it should be for readability and consistency, not because the engine lacks summation.
• It may be more useful later for array-style aggregation than for plain field-to-field math.

Non-Goals

• Do not use eval
• Do not introduce unrestricted JavaScript execution into the safe path
• Do not merge safe expressions with dangerous JS
• Do not make the safe engine Turing-complete
• Do not add helpers that are overly broad or hard to reason about

Implementation Themes

• Extend the tokenizer/parser only where needed
• Keep the AST evaluator deterministic and bounded
• Preserve backward compatibility with existing safe expressions
• Add tests for each new helper and syntax form
• Update documentation to clearly separate:
  • safe expressions
  • dangerous JS
  • when to use each

Acceptance Criteria

• Safe expressions support the agreed helper set
• Existing safe expressions continue to work unchanged
• Invalid expressions fail cleanly and predictably
• Expression validation reflects the expanded safe grammar
• Builder/editor UX can surface the new capabilities clearly
• Dangerous JS remains a separate, explicitly gated execution path

Suggested Deliverable Split

• Expand the safe evaluator with a first batch of helpers
• Add tests and docs for the new syntax
• Improve builder/Monaco expression authoring support
• Review whether any additional JS-like helpers are worth making safe in a later phase

[redmine-bridge-mieweb]

Redmine Ticket: #158674 created from GitHub Issue