diff --git a/CHANGELOG.md b/CHANGELOG.md index 256512fb1..bc99b0823 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -74,6 +74,9 @@ Bug fixes for 4.0. ### Fixed - A `HomotopySolver` solve's records ask now includes its start points (#498). It named the homotopy, target, settings and seed but not the given start points, so with recording on (the default) a second solve of the same homotopy from different start points was the same ask, and was answered from the record of the first: it silently returned the first solve's endpoints. The start system now reports the identity of start points it was given (`StartSystem::GivenStartIdentity`, a digest of their exact values), and the solve adds it to its ask. A solve from given start points therefore has a different run id than in 4.0.0, so records written by 4.0.0 for such solves are asked again rather than recalled. +- Differentiating an expression differentiates each distinct subexpression once, however many paths in the expression reach it (#505). It used to differentiate a shared subexpression once per path, so a determinant built by cofactor expansion with its sub-minors shared took time proportional to k! -- 24 s for the three partial derivatives of a 7 x 7 determinant of quadratics, all of it paid on a `System`'s first evaluation, which builds its Jacobian. `Node::Differentiate` now keeps each derivative for the length of the outermost call, and a Jacobian is built under one `node::DerivativeScope`, so subexpressions its functions share are differentiated once among all of them. A node type's own rule is now `DifferentiateImpl`; a Python subclass of `Node` still overrides `Differentiate`. +- Building a sum or product no longer walks its operands (#505). Putting the operands in canonical order asks each one's variables, multidegree and, on a tie, canonical encoding, and each answer was a walk of the operand's DAG -- of its whole expansion, for `Degree(v)` under a negation -- so building an expression cost the square of its size, or worse. Each node now keeps a degree record, made once from its operands' records: its variables, its degree and multidegree in each variable it mentions, its total degree, and its answer for every variable it does not mention. `Degree`, `MultiDegree` and `GatherVariables` read it. On the Barth sextic's 50 deflated critical-point systems, the first evaluations took 121 s and take 0.6 s. A node type's degree rule is now `DegreeImpl`; a Python subclass of `Node` still overrides `Degree`. +- **The canonicalization tie-break is a content key, and the system encoding version is `b2sysenc/3`** (#505, ADR-0073). Operands that tie on multidegree were ordered by their whole canonical encoding, which costs a walk of the operand. They are now ordered by a content key: the SHA-256 of the operand's own canonical encoding with each of its operands written as that operand's key -- a Merkle hash, made from the operands' keys once per node. The first evaluation of a 7 x 7 determinant of quadratics took 24.6 s in 4.0.0 and takes 0.04 s. Where operands tie, their order in a system's canonical encoding changes, so the encoding version moves to `b2sysenc/3` and every system digest changes: records written by 4.0.0 are asked again rather than recalled, and `System.from_canonical` refuses `b2sysenc/2` text. _______________________________________________________________________________ diff --git a/core/include/bertini2/function_tree/canonical_encoding.hpp b/core/include/bertini2/function_tree/canonical_encoding.hpp index bb5eeb2f4..7f410e8db 100644 --- a/core/include/bertini2/function_tree/canonical_encoding.hpp +++ b/core/include/bertini2/function_tree/canonical_encoding.hpp @@ -101,5 +101,17 @@ void EncodeCanonical(std::shared_ptr const& root, std::ostream& out, Encod */ std::string CanonicalEncoding(std::shared_ptr const& root); +/** +\brief The root's own canonical encoding, with each operand written as `@` and its content key. + +The text a content key is the SHA-256 of (Node::ContentKey): the root's kind, payload and +operands in the format above, but an operand is not expanded -- it is `@`. So a +node's key is made from its operands' keys, once per node, and equal content gives equal keys. + +\param root The node. +\return The shallow encoding text. +*/ +std::string ShallowCanonicalEncoding(std::shared_ptr const& root); + } // namespace node } // namespace bertini diff --git a/core/include/bertini2/function_tree/node.hpp b/core/include/bertini2/function_tree/node.hpp index 29d1fedcd..702bfbaf9 100755 --- a/core/include/bertini2/function_tree/node.hpp +++ b/core/include/bertini2/function_tree/node.hpp @@ -237,11 +237,17 @@ class Node : public VisitableBase<>, public std::enable_shared_from_this /** \brief Compute the derivative with respect to a single variable. - Virtual method for differentiating the node. If no variable is passed, produces a Jacobian tree when all is said and done, which is used for evaluating the Jacobian. + If no variable is passed, produces a Jacobian tree when all is said and done, which is used for evaluating the Jacobian. + Each distinct node is differentiated once per variable, however many paths reach it: the + derivative of a subexpression is kept for as long as the outermost call (or an enclosing + DerivativeScope) lasts, and every later request for it is answered from there. A node type + supplies its own rule as DifferentiateImpl, which calls this on its operands. + + \param v The variable to differentiate with respect to. \return The result of differentiation. Jacobian or regular Node depending on what you passed in. */ - virtual std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const = 0; + std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const; /** \brief Differentiate repeatedly with respect to a single variable. @@ -270,43 +276,71 @@ class Node : public VisitableBase<>, public std::enable_shared_from_this /** Compute the degree, optionally with respect to a single variable. + Answered from the node's degree record (see DegreeRecord), which is made once per node from + its operands' records, so asking costs a lookup however large the expansion beneath the node. + The per-type rule lives in DegreeImpl. + \param v Shared pointer to variable with respect to which you want to compute the degree of the Node. \return The degree. Will be negative if the Node is non-polynomial. */ - virtual int Degree(std::shared_ptr const& v = nullptr) const = 0; - + int Degree(std::shared_ptr const& v = nullptr) const; + /// \brief Per-node-type degree in one variable. Asks its operands through Degree(), so + /// their records answer; a node's own record is made by calling this, so it must never ask + /// Degree() of its own node. + /// \param v The variable, or null for the total degree. + /// \return The degree. Negative if the node is non-polynomial. + virtual int DegreeImpl(std::shared_ptr const& v) const = 0; - /** - Compute the multidegree with respect to a variable group. This is for homogenization, and testing for homogeneity. - - \return A vector containing the degrees. Negative entries indicate non-polynomiality. - */ /** Compute the multidegree with respect to a variable group. - DAG-AWARE: this is a non-virtual wrapper that memoizes by node identity FOR THE DURATION - OF ONE TOP-LEVEL CALL, so a shared subtree is visited once however many times it is - reached. The per-type work lives in MultiDegreeImpl. - - Why it exists (b2#417): the node graph is hash-consed, so a small DAG can stand for an - enormous expansion -- measured, a 46-node expression whose printed form is 196 KB. A - plain recursive walk visits the TREE, so it costs O(expansion) where the structure is - O(DAG). `CanonicalizeNaryOperands` calls this for every operand of every n-ary - construction, which is how a 46-node expression came to cost 13 ms to multiply by a - variable, and how a cofactor expansion inside a deflation stage came to run for hours - with a DAG that never exceeded 1102 nodes. - - Caching the RESULT between calls does not fix this and was measured 29% SLOWER -- filling - such a cache costs the same walk it is trying to avoid, so it only pays from a node's - second mention. The memo has to be inside the traversal, which is what this is. + Answered from the node's degree record, like Degree. The per-type rule lives in + MultiDegreeImpl. + + Why it matters (b2#417, #505): the node graph is hash-consed, so a small DAG can stand for + an enormous expansion -- measured, a 46-node expression whose printed form is 196 KB. + `CanonicalizeNaryOperands` asks this of every operand of every sum and product it builds. + A plain recursive walk visits the TREE, so it cost O(expansion), which is how a cofactor + expansion inside a deflation stage came to run for hours with a DAG that never exceeded + 1102 nodes. A walk memoized within one call (#417) costs O(DAG), and asked once per + construction that is still O(DAG^2) for building an expression: the derivative of a 7 x 7 + determinant took seconds. The record makes each construction's question O(operands). + + #417 measured a cache of RESULTS between calls 29% slower than the memoized walk, because + filling it cost the same walk. The record does not walk: it is made from the operands' + records, which already exist for any operand that has been built into a sum or product. */ std::vector MultiDegree(VariableGroup const& vars) const; - /// \brief Per-node-type multidegree. Recurses through MultiDegree(), so children are - /// memoized; never call this directly unless you want the unmemoized walk. + /// \brief Per-node-type multidegree. Asks its operands through MultiDegree(), and its own + /// node only through DegreeImpl, never Degree() (see DegreeImpl). + /// \param vars The variable group. + /// \return One degree per variable of the group. virtual std::vector MultiDegreeImpl(VariableGroup const& vars) const = 0; + /** + \brief The variables in this expression, distinct and sorted by name (GatherVariables). + + Part of the node's degree record: made once per node, from its operands' variables. + + \return The variables. + */ + VariableGroup const& Variables() const; + + /** + \brief This node's content key: the SHA-256, in hex, of its shallow canonical encoding. + + The shallow encoding (ShallowCanonicalEncoding) is the node's own canonical encoding with + each operand written as its content key, so the key is a Merkle hash of the tree's content: + made from the operands' keys, once per node, and equal for equal content. Canonicalization + orders the operands of a sum or product by it when their multidegrees tie (ADR-0073). + Computed on the first request and kept: a node is immutable. + + \return The content key, 64 hex digits. + */ + std::string const& ContentKey() const; + /** Compute the overall degree with respect to a variable group. @@ -376,10 +410,50 @@ class Node : public VisitableBase<>, public std::enable_shared_from_this /// EvalProgram(). Lazily populated by EvalExpression; transient, not serialized. mutable std::shared_ptr eval_program_; + /** + \brief What Variables, Degree and MultiDegree report about this node, made once per node. + + Its variables; for each variable it mentions (its variables and its differentials' + variables), its degree and multidegree entry in that variable; its total degree; and one + degree and one multidegree entry for every variable it does not mention. The last are a + single pair because the degree rules compare a variable only with the variables a subtree + mentions, so every unmentioned variable is answered alike -- not always 0: a negative + constant power reports -1 in every variable. Made from the operands' records, so it costs + O(operands x mentioned variables), never a walk. Defined in node.cpp. + */ + struct DegreeRecord; + + /// The degree record, once made (see Record()). Published once, atomically, and never + /// replaced, so a reference into it lives as long as the node. Transient, not serialized. + mutable std::shared_ptr degree_record_; + + /// The content key, once computed (see ContentKey()). Published like degree_record_. + mutable std::shared_ptr content_key_; + + /// \brief This node's degree record, made on first request from its operands' records. + /// \return The record. + DegreeRecord const& Record() const; + + /// \brief A variable that is in no expression, for the record's answer about unmentioned variables. + /// \return The variable. + static std::shared_ptr const& Unmentioned(); + /// Compute this node's structural hash. Default: identity (the object address), so /// distinct nodes hash distinctly. Value/operator nodes override to be structural. virtual std::size_t HashImpl() const; + /** + \brief This node type's differentiation rule, applied to this node alone. + + Called by Differentiate only when the derivative is not already known. Differentiate an + operand by calling its Differentiate, never its DifferentiateImpl, so that a subexpression + shared by many operands is differentiated once. + + \param v The variable to differentiate with respect to. + \return The derivative of this node. + */ + virtual std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const = 0; + /// Combine an extra value into a running hash (boost::hash_combine recipe). static void HashCombine(std::size_t& seed, std::size_t value) { @@ -402,6 +476,34 @@ class Node : public VisitableBase<>, public std::enable_shared_from_this }; // re: class node + /** + \brief While one of these is alive, every derivative computed on this thread is kept and reused. + + Node::Differentiate opens one for the length of each outermost call, so a subexpression + reached by many paths is differentiated once, not once per path: the sub-minors of a + determinant built by cofactor expansion used to be differentiated k! times at the top (#505). + Open one around a loop that differentiates many expressions, such as a whole Jacobian, so that + subexpressions they share are differentiated once among all of them. Scopes nest: an inner one + uses the outer one's derivatives. It keeps every node it has differentiated, and every + derivative, alive until it closes. + */ + class DerivativeScope + { + public: + /// \brief Start keeping derivatives on this thread, unless an enclosing scope already is. + DerivativeScope(); + /// \brief Stop keeping them and release them, if this scope started it. + ~DerivativeScope(); + DerivativeScope(DerivativeScope const&) = delete; + DerivativeScope& operator=(DerivativeScope const&) = delete; + + /// \cond DERIVATIVE_SCOPE_DETAIL + struct Derivatives; + /// \endcond + private: + std::unique_ptr owned_; ///< the derivatives, when this scope is the outermost + }; + /** a single layer of indirection, though which to call the overridden virtual print() method which must be defined for each non-abstract Node type. */ diff --git a/core/include/bertini2/function_tree/operators/arithmetic.hpp b/core/include/bertini2/function_tree/operators/arithmetic.hpp index d6201705c..4e7415493 100644 --- a/core/include/bertini2/function_tree/operators/arithmetic.hpp +++ b/core/include/bertini2/function_tree/operators/arithmetic.hpp @@ -171,12 +171,13 @@ namespace node{ /** Return SumOperator whose children are derivatives of the children, omitted as possible */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; /** Compute the degree of a node. For sum functions, the degree is the max among summands. */ - int Degree(std::shared_ptr const& v = nullptr) const override; + int DegreeImpl(std::shared_ptr const& v) const override; + using Node::Degree; ///< the memoized Degree(v), which this class's Degree(vars) would otherwise hide int Degree(VariableGroup const& vars) const override; @@ -305,10 +306,12 @@ namespace node{ /** Returns negative of derivative of child. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; + + using Node::Degree; ///< the memoized Degree(v), which this class's Degree(vars) would otherwise hide /// \brief Degree in one variable: negation preserves its operand's degree. - int Degree(std::shared_ptr const& v = nullptr) const override + int DegreeImpl(std::shared_ptr const& v) const override { return operand_->Degree(v); } @@ -465,12 +468,13 @@ namespace node{ /** Differentiates using the product rule. If there is division, consider as ^(-1) and use chain rule. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; /** Compute the degree of a node. For trig functions, the degree is 0 if the argument is constant, otherwise it's undefined, and we return nan. */ - int Degree(std::shared_ptr const& v = nullptr) const override; + int DegreeImpl(std::shared_ptr const& v) const override; + using Node::Degree; ///< the memoized Degree(v), which this class's Degree(vars) would otherwise hide int Degree(VariableGroup const& vars) const override; @@ -621,12 +625,13 @@ namespace node{ /** Differentiates with the power rule. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; /** Compute the degree of a node. For power functions, the degree depends on the degree of the power. If the exponent is constant, then the degree is actually a number. If the exponent is non-constant, then the degree is ill-defined. */ - int Degree(std::shared_ptr const& v = nullptr) const override; + int DegreeImpl(std::shared_ptr const& v) const override; + using Node::Degree; ///< the memoized Degree(v), which this class's Degree(vars) would otherwise hide int Degree(VariableGroup const& vars) const override; @@ -749,12 +754,13 @@ namespace node{ /** \brief Differentiate */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; /** Compute the degree of a node. For integer power functions, the degree is the product of the degree of the argument, and the power. */ - int Degree(std::shared_ptr const& v = nullptr) const override; + int DegreeImpl(std::shared_ptr const& v) const override; + using Node::Degree; ///< the memoized Degree(v), which this class's Degree(vars) would otherwise hide int Degree(VariableGroup const& vars) const override; @@ -872,7 +878,7 @@ namespace node{ /** Differentiates the square root function. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; virtual ~SqrtOperator() = default; @@ -942,7 +948,7 @@ namespace node{ /** Differentiates the exponential function. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; virtual ~ExpOperator() = default; @@ -999,7 +1005,7 @@ namespace node{ /** Differentiates the exponential function. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; virtual ~LogOperator() = default; diff --git a/core/include/bertini2/function_tree/operators/operator.hpp b/core/include/bertini2/function_tree/operators/operator.hpp index cb4cf8c64..a4024a594 100755 --- a/core/include/bertini2/function_tree/operators/operator.hpp +++ b/core/include/bertini2/function_tree/operators/operator.hpp @@ -107,7 +107,8 @@ namespace node{ -1 otherwise (not a polynomial). The degree-preserving unaries, NegateOperator and IntegerPowerOperator, override. */ - int Degree(std::shared_ptr const& v = nullptr) const override; + int DegreeImpl(std::shared_ptr const& v) const override; + using Node::Degree; ///< the memoized Degree(v), which this class's Degree(vars) would otherwise hide /// \brief Degree with respect to a variable group, by the same rule as the one-variable form. int Degree(VariableGroup const& vars) const override; diff --git a/core/include/bertini2/function_tree/operators/trig.hpp b/core/include/bertini2/function_tree/operators/trig.hpp index 7122faec7..ba7f55ab0 100644 --- a/core/include/bertini2/function_tree/operators/trig.hpp +++ b/core/include/bertini2/function_tree/operators/trig.hpp @@ -106,7 +106,7 @@ namespace node{ /** Differentiates the sine function. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; virtual ~SinOperator() = default; @@ -163,7 +163,7 @@ namespace node{ /** Differentiates the sine function. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; virtual ~ArcSinOperator() = default; @@ -230,7 +230,7 @@ namespace node{ /** Differentiates the cosine function. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; virtual ~CosOperator() = default; @@ -290,7 +290,7 @@ namespace node{ /** Differentiates the cosine function. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; virtual ~ArcCosOperator() = default; @@ -357,7 +357,7 @@ namespace node{ /** Differentiates the tangent function. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; virtual ~TanOperator() = default; @@ -416,7 +416,7 @@ namespace node{ /** Differentiates the tangent function. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; virtual ~ArcTanOperator() = default; diff --git a/core/include/bertini2/function_tree/roots/named_expression.hpp b/core/include/bertini2/function_tree/roots/named_expression.hpp index 3b41e5222..308a11cee 100644 --- a/core/include/bertini2/function_tree/roots/named_expression.hpp +++ b/core/include/bertini2/function_tree/roots/named_expression.hpp @@ -88,10 +88,11 @@ namespace node{ const std::shared_ptr& EntryNode() const; /// differentiate the wrapped expression - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; /// the degree is the degree of the wrapped expression - int Degree(std::shared_ptr const& v = nullptr) const override; + int DegreeImpl(std::shared_ptr const& v) const override; + using Node::Degree; ///< the memoized Degree(v), which this class's Degree(vars) would otherwise hide int Degree(VariableGroup const& vars) const override; std::vector MultiDegreeImpl(VariableGroup const& vars) const override; diff --git a/core/include/bertini2/function_tree/symbols/differential.hpp b/core/include/bertini2/function_tree/symbols/differential.hpp index 25d9fb762..dc19ac2a2 100644 --- a/core/include/bertini2/function_tree/symbols/differential.hpp +++ b/core/include/bertini2/function_tree/symbols/differential.hpp @@ -83,7 +83,7 @@ namespace node{ /** Differentiates a number. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; virtual ~Differential() = default; @@ -93,7 +93,8 @@ namespace node{ /** Compute the degree with respect to a single variable. For differentials, the degree is 0. */ - int Degree(std::shared_ptr const& v = nullptr) const override; + int DegreeImpl(std::shared_ptr const& v) const override; + using Node::Degree; ///< the memoized Degree(v), which this class's Degree(vars) would otherwise hide int Degree(VariableGroup const& vars) const override; diff --git a/core/include/bertini2/function_tree/symbols/number.hpp b/core/include/bertini2/function_tree/symbols/number.hpp index 2d65ca021..9be5c09de 100755 --- a/core/include/bertini2/function_tree/symbols/number.hpp +++ b/core/include/bertini2/function_tree/symbols/number.hpp @@ -67,10 +67,11 @@ namespace node{ The degree of a number is always 0. It's a number. */ inline - int Degree(std::shared_ptr const& /*v*/ = nullptr) const override + int DegreeImpl(std::shared_ptr const& /*v*/) const override { return 0; } + using Node::Degree; ///< the memoized Degree(v), which this class's Degree(vars) would otherwise hide /** \brief Get the degree of this node. @@ -118,7 +119,7 @@ namespace node{ /** \brief Differentiate a number. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; protected: diff --git a/core/include/bertini2/function_tree/symbols/variable.hpp b/core/include/bertini2/function_tree/symbols/variable.hpp index 32ace2b8e..9f8a20768 100755 --- a/core/include/bertini2/function_tree/symbols/variable.hpp +++ b/core/include/bertini2/function_tree/symbols/variable.hpp @@ -89,7 +89,7 @@ namespace node{ /** Differentiates a variable. */ - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const override; + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override; /** Substitutes this variable: if its name is a key in `substitutions`, returns the associated @@ -106,7 +106,8 @@ namespace node{ If this is the variable, then the degree is 1. Otherwise, 0. */ - int Degree(std::shared_ptr const& v = nullptr) const override; + int DegreeImpl(std::shared_ptr const& v) const override; + using Node::Degree; ///< the memoized Degree(v), which this class's Degree(vars) would otherwise hide int Degree(VariableGroup const& vars) const override; @@ -127,6 +128,9 @@ namespace node{ private: friend class boost::serialization::access; + // a node's degree record asks about one variable that is in no expression: built with the + // default constructor and never interned, so no name a user can give collides with it + friend class Node; template void serialize(Archive& ar, const unsigned /*version*/) { diff --git a/core/include/bertini2/system/blocks/polynomial_block.hpp b/core/include/bertini2/system/blocks/polynomial_block.hpp index f225103ee..99c56ca99 100644 --- a/core/include/bertini2/system/blocks/polynomial_block.hpp +++ b/core/include/bertini2/system/blocks/polynomial_block.hpp @@ -287,6 +287,9 @@ class PolynomialBlock void DifferentiateUsingDerivatives() const { + // one scope for the whole Jacobian: functions that share subexpressions (the minors a + // deflation appends share their sub-minors) differentiate each of them once among them + node::DerivativeScope shared_derivatives; const size_t n = functions_.size(); space_derivatives_.resize(n * variables_.size()); for (size_t jj = 0; jj < variables_.size(); ++jj) diff --git a/core/include/bertini2/system/system.hpp b/core/include/bertini2/system/system.hpp index 6f342c39f..41badd037 100644 --- a/core/include/bertini2/system/system.hpp +++ b/core/include/bertini2/system/system.hpp @@ -2675,9 +2675,10 @@ namespace bertini { the same commit) whenever the emitted text changes in any way, including the ORDER the session canonicalization gives operands: the encoder writes them in that order. History: `b2sysenc/1` (ADR-0042); `b2sysenc/2` when the canonicalization tie-break moved - from the printed form to the canonical encoding (ADR-0059). + from the printed form to the canonical encoding (ADR-0059); `b2sysenc/3` when it moved to + the content key, a Merkle hash of the canonical encoding (ADR-0073). */ - inline constexpr char SystemEncodingVersion[] = "b2sysenc/2"; + inline constexpr char SystemEncodingVersion[] = "b2sysenc/3"; /// \cond INTERNAL // Explicit instantiation declarations for the two concrete numeric types. diff --git a/core/src/function_tree/canonical.cpp b/core/src/function_tree/canonical.cpp index 0421c0e8d..cb9cfa3b3 100644 --- a/core/src/function_tree/canonical.cpp +++ b/core/src/function_tree/canonical.cpp @@ -95,10 +95,12 @@ namespace { // subtree is a back-reference after its first mention -- linear in the size of the DAG, // not of the expansion. It used to be the PRINTED form, which walks the tree (a 46-node // DAG printed as 196 KB; b2#417) and made every cosmetic printer change a digest event - // (ADR-0059). Printing is for people; identity is the encoding. - std::string ContentKeyOf(std::shared_ptr const& n) + // (ADR-0059). Printing is for people; identity is the encoding. Each node computes its + // own once and keeps it (Node::ContentKey), so an operand compared in many constructions + // is encoded once, not once per construction (#505). + std::string const& ContentKeyOf(std::shared_ptr const& n) { - return CanonicalEncoding(n); + return n->ContentKey(); } // A product operand decomposed into (base, integer exponent): x -> (x,1), x^k -> (x,k). @@ -265,17 +267,11 @@ void CanonicalizeNaryOperands(std::vector>& operands, // The content key is only the TIE-BREAK, consulted when two operands have equal // multidegree. Most comparisons are decided by the multidegree and never ask, so it is - // computed on demand and memoized for the sort -- one encoding per operand at most. - std::vector content_keys(N); - std::vector have_key(N, false); + // computed on demand -- and kept by the operand, so once per node however many + // constructions it is an operand of. auto key_of = [&](std::size_t i) -> std::string const& { - if (!have_key[i]) - { - content_keys[i] = ContentKeyOf(operands[i]); - have_key[i] = true; - } - return content_keys[i]; + return ContentKeyOf(operands[i]); }; const MonomialOrder order = CurrentMonomialOrder(); diff --git a/core/src/function_tree/canonical_encoding.cpp b/core/src/function_tree/canonical_encoding.cpp index 9def4f09d..b7ad8bfda 100644 --- a/core/src/function_tree/canonical_encoding.cpp +++ b/core/src/function_tree/canonical_encoding.cpp @@ -89,12 +89,19 @@ namespace { public Visitor { public: - CanonicalEncoder(std::ostream& out, EncodingContext& ctx) : out_(out), ctx_(ctx) + CanonicalEncoder(std::ostream& out, EncodingContext& ctx, bool shallow = false) + : out_(out), ctx_(ctx), shallow_(shallow) {} - // Encode one node: back-reference if already seen, else index it and dispatch. + // Encode one node: back-reference if already seen, else index it and dispatch. In a + // shallow encoding only the root is expanded; an operand is its content key. void Encode(std::shared_ptr const& n) { + if (shallow_ && !ctx_.seen.empty()) + { + out_ << '@' << n->ContentKey(); + return; + } auto it = ctx_.seen.find(n.get()); if (it != ctx_.seen.end()) { @@ -218,6 +225,7 @@ namespace { std::ostream& out_; EncodingContext& ctx_; + bool shallow_; }; } // unnamed namespace @@ -239,5 +247,16 @@ std::string CanonicalEncoding(std::shared_ptr const& root) return out.str(); } +std::string ShallowCanonicalEncoding(std::shared_ptr const& root) +{ + if (!root) + throw std::invalid_argument("ShallowCanonicalEncoding: null root"); + std::ostringstream out; + EncodingContext ctx; + CanonicalEncoder encoder(out, ctx, true); + encoder.Encode(root); + return out.str(); +} + } // namespace node } // namespace bertini diff --git a/core/src/function_tree/gather.cpp b/core/src/function_tree/gather.cpp index cacb78d86..889350d28 100644 --- a/core/src/function_tree/gather.cpp +++ b/core/src/function_tree/gather.cpp @@ -20,24 +20,36 @@ // additional terms in the b2/licenses/ directory. #include "bertini2/function_tree/gather.hpp" -#include "bertini2/function_tree/find.hpp" #include "bertini2/function_tree.hpp" +#include + namespace bertini { namespace node { - // GatherVariables is the variable-specific case of the general Find (sympy's find): the - // one traversal lives in find.cpp, and these delegate to Find. + // GatherVariables is what Find finds -- distinct, sorted by name -- but read from + // each node's degree record (Node::Variables), which is made once per node from its + // operands' (#505): canonicalization asks it of every operand of every sum and product. VariableGroup GatherVariables(std::shared_ptr const& n) { - return Find(n); + if (!n) + return {}; + return n->Variables(); } VariableGroup GatherVariables(std::vector> const& functions) { - std::vector> roots(functions.begin(), functions.end()); - return Find(roots); + VariableGroup out; + for (auto const& f : functions) + if (f) + out.insert(out.end(), f->Variables().begin(), f->Variables().end()); + std::sort(out.begin(), out.end(), + [](std::shared_ptr const& a, std::shared_ptr const& b) { return a->name() < b->name(); }); + out.erase(std::unique(out.begin(), out.end(), + [](std::shared_ptr const& a, std::shared_ptr const& b) { return a.get() == b.get(); }), + out.end()); + return out; } } // namespace node diff --git a/core/src/function_tree/node.cpp b/core/src/function_tree/node.cpp index 4d385ea13..cf2593591 100644 --- a/core/src/function_tree/node.cpp +++ b/core/src/function_tree/node.cpp @@ -20,7 +20,10 @@ // additional terms in the b2/licenses/ directory. #include "bertini2/function_tree.hpp" +#include "bertini2/function_tree/canonical_encoding.hpp" +#include "bertini2/detail/sha256.hpp" +#include #include #include #include @@ -116,7 +119,73 @@ namespace node{ return Degree(v)>=0; } - // Convenience: repeated single-variable differentiation. Folds the virtual one-variable + // ---- differentiation, each distinct node once (#505) ---- + + // The derivatives kept by the outermost open DerivativeScope on a thread, keyed by + // (node, variable). Each entry holds its node and variable as well as the derivative: a key + // is an address, and a node that died while the scope was open could have its address reused + // by another node, which would then be answered with the dead node's derivative. + struct DerivativeScope::Derivatives + { + struct Key + { + Node const* node; + Variable const* variable; + bool operator==(Key const& other) const { return node == other.node && variable == other.variable; } + }; + struct KeyHash + { + std::size_t operator()(Key const& k) const + { + std::size_t seed = std::hash{}(k.node); + seed ^= std::hash{}(k.variable) + 0x9e3779b97f4a7c15ULL + (seed << 6) + (seed >> 2); + return seed; + } + }; + struct Entry + { + std::shared_ptr node; + std::shared_ptr variable; + std::shared_ptr derivative; + }; + std::unordered_map entries; + }; + + namespace { + thread_local DerivativeScope::Derivatives* open_derivatives = nullptr; + } + + DerivativeScope::DerivativeScope() + { + if (!open_derivatives) + { + owned_ = std::make_unique(); + open_derivatives = owned_.get(); + } + } + + DerivativeScope::~DerivativeScope() + { + if (owned_) + open_derivatives = nullptr; + } + + std::shared_ptr Node::Differentiate(std::shared_ptr const& v) const + { + DerivativeScope scope; + auto& entries = open_derivatives->entries; + DerivativeScope::Derivatives::Key const key{this, v.get()}; + auto found = entries.find(key); + if (found != entries.end()) + return found->second.derivative; + auto derivative = DifferentiateImpl(v); + // the rule differentiated other nodes meanwhile and may have rehashed the table, so + // insert afresh rather than through anything found before it ran + entries.emplace(key, DerivativeScope::Derivatives::Entry{shared_from_this(), v, derivative}); + return derivative; + } + + // Convenience: repeated single-variable differentiation. Folds the one-variable // Differentiate `count` times; count==0 is the identity (the node itself). std::shared_ptr Node::Differentiate(std::shared_ptr const& v, unsigned count) const { @@ -129,7 +198,7 @@ namespace node{ } // Convenience: sequential differentiation wrt each variable in the group (mixed partials). - // Folds the virtual one-variable Differentiate over `vars` in order; empty is the identity. + // Folds the one-variable Differentiate over `vars` in order; empty is the identity. std::shared_ptr Node::Differentiate(VariableGroup const& vars) const { if (vars.empty()) @@ -183,39 +252,154 @@ namespace node{ return candidate; } + // ---- the degree record: variables, degrees and multidegrees, once per node (#505) ---- + + struct Node::DegreeRecord + { + VariableGroup variables; ///< distinct, sorted by name (GatherVariables) + VariableGroup mentioned; ///< variables and differentials' variables, sorted by address + std::vector degree; ///< Degree(v) for each mentioned v + std::vector multidegree; ///< the MultiDegree entry for each mentioned v + int total_degree = 0; ///< Degree(nullptr) + int unmentioned_degree = 0; ///< Degree(v) for any v not mentioned + int unmentioned_multidegree = 0; ///< the MultiDegree entry for any v not mentioned + + // the index of v among the mentioned variables, or -1 + long IndexOf(Variable const* v) const + { + auto it = std::lower_bound(mentioned.begin(), mentioned.end(), v, + [](std::shared_ptr const& a, Variable const* b) { return a.get() < b; }); + return (it != mentioned.end() && it->get() == v) ? it - mentioned.begin() : -1; + } + }; + + // A variable no expression contains: asking a node about it asks about every variable the + // node does not mention. Made with the default constructor and never interned, so no + // variable a user makes, whatever its name, is this one. + std::shared_ptr const& Node::Unmentioned() + { + static std::shared_ptr const unmentioned(new Variable()); + return unmentioned; + } + namespace { - /// Memo for one top-level MultiDegree call: node identity -> its degrees. Thread-local - /// because node graphs are shared across threads but a traversal is not. Null when no - /// traversal is in flight, which is how the outermost call knows to own the memo. - thread_local std::unordered_map>* multidegree_memo = nullptr; - /// The variable group the in-flight memo was built for. A multidegree is only - /// meaningful relative to a variable group, so a nested call asking about a DIFFERENT - /// group must not read this memo -- it computes unmemoized instead. Within one - /// traversal the same `vars` reference is handed down, so identity is the right test. - thread_local VariableGroup const* multidegree_memo_vars = nullptr; + + // Each operand of n, by the public child accessors (the same descent as Find). + template + void ForEachOperand(Node const& n, F&& f) + { + if (auto h = dynamic_cast(&n)) + f(h->EntryNode()); + else if (auto nary = dynamic_cast(&n)) + for (auto const& child : nary->Operands()) + f(child); + else if (auto u = dynamic_cast(&n)) + f(u->Operand()); + else if (auto p = dynamic_cast(&n)) + { + f(p->GetBase()); + f(p->GetExponent()); + } + } + + void SortAndDedupe(VariableGroup& vars, bool by_name) + { + if (by_name) + std::sort(vars.begin(), vars.end(), + [](std::shared_ptr const& a, std::shared_ptr const& b) { return a->name() < b->name(); }); + else + std::sort(vars.begin(), vars.end(), + [](std::shared_ptr const& a, std::shared_ptr const& b) { return a.get() < b.get(); }); + vars.erase(std::unique(vars.begin(), vars.end(), + [](std::shared_ptr const& a, std::shared_ptr const& b) { return a.get() == b.get(); }), + vars.end()); + } } - std::vector Node::MultiDegree(VariableGroup const& vars) const + Node::DegreeRecord const& Node::Record() const { - if (multidegree_memo && multidegree_memo_vars != &vars) - return this->MultiDegreeImpl(vars); // different variable group: do not memoize + if (auto made = std::atomic_load(°ree_record_)) + return *made; - if (multidegree_memo) // inside a traversal: consult the memo + auto record = std::make_shared(); + ForEachOperand(*this, [&](auto const& operand) { - auto found = multidegree_memo->find(this); - if (found != multidegree_memo->end()) - return found->second; - auto computed = this->MultiDegreeImpl(vars); - (*multidegree_memo)[this] = computed; - return computed; + if (!operand) + return; + auto const& r = operand->Record(); + record->variables.insert(record->variables.end(), r.variables.begin(), r.variables.end()); + record->mentioned.insert(record->mentioned.end(), r.mentioned.begin(), r.mentioned.end()); + }); + if (dynamic_cast(this)) + { + auto as_variable = std::const_pointer_cast(std::static_pointer_cast(shared_from_this())); + record->variables.push_back(as_variable); + record->mentioned.push_back(as_variable); } + if (auto d = dynamic_cast(this)) + if (d->GetVariable()) + record->mentioned.push_back(std::const_pointer_cast(d->GetVariable())); + SortAndDedupe(record->variables, true); + SortAndDedupe(record->mentioned, false); + + // The rules ask the operands' Degree and MultiDegree, which their records answer, and + // never this node's own (DegreeImpl's contract), so nothing here recurses into this. + record->degree.reserve(record->mentioned.size()); + for (auto const& v : record->mentioned) + record->degree.push_back(this->DegreeImpl(v)); + record->total_degree = this->DegreeImpl(nullptr); + record->unmentioned_degree = this->DegreeImpl(Unmentioned()); + record->multidegree = this->MultiDegreeImpl(record->mentioned); + record->unmentioned_multidegree = this->MultiDegreeImpl(VariableGroup{Unmentioned()}).front(); + + // publish once: a thread that lost the race uses the winner's, so a reference into the + // record stays valid for the node's life + std::shared_ptr expected; + std::shared_ptr mine = record; + if (!std::atomic_compare_exchange_strong(°ree_record_, &expected, mine)) + return *expected; + return *mine; + } - // outermost call: own the memo, and clear it however we leave - std::unordered_map> memo; - multidegree_memo = &memo; - multidegree_memo_vars = &vars; - struct Guard { ~Guard() { multidegree_memo = nullptr; multidegree_memo_vars = nullptr; } } guard; - return this->MultiDegreeImpl(vars); + VariableGroup const& Node::Variables() const + { + return Record().variables; + } + + int Node::Degree(std::shared_ptr const& v) const + { + auto const& record = Record(); + if (!v) + return record.total_degree; + auto index = record.IndexOf(v.get()); + return index < 0 ? record.unmentioned_degree : record.degree[index]; + } + + std::vector Node::MultiDegree(VariableGroup const& vars) const + { + auto const& record = Record(); + std::vector degrees; + degrees.reserve(vars.size()); + for (auto const& v : vars) + { + auto index = record.IndexOf(v.get()); + degrees.push_back(index < 0 ? record.unmentioned_multidegree : record.multidegree[index]); + } + return degrees; + } + + std::string const& Node::ContentKey() const + { + if (auto made = std::atomic_load(&content_key_)) + return *made; + // the operands' keys are made first, by the shallow encoding asking for them, so a key + // costs O(operands): a Merkle hash of the canonical encoding + std::shared_ptr mine = std::make_shared( + detail::Sha256(ShallowCanonicalEncoding(std::const_pointer_cast(shared_from_this()))).Hex()); + std::shared_ptr expected; + if (!std::atomic_compare_exchange_strong(&content_key_, &expected, mine)) + return *expected; + return *mine; } } // namespace node diff --git a/core/src/function_tree/operators/arithmetic.cpp b/core/src/function_tree/operators/arithmetic.cpp index 5cbfdd266..1759de607 100644 --- a/core/src/function_tree/operators/arithmetic.cpp +++ b/core/src/function_tree/operators/arithmetic.cpp @@ -540,7 +540,7 @@ void SumOperator::print(std::ostream & target) const } -std::shared_ptr SumOperator::Differentiate(std::shared_ptr const& v) const +std::shared_ptr SumOperator::DifferentiateImpl(std::shared_ptr const& v) const { std::vector, bool>> terms; terms.reserve(operands_.size()); @@ -554,7 +554,7 @@ std::shared_ptr SumOperator::Differentiate(std::shared_ptr const return SimplifiedSum(terms); } -int SumOperator::Degree(std::shared_ptr const& v) const +int SumOperator::DegreeImpl(std::shared_ptr const& v) const { int deg = 0; @@ -736,7 +736,7 @@ void NegateOperator::print(std::ostream & target) const PrintOperand(target, operand_, operand_->Precedence() <= PrecNegate); } -std::shared_ptr NegateOperator::Differentiate(std::shared_ptr const& v) const +std::shared_ptr NegateOperator::DifferentiateImpl(std::shared_ptr const& v) const { return SimplifiedNegate(operand_->Differentiate(v)); } @@ -806,7 +806,7 @@ void MultOperator::print(std::ostream & target) const -std::shared_ptr MultOperator::Differentiate(std::shared_ptr const& v) const +std::shared_ptr MultOperator::DifferentiateImpl(std::shared_ptr const& v) const { std::vector, bool>> sum_terms; // this loop implements the generic product rule, perhaps inefficiently. @@ -835,7 +835,7 @@ std::shared_ptr MultOperator::Differentiate(std::shared_ptr cons return SimplifiedSum(sum_terms); } -int MultOperator::Degree(std::shared_ptr const& v) const +int MultOperator::DegreeImpl(std::shared_ptr const& v) const { int deg = 0; for (auto iter = operands_.begin(); iter!= operands_.end(); iter++) @@ -974,7 +974,7 @@ void PowerOperator::print(std::ostream & target) const } -std::shared_ptr PowerOperator::Differentiate(std::shared_ptr const& v) const +std::shared_ptr PowerOperator::DifferentiateImpl(std::shared_ptr const& v) const { // d/dv (base^exp) = exp * base^(exp-1) * base'. When the exponent is an integer literal, fold // exp-1 to a literal Integer (e.g. 4 -> 3) instead of leaving a computed `exponent - 1` node -- @@ -1008,7 +1008,7 @@ namespace { } } -int PowerOperator::Degree(std::shared_ptr const& v) const +int PowerOperator::DegreeImpl(std::shared_ptr const& v) const { auto base_deg = base_->Degree(v); @@ -1101,7 +1101,7 @@ std::vector PowerOperator::MultiDegreeImpl(VariableGroup const& vars) const std::vector deg(vars.size(),0); for (auto iter = vars.begin(); iter!= vars.end(); ++iter) { - *(deg.begin()+(iter-vars.begin())) = this->Degree(*iter); + *(deg.begin()+(iter-vars.begin())) = this->DegreeImpl(*iter); // never Degree(): this node's record is being made from this } return deg; } @@ -1289,7 +1289,7 @@ void IntegerPowerOperator::print(std::ostream & target) const -std::shared_ptr IntegerPowerOperator::Differentiate(std::shared_ptr const& v) const +std::shared_ptr IntegerPowerOperator::DifferentiateImpl(std::shared_ptr const& v) const { if (exponent_==0) return Integer::Make(0); @@ -1306,7 +1306,7 @@ std::shared_ptr IntegerPowerOperator::Differentiate(std::shared_ptr const& v) const +int IntegerPowerOperator::DegreeImpl(std::shared_ptr const& v) const { auto base_deg = operand_->Degree(v); if (base_deg<0) @@ -1357,7 +1357,7 @@ void SqrtOperator::print(std::ostream & target) const -std::shared_ptr SqrtOperator::Differentiate(std::shared_ptr const& v) const +std::shared_ptr SqrtOperator::DifferentiateImpl(std::shared_ptr const& v) const { return SimplifiedMult({ {Rational::Make(mpq_rational(1,2),0), true}, @@ -1403,7 +1403,7 @@ void ExpOperator::print(std::ostream & target) const -std::shared_ptr ExpOperator::Differentiate(std::shared_ptr const& v) const +std::shared_ptr ExpOperator::DifferentiateImpl(std::shared_ptr const& v) const { return SimplifiedMult({ {exp(operand_), true}, @@ -1443,7 +1443,7 @@ void LogOperator::print(std::ostream & target) const -std::shared_ptr LogOperator::Differentiate(std::shared_ptr const& v) const +std::shared_ptr LogOperator::DifferentiateImpl(std::shared_ptr const& v) const { return SimplifiedMult({ {operand_->Differentiate(v), true}, diff --git a/core/src/function_tree/operators/operator.cpp b/core/src/function_tree/operators/operator.cpp index 5d28187a8..c03418fc7 100644 --- a/core/src/function_tree/operators/operator.cpp +++ b/core/src/function_tree/operators/operator.cpp @@ -48,7 +48,7 @@ std::shared_ptr UnaryOperator::Operand() const // degrees, which equals the total degree only for a single monomial: NegateOperator inherited // it and reported -(x^2+y^2) as degree 4, inflating System::DegreeBound() -- the same defect // #397 found in PowerOperator.) -int UnaryOperator::Degree(std::shared_ptr const& v) const +int UnaryOperator::DegreeImpl(std::shared_ptr const& v) const { return operand_->Degree(v) == 0 ? 0 : -1; } @@ -64,7 +64,7 @@ std::vector UnaryOperator::MultiDegreeImpl(VariableGroup const& vars) const std::vector deg(vars.size()); for (auto iter = vars.begin(); iter!= vars.end(); ++iter) { - *(deg.begin()+(iter-vars.begin())) = this->Degree(*iter); + *(deg.begin()+(iter-vars.begin())) = this->DegreeImpl(*iter); // never Degree(): this node's record is being made from this } return deg; } diff --git a/core/src/function_tree/operators/trig.cpp b/core/src/function_tree/operators/trig.cpp index 23e0f91ae..5ede76876 100644 --- a/core/src/function_tree/operators/trig.cpp +++ b/core/src/function_tree/operators/trig.cpp @@ -33,7 +33,7 @@ namespace node{ } - std::shared_ptr SinOperator::Differentiate(std::shared_ptr const& v) const + std::shared_ptr SinOperator::DifferentiateImpl(std::shared_ptr const& v) const { return SimplifiedMult({ {cos(operand_), true}, @@ -53,7 +53,7 @@ namespace node{ target << ")"; } - std::shared_ptr ArcSinOperator::Differentiate(std::shared_ptr const& v) const + std::shared_ptr ArcSinOperator::DifferentiateImpl(std::shared_ptr const& v) const { return SimplifiedMult({ {operand_->Differentiate(v), true}, @@ -74,7 +74,7 @@ namespace node{ target << ")"; } - std::shared_ptr CosOperator::Differentiate(std::shared_ptr const& v) const + std::shared_ptr CosOperator::DifferentiateImpl(std::shared_ptr const& v) const { return SimplifiedNegate(SimplifiedMult({ {sin(operand_), true}, @@ -96,7 +96,7 @@ namespace node{ } - std::shared_ptr ArcCosOperator::Differentiate(std::shared_ptr const& v) const + std::shared_ptr ArcCosOperator::DifferentiateImpl(std::shared_ptr const& v) const { return SimplifiedNegate(SimplifiedMult({ {operand_->Differentiate(v), true}, @@ -117,7 +117,7 @@ namespace node{ target << ")"; } - std::shared_ptr TanOperator::Differentiate(std::shared_ptr const& v) const + std::shared_ptr TanOperator::DifferentiateImpl(std::shared_ptr const& v) const { return SimplifiedMult({ {operand_->Differentiate(v), true}, @@ -139,7 +139,7 @@ namespace node{ } - std::shared_ptr ArcTanOperator::Differentiate(std::shared_ptr const& v) const + std::shared_ptr ArcTanOperator::DifferentiateImpl(std::shared_ptr const& v) const { return SimplifiedMult({ {operand_->Differentiate(v), true}, diff --git a/core/src/function_tree/roots/named_expression.cpp b/core/src/function_tree/roots/named_expression.cpp index bcc3a55d2..b37ca7182 100644 --- a/core/src/function_tree/roots/named_expression.cpp +++ b/core/src/function_tree/roots/named_expression.cpp @@ -47,12 +47,12 @@ void NamedExpression::EnsureNotEmpty() const } } -std::shared_ptr NamedExpression::Differentiate(std::shared_ptr const& v) const +std::shared_ptr NamedExpression::DifferentiateImpl(std::shared_ptr const& v) const { return entry_node_->Differentiate(v); } -int NamedExpression::Degree(std::shared_ptr const& v) const +int NamedExpression::DegreeImpl(std::shared_ptr const& v) const { return entry_node_->Degree(v); } @@ -69,7 +69,7 @@ std::vector NamedExpression::MultiDegreeImpl(VariableGroup const& vars) con std::vector deg(vars.size()); for (auto iter = vars.begin(); iter!= vars.end(); ++iter) { - *(deg.begin()+(iter-vars.begin())) = this->Degree(*iter); + *(deg.begin()+(iter-vars.begin())) = this->DegreeImpl(*iter); // never Degree(): this node's record is being made from this } return deg; } diff --git a/core/src/function_tree/symbols/differential.cpp b/core/src/function_tree/symbols/differential.cpp index 06d9b59c7..a51b73f31 100644 --- a/core/src/function_tree/symbols/differential.cpp +++ b/core/src/function_tree/symbols/differential.cpp @@ -43,7 +43,7 @@ void Differential::print(std::ostream & target) const } -std::shared_ptr Differential::Differentiate(std::shared_ptr const& /*v*/) const +std::shared_ptr Differential::DifferentiateImpl(std::shared_ptr const& /*v*/) const { throw std::runtime_error("differentiating a differential is not correctly implemented yet"); return Integer::Make(0); @@ -52,7 +52,7 @@ std::shared_ptr Differential::Differentiate(std::shared_ptr cons /** Compute the degree with respect to a single variable. For differentials, the degree is 0. */ -int Differential::Degree(std::shared_ptr const& /*v*/) const +int Differential::DegreeImpl(std::shared_ptr const& /*v*/) const { return 0; } diff --git a/core/src/function_tree/symbols/number.cpp b/core/src/function_tree/symbols/number.cpp index 2b421733c..98bd1e04e 100644 --- a/core/src/function_tree/symbols/number.cpp +++ b/core/src/function_tree/symbols/number.cpp @@ -31,7 +31,7 @@ namespace bertini{ namespace node{ using ::pow; -std::shared_ptr Number::Differentiate(std::shared_ptr const& /*v*/) const +std::shared_ptr Number::DifferentiateImpl(std::shared_ptr const& /*v*/) const { return Integer::Make(0); } diff --git a/core/src/function_tree/symbols/variable.cpp b/core/src/function_tree/symbols/variable.cpp index 61080258a..64e64b6e5 100644 --- a/core/src/function_tree/symbols/variable.cpp +++ b/core/src/function_tree/symbols/variable.cpp @@ -42,7 +42,7 @@ Variable::Variable(std::string new_name) : NamedSymbol(new_name) Variable::Variable() : NamedSymbol("unnamed_variable_be_scared") { } -std::shared_ptr Variable::Differentiate(std::shared_ptr const& v) const +std::shared_ptr Variable::DifferentiateImpl(std::shared_ptr const& v) const { if (v==nullptr) return Differential::Make(std::static_pointer_cast(shared_from_this()), name()); @@ -58,7 +58,7 @@ std::shared_ptr Variable::Subs(SubstitutionMap const& substitutions) const return std::const_pointer_cast(shared_from_this()); } -int Variable::Degree(std::shared_ptr const& v) const +int Variable::DegreeImpl(std::shared_ptr const& v) const { if (v) { diff --git a/core/src/system/system.cpp b/core/src/system/system.cpp index 0852cc9b5..407ace025 100644 --- a/core/src/system/system.cpp +++ b/core/src/system/system.cpp @@ -359,6 +359,7 @@ namespace bertini J.rows = functions.size(); J.cols = variables.size(); J.entries.reserve(J.rows * J.cols); + node::DerivativeScope shared_derivatives; // subexpressions the functions share, once for (auto const& f : functions) for (auto const& v : variables) J.entries.push_back(f->Differentiate(v)); // J[i,j] = d f_i / d v_j, row-major diff --git a/core/test/classes/data/system_digest_fixture.txt b/core/test/classes/data/system_digest_fixture.txt index 680cfdc98..b53e34415 100644 --- a/core/test/classes/data/system_digest_fixture.txt +++ b/core/test/classes/data/system_digest_fixture.txt @@ -1,5 +1,6 @@ -circle_line 68a4e3459c90d5be42b61ca8f324dd01a321ffffefc7b5f139df8655db25ec12 -circle_line_homogenized 416f3f50ee4566e4764e25b148ad41d21a6d642d1f4a59fb268d79ac250a2936 -rational_coeffs 335c392550a5b4ba2067a02b5a31ae9a719e4c456e28a619938fb563d422b276 -single_var 36cdb679d8525079ebf47d8c6b574ed37df9f4e6a8d46e622b1cf8a88523a5b4 -transcendental 3fd785b806405fc3e1b545bb13c9bc5078616bcc64a68adcb54d41c77329c9fa +circle_line 3c549e567ee345313dcb52fbff04731d45b01d6bd98b3e231bf84b679fc9f948 +circle_line_homogenized 0cb02d9d785c0b99fb24596937403c728210f3cdc21655825a0b7b0546d702f3 +rational_coeffs 507d6dac49c514cac1bf05460f6850cf7e36399a829c811694fb8f98b8b8f54f +single_var 66042474b8755a4140e6334c76a65024fe97ac90b7721920f3c981fb14f8ecb0 +tied_terms fec32472868b9e5cd9b9458d8b3011b5ea4ec43b52efc83dae8861f270c94905 +transcendental 33c74d05d923c4469e2f628018734c1eb055b005d005d04245e410df51f181fb diff --git a/core/test/classes/data/system_encoding_versions.txt b/core/test/classes/data/system_encoding_versions.txt index 51d523c9b..778ee50c2 100644 --- a/core/test/classes/data/system_encoding_versions.txt +++ b/core/test/classes/data/system_encoding_versions.txt @@ -10,3 +10,4 @@ # never edited again. See core/test/utility/encoding_registry.hpp. b2sysenc/1 50e21eb3ad705b6ae31b99a12c8ed6b8cb993f76a947791da05127d5bb8b805f 3.0.0 b2sysenc/2 038d9088d5d2ddbb854a2ef7828ceddb5a3f8c6d0f34cd2ad06a87e2f62a6d24 4.0.0 +b2sysenc/3 69b3a623023d071853e3b339d87320ec873c92e4a39475af37d9375a3b78aa36 4.0.1 diff --git a/core/test/classes/degrees_test.cpp b/core/test/classes/degrees_test.cpp index 1121a48a1..164ba5fb5 100644 --- a/core/test/classes/degrees_test.cpp +++ b/core/test/classes/degrees_test.cpp @@ -34,6 +34,8 @@ Block degree contract under test: BlendBlock -- the elementwise max of its operands' degrees */ +#include + #include #include "bertini2/system/system.hpp" @@ -418,4 +420,64 @@ BOOST_AUTO_TEST_CASE(non_polynomial_unary_operators_share_one_degree_rule) } } +// Degree(v) visits each distinct node once per call, as MultiDegree does (#505). A negation's +// multidegree is computed from Degree(v) of its operand, once per variable, so before Degree(v) +// was memoized every negation reopened a walk of the whole expansion beneath it -- 2^26 copies +// of x here, in an expression of 26 levels. Canonicalizing every sum and product asks for the +// multidegree of each operand, so this cost was paid on every construction above a negation. +BOOST_AUTO_TEST_CASE(degree_visits_each_shared_subexpression_once) +{ + Var x = Variable::Make("x"), y = Variable::Make("y"), z = Variable::Make("z"); + const int levels = 26; + std::shared_ptr e = x; + for (int i = 0; i < levels; ++i) + e = e*y + e*z; // x (y + z)^levels, each level using e twice + auto negated = -e; + + auto start = std::chrono::steady_clock::now(); + auto multidegree = negated->MultiDegree(VariableGroup{x, y, z}); + int degree_in_y = negated->Degree(y); + int total_degree = negated->Degree(); + double seconds = std::chrono::duration(std::chrono::steady_clock::now() - start).count(); + BOOST_CHECK_MESSAGE(seconds < 2.0, "the degrees took " << seconds << " s: a shared " + "subexpression is being visited once per path to it"); + + BOOST_CHECK_EQUAL(multidegree.size(), 3); + BOOST_CHECK_EQUAL(multidegree[0], 1); + BOOST_CHECK_EQUAL(multidegree[1], levels); + BOOST_CHECK_EQUAL(multidegree[2], levels); + BOOST_CHECK_EQUAL(degree_in_y, levels); + BOOST_CHECK_EQUAL(total_degree, levels + 1); +} + +// A node's degree record answers every variable it does not mention with one value, and that +// value is not always 0: a power with a negative constant exponent is not a polynomial, so it +// reports -1 in every variable, including ones it does not contain -- and so does a sum or +// product over it. The record must keep that answer, not assume 0. +BOOST_AUTO_TEST_CASE(an_unmentioned_variable_gets_the_degree_rules_answer) +{ + Var x = Variable::Make("x"), y = Variable::Make("y"), z = Variable::Make("z"); + auto inverse_square = pow(x, Integer::Make(-2)); // a generic power, not an integer power + BOOST_CHECK_EQUAL(inverse_square->Degree(x), -1); + BOOST_CHECK_EQUAL(inverse_square->Degree(z), -1); // z is not in it + auto md = inverse_square->MultiDegree(VariableGroup{x, y, z}); + BOOST_CHECK_EQUAL(md[0], -1); + BOOST_CHECK_EQUAL(md[1], -1); + BOOST_CHECK_EQUAL(md[2], -1); + + auto sum = inverse_square + y; + BOOST_CHECK_EQUAL(sum->Degree(y), -1); + BOOST_CHECK_EQUAL(sum->Degree(z), -1); + + // and a polynomial answers 0 for a variable it does not contain + auto poly = x*x*y + y; + BOOST_CHECK_EQUAL(poly->Degree(z), 0); + BOOST_CHECK_EQUAL(poly->Degree(x), 2); + BOOST_CHECK_EQUAL(poly->Degree(), 3); + auto pmd = poly->MultiDegree(VariableGroup{z, y, x}); + BOOST_CHECK_EQUAL(pmd[0], 0); + BOOST_CHECK_EQUAL(pmd[1], 1); + BOOST_CHECK_EQUAL(pmd[2], 2); +} + BOOST_AUTO_TEST_SUITE_END() diff --git a/core/test/classes/function_tree_transform.cpp b/core/test/classes/function_tree_transform.cpp index 665570d7e..3c47dd1e7 100644 --- a/core/test/classes/function_tree_transform.cpp +++ b/core/test/classes/function_tree_transform.cpp @@ -26,6 +26,7 @@ #include #include +#include #include #include @@ -403,7 +404,7 @@ BOOST_AUTO_TEST_SUITE_END() // transform // The convenience Differentiate overloads: repeated single-variable, and sequential over a -// group. Both fold the virtual one-variable Differentiate, so they are checked by value at a +// group. Both fold the one-variable Differentiate, so they are checked by value at a // point (distinct non-identity coordinates x=3, y=5 so each term is separable). BOOST_AUTO_TEST_SUITE(differentiate_overloads) @@ -464,6 +465,66 @@ BOOST_AUTO_TEST_CASE(empty_group_is_identity) BOOST_AUTO_TEST_SUITE_END() // differentiate_overloads +// Differentiation visits each distinct node once per variable, however many paths reach it +// (#505). Each level of e_{i+1} = e_i y + e_i z uses e_i twice, so 26 levels make an expression +// whose DAG has 26 levels and whose tree has 2^26 copies of x. Differentiating once per path +// took minutes; once per node, milliseconds. +BOOST_AUTO_TEST_SUITE(differentiate_shared_subexpressions) + +BOOST_AUTO_TEST_CASE(each_shared_subexpression_is_differentiated_once) +{ + auto x = Variable::Make("x"); + auto y = Variable::Make("y"); + auto z = Variable::Make("z"); + const int levels = 26; + Nd e = x; + for (int i = 0; i < levels; ++i) + e = e*y + e*z; // e_levels = x (y + z)^levels + + auto start = std::chrono::steady_clock::now(); + auto ddx = e->Differentiate(x); // (y + z)^levels + auto ddy = e->Differentiate(y); // levels x (y + z)^(levels - 1) + double seconds = std::chrono::duration(std::chrono::steady_clock::now() - start).count(); + BOOST_CHECK_MESSAGE(seconds < 2.0, "differentiating took " << seconds << " s: a shared " + "subexpression is being differentiated once per path to it"); + + std::map pt{ {"x", complex_dbl(3,0)}, {"y", complex_dbl(0.6,0)}, {"z", complex_dbl(0.5,0)} }; + const double s = 1.1; // y + z at pt + BOOST_CHECK_CLOSE(std::real(EvalAt(ddx, pt)), std::pow(s, levels), 1e-10); + BOOST_CHECK_CLOSE(std::real(EvalAt(ddy, pt)), levels * 3 * std::pow(s, levels - 1), 1e-10); +} + +// A scope opened around several differentiations keeps one memo for all of them, and nested +// scopes and calls use the outermost one: the derivatives are those of separate calls. +BOOST_AUTO_TEST_CASE(a_derivative_scope_gives_the_derivatives_of_separate_calls) +{ + auto x = Variable::Make("x"); + auto y = Variable::Make("y"); + Nd shared = x*x*y + y; + Nd f = shared*shared; + Nd g = shared + x; + + Nd df_alone = f->Differentiate(x); + Nd dg_alone = g->Differentiate(y); + Nd df_scoped, dg_scoped; + { + bertini::node::DerivativeScope outer; + bertini::node::DerivativeScope inner; + df_scoped = f->Differentiate(x); + dg_scoped = g->Differentiate(y); + } + std::map pt{ {"x", complex_dbl(3,0)}, {"y", complex_dbl(5,0)} }; + BOOST_CHECK_SMALL(std::abs(EvalAt(df_scoped, pt) - EvalAt(df_alone, pt)), 1e-9); + BOOST_CHECK_SMALL(std::abs(EvalAt(dg_scoped, pt) - EvalAt(dg_alone, pt)), 1e-12); + // d/dx (x^2 y + y)^2 = 2 (x^2 y + y)(2 x y) at (3, 5): 2 * 50 * 30 + BOOST_CHECK_SMALL(std::abs(EvalAt(df_scoped, pt) - complex_dbl(3000,0)), 1e-9); + // d/dy (x^2 y + y + x) = x^2 + 1 + BOOST_CHECK_SMALL(std::abs(EvalAt(dg_scoped, pt) - complex_dbl(10,0)), 1e-12); +} + +BOOST_AUTO_TEST_SUITE_END() // differentiate_shared_subexpressions + + // Symbolic substitution Node::Subs (variable -> node). Simultaneous, single-pass; results are // simplified (constants fold). Checked by value at a point (distinct non-identity coordinates). BOOST_AUTO_TEST_SUITE(substitution) diff --git a/core/test/classes/seeded_randomness_test.cpp b/core/test/classes/seeded_randomness_test.cpp index ab0c8a84c..ff73f4ded 100644 --- a/core/test/classes/seeded_randomness_test.cpp +++ b/core/test/classes/seeded_randomness_test.cpp @@ -260,10 +260,9 @@ BOOST_AUTO_TEST_CASE(seed_42_homotopy_digest_is_pinned_cross_platform) auto const hex = homotopy.ContentDigest().Hex(); std::cout << "[stage] homotopy: " << hex << "\n"; - // re-pinned for b2sysenc/2, which is edited in place while 3.5 is unreleased (ADR-0061), so - // this moves whenever the encoding does until release -- most recently for the auxiliary - // coordinate marking joining the encoding (b2#403) - char const* const pinned = "ae1c4276324dc93676766a583d97cba494ae4c3b1bb8f51a9fa8a8d530eeed02"; + // re-pinned for b2sysenc/3 (the content-key tie-break, ADR-0073), which is edited in place + // while 4.0.1 is unreleased (ADR-0061), so this moves whenever the encoding does until release + char const* const pinned = "2508128366cab73dd135a084eeba2a28dec4866bf11205874d32c6fc6e5a1a0a"; if (hex != pinned) { std::cout << "[diagnostic] canonical encoding of the mismatching homotopy follows\n" diff --git a/core/test/classes/system_identity_test.cpp b/core/test/classes/system_identity_test.cpp index 3b10c4684..48b5a7627 100644 --- a/core/test/classes/system_identity_test.cpp +++ b/core/test/classes/system_identity_test.cpp @@ -60,6 +60,15 @@ System Parse(std::string const& str) std::string const kCircleLine = "function f,g; variable_group x,y; f = x^2+y^2-1; g = x-y;"; +// Operands that tie on multidegree -- f's four products are all (1,1), g's four terms all +// non-polynomial -- so their order in the encoding is the tie-break's (the content key, +// ADR-0073). Without a recipe like this the fixture and the registry could not see a change to +// the tie-break; with four tied operands, two tie-breaks agree on their order by chance only +// once in 24. +std::string const kTiedTerms = "function f,g; variable_group x,y; " + "f = (x+1)*(y+2) + (x+2)*(y+1) + (x+3)*(y+4) + (x+4)*(y+3); " + "g = sin(x) + cos(x) + sin(y) + cos(y);"; + // Pin the session canonicalization settings for the golden fixture (they are part of the // encoding header, so unpinned settings would honestly change the digest -- the fixture // pins one configuration). RAII-restores the previous settings. @@ -724,6 +733,7 @@ BOOST_AUTO_TEST_CASE(golden_digests_match_committed_fixture) recipes.emplace("circle_line", Parse(kCircleLine)); recipes.emplace("rational_coeffs", Parse("function f; variable_group x,y; f = (1/3)*x^3 - y + 2;")); recipes.emplace("transcendental", Parse("function f; variable_group x; f = sin(x) + 3*exp(x);")); + recipes.emplace("tied_terms", Parse(kTiedTerms)); { auto homogenized = Parse(kCircleLine); homogenized.Homogenize(); // deterministic: no random coefficients involved @@ -778,6 +788,7 @@ BOOST_AUTO_TEST_CASE(encoding_version_is_bumped_when_the_encoding_changes) recipes.emplace("circle_line", Parse(kCircleLine)); recipes.emplace("rational_coeffs", Parse("function f; variable_group x,y; f = (1/3)*x^3 - y + 2;")); recipes.emplace("transcendental", Parse("function f; variable_group x; f = sin(x) + 3*exp(x);")); + recipes.emplace("tied_terms", Parse(kTiedTerms)); { auto homogenized = Parse(kCircleLine); homogenized.Homogenize(); diff --git a/docs/adr/0059-identity-is-the-encoding-printing-is-a-dialect-family.md b/docs/adr/0059-identity-is-the-encoding-printing-is-a-dialect-family.md index 21272b18d..4fa6c1837 100644 --- a/docs/adr/0059-identity-is-the-encoding-printing-is-a-dialect-family.md +++ b/docs/adr/0059-identity-is-the-encoding-printing-is-a-dialect-family.md @@ -1,6 +1,6 @@ # 0059 — Identity is the canonical encoding; printing is a family of dialects, one per audience -**Status:** Accepted +**Status:** Accepted; decision 1's tie-break key is replaced by ADR-0073 (a content key derived from the encoding, `b2sysenc/3`) **Date:** 2026-09-17 ## Context diff --git a/docs/adr/0073-a-node-keeps-what-canonicalization-asks-of-it.md b/docs/adr/0073-a-node-keeps-what-canonicalization-asks-of-it.md new file mode 100644 index 000000000..47520dd9c --- /dev/null +++ b/docs/adr/0073-a-node-keeps-what-canonicalization-asks-of-it.md @@ -0,0 +1,34 @@ +# ADR-0073: A node keeps what canonicalization asks of it; differentiation visits each node once + +**Status:** Accepted +**Date:** 2026-10-11 + +## Context + +Building a sum or a product puts its operands in canonical order (ADR-0059), and to do that it asks each operand three things: its variables, its multidegree, and, when two multidegrees tie, its canonical encoding. Each answer was a walk of the operand. #417/#418 made the multidegree walk visit each node once per call, so a question cost the operand's DAG rather than its expansion, and recorded that a cache of results between calls had measured 29% slower, because filling it cost the same walk. + +That was still a walk per construction. Differentiation builds as many nodes as the expression has, each over operands it has just built, so building a derivative cost the square of its size. Two holes made it worse: + +- `Differentiate` had no memo at all. A subexpression reached by many paths was differentiated once per path: the sub-minors of a determinant built by cofactor expansion, k! times at the top. +- A unary operator's multidegree is computed from `Degree(v)` of its operand, and `Degree(v)` walked the whole expansion. So every construction above a negation reopened the walk #418 had closed. + +Deflation appends Jacobian minors, so these costs landed on every singular point a decomposition sharpens: on the Barth sextic, 121 s of first evaluations for 50 deflated systems, against 0.1 s of evaluating them (#505). + +## Decision + +- **`Differentiate` is non-virtual and keeps each derivative for the outermost call.** A node type's rule is `DifferentiateImpl`, which differentiates its operands by calling `Differentiate`. A `node::DerivativeScope` extends the memo across a loop; both Jacobian builds open one, so functions that share subexpressions differentiate each of them once among them. +- **Each node keeps a degree record, made once from its operands' records.** It holds the node's variables (what `GatherVariables` returns), its degree and multidegree entry in each variable it mentions (its variables and its differentials' variables), its total degree, and one degree and one multidegree entry for every variable it does not mention. `Degree`, `MultiDegree` and `GatherVariables` read it. It is made without a walk, so #418's measurement does not apply to it. +- **The answer for an unmentioned variable is computed, not assumed.** It is not always 0: a power with a negative constant exponent reports -1 in every variable. The degree rules compare a variable only by identity with the variables a subtree mentions, so every unmentioned variable gets the same answer, and the record computes it once by asking the rules about a variable that is in no expression (a default-constructed `Variable`, never interned, so no user's name can collide with it). +- **The tie-break key is a content key, a Merkle hash of the canonical encoding** (this replaces ADR-0059's key, the operand's whole canonical encoding). A node's key is the SHA-256 of its shallow canonical encoding: its own kind, payload and operands in the ADR-0042 format, each operand written as `@` and that operand's key. It is made from the operands' keys, once per node, and kept. The whole encoding could not be: its back-references number the whole tree, so it is not made from the operands' encodings, and computing it once per node still walked each node's DAG -- O(DAG^2) to build an expression, 1.4 s for the derivative of a 7 x 7 determinant against 0.04 s with the key. The key is still derived from the encoding, as ADR-0059 asks of any ordering criterion, exact, and the same on every platform. +- **Degree records and content keys are published once, by atomic compare-and-swap.** Systems compile lazily on worker threads, so two threads may make the same record; the first one published wins, and a reference into it lives as long as the node. + +## Consequences + +- A node type's `DegreeImpl` and `MultiDegreeImpl` must never ask `Degree` or `MultiDegree` of their own node: the record is being made from them, and the request would recurse until the stack overflows. Ask `DegreeImpl` of `this` instead, as the unary, power and named-expression rules do. +- Do not replace the unmentioned answer with 0, and do not key the sentinel variable by a name. +- Do not go back to walking an operand to put a sum or product in order, or to a memo that lasts one call: a construction's question should cost O(operands). +- Do not make `Differentiate` or `Degree(v)` virtual again; override the `Impl`. +- Where operands tie on multidegree, their canonical order changes, so the system encoding version is `b2sysenc/3` and every system digest moved in 4.0.1: records written by 4.0.0 are asked again, not recalled. The golden fixture has a recipe whose operands tie (`tied_terms`), so a later change to the tie-break moves the registry's keyspace hash; before it, no recipe had a tie and the registry could not see one. +- Do not order operands by anything that walks them, the full encoding included. A new tie-break must be made from the operands' keys, or it is a walk again. +- A collision of two different operands' keys would leave them in input order: a valid expression, not a canonical one. At SHA-256 that is not a practical concern. +- Memory: every node that has been an operand of a canonicalized sum or product holds a record, proportional to the variables it mentions, and, if it was ever in a tie, its 64-character content key. diff --git a/docs/adr/README.md b/docs/adr/README.md index 350c6ef2f..568c6ff8d 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -75,7 +75,7 @@ Each ADR follows the template: | [0056](0056-metadata-for-matches-representatives-final-tolerance.md) | metadata_for matches against multiplicity REPRESENTATIVES at final_tolerance (not the complete copy set at the looser same-point tolerance), so a solver result fed straight back resolves to its representative and is never spuriously "ambiguous"; representatives are ≥ same_point_tolerance apart so the tight window holds at most one; don't restore the old default | Core / nag_algorithms | | [0057](0057-precision-follows-the-point-not-the-system.md) | Precision follows the POINT: a System carries NO precision at all (no setter, no getter, no member, nothing to prepare or fan out) -- every evaluable type (4 blocks, patch, SLP) self-aligns to the point it is handed under one uniform SyncPrecision, and coefficient generation is a construct-time concern; evaluation aligns instead of refusing, which used to wedge a System with no escape; archive format changed (precision_ no longer serialized); don't re-add a setter, a refusal, or a "prepare" entry point | Core / precision | | [0058](0058-default-endgame-powerseries.md) | The default endgame is POWER SERIES, matching Bertini 1's documented default (EndgameNum 1), at every choice point (config default, classic parser fallback, CLI, Python factories and solve); measured: Cauchy 190 s vs power series 2.5 s on a regeneration workload, from slow divergers that only Cauchy mishandles; Cauchy stays available by explicit request; don't flip the default back for a workload where Cauchy wins -- pass endgame='cauchy' there | Core / endgames | -| [0059](0059-identity-is-the-encoding-printing-is-a-dialect-family.md) | Identity is the canonical ENCODING, never printed text: the canonicalization tie-break and like-factor key are the node's canonical encoding (lazy, memoized; linear in the DAG -- the second half of #417), b2sysenc bumped to /2; printing is a FAMILY of dialects on the stream -- Classic (`^`, `(re+im*I)`, the Bertini 1 spelling), Python readable (`str`, `**`), Python exact (`repr`, `real_mp('d', p)` / `Complex('re','im', p)` / `Rational('p/q')`, eval rebuilds at full precision); the `(re,im)` form and both Python shims are gone; don't put printed text back into anything a digest depends on, and add a dialect for another system rather than a regex | Core / function_tree | +| [0059](0059-identity-is-the-encoding-printing-is-a-dialect-family.md) | Identity is the canonical ENCODING, never printed text: the canonicalization tie-break and like-factor key are the node's canonical encoding (lazy, memoized; linear in the DAG -- the second half of #417), b2sysenc bumped to /2; printing is a FAMILY of dialects on the stream -- Classic (`^`, `(re+im*I)`, the Bertini 1 spelling), Python readable (`str`, `**`), Python exact (`repr`, `real_mp('d', p)` / `Complex('re','im', p)` / `Rational('p/q')`, eval rebuilds at full precision); the `(re,im)` form and both Python shims are gone; don't put printed text back into anything a digest depends on, and add a dialect for another system rather than a regex _(tie-break key replaced by 0073)_ | Core / function_tree | | [0060](0060-stopping-a-solve-and-reusing-what-it-left.md) | Stopping a solve and reusing what it left: ONE cooperative stop (process-wide request + per-tracker wall-clock DEADLINE) checked between steps, never a watcher thread or an exception across a worker (overrun = one step); every path stamped with where it got to (time, `latest_path_point`, step tally, precision -- no observer, the tracker already knows), abandoned paths RECORDED, never-started ones not; wall-clock budgets OUT of the ask; reuse rule "a record is reusable when the run asks no more of that path than the run that produced it", applied through the user's `RecallPolicy` in its own `RecordsConfig` (Nothing / Completed / Everything); whole-solve budget reads as an interrupt; failed endgame's solution slot is EMPTY (was the previous path's approximation); append new SuccessCodes at the END | Core / trackers, nag_algorithms, records | | [0061](0061-one-encoding-version-per-release.md) | At most ONE encoding version per RELEASED library version: a registry line is edited IN PLACE while its release is unreleased (new hash, same token) and is append-only once it has shipped, so the number counts encodings that were really written rather than development events (`b2cfgenc/1` and `/2` never shipped at all); every line records the release it first shipped in, no two may claim the same one, and a `.devN` suffix buys no version; the `b2cfgenc` registry gains a COMPOSITION column, because the keyspace hash cannot see which configs the settings text folds in (`/3` and `/4` share a keyspace -- `/4` WAS the composition changing) | Core / records, identity, tests | | [0062](0062-the-system-judges-its-own-points.md) | The SYSTEM judges its own points: `IsFinite`/`IsReal` live on System, and a system declares which of its coordinates are AUXILIARY (by variable group and/or index) -- carried for the construction's sake, so their magnitude is not a quantity anybody chose. Path truncation, the endgame's `Security::max_norm` and the finite/real classification are one question with three thresholds, and all route through the chokepoint that already called itself canonical; two of them had been hand-rolling it. The declaration IS content identity (`b2sysenc`), since the tracker truncates on what is not auxiliary. Rejected: a caller-supplied predicate (cannot join the ask), first-group-by-default (every group is geometric in an mhom system), a config plumbed to three layers. Note: System's copy constructor is a hand-maintained mirror -- a forgotten member is dropped by every `Clone` | Core / system, trackers, endgames, nag_algorithms | @@ -89,3 +89,4 @@ Each ADR follows the template: | [0070](0070-b2-reads-its-own-thread-count-variable.md) | b2 reads ITS OWN thread-count variable, `BERTINI_NUM_THREADS`, and ignores `OMP_NUM_THREADS`, which numpy's OpenBLAS also reads (pinning one pinned the other); one rule for a standalone solve and each MPI rank: the variable, then `num_threads`, then every CPU the process may run on (the affinity mask on Linux), OpenMP's default; MPI ranks no longer default to one thread; don't read `OMP_NUM_THREADS` again, even as a fallback, and don't give ranks a different default | Core / parallel, nag_algorithms; CLI; docs | | [0071](0071-a-path-does-not-depend-on-the-paths-before-it.md) | A path DOES NOT DEPEND ON THE PATHS BEFORE IT: its result is a function of the systems, settings, seed and index, met by state hygiene in the tracker and endgame rather than a fresh object per path (which would leave the trap for hand users); a track sets what it uses from its own inputs, never a leftover member (the first step was built at the previous track's precision); random directions are stored as drawn and every working copy derived, never rounded in place; per-path directions are drawn at run start after the reseed, never lazily; per-lane containers cleared in every lane; threaded, MPI and serial solves are bit-identical; a record starts afresh exactly where its span starts: the per-track precision recorders at `Initializing`, never unsubscribing, and the solver's per-path metadata from `PathPrecisionRecorder`, reset only at path start | Core / trackers, endgames, nag_algorithms | | [0072](0072-seed-zero-is-a-seed.md) | SEED 0 IS A SEED in the library API (`SetGlobalSeed(0)`, `set_random_seed(0)`), as in numpy and Python's random; entropy is its own request (`SetGlobalSeedFromEntropy()`, `set_random_seed()` / `None`) and returns the seed it chose; the classic input format keeps Bertini 1's meaning, `randomseed: 0` = draw from entropy, via `ApplyClassicRandomSeed`; so seed 0 alone does not carry between the two interfaces; "not set" is its own flag; derived seeds still avoid 0 so they replay through classic input; don't make the API's 0 mean entropy again | Core / random, blackbox; Python bindings | +| [0073](0073-a-node-keeps-what-canonicalization-asks-of-it.md) | A NODE KEEPS WHAT CANONICALIZATION ASKS OF IT, and differentiation visits each node once: `Differentiate` is non-virtual with a memo for the outermost call (`DifferentiateImpl` is the rule; a `DerivativeScope` spans a Jacobian); each node makes a degree record once from its operands' -- variables, degree and multidegree per mentioned variable, total degree, and the rules' answer for every unmentioned variable (not always 0: a negative constant power is -1 everywhere; computed with a never-interned sentinel variable) -- and the tie-break is a CONTENT KEY, the SHA-256 of the node's shallow canonical encoding (operands written as their keys), a Merkle hash made once per node (replaces ADR-0059's key, the whole encoding, which walked); published once by atomic compare-and-swap; building a sum or product no longer walks its operands (#505: Barth sextic deflations 121 s to 0.6 s, a 7 x 7 determinant's first evaluation 24.6 s to 0.04 s); tied operands reorder, so `b2sysenc/3` and every digest moves in 4.0.1; a `DegreeImpl` never asks its own node's `Degree`; don't assume 0, don't order by anything that walks, don't make `Differentiate`/`Degree(v)` virtual again | Core / function tree; system identity | diff --git a/python_bindings/src/node_export.cpp b/python_bindings/src/node_export.cpp index 60bd909bb..5ac9fbf9b 100644 --- a/python_bindings/src/node_export.cpp +++ b/python_bindings/src/node_export.cpp @@ -38,10 +38,10 @@ namespace bertini{ // Wrapper struct to allow derived classes to overide methods in python struct NodeWrap : Node, wrapper { - int Degree(std::shared_ptr const& v = nullptr) const {return this->get_override("Degree")(v); } + int DegreeImpl(std::shared_ptr const& v) const override {return this->get_override("Degree")(v); } int Degree(VariableGroup const& vars) const {return this->get_override("Degree")(vars); } - std::shared_ptr Differentiate(std::shared_ptr const& v = nullptr) const {return this->get_override("Differentiate")(v); } + std::shared_ptr DifferentiateImpl(std::shared_ptr const& v) const override {return this->get_override("Differentiate")(v); } std::vector MultiDegree(VariableGroup const& vars) const {return this->get_override("MultiDegree")(vars); } diff --git a/python_bindings/src/system_export.cpp b/python_bindings/src/system_export.cpp index 995202c0a..4fb2677f8 100644 --- a/python_bindings/src/system_export.cpp +++ b/python_bindings/src/system_export.cpp @@ -120,7 +120,7 @@ namespace bertini{ .def("canonical_encoding", +[](SystemBaseT const& self){ return self.CanonicalEncodingText(); }, (arg("self")), - "The exact canonical encoding text of the system: the text content_digest() is the SHA-256 of, and the text the records archive stores for every system and homotopy. Versioned (its first word is the format version, e.g. b2sysenc/2) and complete: variable groups, path variable, every block with its exact coefficients, the patch, and operand systems inside randomization and blend blocks. System.from_canonical(text) rebuilds an equal system from it.") + "The exact canonical encoding text of the system: the text content_digest() is the SHA-256 of, and the text the records archive stores for every system and homotopy. Versioned (its first word is the format version, e.g. b2sysenc/3) and complete: variable groups, path variable, every block with its exact coefficients, the patch, and operand systems inside randomization and blend blocks. System.from_canonical(text) rebuilds an equal system from it.") .def("from_canonical", +[](std::string const& text){ return std::make_shared(System::FromCanonicalEncoding(text)); }, (arg("text")),