Export models with __slots__ on the generated Space classes - #273
Merged
Merged
Conversation
CPython keeps slotted attributes in a fixed-size array instead of an instance dictionary and specialises the bytecode that reads them, so an exported model runs faster and takes less memory. Measured on BasicTerm_SC of lifelib's basiclife library, 3,000 model points per round and the best of three rounds: 3.68 s -> 2.90 s on Python 3.13.9 (-21%) and 3.22 s -> 2.31 s on Python 3.14.6 (-28%), with an ItemSpace of Projection taking 616 bytes instead of 1,632. The new use_slots keyword of export_model(), Model.export() and Exporter() turns the declarations off. Its output is byte-identical to what modelx v0.32.0 generates. A name missing from __slots__ is an AttributeError at run time rather than a compile-time error, so each slot name is collected next to the statement that assigns the attribute, and ref_names/space_names are now the single source of truth for the References and the child Spaces. __call__ propagates a Space's parameters onto every Space in its subtree and copies the parameters of every enclosing ItemSpace root as well, so a Space class also declares the parameters of every parameterized ancestor. __slots__ cannot hold a name that the class body binds, so the exporter now rejects a Cells sharing its name with a parameter of its own or of an enclosing Space. That name has always been exported incorrectly, since the parameter value is assigned over the method. BaseMxObject, BaseParent and BaseSpace in the _mx_sys.py template declare __slots__ = () in both modes; a subclass declaring none of its own still gets a __dict__. BaseModel and the generated model class are left alone. No __weakref__ slot is declared, so Space objects in an export made with use_slots=True are not weak-referenceable, and modelx-cython needs use_slots=False until it is updated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A Space parameter is the only name that reaches __slots__ with a leading underscore, because modelx rejects one in the name of a Cells, a Reference or a Space. Two kinds of parameter name needed handling: * A parameter named after a member the generated class inherits from the _mx_sys template. CPython raises nothing here, unlike a name bound in the class body: the slot silently shadows the member. A parameter named _cells would take over the property that populates _mx_cells, turning today's loud "property has no setter" into silent corruption, and one named _mx_walk would newly break a model that runs today. The reserved set is now read from _mx_sys.BaseSpace.__mro__ rather than listed, so it cannot fall behind the template. * A parameter of the form __x. CPython mangles it with the class whose body it appears in, and mangles a __slots__ string with the class that declares it, so _mx_assign_params writes _c_<Owner>__x while a descendant declared _c_<Descendant>__x. Creating any ItemSpace of such a Space raised AttributeError although it works today. A descendant now declares the owner's form of the name, which is exactly the name the assignment writes. Also document, and pin with a test, that the attributes of a Space class are fixed at export time: a macro or a formula assigning a Reference the model does not already have raises AttributeError under __slots__. The memo's "macros are not affected" holds for where macros live, not for what they do, and the name cannot be known when the class is generated. Record the modelx-cython break in devnotes/DependentPackages.md alongside the other export-format couplings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A Cells docstring line reading '__slots__ = (' at column 0 lands at the
template's own indent in the generated module, so the unanchored pattern
could have removed it from one side of the comparison.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Python normalises every identifier in a source file to NFKC, but a __slots__ entry is a string and is only mangled, never normalised. str.isidentifier() accepts a non-normalised name and modelx keeps it verbatim, so a Reference named with a fullwidth letter - what a name pasted out of a spreadsheet carries - was assigned as `self.Rate` and declared as `self.<FULLWIDTH R>ate`. The exported package then failed to import at all, while the same model exported with use_slots=False imports and runs. Slot names, the names guarded against the class body, and the class name used for private-name mangling are now all normalised. Also correct the guard's message and its documentation. The message claimed the exported model is incorrect for the name either way, which is false for a model-level Reference aliasing a Cells of the same name: `m.rate = m.S.rate` generates `self.rate = self.rate`, which resolves to the same bound method and so is a no-op. That model exports correctly today and cannot be slotted at all, so the export is still refused, but only because __slots__ cannot declare a name the class body binds. The export_model docstring and the release note did not mention the ValueError at all, or covered only the Cells-versus- parameter half of it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generated Space classes now declare
__slots__. CPython keeps slotted attributes in a fixed-size array instead of an instance dictionary and specialises the bytecode that reads them, so an exported model runs faster and takes less memory. A newuse_slotskeyword onexport_model(),Model.export()andExporter()turns the declarations off.The motivation is a CPython 3.14 change (
5d3201fe/ gh-123219) that stopped specialisingLOAD_ATTRfor instance attributes outside a type's 30-entry shared-key table._c_Projectionof lifelib'sBasicTerm_SChas 73 attributes, so all 39_v_*memo caches fell off the fast path.__slots__is not just a 3.14 workaround, though — it is a solid win on 3.13 too.Measurements
lifelib
basiclife/BasicTerm_SC, 3,000 model points per round, best of three, identical checksums both ways:__slots__What is in the diff
use_slots=Trueonexport_model(),Model.export()andExporter.__init__, threaded toSpaceTranslator.__slots__ = ()onBaseMxObject/BaseParent/BaseSpacein the_mx_sys.pytemplate, in both modes. A subclass that declares no__slots__of its own still gets a__dict__, souse_slots=Falseis unaffected.BaseModeland the generated model class are left alone deliberately — the model class carries arbitrary Reference names and there is one instance.{slots_decl}placeholder inSpaceTranslator.class_template, expanding to the empty string underuse_slots=False.__weakref__slot, by decision.A name missing from
__slots__is anAttributeErrorat run time, not a compile-time error, so each slot name is collected next to the statement that assigns the attribute.ref_names()andspace_names()are now the single source of truth for the References and the child Spaces, shared by the assignment emitters and the slot list.The propagation rule
__call__assigns a Space's own parameters onto every Space in its subtree through_mx_assign_params, and replays every enclosing ItemSpace root's_mx_copy_paramsas well. A Space class therefore declares the parameters of every parameterized ancestor, not only its own —_c_SubChildof theNestedParamssample needs slots forParent'sxandChild'syalthough it has no parameters of its own.Names
__slots__cannot expressA Space parameter is the only name that reaches
__slots__with a leading underscore — modelx rejects one in the name of a Reference or a Space, and a Cells name only ever arrives behind a_v_or_has_prefix. Three cases needed handling:type.__new__raise, so the whole_mx_classes.pyfails to import. The exporter now raisesValueErrorat export time instead, naming the Space and the name and pointing atuse_slots=False._mx_systemplate (_cells,_mx_walk, …) gets no error from CPython at all: the slot silently shadows the member. A parameter named_cellswould take over the property that populates_mx_cells. The reserved set is read from_mx_sys.BaseSpace.__mro__rather than listed, so it cannot fall behind the template.__x) is mangled with the class body it is written from, and a__slots__string is mangled with the class that declares it, so a descendant must declare_c_<Owner>__x. Without that, creating any ItemSpace of such a Space raisedAttributeErroralthough it works today.__slots__string is only mangled, never normalised.str.isidentifier()accepts a fullwidth letter and modelx keeps the name verbatim, so a Reference namedRatewas assigned asself.Rateand declared as'Rate', and the exported package did not import at all. Slot names, the guarded names and the mangling class name are all normalised now.Verification
use_slots=Falseis byte-identical to v0.32.0 across all 44 generated modules of 10 sample models. Only_mx_sys.pydiffers, in both modes, because it is copied verbatim and now carries__slots__ = ()on the base classes.__slots__covers every attribute ause_slots=Falseexport actually assigns at run time, checked over 12 sample models and 6 lifelib models includingTradLife_A(14 classes) andIntegratedLife(9), with the models exercised until ItemSpaces are created three levels deep.modelx/tests/export/test_slots.py: the slots-cover-__dict__equivalence test over every sample plus a model with nested Spaces under a parameterized Space, multiple parameters, cacheless Cells, model-level globals, macros and every Reference kindref_value()handles; a static per-class assignment scan; byte-identity of the two modes; no__dict__under slots; and one test per rejected name.For the reviewer
Two consequences are documented rather than guarded, both loud and both with an obvious workaround:
AttributeErrorin the exported model. The name is not knowable when the class is generated. The task memo's "macros are not affected" is true about where macros live, not about what they do.__dict__, and cannot be pickled with pickle protocol 0 or 1.Two things worth an explicit decision:
ValueErrorrefuses one model that exports correctly today.m.alias = m.S.cells— a model-level Reference aliasing a Cells — generatesself.rate = self.rate, which resolves to the same bound method and is a no-op, so v0.32.0 exports it correctly. It cannot be slotted at all while the class body defines the method, so the export is refused; the message no longer claims the export was already wrong. Falling back to an unslotted class for that one Space would be the alternative.modelx-cythonbreaks onuse_slots=Trueoutput —tracer.pyreads the traced Space's__dict__to discover its refs. That single line is the only blocker; compiled models were never affected by the CPython regression in the first place, since@cy.cclassturns attributes into C struct fields. Fixing it is out of scope here and is recorded indevnotes/DependentPackages.md§2.2, but withuse_slots=Trueas the default the two releases need coordinating.🤖 Generated with Claude Code