Repository navigation
A record inside a list group (CELL) sends GroupListOverlay.Factory into an endless loop that ends in OutOfMemoryException #1612
Description
Problem
A plugin whose top-level CELL group holds a record where a cell block group belongs makes Mutagen's overlay loop until the process runs out of memory. A malformed plugin should fail with a parse error that names the bad record, as other groups do. Instead it takes minutes and kills the reader.
For comparison, the same stray record in an ordinary top-level group, such as NPC_ or QUST, fails at once: RecordException: Unexpected type encountered when parsing MajorRecord locations: MISC.
Reproduce (Fallout 4)
With Modbench's RawPlugin test helper:
RawPlugin.Plugin(RawPlugin.Tes4(), RawPlugin.Group("CELL", RawPlugin.Misc(0x802, "Stray")))Open the plugin with Fallout4Mod.CreateFromBinaryOverlay and enumerate its records, or read mod.Cells. After about 80 s:
System.OutOfMemoryException : Array dimensions exceeded supported range.
at System.Collections.Generic.List`1.AddWithResize(T item)
at Mutagen.Bethesda.Plugins.Binary.Overlay.GroupListOverlay`1.Factory(IBinaryReadStream stream, ReadOnlyMemorySlice`1 data, BinaryOverlayFactoryPackage package, ObjectType objectType, Int32 offset)
at Mutagen.Bethesda.Fallout4.Fallout4ListGroupBinaryOverlay`1.CustomFactoryEnd(OverlayStream stream, Int32 finalPos, Int32 offset)
at Mutagen.Bethesda.Fallout4.Fallout4ListGroupBinaryOverlay`1.Fallout4ListGroupFactory(OverlayStream stream, BinaryOverlayFactoryPackage package, TypedParseParams translationParams)
Likely cause
In Mutagen.Bethesda.Core/Plugins/Binary/Overlay/AListGroupOverlay.cs, GroupListOverlay<T>.Factory reads every entry as a VariableHeader of objectType (a group) and advances by meta.TotalLength:
while (stream.Position < finalPos)
{
VariableHeader meta = package.MetaData.Constants.VariableHeader(stream.RemainingMemory, objectType);
locations.Add(checked((int)stream.Position - offset));
stream.Position += checked((int)meta.TotalLength);
}It never checks that the entry is a GRUP. A record read as a group header advances by the wrong length, and once a length reads 0 the position never moves, so locations grows without end. The fix is probably to check the header's signature, and to throw a RecordException naming the record type when an entry is not a group or makes no progress, as GroupMajorRecordCacheWrapper.Factory does for ordinary groups.
Found while testing Modbench's fallback for a walk Mutagen stops partway. The same loop applies to every game's list groups that use this factory.
Activity
WhiskyTangoFawks commentedon Oct 12, 2026
Upstream check, 2026-10-12: no issue or PR on Mutagen-Modding/Mutagen matches this (searched its issues and PRs for the record type, the exception and the code named above). Upstream
devis still 414371ee, the base of alpha.67, so nothing unreleased fixes it either. It is ours to report and fix, through the fork and the stack (#668's D1 and D2).WhiskyTangoFawks commentedon Oct 12, 2026
Fixed on the fork, verified on the stack:
fix-1612-list-group-stray-record67a4c6074, inmodbench-stackbe831c2fe, packed as 0.55.0-modbench.12 and pinned on Modbench'smutagen-stackbranch.Cause.
GroupListOverlay<T>.Factoryread every entry of a list group as a group header and stepped by its length. A record in that place was read as a group. The loop stepped by the record's content length, and once a length read 0 it stopped moving. A realGRUPwhose length is shorter than its header stalled the same way.Fix. A non-
GRUPentry throwsRecordException: Unexpected type encountered when parsing group locations: MISC, as the overlay of an ordinary group does. A group too short to hold its header throwsMalformedDataException. Every caller passedObjectType.Group, so that parameter is gone. The full import still skips the stray record silently, for list groups and ordinary groups alike. That is pre-existing behaviour and is not changed here.Tests (
ListGroupOverlayTests, Fallout 4 bytes built by hand):Evidence. The stack suite passes (3457 + 790). The Modbench gates pass on
mutagen-stack.Draft upstream PR
List group overlay throws on an entry that is not a group
GroupListOverlay<T>.Factoryreads each entry of a list group (CELL, and each game's equivalent) as a group header. A major record in that position was read as a group and stepped by its content length. Once a length read 0, the loop stopped advancing andlocationsgrew untilOutOfMemoryException, after about 80 s on Fallout 4.It now throws a
RecordExceptionnaming the record type, matchingGroupMajorRecordCacheWrapper.Factoryfor ordinary groups. It throws aMalformedDataExceptionfor a group shorter than its header. TheobjectTypeparameter is removed, since every caller passedObjectType.Group. Tests build the plugin bytes by hand.WhiskyTangoFawks commentedon Oct 12, 2026
Upstream PR opened: Mutagen-Modding/Mutagen#717. It names the full import's silent skip and a remaining zero-length stall in
PluginBinaryOverlay.ParseRecordLocations(sub-blocks) as left alone. Stays open until a release ships it andmain's pin moves.A plugin whose read runs out of memory is that plugin's Failed to read