Skip to content

core: treat malformed definition/data messages as corruption, not crashes - #128

Merged
bkueng merged 2 commits into
PX4:mainfrom
lfvaranda:fix/malformed-ulog-robustness
Oct 20, 2026
Merged

bkueng merged 2 commits into
PX4:mainfrom
lfvaranda:fix/malformed-ulog-robustness

Conversation

@lfvaranda

@lfvaranda lfvaranda commented Oct 13, 2026 •

Copy link
Copy Markdown

What

ULog() raises uncaught exceptions on malformed .ulg input, aborting the parse (reported in #127). The definitions phase only guarded IndexError, and the data phase only guarded IndexError / end-of-file struct.error, so several failure modes escaped:

Exception Site Cause
KeyError (dominant) _parse_nested_type ADD_LOGGED_MSG referencing an undefined / nested format
ValueError _extract_type malformed array spec (int(...)); also an unclamped array_size → range() DoS
struct.error _MessageInfo / _MessageFlagBits / _MessageHeader struct.unpack in the definitions phase (only IndexError was caught there)
OverflowError _load_file file.seek() with a bogus appended offset from FLAG_BITS

Fix

Treat these as corruption, consistent with the existing IndexError handling:

  • _read_file_definitions: move the header parse into the try and broaden the handler to (IndexError, ValueError, KeyError, OverflowError, struct.error).
  • _read_file_data: broaden the per-message handler to also catch KeyError, ValueError, OverflowError.
  • _extract_type: reject an out-of-range array_size (also prevents a range()-based resource-exhaustion DoS).
  • _load_file: guard the appended-data seek() against a bogus offset.

String decoding errors with disable_str_exceptions=False (UnicodeDecodeError) still propagate, as before.

Valid logs are unaffected; only malformed records are skipped / flagged (file_corruption stays True).

Dedicated exceptions

Intentional refusals now use dedicated exceptions (new pyulog/exceptions.py, exported from pyulog) instead of generic built-ins:

Case Before Now
invalid file header TypeError ULogException
unknown incompatible flag ValueError / NotImplementedError ULogIncompatibleFlagException (subclass of ULogException)

Callers can catch every intentional refusal with a single except ULogException.

Verification

Fuzzing ULog() on mutated valid logs (seed=1, 4000 iterations):

parses OK distinct escaping exceptions
before 3255 / 4000 10
after 3971 / 4000 3

The 3 that remain are the intended refusals above (ULogException for an invalid header, ULogIncompatibleFlagException at the two incompatible-flag checks). The genuine robustness gaps (KeyError, _extract_type ValueError, definitions-phase struct.error, _load_file OverflowError) are gone.

New tests in test/test_ulog.py cover both exceptions, corrupt definitions and strict string parsing; pytest and pylint pass.

Fixes #127

@lfvaranda
lfvaranda force-pushed the fix/malformed-ulog-robustness branch from 1199655 to 1b83cfa Compare October 13, 2026 18:06
…shes

ULog() raised uncaught exceptions on malformed .ulg files: KeyError
(_parse_nested_type, an ADD_LOGGED_MSG referencing an undefined/nested
format), ValueError (_extract_type array spec), struct.error (the
definitions phase only caught IndexError), and OverflowError (file.seek
with a bogus appended offset from FLAG_BITS). Any service parsing an
untrusted log was taken down unless every call was wrapped in try/except.

- _read_file_definitions: move the header parse into the try and broaden
  the handler to (IndexError, ValueError, KeyError, OverflowError,
  struct.error), treating a malformed definition message as corruption
  like _read_file_data already does. The intentional "unknown incompatible
  flag" aborts (ValueError / NotImplementedError) are preserved and still
  propagate.
- _read_file_data: broaden the per-message handler to also catch KeyError,
  ValueError and OverflowError.
- _extract_type: reject an out-of-range array size (also prevents a
  range()-based resource-exhaustion DoS).
- _load_file: guard the appended-data seek against a bogus offset.

Fuzzing ULog() on mutated valid logs (seed=1, 4000 iterations) went from
10 distinct escaping exception sites to 3 - and those 3 are the library's
intended invalid-header / incompatible-flag refusals.

Fixes PX4#127
@lfvaranda
lfvaranda force-pushed the fix/malformed-ulog-robustness branch from 1b83cfa to a6b45a4 Compare October 13, 2026 18:12

@bkueng bkueng left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flight review already has a broad exception capture around parsing.
This makes sense nonetheless though.
And I would directly add a dedicated ULogException while you're at it (with a separate class for the incompatible flag case).

Comment thread pyulog/core.py Outdated
except (IndexError, ValueError, KeyError, OverflowError, struct.error) as exc:
# Keep the intentional incompatible-flag abort ("cannot parse the
# log"); treat any other failure as corruption, like _read_file_data.
if isinstance(exc, ValueError) and 'cannot parse the log' in str(exc):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a dedicated exception so no string check is required?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 0d297a9: the incompatible-flag case now raises ULogIncompatibleFlagException (a ULogException subclass), which the corruption handler doesn't catch, so no string check is needed.

Use dedicated exceptions instead of matching the error message to tell
the intentional incompatible-flag abort apart from file corruption.

- Add pyulog/exceptions.py with ULogException (the file cannot be
  parsed) and its subclass ULogIncompatibleFlagException, both exported
  from pyulog. They live in their own module as core.py is right at
  pylint's max-module-lines.
- Raise ULogException for an invalid file header (was TypeError) and
  ULogIncompatibleFlagException for an unknown incompatible flag (was
  ValueError / NotImplementedError). They are not caught by the
  corruption handlers, so the string check is gone.
- Let UnicodeDecodeError propagate again: it is a ValueError subclass,
  so the broadened handlers turned string errors into corruption even
  with disable_str_exceptions=False.
- Only guard the appended-data seek(), not _read_file_data().
- Add tests for both exceptions, corrupt definitions and strict string
  parsing.
@lfvaranda

Copy link
Copy Markdown
Author

Thanks for the review! Good to know Flight Review already catches this, so the PR is mainly about robustness for other users of the library.

Updated in 0d297a9:

  • Added ULogException and its subclass ULogIncompatibleFlagException in a new pyulog/exceptions.py, exported from pyulog (I kept them out of core.py, which is right at pylint's max-module-lines).
  • An invalid header now raises ULogException (was TypeError), and an unknown incompatible flag raises ULogIncompatibleFlagException (was ValueError / NotImplementedError). Neither is caught by the corruption handlers, so the string check is gone.
  • While at it, I noticed the broadened handlers also swallowed UnicodeDecodeError (a ValueError subclass), so string errors were treated as corruption even with disable_str_exceptions=False. It propagates again, as before this PR.
  • Added tests for both exceptions, corrupt definitions and strict string parsing.

Note that this changes the exception types for these cases, so a caller catching TypeError / ValueError there would need ULogException instead. If you prefer, I can also derive the new classes from the old built-in types for backward compatibility.

@bkueng bkueng left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

Note that this changes the exception types for these cases, so a caller catching TypeError / ValueError there would need ULogException instead. If you prefer, I can also derive the new classes from the old built-in types for backward compatibility.

I'll mention it in the release notes, that should be sufficient

@bkueng bkueng left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

Note that this changes the exception types for these cases, so a caller catching TypeError / ValueError there would need ULogException instead. If you prefer, I can also derive the new classes from the old built-in types for backward compatibility.

I'll mention it in the release notes, that should be sufficient

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ULog parser raises uncaught exceptions (KeyError/ValueError/struct.error/OverflowError) on malformed .ulg — DoS for services parsing untrusted logs

2 participants