Repository navigation
Conversation
Both regexes run on the validated data and backtrack quadratically on long invalid strings. After the @, [^@]+\.[^@]+\Z retries every dot when the string ends in a second @, and the email lookahead (?!.*\.\..*@) rescans the rest of the string for each '..'. Rejecting 'a@' + 'a.' * 40000 + '@' (80 KB) took 24 s for email, and 'a@a' + '.' * 80000 + '@' took 61 s for idn-email. Match the local part as dot-separated runs of [^@.] and check for a dot inside the domain with a single lookahead, so both regexes run in linear time. idn-email accepts the same strings as before. email now also rejects '..' in the local part when a newline sits between it and the @, which the old lookahead's '.' could not see.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
emailandidn-emailformat regexes backtrack quadratically on long invalid stringsemailmasteremailhereidn-emailmasteridn-emailhereAfter the
@,[^@]+\.[^@]+\Zretries every dot once the trailing@makes\Zfail, and theemaillookahead(?!.*\.\..*@)rescans the rest of the string for each... Format checks run on the data being validated, so a single 80 KB string field in a request body keeps the validator (and the GIL) busy for about a minute.The local part is now matched as dot-separated runs of
[^@.], and the "dot inside the domain" requirement is a single lookahead,(?=[^@][^@.]*\.[^@]), so there are no overlapping choices left to retry. Comparing old and new regexes over every string up to length 10 built froma,.,@,\n(plus 300k random strings),idn-emailaccepts exactly the same strings.emaildiffers only when a newline sits between a..in the local part and the@(e.g.'a..\nb@c.d'): the old lookahead's.could not see past the newline and accepted it, the new regex rejects it.tests/test_format.pygetsemail/idn-emailcases and a long-input test that fails onmaster(about 45 s) and passes here. The full suite passes on Python 3.10 and 3.12, and pylint output is unchanged.CHANGELOG.txthas an entry under UNRELEASED.