[omkar-334]

Summary

Older libraries often document parameter types in Sphinx fields such as :param flask.Flask|flask.Blueprint app: instead of annotating them, so Pyrefly treats those parameters as Any.

This adds an opt-in infer-param-types-from-docstring option (also --infer-param-types-from-docstring, default false). When it is on, the type from a :param <type> <name>: or :type <name>: <type> field becomes the type of an unannotated parameter. :type wins if a parameter has both. For __init__ the class docstring is read as well, since that is where Sphinx documents constructor parameters, and the method's own docstring takes priority.

A docstring type is only a hint:

• It is never a declared type, so an explicit annotation always wins and assigning to the parameter in the body is not an error.
• It is not checked against the default value, and a parameter that defaults to None becomes T | None.
• A type that cannot be understood is dropped without an error. That covers prose such as list of str, unknown names, and types that are not valid for a parameter.

Google and NumPy docstrings are not handled here and can be added on top of this.

Fixes #1384

• I am a human, and I fully understand the code I am submitting.

Test Plan

Added type checker tests in pyrefly/lib/test/docstring_param_types.rs (off by default, :param and :type, None defaults, unions and attributes, local classes, __init__ with a class docstring, annotations winning, and types that are ignored), parser tests in docstring.rs, and a config test. The pyrefly, pyrefly_python and pyrefly_config test suites pass, except for check_writes_cinderx_and_glean_reports_together and deep_phi_chain_term_expr, which also fail on an unmodified main in my environment. Formatting and lint, conformance, and the jsonschema check are clean.

[github-actions]

✅ This PR meets all requirements from our pull request checklist.

Optional recommendations:

• This PR is very large, which makes it slow and difficult to review. We strongly recommend splitting it into smaller PRs that can be merged independently. If that isn't possible, please structure it as a series of small commits. If you've already split it as far as is practical, no action is needed.

Note: this comment updates automatically after changes to the PR. Updates may take up to half an hour.

[meta-codesync]

This pull request has been imported. If you are a Meta employee, you can view this in D129028362. (Because this pull request was imported automatically, there will not be any future comments.)

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

[omkar-334]

cc @asukaminato0721

[rchen152]

Thanks for the PR! From the typing side, this looks very good; I found one minor typing issue (see in-line comment). IMO it doesn't necessarily need to be addressed before merging, but it looks like a relatively simple fix.

How this interacts with the language server is where things get a little hairy. I asked claude to test a number of lsp scenarios by using a mini hand-written client, and it seems like with the new flag turned on, find-references and renames will see/update docstring types across files but not within the same file. I don't have a strong preference for whether we fix this by always excluding docstring types or always including them, but we should be consistent. I'll include claude's notes below on how to implement either approach.

As a side note, thank you for organizing this into easy-to-review commits :)

AI notes on an LSP fix:

Option 1: Ignore docstring types in IDE navigation

1. Record the range of each docstring type expression on Bindings.
   • Add a docstring_type_ranges: Vec<TextRange> field next to annotated_metadata_ranges / unused_imports (pyrefly/lib/binding/bindings.rs:238) and on BindingsBuilder (:340).
   • Initialize it in both constructors (:468, :861), move it across in Bindings::new (:1018), and add an is_in_docstring_type(range) accessor next to unused_imports() (:557).
   • Push expr.range() after ensure_type in bind_docstring_param_types (pyrefly/lib/binding/function.rs:429).
2. Skip those ranges when building the index of names defined in other modules (pyrefly/lib/alt/answers.rs:1453), by adding && !bindings.is_in_docstring_type(reference_range) to the existing check on the reference text.
3. Return early from record_attribute_definition_index (pyrefly/lib/alt/answers.rs:1767) when attribute_reference_range is in a docstring type. This covers attribute accesses like m.Foo, which alt/expr.rs:2765 records when the docstring annotation is solved.

The syntax-tree scans for same-module references and for the star-import fix already ignore docstrings, so nothing else changes. One difference remains: lookup_name (pyrefly/lib/binding/bindings.rs:2029, :2064) still counts a docstring name as a use of its import. I'd keep that, because otherwise removing an "unused" import would silently break the docstring type.

Option 2: Include docstring types everywhere

1. Same-module references to a name: in local_variable_references_from_local_definition, after the syntax-tree walk (pyrefly/lib/state/lsp.rs:4971), also run the visitor over every BindingAnnotation::DocstringParam expression in the bindings. The is_valid_use check works unchanged, because ensure_type already created name-use keys for these names.
2. Same-module references to an attribute: do the same in local_attribute_references_from_local_definition (pyrefly/lib/state/lsp.rs:4794). This also needs a recorded type for the expression before the dot (get_type_trace(attribute.value.range()), :4805), and I'm not sure solving the docstring annotation records one.
3. Star-import conversion: also visit docstring expressions in collect_star_imported_names (pyrefly/lib/state/lsp/quick_fixes/convert_star_import.rs:159), so names that only docstrings use are kept.
4. Starting from inside a docstring: identifier_at (pyrefly/lib/state/lsp.rs:1079) only scans the syntax tree (:1274). So rename, find-references, hover, and go-to-definition do nothing when the cursor is in a docstring. Full consistency would mean also checking the docstring expressions there.

Unused-import tracking already counts docstring uses, so it's consistent with this option as is.

[rchen152]

The second argument to ensure_type is a collector for legacy type variables that makes them resolve correctly in signatures. Passing in None means that if a docstring references a legacy type variable, we'll infer the type as TypeVar[T] (essentially, the definition of the legacy type variable) rather than T (the type parameter).

Concretely, with the new flag:

from typing import TypeVar, reveal_type
T = TypeVar("T")
def ident(x):
    """
    :param T x: the x
    """
    return x
ident(1) # error!

A possible fix is to bind docstring types inside function_header, with the same LegacyTParamCollector as real annotations.