Skip to content

Fix typing, typevar, genericalias, and symboltable#7091

Merged
youknowone merged 1 commit intoRustPython:mainfrom
youknowone:typing
Feb 14, 2026
Merged

Fix typing, typevar, genericalias, and symboltable#7091
youknowone merged 1 commit intoRustPython:mainfrom
youknowone:typing

Conversation

@youknowone
Copy link
Member

@youknowone youknowone commented Feb 11, 2026

Summary by CodeRabbit

  • Refactor

    • Generic types: improved handling of TypeVarTuple so generic subscripting builds correct arguments.
    • Symbol handling: selective, per-scope name mangling for type-parameter and class contexts for consistent exported names.
    • Representation formatting: unified variance formatting for TypeVar and ParamSpec repr outputs.
    • Type semantics: NoDefault now treated with stricter immutability semantics.
    • Iteration behavior: objects with iter explicitly set to None now raise a TypeError when iterated.
  • Chores

    • Added "typevartuples" to the spellchecker dictionary.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 11, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

SymbolTable gains optional per-scope mangled_names and name resolution now calls maybe_mangle_name(class_name, mangled_names, name). Type-parameter scope handling and default-parameter scanning were adjusted. GenericAlias and typing internals updated to unpack TypeVarTuple arguments; iterator slot and TypeVar/ParamSpec repr logic were refactored.

Changes

Cohort / File(s) Summary
Compiler callsite
crates/codegen/src/compile.rs
Use current_symbol_table().mangled_names and call maybe_mangle_name(...) instead of unconditional mangle_name(...).
Symbol table & type-parameter scope logic
crates/codegen/src/symboltable.rs
Add mangled_names: Option<IndexSet<String>>; inherit/initialize per-scope; enter_type_param_block(..., for_class: bool); add maybe_mangle_name(...), scan_parameter_defaults(...), skip_defaults handling, and register TypeParam names into mangled_names.
GenericAlias TypeVarTuple handling
crates/vm/src/builtins/genericalias.rs
Preprocess type_params into params, call unpack_typevartuples(...), and pass unpacked args into _GenericAlias(Generic, args).
TypeVar / ParamSpec refactor
crates/vm/src/stdlib/typevar.rs
Introduce variance_repr helper; unify repr logic and comparison handling; adjust bound processing via type_check(...).transpose().
Typing visibility & flags
crates/vm/src/stdlib/typing.rs
Change NoDefault pyclass flag from BASETYPE to IMMUTABLETYPE; promote unpack_typevartuples to pub(crate) and adjust signature formatting.
Iterator slot behavior
crates/vm/src/types/slot.rs
iter_wrapper treats an explicit __iter__ = None on the class as "not iterable" and raises TypeError in that case.
Misc
.cspell.json
Add "typevartuples" to spelling dictionary.

Sequence Diagram

sequenceDiagram
    participant Compiler as Compiler (compile.rs)
    participant SymTable as SymbolTable (symboltable.rs)
    participant Resolver as maybe_mangle_name

    Compiler->>SymTable: enter_type_param_block(for_class)
    SymTable->>SymTable: init/propagate mangled_names (Option\<Set\>)
    Compiler->>SymTable: register_name(name, usage)
    alt usage == TypeParam
        SymTable->>SymTable: add name to mangled_names
    end
    Compiler->>Resolver: maybe_mangle_name(class_name, mangled_names, name)
    alt mangled_names is Some and contains name
        Resolver->>Resolver: call mangle_name(class_name, name)
        Resolver-->>Compiler: return mangled name
    else
        Resolver-->>Compiler: return original name
    end
    Compiler->>SymTable: insert/lookup resolved name
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested reviewers

  • arihant2math

Poem

🐰 I hopped through scopes both wide and narrow,
Collected names in a tiny straw-barrow,
Some I wrapped in mangled, secret art,
Left others plain — each played its part,
Typevars twitched and danced, a merry sparrow.

🚥 Pre-merge checks | ✅ 2 | ❌ 2
❌ Failed checks (1 warning, 1 inconclusive)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 30.77% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title is vague and generic, listing multiple modules without describing the specific changes or improvements being made. Replace with a more specific title that describes the primary change, such as 'Implement selective mangling for generic class type parameters' or 'Refactor type parameter handling and typing module improvements'.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Merge Conflict Detection ✅ Passed ✅ No merge conflicts detected when merging into main

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

No actionable comments were generated in the recent review. 🎉


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@youknowone youknowone force-pushed the typing branch 2 times, most recently from 07fbdd0 to cf798e7 Compare February 13, 2026 05:13
@youknowone youknowone changed the title fix more typing Fix typing, typevar, genericalias, and symboltable Feb 13, 2026
@youknowone youknowone marked this pull request as ready for review February 13, 2026 09:09
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
crates/codegen/src/symboltable.rs (1)

1279-1322: ⚠️ Potential issue | 🔴 Critical

Base class names are mangled incorrectly due to class_name being set when scanning bases (regression).

In the old code, self.class_name = prev_class was restored immediately after self.leave_scope() and before scanning bases. The new code moved this restoration to after base scanning, leaving class_name set to the current class during base expression evaluation.

This breaks name mangling for nested classes with private base names:

Non-generic class: Bases are scanned with class_name = "Child" and mangled_names = None, so __Private gets mangled as _Child__Private. CPython mangles it as _Outer__Private (bases are evaluated in the enclosing scope).

Generic class: Bases are scanned with class_name = "Child" and mangled_names = Some({T}). Since __Private is not in the set, maybe_mangle_name returns it unmangled. CPython would mangle it as _Outer__Private.

Root cause: The refactored code moved self.class_name = prev_class (restoration of the enclosing class name) to line 1320, after bases are scanned (lines 1310–1315), whereas the old code restored it before scanning bases.

Fix: Restore self.class_name to prev_class (the enclosing class name) immediately after self.leave_scope() and before scanning base arguments.

🧹 Nitpick comments (2)
crates/vm/src/builtins/genericalias.rs (1)

737-766: Duplicated logic with unpack_typevartuples in typing.rs.

This TypeVarTuple unpacking logic (check downcast_ref::<TypeVarTuple>, wrap with Unpack.__getitem__) is essentially identical to unpack_typevartuples in crates/vm/src/stdlib/typing.rs (lines 469–490). Consider extracting a shared helper to avoid maintaining both copies.

Additionally, new_args (line 738) is allocated even when needs_unpack is false and the else branch returns params without using it. Move the allocation inside the if needs_unpack block.

Suggested refactor
-    // Unpack TypeVarTuples: wrap each TypeVarTuple in Unpack[...]
-    let mut new_args = Vec::with_capacity(params.len());
-    let mut needs_unpack = false;
-    for param in params.iter() {
-        if param
-            .downcast_ref::<crate::stdlib::typing::TypeVarTuple>()
-            .is_some()
-        {
-            needs_unpack = true;
-            break;
-        }
-    }
-
-    let args = if needs_unpack {
-        let unpack_cls = typing_module.get_attr("Unpack", vm)?;
-        for param in params.iter() {
-            if param
-                .downcast_ref::<crate::stdlib::typing::TypeVarTuple>()
-                .is_some()
-            {
-                let unpacked = vm.call_method(&unpack_cls, "__getitem__", (param.clone(),))?;
-                new_args.push(unpacked);
-            } else {
-                new_args.push(param.clone());
-            }
-        }
-        PyTuple::new_ref(new_args, &vm.ctx)
-    } else {
-        params
-    };
+    // Unpack TypeVarTuples: wrap each TypeVarTuple in Unpack[...]
+    let has_tvt = params
+        .iter()
+        .any(|p| p.downcast_ref::<crate::stdlib::typing::TypeVarTuple>().is_some());
+
+    let args = if has_tvt {
+        let unpack_cls = typing_module.get_attr("Unpack", vm)?;
+        let new_args: Vec<PyObjectRef> = params
+            .iter()
+            .map(|p| {
+                if p.downcast_ref::<crate::stdlib::typing::TypeVarTuple>().is_some() {
+                    vm.call_method(&unpack_cls, "__getitem__", (p.clone(),))
+                } else {
+                    Ok(p.clone())
+                }
+            })
+            .collect::<PyResult<_>>()?;
+        PyTuple::new_ref(new_args, &vm.ctx)
+    } else {
+        params
+    };

This also aligns the style with the existing unpack_typevartuples in crates/vm/src/stdlib/typing.rs (line 479–488), which uses the iterator/map/collect pattern.

crates/vm/src/stdlib/typevar.rs (1)

670-685: ParamSpec and TypeVar repr implementations are identical — consider extracting a shared helper.

The variance-prefix formatting logic (lines 674–684) is a direct copy of TypeVar's repr_str (lines 273–283). A small shared function like format_variance_prefix(name, infer_variance, covariant, contravariant) would eliminate the duplication.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
crates/codegen/src/symboltable.rs (1)

2631-2662: ⚠️ Potential issue | 🟠 Major

maybe_mangle_name logic is correct, but mangle_name implementation diverges from existing code.

The mangle_name in symboltable.rs (line 2640) uses strip_prefix(|c| c == '_') to strip only one leading underscore, while the existing mangle_name in crates/vm/src/builtins/type.rs (line 2258) uses trim_start_matches('_') to strip all leading underscores. This inconsistency was introduced in this PR (commit 3343b1c), not pre-existing. The comment in symboltable.rs says "strip leading underscore" (singular), while type.rs says "Strip leading underscores" (plural), indicating intent divergence. CPython strips all leading underscores during name mangling. Both implementations should be consistent; symboltable.rs should use trim_start_matches('_') to match type.rs and CPython semantics.

🧹 Nitpick comments (2)
crates/vm/src/builtins/genericalias.rs (1)

737-759: Eliminate duplicate TypeVarTuple-unpacking logic by reusing unpack_typevartuples.

The logic at lines 737–759 duplicates the exact same TypeVarTuple detection and wrapping pattern already implemented in crate::stdlib::typing::unpack_typevartuples() (lines 470–490). The only syntactic difference is downcast_ref::<…>().is_some() vs downcastable::<…>(), which are equivalent.

To maintain a single source of truth, make unpack_typevartuples pub(crate) in typing.rs and call it here instead:

-    let has_tvt = params
-        .iter()
-        .any(|p| p.downcast_ref::<crate::stdlib::typing::TypeVarTuple>().is_some());
-
-    let args = if has_tvt {
-        let unpack_cls = typing_module.get_attr("Unpack", vm)?;
-        let new_args: Vec<PyObjectRef> = params
-            .iter()
-            .map(|p| {
-                if p.downcast_ref::<crate::stdlib::typing::TypeVarTuple>()
-                    .is_some()
-                {
-                    vm.call_method(&unpack_cls, "__getitem__", (p.clone(),))
-                } else {
-                    Ok(p.clone())
-                }
-            })
-            .collect::<PyResult<_>>()?;
-        PyTuple::new_ref(new_args, &vm.ctx)
-    } else {
-        params
-    };
+    let args = crate::stdlib::typing::unpack_typevartuples(&params, vm)?;
crates/codegen/src/symboltable.rs (1)

1279-1322: class_name save/restore is correct for nested classes and the generic path.

One minor observation: when type_params.is_some(), line 1300 re-sets class_name to the same value already set at line 1288. This is harmless (and needed for the non-generic path), but you could guard the second set with if type_params.is_none() if you want to avoid the redundant allocation.

Optional: avoid redundant allocation when type_params is present
-                // Set class_name for mangling in class body
-                self.class_name = Some(name.to_string());
+                // Set class_name for mangling in class body (already set for generic path)
+                if type_params.is_none() {
+                    self.class_name = Some(name.to_string());
+                }

@github-actions
Copy link
Contributor

Code has been automatically formatted

The code in this PR has been formatted using:

  • cargo fmt --all
    Please pull the latest changes before pushing again:
git pull origin typing

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
crates/codegen/src/symboltable.rs (1)

1279-1321: ⚠️ Potential issue | 🟠 Major

Base classes incorrectly mangled with current class name in non-generic classes.

After leave_scope() at line 1305 (exiting the class body), self.class_name remains set to the current class name (e.g., "__Inner"). When bases and keywords are scanned at lines 1309–1313, they are evaluated in the enclosing scope's symbol table, but name mangling uses the wrong class context.

For example, in:

class Outer:
    class __Inner(__Base):  # __Base should mangle with "Outer", not "__Inner"
        pass

The base class __Base would incorrectly be tracked as mangled with __Inner, producing _Inner__Base instead of _Outer__Base.

For generic classes this is not an issue since bases are scanned within the type_param scope (before line 1316's leave_scope). The fix is to restore self.class_name to prev_class before scanning bases in the non-generic case:

Proposed fix
             self.leave_scope();
             self.in_conditional_block = saved_in_conditional;
+            // Restore class_name to enclosing class for base/keyword scanning in non-generic case
+            if type_params.is_none() {
+                self.class_name = prev_class.clone();
+            }
             // Bases/keywords are scanned in type_param scope (if generic)
-            // class_name is still set, so mangling works via mangled_names filtering
+            // or enclosing scope (if non-generic)
             if let Some(arguments) = arguments {
🤖 Fix all issues with AI agents
In `@crates/codegen/src/symboltable.rs`:
- Around line 2638-2639: After trimming leading underscores from class_name (let
class_name = class_name.trim_start_matches('_');), add a check for an empty
result and skip mangling if empty: if class_name.is_empty() { return
original_name.to_string(); } so that classes made only of underscores (e.g.,
"__") do not trigger private-name mangling; update the code path that builds the
mangled identifier to return the unmodified identifier when class_name is empty.
🧹 Nitpick comments (1)
crates/vm/src/types/slot.rs (1)

536-545: Double MRO lookup for __iter__ in the non-None path.

When __iter__ exists and is not None, cls.get_attr() (line 539) performs an MRO lookup, and then vm.call_special_method() (line 544) performs a second MRO lookup for the same __iter__. This is functionally correct but redundant.

Since this wrapper is only invoked for heap types with a Python-defined __iter__, the overhead is likely negligible in practice. Consider caching the result if profiling ever shows this as a concern.

♻️ Optional: reuse the looked-up attribute to avoid the second MRO walk
 fn iter_wrapper(zelf: PyObjectRef, vm: &VirtualMachine) -> PyResult {
     // slot_tp_iter: if __iter__ is None, the type is explicitly not iterable
     let cls = zelf.class();
     let iter_attr = cls.get_attr(identifier!(vm, __iter__));
     match iter_attr {
         Some(attr) if vm.is_none(&attr) => {
             Err(vm.new_type_error(format!("'{}' object is not iterable", cls.name())))
         }
-        _ => vm.call_special_method(&zelf, identifier!(vm, __iter__), ()),
+        Some(attr) => attr.call((zelf,), vm),
+        None => vm.call_special_method(&zelf, identifier!(vm, __iter__), ()),
     }
 }

Comment on lines +2638 to +2639
// Strip leading underscores from class name
let class_name = class_name.trim_start_matches('_');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# First, check the actual code at lines 2638-2661 in symboltable.rs
sed -n '2630,2670p' crates/codegen/src/symboltable.rs

Repository: RustPython/RustPython

Length of output: 1120


🏁 Script executed:

#!/bin/bash
# Search for mangle_name usage and test cases
rg -n "mangle_name|mangle" crates/codegen/src/symboltable.rs | head -30
# Check for test files related to name mangling
fd -e rs -e py | xargs grep -l "mangle" | head -10

Repository: RustPython/RustPython

Length of output: 1805


🏁 Script executed:

#!/bin/bash
# Search for test files in the codegen directory
fd -e rs crates/codegen/src/ | xargs grep -l "test.*mangle\|mangle.*test" 2>/dev/null
# Search for any tests that use double-underscore names (which trigger mangling)
rg -n "__.*__" crates/codegen/src/ | grep -i "test\|def\|class" | head -20
# Check if there are any Python test files related to name mangling
fd test crates/codegen/ -type f

Repository: RustPython/RustPython

Length of output: 2865


🏁 Script executed:

#!/bin/bash
# Search for Python test files that might test name mangling
find . -path ./target -prune -o -name "*.py" -type f -exec grep -l "^class _" {} \; | head -10

# Check if there are any tests for name mangling behavior
rg -i "name.*mangle\|mangle.*test" Lib/test/ | head -20

# Also check the Python standard library for all-underscore class names
rg "^class _[_]+[^_]" --type py | head -10

Repository: RustPython/RustPython

Length of output: 5821


🏁 Script executed:

#!/bin/bash
# Check the context around the all-underscore class definitions
grep -B2 -A10 "^class __:" Lib/test/pyclbr_input.py
echo "---"
grep -B2 -A10 "^class ___:" Lib/test/pyclbr_input.py
echo "---"
grep -B2 -A10 "^class ___(object):" Lib/test/test_copyreg.py

Repository: RustPython/RustPython

Length of output: 772


🏁 Script executed:

#!/bin/bash
# Create a simple test script to verify CPython's behavior
python3 << 'EOF'
# Test CPython's name mangling with all-underscore class names
class __:
    def __foo(self):
        return "foo"
    
class ___:
    def __bar(self):
        return "bar"

# Check what the actual method names are after mangling
import inspect

# For class __
cls1 = __
methods1 = [m for m in dir(cls1) if not m.startswith('_') or m.startswith('__')]
print("Class __ methods:", [m for m in dir(cls1) if 'foo' in m])

# For class ___
cls2 = ___
print("Class ___ methods:", [m for m in dir(cls2) if 'bar' in m])

# The key question: are the names mangled or not?
# If they're mangled, they'd have the class name prefix
# If not mangled, they'd stay as __foo and __bar
EOF

Repository: RustPython/RustPython

Length of output: 121


🏁 Script executed:

#!/bin/bash
# Search for tests that use all-underscore class names with double-underscore methods
rg -n "class __|class ___" extra_tests/ Lib/test/ --type py

# Check if pyclbr_input.py is actually tested
rg -n "pyclbr_input" . --type py | head -10

# Check if there are specific tests for name mangling behavior
rg -n "mangle|__.*__" extra_tests/ --type py | grep -i "class\|def" | head -20

Repository: RustPython/RustPython

Length of output: 3837


Add empty-string check after stripping leading underscores from class name.

When class_name consists entirely of underscores (e.g., "__" or "___"), trim_start_matches('_') produces an empty string. CPython's mangling logic skips mangling in this case, but the current code proceeds to produce an incorrect mangled name.

For example, class __: with a method def __foo(self): should remain __foo (unmangled), not become ___foo.

Proposed fix
     // Strip leading underscores from class name
     let class_name = class_name.trim_start_matches('_');
+    if class_name.is_empty() {
+        return name.into();
+    }
     let mut ret = String::with_capacity(1 + class_name.len() + name.len());
🤖 Prompt for AI Agents
In `@crates/codegen/src/symboltable.rs` around lines 2638 - 2639, After trimming
leading underscores from class_name (let class_name =
class_name.trim_start_matches('_');), add a check for an empty result and skip
mangling if empty: if class_name.is_empty() { return original_name.to_string();
} so that classes made only of underscores (e.g., "__") do not trigger
private-name mangling; update the code path that builds the mangled identifier
to return the unmodified identifier when class_name is empty.

- TypeVar/ParamSpec repr: use infer_variance flag
- ParamSpec: add type_check on bound argument
- ParamSpecArgs/Kwargs: use equality instead of identity
- NoDefault: change to IMMUTABLETYPE flag
- subscript_generic: wrap TypeVarTuple in Unpack
- symboltable: selective name mangling in type param scopes
- symboltable: fix double default scanning for non-generic fns
- Unmark 4 passing tests in test_type_params
@github-actions
Copy link
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[ ] lib: cpython/Lib/concurrent
[ ] test: cpython/Lib/test/test_concurrent_futures (TODO: 18)
[ ] test: cpython/Lib/test/test_interpreters
[ ] test: cpython/Lib/test/test__interpreters.py
[ ] test: cpython/Lib/test/test__interpchannels.py
[ ] test: cpython/Lib/test/test_crossinterp.py

dependencies:

  • concurrent (native: _interpqueues, _interpreters, concurrent.futures, concurrent.futures._base, itertools, multiprocessing.connection, multiprocessing.queues, sys, time)
    • collections (native: _weakref, itertools, sys)
    • logging (native: atexit, errno, logging.handlers, sys, time)
    • multiprocessing (native: _multiprocessing, _winapi, array, atexit, collections.abc, errno, itertools, mmap, msvcrt, sys, time)
    • pickle (native: itertools, sys)
    • types
    • functools, os, queue, threading, traceback, weakref

dependent tests: (11 tests)

  • concurrent: test_asyncio test_concurrent_futures test_context test_genericalias
    • asyncio: test_asyncio test_contextlib_async test_inspect test_logging test_os test_sys_settrace test_unittest

[x] lib: cpython/Lib/typing.py
[ ] test: cpython/Lib/test/test_typing.py (TODO: 4)
[x] test: cpython/Lib/test/test_type_aliases.py
[x] test: cpython/Lib/test/test_type_annotations.py (TODO: 1)
[x] test: cpython/Lib/test/test_type_params.py (TODO: 6)
[x] test: cpython/Lib/test/test_genericalias.py

dependencies:

  • typing

dependent tests: (14 tests)

  • typing: test_annotationlib test_builtin test_enum test_fractions test_funcattrs test_functools test_genericalias test_grammar test_isinstance test_pydoc test_type_aliases test_type_params test_types test_typing

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@youknowone youknowone merged commit 38283be into RustPython:main Feb 14, 2026
14 checks passed
@youknowone youknowone deleted the typing branch February 14, 2026 00:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant