fn tokenize(expr: &str) -> Vec<Token>Expand description
Tokenizes a Python type expression into tokens.
Handles:
- Identifiers:
ClassA,int,str - Dotted paths:
typing.Optional,collections.abc.Callable - Brackets:
[,],(,) - Special characters:
,,|,... - String literals:
"ForwardRef" - Whitespace preservation