◐ Shell
reader mode source ↗
Skip to content

Updated token + weakref libraries + weakref test - v3.13.11#6633

Merged
youknowone merged 6 commits into
RustPython:mainfrom
terryluan12:update_token_weakref
Jan 4, 2026
Merged

Updated token + weakref libraries + weakref test - v3.13.11#6633
youknowone merged 6 commits into
RustPython:mainfrom
terryluan12:update_token_weakref

Conversation

@terryluan12

@terryluan12 terryluan12 commented Jan 3, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@coderabbitai

coderabbitai Bot commented Jan 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters (4)
  • Lib/test/_test_multiprocessing.py is excluded by !Lib/**
  • Lib/test/test_weakref.py is excluded by !Lib/**
  • Lib/token.py is excluded by !Lib/**
  • Lib/weakref.py is excluded by !Lib/**

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@terryluan12 terryluan12 changed the title Updated token + weakref libraries Jan 3, 2026
@terryluan12

terryluan12 commented Jan 3, 2026

Copy link
Copy Markdown
Contributor Author

I could not find a token test file

@youknowone

youknowone commented Jan 3, 2026

Copy link
Copy Markdown
Member

by searching import token, test_tokenize seems to test it

@terryluan12

terryluan12 commented Jan 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

by searching import token, test_tokenize seems to test it

I saw that, & assumed that it would test out the tokenize library, but as I'm writing it out I'm realizing I should probably jointly update the two libraries. Will add it, thanks!

@terryluan12 terryluan12 changed the title Updated token + weakref libraries + weakref test Jan 3, 2026
@terryluan12

terryluan12 commented Jan 3, 2026

Copy link
Copy Markdown
Contributor Author

So it seems a rewrite of tokenize has been done. the _tokenize function was moved to a precompiled library, and logic has been changed, so it's not as simple as keeping the older _tokenize version. Is it okay to merge, without making any tokenization migrations?

@terryluan12

terryluan12 commented Jan 3, 2026

Copy link
Copy Markdown
Contributor Author

It's a bit of a different error, but waiting until #6635 is merged until debugging

@terryluan12 terryluan12 mentioned this pull request Jan 3, 2026
Closed
@youknowone youknowone force-pushed the update_token_weakref branch from cd1bf05 to c961277 Compare January 4, 2026 00:27
youknowone
youknowone approved these changes Jan 4, 2026
View reviewed changes

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

👍

Hide details View details @youknowone youknowone merged commit 68d65a7 into RustPython:main Jan 4, 2026
12 of 13 checks passed
terryluan12 added a commit to terryluan12/RustPython that referenced this pull request Jan 5, 2026
@terryluan12
Updated token + weakref libraries + weakref test - v3.13.11 (RustPyth… …
313e630 
…on#6633)

* Updated the token library

* Updated the weakref library

* Annotated the failing weakref test
terryluan12 added a commit to terryluan12/RustPython that referenced this pull request Jan 5, 2026
@terryluan12
Updated token + weakref libraries + weakref test - v3.13.11 (RustPyth… …
211f508 
…on#6633)

* Updated the token library

* Updated the weakref library

* Annotated the failing weakref test
terryluan12 added a commit to terryluan12/RustPython that referenced this pull request Jan 5, 2026
@terryluan12
Updated token + weakref libraries + weakref test - v3.13.11 (RustPyth… …
e4d29fd 
…on#6633)

* Updated the token library

* Updated the weakref library

* Annotated the failing weakref test
@terryluan12 terryluan12 deleted the update_token_weakref branch January 5, 2026 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@youknowone youknowone youknowone approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@terryluan12 @youknowone