◐ Shell
reader mode source ↗
Skip to content

bpo-40059: tomllib#31498

Merged
encukou merged 13 commits into
python:mainfrom
hukkin:tomllib
Mar 8, 2022
Merged

bpo-40059: tomllib#31498
encukou merged 13 commits into
python:mainfrom
hukkin:tomllib

Conversation

@hukkin

@hukkin hukkin commented Feb 22, 2022 •
edited by encukou
Loading

Copy link
Copy Markdown
Contributor

This adds a new standard library module, tomllib, for parsing TOML. The recently accepted PEP 680 -- tomllib is relevant here.

This PR has already seen some review in a PR under my personal fork: hukkin#2 (thanks to @encukou, @merwok, @hauntsaninja, @JelleZijlstra (I hope I'm not forgetting anyone)).

The implementation is based on Tomli which I plan to keep maintaining as a backport for Python versions 3.7, 3.8, 3.9 and 3.10, until finally Python 3.10 goes EOL.

Steps taken (converting tomli to tomllib)

  • Move everything in tomli:src/tomli to Lib/tomllib. Exclude py.typed.

  • Remove __version__ = ... line from Lib/tomllib/__init__.py

  • Move everything in tomli:tests to Lib/test/test_tomllib. Exclude the following test data dirs recursively:

    • tomli:tests/data/invalid/_external/
    • tomli:tests/data/valid/_external/

  • Create Lib/test/test_tomllib/__main__.py:

    import unittest

from . import load_tests


unittest.main()

  • Add the following to Lib/test/test_tomllib/__init__.py:

    import os
from test.support import load_package_tests

def load_tests(*args):
    return load_package_tests(os.path.dirname(__file__), *args)

    Also change import tomli as tomllib to import tomllib.

  • In cpython/Lib/tomllib/_parser.py replace __fp with fp and __s with s. Add the / to load and loads function signatures.

  • Run make regen-stdlib-module-names

  • Create Doc/library/tomllib.rst and reference it in Doc/library/fileformats.rst

edit: For reference, there's one more step – Add tomllib to Makefile.pre.in

https://bugs.python.org/issue40059

@hukkin hukkin mentioned this pull request Feb 22, 2022
Closed
@hugovk hugovk added the type-feature A feature request or enhancement label Feb 22, 2022
@hukkin

hukkin commented Feb 22, 2022

Copy link
Copy Markdown
Contributor Author

A question: upstream (Tomli) is formatted with Black, using Black's defaults. This means a line length of 88. Should I reformat with line length at 79?

@hugovk

hugovk commented Feb 22, 2022

Copy link
Copy Markdown
Member

This will need NEWS and "What's new" entries:

You can add this to https://github.com/python/cpython/blob/main/.github/CODEOWNERS

@mgorny

mgorny commented Feb 22, 2022

Copy link
Copy Markdown
Contributor

Perhaps it'd make sense to include the tomli version used (or even the git commit hash), to make future syncing easier.

@hukkin

hukkin commented Feb 22, 2022

Copy link
Copy Markdown
Contributor Author

Perhaps it'd make sense to include the tomli version used (or even the git commit hash), to make future syncing easier.

It was previously suggested that I remove Tomli version from the standard library port: hukkin#2 (comment)

I plan to include a migration guide in Tomli repository, also including commit hash that tracks what is currently in the stdlib.

@TeamSpen210

TeamSpen210 commented Feb 22, 2022

Copy link
Copy Markdown

Would it be good to mention in the docs why load() takes only binary files? The encoding requirement probably isn't obvious for first-time users.

@hukkin hukkin mentioned this pull request Feb 22, 2022
Closed
@encukou

encukou commented Feb 23, 2022

Copy link
Copy Markdown
Member

I should get to the review this week or the next.

A question: upstream (Tomli) is formatted with Black, using Black's defaults. This means a line length of 88. Should I reformat with line length at 79?

No, I don't think that's worth it. And anyway, line length is not the only point where Black disagrees with PEP8 (starting with, like, the whole philosophy).
But future edits probably won't use Black.

Would it be good to mention in the docs why load() takes only binary files? The encoding requirement probably isn't obvious for first-time users.

Let's leave docs improvements to future PRs, so they get a proper discussion and don't delay this PR?

@astrojuanlu astrojuanlu mentioned this pull request Feb 23, 2022
Merged
@Contextualist Contextualist mentioned this pull request Feb 27, 2022
Merged
@hukkin @encukou
Update .github/CODEOWNERS …
d5e8053 
Co-authored-by: Petr Viktorin <encukou@gmail.com>
encukou
encukou approved these changes Mar 2, 2022
View reviewed changes

@encukou encukou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

I have a few test nitpicks, but I'm happy to merge this!

12 hidden items Load more…
@encukou encukou added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 3, 2022
@bedevere-bot

bedevere-bot commented Mar 3, 2022

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @encukou for commit 2898cc3 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 3, 2022
@brandtbucher brandtbucher mentioned this pull request Mar 3, 2022
Merged
@hukkin hukkin mentioned this pull request Mar 3, 2022
Merged
@hukkin

hukkin commented Mar 4, 2022

Copy link
Copy Markdown
Contributor Author

It seems the failing CI job (AMD64 Arch Linux Usan PR) errors in other PRs too so should be unrelated to this PR.

@encukou

encukou commented Mar 4, 2022

Copy link
Copy Markdown
Member

The alpha 6 release is a bit bumpy and I don't want to destabilize it, so I'm holding off the merge until it's out.
I tested with all buildbots to see if there's an unforeseen platform-specific issue. It's common to see a few buildbots fail.

@encukou encukou merged commit 591f675 into python:main Mar 8, 2022
@encukou

encukou commented Mar 8, 2022

Copy link
Copy Markdown
Member

Let's get it in!

@hukkin hukkin deleted the tomllib branch March 8, 2022 08:29
@mkniewallner mkniewallner mentioned this pull request Mar 8, 2022
Merged
@messense messense mentioned this pull request Mar 15, 2022
Closed
2 tasks
@DanielNoord DanielNoord mentioned this pull request Mar 15, 2022
Closed
@Pierre-Sassoulas Pierre-Sassoulas mentioned this pull request Mar 15, 2022
Closed
@mgorny mgorny mannequin mentioned this pull request Apr 10, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hauntsaninja hauntsaninja hauntsaninja left review comments

@encukou encukou encukou approved these changes

@pganssle pganssle Awaiting requested review from pganssle

@abalkin abalkin Awaiting requested review from abalkin

Assignees

No one assigned

Labels

type-feature A feature request or enhancement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@hukkin @hugovk @mgorny @TeamSpen210 @encukou @bedevere-bot @hauntsaninja @the-knights-who-say-ni