Message 390599 - Python tracker

➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author	methane
Recipients	methane
Date	2021-04-09.08:49:42
SpamBayes Score	-1.0
Marked as misclassified	Yes
Message-id	<1617958183.01.0.290245206759.issue43787@roundup.psfhosted.org>
In-reply-to

Content
__iter__ method of BZ2File, GzipFile, and LZMAFile is IOBase.__iter__. It calls `readline()` for each line. Since `readline()` is defined as Python function, it is slower than C iterator. Adding custom __iter__ method that delegates to underlying buffer __iter__ makes `for line in file` 2x faster. def __iter__(self): self._check_can_read() return self._buffer.__iter__() --- The original issue is reported here. https://discuss.python.org/t/non-optimal-bz2-reading-speed/6869 This issue is relating to #43785.

__iter__ method of BZ2File, GzipFile, and LZMAFile is IOBase.__iter__. It calls `readline()` for each line.

Since `readline()` is defined as Python function, it is slower than C iterator. Adding custom __iter__ method that delegates to underlying buffer __iter__ makes `for line in file` 2x faster.

    def __iter__(self):
        self._check_can_read()
        return self._buffer.__iter__()

---

The original issue is reported here.
https://discuss.python.org/t/non-optimal-bz2-reading-speed/6869
This issue is relating to #43785.

History
Date	User	Action	Args
2021-04-09 08:49:43	methane	set	recipients: + methane
2021-04-09 08:49:43	methane	set	messageid: <1617958183.01.0.290245206759.issue43787@roundup.psfhosted.org>
2021-04-09 08:49:42	methane	link	issue43787 messages
2021-04-09 08:49:42	methane	create