Issue 44822: [sqlite3] Do not truncate results of user functions and aggregates on the first NUL
Created on 2021-08-04 08:29 by erlendaasland, last changed 2022-04-11 14:59 by admin. This issue is now closed.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 27588 | merged | erlendaasland, 2021-08-04 08:33 | |
| PR 27611 | merged | miss-islington, 2021-08-05 07:22 | |
| PR 27639 | merged | erlendaasland, 2021-08-06 18:52 | |
| Messages (8) | |||
|---|---|---|---|
| msg398865 - (view) | Author: Erlend E. Aasland (erlendaasland) *  |
Date: 2021-08-04 08:29 | |
The third argument to sqlite3_result_text() is the length of the string passed as the second argument. Currently, we pass -1, so SQLite has to invoke strlen() to compute the length of the passed string. Suggesting to use PyUnicode_AsUTF8AndSize() iso. PyUnicode_AsUTF8() and pass the string size to avoid the superfluous strlen(). See also: - https://sqlite.org/c3ref/result_blob.html |
|||
| msg398869 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) *  |
Date: 2021-08-04 09:01 | |
The difference between specifying negative and non-negative third argument of sqlite3_result_text() is that in the latter case the result can contain embedded NUL characters. Could you please add a test for string containing embedded NUL? Letting SQLite compute string length would work incorrect in case of embedded NULs, so I think that we should raise explicit error if sz > INT_MAX. |
|||
| msg398873 - (view) | Author: Erlend E. Aasland (erlendaasland) * |
Date: 2021-08-04 09:06 | |
> Could you please add a test for string containing embedded NUL? Of course, thanks for the heads up. > I think that we should raise explicit error if sz > INT_MAX. Yes, I thought about this. I do agree that raising OverflowError would be better. Thanks. |
|||
| msg398881 - (view) | Author: Erlend E. Aasland (erlendaasland) * |
Date: 2021-08-04 10:03 | |
Serhiy, I've updated the PR, if you want to take a look. Thanks for your feedback. |
|||
| msg398971 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2021-08-05 07:22 | |
New changeset 8f010dc920e1f6dc6a357e7cc1460a7a567c05c6 by Erlend Egeberg Aasland in branch 'main': bpo-44822: Don't truncate `str`s with embedded NULL chars returned by `sqlite3` UDF callbacks (GH-27588) https://github.com/python/cpython/commit/8f010dc920e1f6dc6a357e7cc1460a7a567c05c6 |
|||
| msg399115 - (view) | Author: miss-islington (miss-islington) | Date: 2021-08-06 18:00 | |
New changeset 2b1e713f877102bbca299f0f5d7db969d78db49f by Miss Islington (bot) in branch '3.10': bpo-44822: Don't truncate `str`s with embedded NULL chars returned by `sqlite3` UDF callbacks (GH-27588) https://github.com/python/cpython/commit/2b1e713f877102bbca299f0f5d7db969d78db49f |
|||
| msg399146 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2021-08-06 21:02 | |
New changeset c352412123140e79dcce6188d17e3e6dbc3f4144 by Erlend Egeberg Aasland in branch '3.9': [3.9] bpo-44822: Don't truncate `str`s with embedded NULL chars returned by `sqlite3` UDF callbacks (GH-27588). (GH-27639) https://github.com/python/cpython/commit/c352412123140e79dcce6188d17e3e6dbc3f4144 |
|||
| msg399149 - (view) | Author: Erlend E. Aasland (erlendaasland) * |
Date: 2021-08-06 21:08 | |
Thanks, Serhiy! |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:59:48 | admin | set | github: 88985 |
| 2021-08-06 21:08:39 | erlendaasland | set | status: open -> closed resolution: fixed messages: + msg399149 stage: patch review -> resolved |
| 2021-08-06 21:02:24 | serhiy.storchaka | set | messages: + msg399146 |
| 2021-08-06 18:52:57 | erlendaasland | set | pull_requests: + pull_request26133 |
| 2021-08-06 18:00:04 | miss-islington | set | messages: + msg399115 |
| 2021-08-05 07:22:30 | serhiy.storchaka | set | messages: + msg398971 |
| 2021-08-05 07:22:17 | miss-islington | set | nosy:
+ miss-islington pull_requests: + pull_request26105 |
| 2021-08-04 10:46:38 | serhiy.storchaka | set | title: [sqlite3] Micro-optimisation: pass string size to sqlite3_result_text() -> [sqlite3] Do not truncate results of user functions and aggregates on the first NUL type: enhancement -> behavior versions: + Python 3.9, Python 3.10 |
| 2021-08-04 10:03:10 | erlendaasland | set | messages: + msg398881 |
| 2021-08-04 09:06:42 | erlendaasland | set | messages: + msg398873 |
| 2021-08-04 09:01:35 | serhiy.storchaka | set | nosy:
+ serhiy.storchaka messages: + msg398869 |
| 2021-08-04 08:33:55 | erlendaasland | set | keywords:
+ patch stage: patch review pull_requests: + pull_request26090 |
| 2021-08-04 08:29:38 | erlendaasland | create | |