bpo-27413: add --no-ensure-ascii argument to json.tool#201
Conversation
|
Here a related issue from the issue tracker: Add an option to json.tool to bypass non-ASCII characters. This is my first time contributing to Python, so let me know if there's anything I need to do related to the issue tracker. |
Sorry, something went wrong.
|
Sorry, please file an issue to bugs.python.org. |
Sorry, something went wrong.
@methane no problem. I'll open an issue on bugs.python.org. Give me a couple days... currently traveling. |
Sorry, something went wrong.
serhiy-storchaka
left a comment
There was a problem hiding this comment.
The main problem -- output characters on the locale that doesn't support them.
Sorry, something went wrong.
e8a7cee to
da24e13
Compare
February 26, 2017 23:16
|
@serhiy-storchaka and @methane, thanks for the review thus far. It's been very helpful. Builds are now passing.
I'm not sure whether I've addressed this. Could you be a little more specific on the locale issues? Is this just something that affects the tests? |
Sorry, something went wrong.
|
$ LC_ALL=en_US.iso88591 ./python -m test.regrtest -v -m test_ensure_ascii test_json
...
======================================================================
FAIL: test_ensure_ascii (test.test_json.test_tool.TestTool)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/serhiy/py/cpython/Lib/test/test_json/test_tool.py", line 137, in test_ensure_ascii
self.assertEqual(expect.splitlines(), json_stdout.splitlines())
AssertionError: Lists differ: [b'"\\u00a7 \\ud83d\\udc0d \\u03b4 \\ud834\\udc37"'] != [b'"\\u00c2\\u00a7 \\u00f0\\u009f\\u0090\\u008d \\[39 chars]b7"']
First differing element 0:
b'"\\u00a7 \\ud83d\\udc0d \\u03b4 \\ud834\\udc37"'
b'"\\u00c2\\u00a7 \\u00f0\\u009f\\u0090\\u008d \\[38 chars]0b7"'
- [b'"\\u00a7 \\ud83d\\udc0d \\u03b4 \\ud834\\udc37"']
+ [b'"\\u00c2\\u00a7 \\u00f0\\u009f\\u0090\\u008d \\u00ce\\u00b4 \\u00f0\\u009d'
+ b'\\u0080\\u00b7"']
----------------------------------------------------------------------Other tests are passed by accident. Input data is encoded with UTF-8 and decoded with locale encoding in Try to pass escaped data not encodable with locale encoding. $ echo '["\u20ac"]' | LC_ALL=en_US.iso88591 ./python -m json.tool --no-ensure-ascii
Traceback (most recent call last):
File "/home/serhiy/py/cpython/Lib/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/serhiy/py/cpython/Lib/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/serhiy/py/cpython/Lib/json/tool.py", line 64, in <module>
main()
File "/home/serhiy/py/cpython/Lib/json/tool.py", line 58, in main
sort_keys=options.sort_keys,
File "/home/serhiy/py/cpython/Lib/json/__init__.py", line 180, in dump
fp.write(chunk)
UnicodeEncodeError: 'latin-1' codec can't encode character '\u20ac' in position 7: ordinal not in range(256) |
Sorry, something went wrong.
|
By the way, |
Sorry, something went wrong.
|
@berkerpeksag @serhiy-storchaka @methane: I'm excited to continue work on this pull request and now have a better grasp of the locale-dependent encoding issues that I still have to deal with. However, I was hoping that we could get the following PRs merged (in the following order since they're dependent):
Of course review thoroughly... just that it seems like review momentum has stalled. |
Sorry, something went wrong.
See python#201 (comment) picked from commit b4e9087)
See #201 (comment) picked from commit b4e9087)
From python#201 which is being split into two pull requests. See http://bugs.python.org/issue29636
|
To try and help move older pull requests forward, we are going through and backfilling 'awaiting' labels on pull requests that are lacking the label. Based on the current reviews, the best we can tell in an automated fashion is that a core developer requested changes to be made to this pull request. If/when the requested changes have been made, please leave a comment that says, |
Sorry, something went wrong.
…ptor" objects, if soft-switching is enabled. The code has been in Stackless forever, but was disabled for some unknown reason.
…ptor" objects, if soft-switching is enabled. The code has been in Stackless forever, but was disabled for some unknown reason.
|
Any chance of this getting merged? |
Sorry, something went wrong.
Work in progress, pending consensus on whether feature should be added
This pull request increases the utility of
python -m json.toolby adding support for settingensure_asciiandindentin thejson.dumpcall.Happy to also address any other arguments that need updating. Or other issues with
json.tool.