Issue 8715: Create PyUnicode_EncodeFSDefault() function

PyUnicode_EncodeFSDefault() is the opposite of PyUnicode_DecodeFSDefault(AndSize)() and is similar to the new function os.fsencode(). As you can see in the patch, it simplifies many functions.

/* Encodes a Unicode object to Py_FileSystemDefaultEncoding with the
   "surrogateescape" error handler and returns a bytes object.

   If Py_FileSystemDefaultEncoding is not set, fall back to UTF-8.
*/

PyAPI_FUNC(PyObject*) PyUnicode_EncodeFSDefault(
    PyObject *unicode
    );

The function unify the behaviour when Py_FileSystemDefaultEncoding is NULL: use UTF-8 whereas import uses ASCII. Other functions did already fall back to UTF-8: PyUnicode_AsEncodedString() uses PyUnicode_GetDefaultEncoding() (hardcoded to utf8 in Python3) if encoding is NULL

The patch does also fix tkinter module initializer (use surrogateescape error handler, instead of strict).

The patch was first attached to issue #8611.

Ooops, I attached the wrong version of the patch. Version 3 changes the documentation (Encodes => Encode).

Notes for myself:
 - "Encodes" and "fallback" in .h documentation  => "Encode", "fall back"
 - bootstrap failure on Windows: import did use default error handler, it uses surrogateescape error handler, but PyUnicode_EncodeString() doesn't have codec "fast-path" for MBCS+surrogateescape.

> bootstrap failure on Windows: import did use default error handler, 
> it uses surrogateescape error handler, but PyUnicode_EncodeString()
> doesn't have codec "fast-path" for MBCS+surrogateescape.

I enabled "shortcuts" in PyUnicode_EncodeString() for any error handler (not only the default error handler, strict) in r81192.

PyUnicode_AsEncodedString() contains a special path for the file system encoding. I don't think that it is still needed, but I don't know how to check that.

    /* During bootstrap, we may need to find the encodings
       package, to load the file system encoding, and require the
       file system encoding in order to load the encodings
       package.

       Break out of this dependency by assuming that the path to
       the encodings module is ASCII-only.  XXX could try wcstombs
       instead, if the file system encoding is the locale's
       encoding. */
    else if (Py_FileSystemDefaultEncoding &&
             strcmp(encoding, Py_FileSystemDefaultEncoding) == 0 &&
             !PyThreadState_GET()->interp->codecs_initialized)
        return PyUnicode_EncodeASCII(PyUnicode_AS_UNICODE(unicode),
                                     PyUnicode_GET_SIZE(unicode),
                                     errors);

Commited as r81194 (py3k), blocked in 3.1 (r81195).

messages: + msg105722