Issue 16379: SQLite error code not exposed to python

The sqlite3 module does not expose the sqlite3 error codes to python. This makes it impossible to detect specific error conditions directly.

Case in point: If a user selects some random file as the database in our application, we can not detect that it is not a valid database file:

$ /opt/python3/bin/python3
Python 3.4.0a0 (default:2d6eec5d01f7, Nov  1 2012, 10:47:27) 
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> conn = sqlite3.connect("/etc/passwd")
>>> from pprint import pprint
>>> try:
...     conn.execute("select * from random_table")
... except Exception as e:
...     pprint({name: getattr(e, name) for name in dir(e)})
...     raise
... 
{'__cause__': None,
 '__class__': <class 'sqlite3.DatabaseError'>,
 '__context__': None,
 '__delattr__': <method-wrapper '__delattr__' of DatabaseError object at 0x7ffc9a13b050>,
 '__dict__': {},
 '__dir__': <built-in method __dir__ of DatabaseError object at 0x7ffc9a13b050>,
 '__doc__': None,
 '__eq__': <method-wrapper '__eq__' of DatabaseError object at 0x7ffc9a13b050>,
 '__format__': <built-in method __format__ of DatabaseError object at 0x7ffc9a13b050>,
 '__ge__': <method-wrapper '__ge__' of DatabaseError object at 0x7ffc9a13b050>,
 '__getattribute__': <method-wrapper '__getattribute__' of DatabaseError object at 0x7ffc9a13b050>,
 '__gt__': <method-wrapper '__gt__' of DatabaseError object at 0x7ffc9a13b050>,
 '__hash__': <method-wrapper '__hash__' of DatabaseError object at 0x7ffc9a13b050>,
 '__init__': <method-wrapper '__init__' of DatabaseError object at 0x7ffc9a13b050>,
 '__le__': <method-wrapper '__le__' of DatabaseError object at 0x7ffc9a13b050>,
 '__lt__': <method-wrapper '__lt__' of DatabaseError object at 0x7ffc9a13b050>,
 '__module__': 'sqlite3',
 '__ne__': <method-wrapper '__ne__' of DatabaseError object at 0x7ffc9a13b050>,
 '__new__': <built-in method __new__ of type object at 0x8267e0>,
 '__reduce__': <built-in method __reduce__ of DatabaseError object at 0x7ffc9a13b050>,
 '__reduce_ex__': <built-in method __reduce_ex__ of DatabaseError object at 0x7ffc9a13b050>,
 '__repr__': <method-wrapper '__repr__' of DatabaseError object at 0x7ffc9a13b050>,
 '__setattr__': <method-wrapper '__setattr__' of DatabaseError object at 0x7ffc9a13b050>,
 '__setstate__': <built-in method __setstate__ of DatabaseError object at 0x7ffc9a13b050>,
 '__sizeof__': <built-in method __sizeof__ of DatabaseError object at 0x7ffc9a13b050>,
 '__str__': <method-wrapper '__str__' of DatabaseError object at 0x7ffc9a13b050>,
 '__subclasshook__': <built-in method __subclasshook__ of type object at 0x1238770>,
 '__suppress_context__': False,
 '__traceback__': <traceback object at 0x7ffc9a138cf8>,
 '__weakref__': None,
 'args': ('file is encrypted or is not a database',),
 'with_traceback': <built-in method with_traceback of DatabaseError object at 0x7ffc9a13b050>}
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
sqlite3.DatabaseError: file is encrypted or is not a database
>>>

Currently, one can only match the error message, with is bad programming style.

The error code for this error is SQLITE_NOTADB, as found in the function sqlite3ErrStr when searching for the message in SQLite's main.c at http://www.sqlite.org/src/artifact/02255cf1da50956c5427c469abddb15bccc4ba09

Unfortunately, the sqlite3 module does not expose the error code itself (neither the actual error code nor the defined error codes) in any way. Errors are handled in Modules/_sqlite/util.c:

http://hg.python.org/cpython/file/2d6eec5d01f7/Modules/_sqlite/util.c#l99

I would like to have the defined error codes available in some mapping inside the sqlite3 module as well as the actual error code inside every sqlite exception e as e.sqlite_errcode

Attached patch that:

- Adds sqlite3.SQLITE_OK constants
- Adds sqlite3.errorcode dictionary
- Adds "sqlite_errcode" attribute to sqlite.Error instances

The patch does not add support for extended result codes
(http://www.sqlite.org/c3ref/c_abort_rollback.html).

I didn't compile-test the Doc/ part of the patch.

New patch, with better docs and less error leaks, per Ezio's review.

A couple of random eyebrow-raisers I noticed while working on v2:

- sqlite3.Warning is a subclass of Exception, rather than sqlite3.Error
  or builtins.Warning.  (Also, the docs say "will raise a Warning",
  intending to refer to sqlite3.Warning, but the lack of markup makes
  this ambiguous --- it would be interpreted as a reference to
  builtins.Warning).

- If _PyUnicode_AsStringAndSize() returns NULL, the code sets a
  sqlite3.Warning exception without first checking what exception is
  already set.  (grep for PYSQLITE_SQL_WRONG_TYPE)

New patch fixing indentation of versionadded markup.

Any update on this? Still seems to be a problem as of 3.4.0.

I propose to also set the SQLite extended error code if this is implemented.

What's the reasoning behind offering a error code to name mapping? This seem problematic to me. In case a newer SQLite version introduces a new error code, this error code cannot be found in the mapping. I propose to leave this out in order to not have this problem.

Otherwise we will have people depending on any error code being able to be found in this mapping.

> What's the reasoning behind offering a error code to name mapping?

Allowing code that runs into an error to print the error name rather than its
numeric value.  This saves whoever reads the error message having to look it up
himself.

> his seem problematic to me. In case a newer SQLite version introduces a new
> error code, this error code cannot be found in the mapping. I propose to
> leave this out in order to not have this problem.
> 
> Otherwise we will have people depending on any error code being able to be
> found in this mapping.

Then people shouldn't depend on the mapping being complete.  Let's keep the
mapping and document that people should only use it as
`sqlite3.errorcode.get(...)`, never as `sqlite3.errorcode[...]`.

Or if that's not a good API, we could encapsulate the incompleteness of the
mapping into a small wrapper function:

   def something(errorcode):
       return sqlite3.errorcode.get(errorcode,
                                    "<sqlite3 error {!d}>".format(errorcode))

I think the error message should display both the numeric code and also the error name if available.
Instead of using a mapping, perhaps an Enum could be used instead.

Attached is a patch based on Daniel last patch with the following changes:

* There is no errorcode mapping.
* The exception object has a error_name attribute.

I think this two changes should solve the API problems raised by Ezio and Gerhard about the error code mapping.

> I propose to also set the SQLite extended error code if this is implemented.

I think that this should be done in a separate patch. I will start working on the extended error code and will upload a patch to issue 24139.

Attached is a new patch with the encoding problem fixed.

What would it take to get this merged?

> What would it take to get this merged?

I've rebased Aviv's PR (GH-1108) onto main and resolved the conflicts. If I get his blessing, I'll open a PR try to land this.

>> What would it take to get this merged?
>
> I've rebased Aviv's PR (GH-1108) onto main and resolved the conflicts.
> If I get his blessing, I'll open a PR try to land this.

I asked Aviv on GH eight days ago, and I haven't heard anything yet, so I'm going forward with this. For the record, here's a link to my question on the PR:
https://github.com/python/cpython/pull/1108#issuecomment-894506420

I would also like to see this feature; it makes sqlite3 development slightly more convenient :)

New changeset 86d8b465231473f850cc5e906013ba8581ddb503 by Erlend Egeberg Aasland in branch 'main':
bpo-16379: expose SQLite error codes and error names in `sqlite3` (GH-27786)
https://github.com/python/cpython/commit/86d8b465231473f850cc5e906013ba8581ddb503

Thanks, Torsten for this nice suggestion, Daniel & Aviv for the initial patches, Gerhard & Ezio for helping improving the API, and Pablo, Asif, Hai Shi, & Michael for reviewing and merging!

New changeset 8deb7afbaaaad847656842375119f8dbef8aea54 by Erlend Egeberg Aasland in branch 'main':
bpo-16379: Fix SQLite version checks in test_module_constants() (GH-28809)
https://github.com/python/cpython/commit/8deb7afbaaaad847656842375119f8dbef8aea54

stage: patch review -> resolved

messages: + msg274064

messages: + msg178601

messages: + msg178597

stage: needs patch