Issue 32419: Add unittest support for pyc projects

This PR makes it possible to "unittest" projects that are all pyc with no __init__.py which is hardcoded in few checks in unittest.loader

Could you tell more about these projects?  In current Python 3 pyc files are in a __pycache__ sub-directory, not directly in the package dir; what tool produces «pyc projects»?

Éric: in python3 if no .py is found but a .pyc is found where the .py is expected, it will be imported and run.  This supports sourceless package distributions, which is something we only sorta-support :)

The change looks simple enough that it might be worth doing, but I'll leave that to the unittest maintainers.  If accepted there should be comments as to why a .pyc is being checked for.

Hi Eric,

Yes it is like David said. For the projects you don't distribute the code
with the deployment package, you end up distributing only pyc files. As of
how to get these files, you can make use of py_compile or compileall
modules. For compileall, however, you have to pass the legacy flag to avoid
the __pycache__ standard.

What I found strange in the unittest code while trying to resolve this
issue is that, it is relying on hardcoded searches for __init__.py to find
if a module is loadable. I would have imagined that there is a standard why
to answer this question.

Best regards,
Bassem

Bassem Girgis, PhD

Email: brgirgis@gmail.com

On Sun, Dec 24, 2017 at 10:53 AM, Serhiy Storchaka <report@bugs.python.org>
wrote:

>
> Change by Serhiy Storchaka <storchaka+cpython@gmail.com>:
>
>
> ----------
> stage:  -> test needed
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue32419>
> _______________________________________
>

There may be now, but I don't think there was when unittest was written.  Also, if someone decided to use namespace packages for tests for some reason, the current check would also probably fail, so it may be worth looking for (or creating if needed) a standard way to check via importlib.

Please let me know what I need to do in order to close this issue and finally merge the code.

Whats the use for *unittest* - a tool to help folk develop - to run a test which is only present in sourceless form?

Oh, and why look for __init__ - in part, because it predates namespace packages, but also because unlike regular imports unittest will do negative things like reading the entire filesystem otherwise.

I Robert,

One use scenario is to run deployment tests for sourceless packages and to
ensure the integrity of the production environment. Very useful in
container environments!

In regards to the looking for the __init__ file, it is obvious that the
logic you mentioned is the reason behind that. However, if the unittest
code used a standard way to figure out "loadable" packages, whatever that
is, it would have captured the fact that python3 can indeed function with
all pyc files.

I would say the fix in here is a dirty one. A better way to solve it is to
call a standard utility which can detect if a package is loadable. This
would resolve this issue and help with any future changes in the standard
as well.

Best regards,
Bassem

Bassem Girgis, PhD

Email: brgirgis@gmail.com

On Sat, Mar 31, 2018 at 8:52 PM, Robert Collins <report@bugs.python.org>
wrote:

>
> Robert Collins <robertc@robertcollins.net> added the comment:
>
> Oh, and why look for __init__ - in part, because it predates namespace
> packages, but also because unlike regular imports unittest will do negative
> things like reading the entire filesystem otherwise.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue32419>
> _______________________________________
>

Seems to be a lot of overlap with Issue 26859 by Xavier. Looks like Xavier included a test case, but Bassemʼs changes on Git Hub seem more thorough in the impementation and doc strings.

components: + Library (Lib), - Extension Modules
versions: - Python 3.6, Python 3.8