[3.4] Backport CI config from master#2475

vstinner

Add .travis.yml for Travis CI
Add .github/ for AppVeyor and CodeCov.

vstinner

Ok, here is my first attempt for enable Travis CI and AppVeyor to the 3.4 branch. It should help to test automatically backported security fixes.

I'm not sure about enabling CodeCov. Do we need CodeCov on 3.4?

Once this PR will work and be merged, I will backport it to the 3.3 branch.

vstinner

The doc job failed with:

Warning, treated as error:
unknown config value 'latex_paper_size' in override, ignoring

=> need to backport the commit e7ffb99.

vstinner

The Travis CI job failed on:

make: *** No rule to make target `regen-all'.  Stop.

Need to remove this from .travis.yml.

At least, the compilation succeeded ;-)

AppVeyor failed on:

C:\projects\cpython>"C:\projects\cpython\PCbuild\\python"  -Wd -E -bb "C:\projects\cpython\PCbuild\..\lib\test\regrtest.py" -uall -u-cpu -rwW --slowest --timeout 1200 -j0   
...
regrtest.py: error: unrecognized arguments: --slowest

Need to replace the option with --slow.

vstinner

I removed CodeCov config from .github/ and tried to fix CI failures.

vstinner

New failure of the Travis CI doc job:

Warning, treated as error:
/home/travis/build/python/cpython/Doc/library/xml.dom.minidom.rst:250:Footnote [#] is not referenced.

vstinner

Failure of the Travis CI python job:

./python  ./Tools/scripts/run_tests.py -j 1 -u all -W --timeout=3600 -j4 -uall,-cpu,-tzdata
...
regrtest.py: error: argument -u/--use: invalid resource: tzdata

zware

Looks mostly fine, a couple of things pointed out below.

vstinner

I removed the Travis CI doc job. It requires too many changes to fix it. The doc wasn't tested, so it's not like it's a regresion. Moreover, security fixes don't touch Doc/ usually, so it should be fine.

zware

I removed the Travis CI doc job. It requires too many changes to fix it.

In that case, let's remove the changes in Doc/ as well.

vstinner

4 tests failed on Windows:

======================================================================
ERROR: test_logincapa_with_client_certfile (test.test_imaplib.RemoteIMAP_SSLTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\projects\cpython\lib\test\test_imaplib.py", line 473, in test_logincapa_with_client_certfile
    _server = self.imap_class(self.host, self.port, certfile=CERTFILE)
  File "C:\projects\cpython\lib\imaplib.py", line 1222, in __init__
    IMAP4.__init__(self, host, port)
  File "C:\projects\cpython\lib\imaplib.py", line 182, in __init__
    self.open(host, port)
  File "C:\projects\cpython\lib\imaplib.py", line 1235, in open
    IMAP4.open(self, host, port)
  File "C:\projects\cpython\lib\imaplib.py", line 258, in open
    self.sock = self._create_socket()
  File "C:\projects\cpython\lib\imaplib.py", line 1227, in _create_socket
    server_hostname=self.host)
  File "C:\projects\cpython\lib\ssl.py", line 368, in wrap_socket
    _context=self)
  File "C:\projects\cpython\lib\ssl.py", line 586, in __init__
    self.do_handshake()
  File "C:\projects\cpython\lib\ssl.py", line 813, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:600)
======================================================================
ERROR: test_logincapa_with_client_ssl_context (test.test_imaplib.RemoteIMAP_SSLTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\projects\cpython\lib\test\test_imaplib.py", line 478, in test_logincapa_with_client_ssl_context
    _server = self.imap_class(self.host, self.port, ssl_context=self.create_ssl_context())
  File "C:\projects\cpython\lib\imaplib.py", line 1222, in __init__
    IMAP4.__init__(self, host, port)
  File "C:\projects\cpython\lib\imaplib.py", line 182, in __init__
    self.open(host, port)
  File "C:\projects\cpython\lib\imaplib.py", line 1235, in open
    IMAP4.open(self, host, port)
  File "C:\projects\cpython\lib\imaplib.py", line 258, in open
    self.sock = self._create_socket()
  File "C:\projects\cpython\lib\imaplib.py", line 1227, in _create_socket
    server_hostname=self.host)
  File "C:\projects\cpython\lib\ssl.py", line 368, in wrap_socket
    _context=self)
  File "C:\projects\cpython\lib\ssl.py", line 586, in __init__
    self.do_handshake()
  File "C:\projects\cpython\lib\ssl.py", line 813, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:600)
----------------------------------------------------------------------

======================================================================
FAIL: test_walk_bottom_up (test.test_os.WalkTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\projects\cpython\lib\test\test_os.py", line 828, in test_walk_bottom_up
    self.sub2_tree)
AssertionError: Tuples differ: ('@test_856_tmp\\TEST1\\SUB2', ['broken_link', 'link'], ['tmp3']) != ('@test_856_tmp\\TEST1\\SUB2', ['link'], ['broken_link', 'tmp3'])
First differing element 1:
['broken_link', 'link']
['link']
- ('@test_856_tmp\\TEST1\\SUB2', ['broken_link', 'link'], ['tmp3'])
?                                                ----------
+ ('@test_856_tmp\\TEST1\\SUB2', ['link'], ['broken_link', 'tmp3'])
?                                ++++++++++
======================================================================
FAIL: test_walk_prune (test.test_os.WalkTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\projects\cpython\lib\test\test_os.py", line 808, in test_walk_prune
    self.assertEqual(all[1], self.sub2_tree)
AssertionError: Tuples differ: ('@test_856_tmp\\TEST1\\SUB2', ['broken_link', 'link'], ['tmp3']) != ('@test_856_tmp\\TEST1\\SUB2', ['link'], ['broken_link', 'tmp3'])
First differing element 1:
['broken_link', 'link']
['link']
- ('@test_856_tmp\\TEST1\\SUB2', ['broken_link', 'link'], ['tmp3'])
?                                                ----------
+ ('@test_856_tmp\\TEST1\\SUB2', ['link'], ['broken_link', 'tmp3'])
?                                ++++++++++
======================================================================
FAIL: test_walk_topdown (test.test_os.WalkTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\projects\cpython\lib\test\test_os.py", line 791, in test_walk_topdown
    self.assertEqual(all[3 - 2 * flipped], self.sub2_tree)
AssertionError: Tuples differ: ('@test_856_tmp\\TEST1\\SUB2', ['broken_link', 'link'], ['tmp3']) != ('@test_856_tmp\\TEST1\\SUB2', ['link'], ['broken_link', 'tmp3'])
First differing element 1:
['broken_link', 'link']
['link']
- ('@test_856_tmp\\TEST1\\SUB2', ['broken_link', 'link'], ['tmp3'])
?                                                ----------
+ ('@test_856_tmp\\TEST1\\SUB2', ['link'], ['broken_link', 'tmp3'])
?                                ++++++++++
----------------------------------------------------------------------

vstinner

On Travis CI, the same 2 test_imaplib tests failed.

vstinner

@zware: "In that case, let's remove the changes in Doc/ as well."

Oh, it's a mistake. I reverted most Doc changes, but I forgot one. It's now fixed.

I backported another fix for test_imaplib which removes the 2 failing tests.

vstinner

Oh yeah! Travis CI now pass. AppVeyor will likely fail in test_os again, I didn't identifier the right commit to cherry-pick yet.

@serhiy-storchaka: Do you recall which commit fixed test_os random failures on os.walk on Windows? see #2475 (comment) failures

serhiy-storchaka

See bpo-23808 and bpo-25911. Seems this issue was not fixed in 3.4.

Maybe removing the True argument from os.symlink('broken', broken_link_path, True) could help.

brettcannon

This review is only for the Travis file.

vstinner

Ok, the Python test suite now pass on Linux (Travis CI) and Windows (AppVeyor).

Summary of the PR:

Backport Travis CI and AppVeyor configuration
Fix 4 failing tests in test_os and test_smtplib: backport 2 changes which only impact tests, not the stdlib
Hum, that's all :-)

This PR doesn't contain:

CodeCov configuration: I consider that it's useless for 3.4, it's not like we want to enhance the code coverage in this branch
CI docs tests: fixing the doc job (fix warnings, not errors, for Sphinx 1.6.1) requires to backport 3 big changes which creates a lot of conflicts, and so there is a high risk of introducing bugs in the doc ... whereas the purpose of a CI is to prevent mistakes :-)

@brettcannon: I really prefer to fix all CI issues in a single PR, and not have to handle 2 or 3 PR to get a working CI.

@zware: Would you mind to review this PR? You worked on these CI recently, so you should be able to check it :-)

@larryhastings: Would you mind to review this PR? It should help to be more confident when merging security fixes in 3.4, since the 3.4 buildbots were removed.

Thanks @serhiy-storchaka, I backported you bpo-23808 fix and it seems like it was just enough ;-)

zware

LGTM, though I have added a suggestion.

brettcannon

@Haypo I have no problem with rolling up the fixes to go with the .travis.yml file. All I meant was that I only looked at the .travis.yml file and not the fixes themselves.

larryhastings

FWIW, there's some automation when cutting releases that removes dotfiles we don't want to leak into Python release tarballs. New dotfiles like .travis.yaml (and ".mention-bot", and ".github") need to be explicitly pruned. I'm gonna follow up and make sure this happens for 3.6.2 final and 3.6.4rc1. In the future, we release managers would appreciate a heads-up warning us when people add new dotfiles to the root of cpython, so we can make sure we don't ship them.

vstinner

FWIW, there's some automation when cutting releases that removes dotfiles we don't want to leak into Python release tarballs.

I guess that you use some scripts, are they public?

brettcannon

@Haypo my guess is https://github.com/python/release-tools, but there's no explicit dot file list that I see there so I may be wrong

larryhastings

Brett's link to release-tools is the right place. The logic to strip the dotfiles is hidden inside. But please don't touch it right now. @ned-deily has done a lot of work on it to update it to Github, but he hasn't checked in those changes yet. I appreciate you being willing to fix release-tools when you add new dotfiles but please wait until release.py doesn't call "hg" anymore ;-)

vstinner

@larryhastings: Please see my summary of this PR, #2475 (comment)

Technically, I don't think that you can merge a PR if the CI fails. So it's not posisble to add the CI config in one PR and then fix failing tests. I could do the opposite, but it's not possible to make a PR depends on another one. Moreover, I used this issue to see which tests on the CI and test my fixes.

vstinner

I just rebased this PR on top of the 3.4 branch, since many security PRs just have been merged. So it would help to check if everything is fine on 3.4 :-)

vstinner

I just rebased this PR on top of the 3.4 branch, since many security PRs just have been merged. So it would help to check if everything is fine on 3.4 :-)

Oh, everything is not fine, test_subprocess fails: http://bugs.python.org/issue30730#msg298218

(Regression of PR #2362, commit fe82c46).

serhiy-storchaka

Oh, I forgot that null character/byte errors were of type TypeError before 3.5. The simplest fix is changing corresponding ValueError in self.assertRaises() to the tuple (ValueError, TypeError). You can add 5e22721 to this PR.

* Add .travis.yml for Travis CI * Add .github/ for AppVeyor and CodeCov.

The regen-all Makefile rule doesn't exist in Python 3.4, only since Python 3.5 and newer (and 2.7).

tzdata resource doesn't exist in Python 3.4.

Fixing Sphinx warnings requires to backport huge intrusive changes like: - commit d97b7dc - commit 5c67933

The public cyrus.andrew.cmu.edu IMAP server (port 993) doesn't accept TLS connection using our self-signed x509 certificate. Remove the two tests which are already skipped. (cherry picked from commit 7895a05)

On Windows a symlink can has the FILE_ATTRIBUTE_DIRECTORY flag. (cherry picked from commit 388b90f)

Fix test_invalid_cmd() and test_invalid_env(), TypeError is raised on Python 3.4. (cherry picked from commit 5e22721)

vstinner

Oh, I forgot that null character/byte errors were of type TypeError before 3.5. The simplest fix is changing corresponding ValueError in self.assertRaises() to the tuple (ValueError, TypeError). You can add 5e22721 to this PR.

It took me one minute to understand that this commit comes from PR #2363 which is not merged yet :-)

I changed the commit message and added it at the end of this PR.

I also rebased this PR on top of 3.4.

vstinner

Yeah, tests pass again \o/

@larryhastings: Ok, it's your turn ;-)

vstinner

Ping @larryhastings: does it now look good to you?

vstinner

(The PR was already approved by 3 core devs, maybe 4 if you include myself.)

larryhastings

I look forward to 3.4 entering the age of automation!

the-knights-who-say-ni added the CLA signed label Jun 28, 2017

vstinner requested review from brettcannon and zware June 28, 2017 14:46

vstinner added the trivial label Jun 28, 2017

zware reviewed Jun 28, 2017

View reviewed changes

brettcannon approved these changes Jun 28, 2017

View reviewed changes

zware approved these changes Jun 28, 2017

View reviewed changes

vstinner mentioned this pull request Jun 28, 2017

[3.3] Backport CI config from master #2481

Merged

larryhastings reviewed Jul 11, 2017

View reviewed changes

vstinner mentioned this pull request Jul 11, 2017

[3.4] bpo-30500: urllib: Simplify splithost by calling into urlparse. (#1849) #2291

Merged

vstinner and others added 10 commits July 13, 2017 13:15

Backport CI config from master …

6922ce8

* Add .travis.yml for Travis CI * Add .github/ for AppVeyor and CodeCov.

Travis CI: remove "make regen-all" check …

b298221

The regen-all Makefile rule doesn't exist in Python 3.4, only since Python 3.5 and newer (and 2.7).

appveyor: replace --slowest with --slow

10f9ce0

Travis CI: remove the GCC coverage job

265f464

Travis CI: remove tzdata resource from regrtest …

681bb41

tzdata resource doesn't exist in Python 3.4.

Travis CI: remove the doc job …

a7ed6fa

Fixing Sphinx warnings requires to backport huge intrusive changes like: - commit d97b7dc - commit 5c67933

appveyor: set version to 3.4.6+

457fb5d

bpo-30231: Remove skipped test_imaplib tests (#1419) (#2193) …

14c13a5

The public cyrus.andrew.cmu.edu IMAP server (port 993) doesn't accept TLS connection using our self-signed x509 certificate. Remove the two tests which are already skipped. (cherry picked from commit 7895a05)

Issues #23808, #25911: Trying to fix walk tests on Windows. …

2143f31

On Windows a symlink can has the FILE_ATTRIBUTE_DIRECTORY flag. (cherry picked from commit 388b90f)

bpo-30730: Fix test_os tests. …

577f926

Fix test_invalid_cmd() and test_invalid_env(), TypeError is raised on Python 3.4. (cherry picked from commit 5e22721)

serhiy-storchaka approved these changes Jul 13, 2017

View reviewed changes

vstinner mentioned this pull request Jul 13, 2017

[3.4] bpo-26617: Ensure gc tracking is off when invoking weakref callbacks. #2695

Merged

brettcannon added skip issue and removed trivial labels Jul 14, 2017

Merge branch '3.4' into travis34

93d03c2

larryhastings merged commit b154917 into python:3.4 Jul 22, 2017

vstinner deleted the travis34 branch August 10, 2017 23:38

vstinner mentioned this pull request Sep 27, 2017

[3.4] bpo-31170: Fix inclusion of expat in Windows build projects #3785

Merged

Conversation

vstinner commented Jun 28, 2017

Uh oh!

vstinner commented Jun 28, 2017

Uh oh!

vstinner commented Jun 28, 2017

Uh oh!

vstinner commented Jun 28, 2017

Uh oh!

vstinner commented Jun 28, 2017

Uh oh!

vstinner commented Jun 28, 2017

Uh oh!

vstinner commented Jun 28, 2017

Uh oh!

zware left a comment

Choose a reason for hiding this comment

Uh oh!

vstinner commented Jun 28, 2017

Uh oh!

zware commented Jun 28, 2017

Uh oh!

vstinner commented Jun 28, 2017

Uh oh!

vstinner commented Jun 28, 2017

Uh oh!

vstinner commented Jun 28, 2017

Uh oh!

vstinner commented Jun 28, 2017

Uh oh!

serhiy-storchaka commented Jun 28, 2017

Uh oh!

brettcannon left a comment

Choose a reason for hiding this comment

Uh oh!

vstinner commented Jun 28, 2017

Uh oh!

zware left a comment

Choose a reason for hiding this comment

Uh oh!

brettcannon commented Jul 5, 2017

Uh oh!

larryhastings commented Jul 11, 2017

Uh oh!

vstinner commented Jul 11, 2017

Uh oh!

brettcannon commented Jul 11, 2017

Uh oh!

larryhastings commented Jul 12, 2017

Uh oh!

vstinner commented Jul 12, 2017

Uh oh!

vstinner commented Jul 12, 2017

Uh oh!

vstinner commented Jul 12, 2017

Uh oh!

serhiy-storchaka commented Jul 12, 2017

Uh oh!

vstinner commented Jul 13, 2017

Uh oh!

vstinner commented Jul 13, 2017

Uh oh!

vstinner commented Jul 21, 2017

Uh oh!

vstinner commented Jul 21, 2017

Uh oh!

larryhastings commented Jul 22, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants