Skip to content
Snippets Groups Projects
New look: Off

Test stem pull requests against all supported tor versions, and tor nightly builds

    • Closed Issue created by teor @teor

    In #27913 (moved), we want to test stem against all supported tor versions, and tor nightly.

    To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information

    Child items ...

    • Activity

    • teor added actualpoints::0.3 ci component::archived/stem points::0.3 priority::medium resolution::invalid severity::normal sponsor::31-can status::closed travis type::enhancement labels

      added actualpoints::0.3 ci component::archived/stem points::0.3 priority::medium resolution::invalid severity::normal sponsor::31-can status::closed travis type::enhancement labels

    • A
      Damian Johnson @atagar ·

      Hi teor, what is the ask from me on this?

    • T
      teor @teor ·
      Author

      Replying to atagar:

      Hi teor, what is the ask from me on this?

      We want to make sure that changes to stem master don't break tor's CI. (I realise it's usually the other way around.)

      I'm going to write a patch to chutney to test against all supported tor versions and tor nightly (see #28170 (closed)). Then I'll copy the tor version matrix into stem's travis CI config, and update this ticket with the branch.

      If you could review and merge my patch to stem's travis ci config, that would be great.

    • A
      Damian Johnson @atagar ·

      We want to make sure that changes to stem master don't break tor's CI.

      Gotcha. I should be made aware of regressions on my side by Stem's Jenkins runs. That tests against tor's tip (not against multiple tor versions) but I've found this tends to be sufficient for bugs on my end.

      https://jenkins.torproject.org/job/stem-tor-ci/

      Then I'll copy the tor version matrix into stem's travis CI config, and update this ticket with the branch.

      Sorry, not sure I quite follow. Tor uses travis, but I don't for Stem.

    • Nick Mathewson
      Nick Mathewson @nickm ·

      Trac:
      Parent: #27913 (moved) to N/A

    • T
      teor @teor ·
      Author

      Replying to atagar:

      We want to make sure that changes to stem master don't break tor's CI.

      Gotcha. I should be made aware of regressions on my side by Stem's Jenkins runs. That tests against tor's tip (not against multiple tor versions) but I've found this tends to be sufficient for bugs on my end.

      https://jenkins.torproject.org/job/stem-tor-ci/

      Then I'll copy the tor version matrix into stem's travis CI config, and update this ticket with the branch.

      Sorry, not sure I quite follow. Tor uses travis, but I don't for Stem.

      The stem repository contains a travis config, and it runs under the torproject's travis: https://travis-ci.org/torproject/stem

    • Nick Mathewson
      Nick Mathewson @nickm ·

      These tickets are all deferred from 0.3.5, since they are about updating CI and/or chutney settings.

      Trac:
      Keywords: ci travis deleted, ci, travis, 041-proposed, 035-deferred-20190115 added
      Milestone: Tor: 0.3.5.x-final to Tor: unspecified

    • T
      teor @teor ·
      Author

      I'd like to do these tickets, but not in the next few months.

      Trac:
      Keywords: N/A deleted, teor-unreached-2019-03-08 added
      Owner: teor to N/A

    • T
      teor @teor ·
      Author

      We might have diagnosed #29437 (moved) faster if we ran Stem CI with python 3.6 and any supported tor version.

      Once I finish #29729 (moved), I can copy chutney's Travis config to a Stem PR, and then drop in the stem test commands.

      Trac:
      Parent: N/A to #29729 (moved)

    • T
      teor @teor ·
      Author

      This ticket can be sponsor 31, because working stem tests help us refactor tor.

      I used the chutney Travis config from #30279 (moved), with some minor edits based on the old stem Travis config.

      See my pull request: https://github.com/torproject/stem/pull/14

      Or the build for the original branch: https://travis-ci.org/teor2345/stem/builds/536613169

      We're seeing the hangs from #29437 (moved) here, as well as a bunch of other issues. I'll open child tickets for them.

      Trac:
      Sponsor: N/A to Sponsor31-can
      Status: assigned to needs_review
      Points: N/A to 0.2
      Actualpoints: N/A to 0.2

    • T
      teor @teor ·
      Author

      Trac:
      Milestone: Tor: unspecified to N/A
      Keywords: teor-unreached-2019-03-08, 035-deferred-20190115, 041-proposed deleted, N/A added

    • A
      Damian Johnson @atagar ·

      Thanks teor! Merged.

      Before resolving this I'd like to check - does it make sense for the travis config to live in this repository? I use Jenkins for my CI. I don't use this configuration, and as such don't have an opinion about what's in it or know if it breaks.

      I'd be happy to merge changes if you'd like to keep it here, but I'm unsure if there's value in requiring me to be in the loop on its maintenance.

    • A
      Damian Johnson @atagar ·

      Oh! And to be clear, if you'd care to suggest that I use Travis rather than Jenkins for my CI I'd be open to discussing that too. Personally I don't have any strong opinions at the moment about what kind of CI Stem uses for itself.

    • T
      teor @teor ·
      Author

      Replying to atagar:

      Before resolving this I'd like to check - does it make sense for the travis config to live in this repository? I use Jenkins for my CI. I don't use this configuration, and as such don't have an opinion about what's in it or know if it breaks.

      I'd be happy to merge changes if you'd like to keep it here, but I'm unsure if there's value in requiring me to be in the loop on its maintenance.

      If the Travis CI config is in the Stem repository, then anyone can run Travis CI on their Stem pull requests. That's the standard config for CIs like Travis and Appveyor.

      Using git for the CI config also allows different CI config for different release branches. That isn't as important for Stem, because your backports are usually pretty small.

      Replying to atagar:

      Oh! And to be clear, if you'd care to suggest that I use Travis rather than Jenkins for my CI I'd be open to discussing that too. Personally I don't have any strong opinions at the moment about what kind of CI Stem uses for itself.

      The network team uses Travis and Appveyor to check our Tor branches. Our Travis and Appveyor tests are more comprehensive, because they run different build options, Windows, Stem, and soon Chutney. Travis and Jenkins also let us run CI on our branches before we merge them.

      We use Jenkins to test rarer architectures, and build binary packages.

      Tor's CI runs a Stem job. To avoid breaking Tor's CI, we need all of Stem's changes to pass Stem's CI. (And we need Stem's CI tests to be roughly equivalent to Tor's CI tests.) So we would really like to have Stem's Travis and Jenkins CIs pass all the time.

      I don't know if you want to check both Travis and Jenkins after you merge, or if you want to standardise on one of them. (Checking multiple CI platforms adds more work.) But we'll check Stem's CI when Tor's Stem job fails, so we can work out if it was Tor or Stem that broke first.

      We can also use Stem's Travis CI to check our Stem branches, but we don't write very many of them.

      So it's totally up to you.

      Let's try it for a while and see how it goes?

    • A
      Damian Johnson @atagar ·

      Thanks for the detailed explanation. Can we configure our Stem CI to email me when there are failures?

      Personally for me my ideal CI tests are...

      1. Run 'run_tests.py --all' with python 2.6.
      2. Run 'run_tests.py --all' with python 2.7.
      3. Run 'run_tests.py --all' with python 3.x.
      4. Run 'run_tests.py --all --target ONLINE' with python 2.7.

      ... and email me when any of these fail. Jenkins provides #2 (closed) but that's it, so swapping over to Travis sounds like it would be an improvement.

    • T
      teor @teor ·
      Author

      I have opened tickets #30650 (closed) through #30653 (closed) for follow up, I'll probably do them all in the same branch, because they all modify .travis.yml.

    • T
      teor @teor ·
      Author

      Here's a pull request with #30650 (closed) and #30653 (closed): https://github.com/torproject/stem/pull/16

    • T
      teor @teor ·
      Author

      Trac:
      Points: 0.2 to 0.3
      Actualpoints: 0.2 to 0.3

    • A
      Damian Johnson @atagar ·

      Thanks teor! Merged.

    • T
      teor @teor ·
      Author

      Trac:
      Parent: #29729 (moved) to N/A

    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    Please register or sign in to reply