|
= CI Failures =
|
|
# CI Failures
|
|
|
|
|
|
[[PageOutline]]
|
|
[[PageOutline]]
|
|
|
|
|
|
This page lists the CI failures for the network team.
|
|
This page lists the CI failures for the network team.
|
|
|
|
|
|
It's based on the [wiki:org/teams/NetworkTeam Network Team's] [wiki:org/teams/NetworkTeam/CoreTorReleases/040Status 0.4.0 Release Status] page, and [wiki:user/teor teor's user page].
|
|
It's based on the [Network Team's](./org/teams/NetworkTeam) [0.4.0 Release Status](./org/teams/NetworkTeam/CoreTorReleases/040Status) page, and [teor's user page](./user/teor).
|
|
|
|
|
|
== CI Builders ==
|
|
## CI Builders
|
|
|
|
|
|
* Jenkins: https://jenkins.torproject.org/view/tor/
|
|
* Jenkins: https://jenkins.torproject.org/view/tor/
|
|
* Travis: https://travis-ci.org/torproject/tor/branches
|
|
* Travis: https://travis-ci.org/torproject/tor/branches
|
... | @@ -16,11 +16,11 @@ Jenkins builds our supported branches and Debian packages for deb.torproject.org |
... | @@ -16,11 +16,11 @@ Jenkins builds our supported branches and Debian packages for deb.torproject.org |
|
|
|
|
|
Travis and Appveyor build GitHub pull requests, personal repositories, and our supported branches. Notifications go to #tor-ci on irc.oftc.net.
|
|
Travis and Appveyor build GitHub pull requests, personal repositories, and our supported branches. Notifications go to #tor-ci on irc.oftc.net.
|
|
|
|
|
|
== Handling CI Failures ==
|
|
## Handling CI Failures
|
|
|
|
|
|
We want to log every CI failure in trac, so we can see which CI issues we need to fix first.
|
|
We want to log every CI failure in trac, so we can see which CI issues we need to fix first.
|
|
|
|
|
|
=== Supported Branches ===
|
|
### Supported Branches
|
|
|
|
|
|
We want CI to always pass on supported branches.
|
|
We want CI to always pass on supported branches.
|
|
|
|
|
... | @@ -33,13 +33,13 @@ When CI fails on: |
... | @@ -33,13 +33,13 @@ When CI fails on: |
|
3. Otherwise, log a ticket tagged "tor-ci-fail" with the information, and mark it as high priority.
|
|
3. Otherwise, log a ticket tagged "tor-ci-fail" with the information, and mark it as high priority.
|
|
4. If the failure is on an earlier supported branch, update the version and backport tags to that branch.
|
|
4. If the failure is on an earlier supported branch, update the version and backport tags to that branch.
|
|
|
|
|
|
=== Pull Requests ===
|
|
### Pull Requests
|
|
|
|
|
|
When CI fails on a pull request, and it was caused by the changes in the pull request, mark the ticket for that pull request as needs_revision.
|
|
When CI fails on a pull request, and it was caused by the changes in the pull request, mark the ticket for that pull request as needs_revision.
|
|
|
|
|
|
== CI Failure Tickets ==
|
|
## CI Failure Tickets
|
|
|
|
|
|
=== Unassigned tickets ===
|
|
### Unassigned tickets
|
|
|
|
|
|
These open tickets are tagged ci-fail, and they don't have an owner.
|
|
These open tickets are tagged ci-fail, and they don't have an owner.
|
|
|
|
|
... | @@ -47,7 +47,7 @@ TODO: who will assign them to the best person to fix the ticket? |
... | @@ -47,7 +47,7 @@ TODO: who will assign them to the best person to fix the ticket? |
|
|
|
|
|
[[TicketQuery(status!=closed,owner=,keywords~=ci-fail,group=owner,format=table,order=changetime,desc=true,col=id|summary|status|owner|reviewer|priority|severity|changetime|sponsor,max=100)]]
|
|
[[TicketQuery(status!=closed,owner=,keywords~=ci-fail,group=owner,format=table,order=changetime,desc=true,col=id|summary|status|owner|reviewer|priority|severity|changetime|sponsor,max=100)]]
|
|
|
|
|
|
== Owner Action tickets ==
|
|
## Owner Action tickets
|
|
|
|
|
|
These tickets are tagged ci-fail, they have an owner, and they are in a status that needs action from the owner.
|
|
These tickets are tagged ci-fail, they have an owner, and they are in a status that needs action from the owner.
|
|
|
|
|
... | @@ -58,19 +58,19 @@ Anyone can change the assignments: |
... | @@ -58,19 +58,19 @@ Anyone can change the assignments: |
|
|
|
|
|
[[TicketQuery(status!=needs_review|merge_ready|closed,owner!=,keywords~=ci-fail,group=owner,format=table,order=changetime,desc=true,col=id|summary|status|owner|reviewer|priority|severity|changetime|sponsor,max=100)]]
|
|
[[TicketQuery(status!=needs_review|merge_ready|closed,owner!=,keywords~=ci-fail,group=owner,format=table,order=changetime,desc=true,col=id|summary|status|owner|reviewer|priority|severity|changetime|sponsor,max=100)]]
|
|
|
|
|
|
== Needs Review tickets ==
|
|
## Needs Review tickets
|
|
|
|
|
|
These tickets are tagged ci-fail, and they are in needs_review.
|
|
These tickets are tagged ci-fail, and they are in needs_review.
|
|
|
|
|
|
[[TicketQuery(status=needs_review,keywords~=ci-fail,group=reviewer,format=table,order=changetime,desc=true,col=id|summary|status|owner|reviewer|priority|severity|changetime|sponsor,max=100)]]
|
|
[[TicketQuery(status=needs_review,keywords~=ci-fail,group=reviewer,format=table,order=changetime,desc=true,col=id|summary|status|owner|reviewer|priority|severity|changetime|sponsor,max=100)]]
|
|
|
|
|
|
== Merge Ready tickets ==
|
|
## Merge Ready tickets
|
|
|
|
|
|
These tickets are tagged ci-fail, and they are in merge_ready.
|
|
These tickets are tagged ci-fail, and they are in merge_ready.
|
|
|
|
|
|
[[TicketQuery(status=merge_ready,keywords~=ci-fail,group=milestone|reviewer,format=table,order=changetime,desc=true,col=id|summary|status|owner|reviewer|priority|severity|changetime|sponsor,max=100)]]
|
|
[[TicketQuery(status=merge_ready,keywords~=ci-fail,group=milestone|reviewer,format=table,order=changetime,desc=true,col=id|summary|status|owner|reviewer|priority|severity|changetime|sponsor,max=100)]]
|
|
|
|
|
|
== CI failures being diagnosed ==
|
|
## CI failures being diagnosed
|
|
|
|
|
|
As of 23 Sep 2019:
|
|
As of 23 Sep 2019:
|
|
|
|
|
... | @@ -83,12 +83,12 @@ This looks like a travis failure. There's a java exception caused by a timeout, |
... | @@ -83,12 +83,12 @@ This looks like a travis failure. There's a java exception caused by a timeout, |
|
|
|
|
|
~~see https://jenkins.torproject.org/job/tor-ci-linux-0.4.1/ARCHITECTURE=amd64,SUITE=buster/28/console
|
|
~~see https://jenkins.torproject.org/job/tor-ci-linux-0.4.1/ARCHITECTURE=amd64,SUITE=buster/28/console
|
|
Buster amd64 is failing with what appears to be~~
|
|
Buster amd64 is failing with what appears to be~~
|
|
{{{
|
|
```
|
|
08:50:53 08:50:53 FAIL: src/test/test_workqueue_efd.sh
|
|
08:50:53 08:50:53 FAIL: src/test/test_workqueue_efd.sh
|
|
08:50:53 ====================================
|
|
08:50:53 ====================================
|
|
08:50:53 08:50:53 Illegal instruction
|
|
08:50:53 08:50:53 Illegal instruction
|
|
08:50:53 FAIL src/test/test_workqueue_efd.sh (exit status: 132)
|
|
08:50:53 FAIL src/test/test_workqueue_efd.sh (exit status: 132)
|
|
}}}
|
|
```
|
|
~~I wonder if this is transient, or if it is a real problem? We didn't change this code any time recently.~~
|
|
~~I wonder if this is transient, or if it is a real problem? We didn't change this code any time recently.~~
|
|
|
|
|
|
* ~~tor-ci-linux-master-clang (new-ish)~~ transient? seems to have fixed itself
|
|
* ~~tor-ci-linux-master-clang (new-ish)~~ transient? seems to have fixed itself
|
... | @@ -99,11 +99,11 @@ Buster amd64 is failing with what appears to be~~ |
... | @@ -99,11 +99,11 @@ Buster amd64 is failing with what appears to be~~ |
|
|
|
|
|
~~Looks like there's now a complaint about an unrecognized warning flag? -catalyst~~ see #31769
|
|
~~Looks like there's now a complaint about an unrecognized warning flag? -catalyst~~ see #31769
|
|
|
|
|
|
{{{
|
|
```
|
|
15:39:37 cc1: error: command line option '-Wextra-semi' is valid for C++/ObjC++ but not for C [-Werror]
|
|
15:39:37 cc1: error: command line option '-Wextra-semi' is valid for C++/ObjC++ but not for C [-Werror]
|
|
15:39:37 cc1: all warnings being treated as errors
|
|
15:39:37 cc1: all warnings being treated as errors
|
|
15:39:37 make[1]: *** [Makefile:11538: src/feature/control/control_bootstrap.o] Error 1
|
|
15:39:37 make[1]: *** [Makefile:11538: src/feature/control/control_bootstrap.o] Error 1
|
|
}}}
|
|
```
|
|
|
|
|
|
* tor-ci-mingwcross-* (old-ish)
|
|
* tor-ci-mingwcross-* (old-ish)
|
|
|
|
|
... | @@ -119,9 +119,9 @@ https://jenkins.torproject.org/job/tor-ci-windows-master/78/console |
... | @@ -119,9 +119,9 @@ https://jenkins.torproject.org/job/tor-ci-windows-master/78/console |
|
This looks like it could be a real bug: a unit test is failing. It's in "slow/process/callbacks" -- I believe ahf knows that code best.
|
|
This looks like it could be a real bug: a unit test is failing. It's in "slow/process/callbacks" -- I believe ahf knows that code best.
|
|
|
|
|
|
* https://jenkins.torproject.org/view/tor/job/tor-ci-linux-master-expensive-hardening/2527/ARCHITECTURE=amd64,SUITE=bionic/console
|
|
* https://jenkins.torproject.org/view/tor/job/tor-ci-linux-master-expensive-hardening/2527/ARCHITECTURE=amd64,SUITE=bionic/console
|
|
{{{
|
|
```
|
|
07:58:07 addr/virtaddr_persist: [forking]
|
|
07:58:07 addr/virtaddr_persist: [forking]
|
|
07:58:07 FAIL ../tor/src/test/test_addr.c:1415: assert(a OP_EQ "192.168.3.4"): <192.168.7.240> vs <192.168.3.4>
|
|
07:58:07 FAIL ../tor/src/test/test_addr.c:1415: assert(a OP_EQ "192.168.3.4"): <192.168.7.240> vs <192.168.3.4>
|
|
07:58:07 [virtaddr_persist FAILED]
|
|
07:58:07 [virtaddr_persist FAILED]
|
|
}}}
|
|
```
|
|
could be a network virtualization artifact? |
|
could be a network virtualization artifact? |
|
|
|
\ No newline at end of file |