Build some CI jobs with ALL_BUGS_ARE_FATAL

Follow-up to #32141 (moved).

We should probably build with ALL_BUGS_ARE_FATAL for some of the check, stem, and chutney jobs. (But not for coverage or distcheck.)

changed milestone to %Tor: 0.4.4.x-final

added component::core tor/tor milestone::Tor: 0.4.4.x-final points::0.2 priority::medium resolution::fixed reviewer::teor severity::normal status::closed tor-ci type::task labels

It doesn't sound like this is blocking the release of 0.4.2. Should we move it to 0.4.3 with the current backlog of 0.4.2 tickets still being in place?

Trac:
Milestone: Tor: 0.4.2.x-final to Tor: 0.4.3.x-final
Keywords: 042-should deleted, 043-should added

Trac:
Parent: #32141 (moved) to N/A

Trac:
Keywords: 043-should deleted, 043-should tor-ci added

https://github.com/torproject/tor/pull/1777

Let's allow failures for the new CI jobs for time being, as they might be fairly fragile.

Bugs #33546 (moved) and #33544 (moved) were discovered.

Trac:
Status: new to needs_review

Trac:
Reviewer: N/A to nickm

I've added a comment about the way that we're invoking configure here, and one about CFLAGS is handled.

Additionally, I'd like to request additional review from Teor if possible, on the question of CI speed. Teor has done good work recently on CI performance, and I'd like their opinion on:

• whether this will slow down CI much,
• whether there is any sensible workaround for that,
• and whether we should try to apply the workaround before or after merge.

Trac:
Status: needs_review to needs_revision
Reviewer: nickm to teor

The above has been addressed. Test build: https://travis-ci.org/github/rl1987/tor/builds/662084047

The failure on one of the jobs is #33544 (moved).

Trac:
Status: needs_revision to needs_review

Replying to nickm:

I've added a comment about the way that we're invoking configure here, and one about CFLAGS is handled.

Additionally, I'd like to request additional review from Teor if possible, on the question of CI speed. Teor has done good work recently on CI performance, and I'd like their opinion on:

• whether this will slow down CI much,
• whether there is any sensible workaround for that,
• and whether we should try to apply the workaround before or after merge.

Thanks Nick!

I'd like to avoid adding any extra jobs, if possible. Instead, we could:

• try to build all existing jobs with ALL_BUGS_ARE_FATAL, unless there's some reason we want to turn hardening options off (coverage, distcheck)
• make ALL_BUGS_ARE_FATAL apply to Appveyor, as well as Travis

If that causes too many CI failures, we could:

• add a few jobs on Travis. I recommended a list of jobs we could add, without adding too much extra time. (Due to Travis parallelism.)
• stop using ALL_BUGS_ARE_FATAL on Appveyor.

Trac:
Status: needs_review to needs_revision

In general, I'd like to avoid using allow_failure, because it doesn't get us much value:

• it slows down CI, and
• it slows down developers, because we have to manually check every PR for failures, and decide if they are bad or not.

Oh, and if we're skipping some unit tests when ALL_BUGS_ARE_FATAL, let's try to make sure that we still run those tests on some configurations, particularly rare configurations (disabled modules, NSS, macOS).

Made further changes. If this is good now, I can squash/cleanup git history into new branch and open new pull request.

Trac:
Status: needs_revision to needs_review

Almost there, but let's use #ifndef COCCI rather than excluding a whole file from coccinelle.

Trac:
Status: needs_review to needs_revision

Trac:
Status: needs_revision to needs_review

Thanks!

Squashed, removed reverted commit and its revert, squashed some commits that used the old option name, and merged to master.

Trac:
Keywords: 043-should tor-ci deleted, tor-ci added
Status: needs_review to closed
Milestone: Tor: 0.4.3.x-final to Tor: 0.4.4.x-final
Resolution: N/A to fixed

closed

changed time estimate to 1h 36m

mentioned in issue #33544 (moved)

mentioned in issue #33546 (moved)

moved to tpo/core/tor#32143 (closed)