Deploy SPF, DKIM, and DMARC records for all of torproject.org
Once the SPF and DKIM records are in use everywhere, deploy SPF records for all of torproject.org pointing to known mail hosts. Also enforce a domain-wide DMARC policy, at least to get reporting when we have failures.
next steps:
- SPF soft policy everywhere
- DKIM records everywhere (#40989 (closed))
-
DMARC soft record (
is it like SPF that we need a record per mx?no, DMARC is inherited) -
SPF hard policy everywhere, after some grace period? -
DMARC hard policy?
Designs
No child items are currently assigned. Use child items to break down this issue into smaller parts.
- #40981improve mail services
- #40989improve mail services
- #40988improve mail services
- #40987TPA-RFC-71: emergency email deployments, phase B
- #40986improve mail services
Activity
marked this issue as related to #40981 (closed)
marked this issue as related to #40989 (closed)
marked this issue as related to #40988 (closed)
marked this issue as related to #40987 (closed)
marked this issue as related to #40986 (closed)
changed milestone to %improve mail services
assigned to @anarcat
mentioned in issue #40981 (closed)
mentioned in issue #40986 (closed)
i sent this patch to tpa-team for review:
commit 8017c10a3ed51a5511369675b50e74163452096d Author: Antoine Beaupré <anarcat@debian.org> AuthorDate: Tue Dec 6 14:33:57 2022 -0500 global SPF records, soft (tpo/tpa/team#40981) --- torproject.org | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/torproject.org b/torproject.org index a8292b2..c1bc5c7 100644 [ 0001-01-01-global-SPF-records-soft-tpo-tpa-team-40981.patch: inline patch (as text/x-diff) ] --- a/torproject.org +++ b/torproject.org @@ -51,6 +51,23 @@ ns5 24h IN A 89.45.235.22 ; per <PMZ2MDQ3YD_5a66b41a688eb_1d0673fe91b0cb98c51870_sprut@zendesk.com> ; Subject: [Fastly] Update: [Action Required] l.ssl- Re-vetting domains on Fastly shared certs +; mail records +; +; those servers write mail from @torproject.org +; +; TODO: consider adding include:riseup.net include:google.com +@ IN TXT "v=spf1 a:crm-int-01.torproject.org a:eugeni.torproject.org a:submit-01.torproject.org mx ~all" + +; TODO: also considering having a generic _spf record that gets included in +; a few places, and that lists all possible servers instead... +; +; warning: this gets close to the 255 character limit +; +; for now i feel we can just pinpoint the servers sending as +; @torproject.org and have per-service overrides as well +;_spf IN TXT "v=spf1 a:eugeni.torproject.org a:crm-int-01.torproject.org a:submit-01.torproject.org a:gitlab-02.torproject.org a:alberti.torproject.org a:rdsys-01.torproject.org a:rude.torproject.org ~all" + +; TODO: DMARC keys and DKIM records belong here ; services ; ======== @@ -79,6 +96,7 @@ collector IN CNAME colchicifolium collector2 IN CNAME corsicum ; DKIM for CiviCRM, tpo/tpa/team#40986 +; TODO: move up with SPF records crm-2022._domainkey IN TXT ( "v=DKIM1; h=sha256; k=rsa; " "p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtSNGCjHmZnGrnBb9nCsPUc6MjZd5QueGKV+iXwcRNfU0LapFZMi5t7WE/kTPJsRWIF8AMHymNqLA5835m5LwaBBXZdu1utNARKSXDzGsEjxuDiAnSqD0Rb1px1JA+Eex0RC3thYZuyIIAxK31pXxJt2mowXtrhIkuKFB2YpE0yUudKuDZIZZ3YNH025czK/jFLD6TH+5xD9Cej" "H0MB6tE4O41rCjZUjSZ7Ar7BjVID6foCmlbr/3EG7dbzQv6YqH19OX6YgL0UMfG2RhvhWEUNYghS6K88vTelnHx/ShUzIeu05jd6mi9OLCA/Hl2bFRsa0f1ttHKpnzuC+ecn0sWwIDAQAB" ) ; ----- DKIM key 2022 for crm.torproject.org @@ -91,10 +109,10 @@ crm-2022._domainkey.crm IN TXT ( "v=DKIM1; h=sha256; k=rsa; " "H0MB6tE4O41rCjZUjSZ7Ar7BjVID6foCmlbr/3EG7dbzQv6YqH19OX6YgL0UMfG2RhvhWEUNYghS6K88vTelnHx/ShUzIeu05jd6mi9OLCA/Hl2bFRsa0f1ttHKpnzuC+ecn0sWwIDAQAB" ) ; ----- DKIM key 2022 for crm.torproject.org crm IN A 116.202.120.186 ; crm-int-01 IN AAAA 2a01:4f8:fff0:4f:266:37ff:fe4d:f883 - IN TXT "v=spf1 a mx -all" + IN TXT "v=spf1 a mx -all" ; TODO: redundant with @ record? IN MX 0 crm _dmarc.crm IN TXT "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@torproject.org" -crm-int-01 IN TXT "v=spf1 a mx -all" +crm-int-01 IN TXT "v=spf1 a mx -all" ; TODO: redundant with @ record? IN MX 0 crm _dmarc.crm-int-01 IN TXT "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@torproject.org" staging.crm IN CNAME crm-int-01 @@ -114,6 +132,8 @@ test-api.donate IN CNAME crm-ext-01 exonerator IN CNAME materculae gitlab IN CNAME gitlab-02 gitlab-dev IN CNAME gitlab-dev-01 +; TODO: does SPF follow CNAMEs? ie. mail goes out From: gitlab@gitlab.tpo, will the following TXT record work? +gitlab-02 IN TXT "v=spf1 a ~all" ; no one else than gitlab can send for gitlab gettor IN CNAME static git IN CNAME vineale git-rw IN CNAME cupani @@ -123,7 +143,7 @@ kgb-bot IN CNAME chives lists IN A 49.12.57.136 ; eugeni IN AAAA 2a01:4f8:fff0:4f:266:37ff:fe48:41b8 ; eugeni IN MX 10 eugeni - IN TXT "v=spf1 mx ~all" + IN TXT "v=spf1 mx ~all" ; TODO: redundant with @ record? media IN CNAME media-01 metrics-store IN CNAME metrics-store-01 metrics-db IN CNAME metrics-psqlts-01the only feedback i received was from @micah and it was basically:
-
a:mechanism should work as expected, as it's in use at Riseup -
mxmight not be necessary in the root level - CNAMEs might work, but it remains to be tested
- subdomains need to be tested explicitly as well
So I'm going to go ahead and deploy those with "soft" mechanisms so that we can use online tools to test them.
-
-
soft SPF records deployed, without the above
mxrecord.Edited by anarcat -
i tested against https://vamsoft.com/support/tools/spf-policy-tester and port25.com and everything seems to be in order.
i tested mailing from
anarcat@torproject.orgfrom eugeni, submit-01, and crm-int-01, all worked. i also tested anarcat@crm.torproject.org using the the vamsoft tool, and that also works. mentioned in commit wiki-replica@9938aa1e
-
more reports came in, all of them look good except, of course, Google:
anarcat@curie:dmarc[SIGINT]$ dmarc-cat -j 1 google.com\!torproject.org\!1670371200\!1670457599.zip dmarc-cat 0.15.0,parallel/j1 by Ollivier Robert Reporting by: google.com — noreply-dmarc-support@google.com From 2022-12-06 19:00:00 -0500 EST to 2022-12-07 18:59:59 -0500 EST Domain: torproject.org Policy: p=none; dkim=r; spf=r Reports(43): IP Count From RFrom RDKIM RSPF polyanthum.torproject.org. 348 torproject.org torproject.org softfail polyanthum.torproject.org. 66 torproject.org torproject.org softfail bosmailout07.eigbox.net. 50 torproject.org torproject.org pass softfail bosmailout07.eigbox.net. 37 torproject.org torproject.org pass none rdsys-frontend-01.torproject.org. 25 torproject.org torproject.org softfail mail-sor-f69.google.com. 15 torproject.org opentech.fund pass pass eugeni.torproject.org. 12 torproject.org torproject.org pass pass eugeni.torproject.org. 11 lists.torproject.org torproject.org pass pass eugeni.torproject.org. 11 torproject.org torproject.org pass eugeni.torproject.org. 9 torproject.org torproject.org pass pass eugeni.torproject.org. 9 lists.torproject.org torproject.org pass pass eugeni.torproject.org. 8 gitlab.torproject.org gitlab.torproject.org softfail eugeni.torproject.org. 7 gitlab.torproject.org gitlab.torproject.org softfail eugeni.torproject.org. 6 torproject.org torproject.org pass eugeni.torproject.org. 4 eugeni.torproject.org eugeni.torproject.org none mail-sor-f41.google.com. 4 torproject.org gmail.com pass submit-01.torproject.org. 3 torproject.org torproject.org pass pass gitlab-02.torproject.org. 3 gitlab.torproject.org gitlab.torproject.org pass eugeni.torproject.org. 3 torproject.org torproject.org pass pass gitlab-02.torproject.org. 3 gitlab.torproject.org gitlab.torproject.org pass eugeni.torproject.org. 3 torproject.org gmail.com softfail eugeni.torproject.org. 2 torproject.org gmail.com softfail mail-sor-f69.google.com. 2 torproject.org torproject.org pass softfail mx0.riseup.net. 2 torproject.org torproject.org softfail rdsys-frontend-01.torproject.org. 2 torproject.org torproject.org softfail eugeni.torproject.org. 1 torproject.org torproject.org pass pass m68-68.mailgun.net. 1 torproject.org torproject.org pass pass crm-int-01.torproject.org. 1 torproject.org torproject.org pass pass mail-sor-f65.google.com. 1 torproject.org gmail.com pass polyanthum.torproject.org. 1 bridges.torproject.org torproject.org softfail st43p00im-zteg10073501.me.com. 1 polyanthum.torproject.org st43p00im-zteg10073501.me.com none crm-int-01.torproject.org. 1 torproject.org torproject.org pass pass mxgw4.iitr.ac.in. 1 torproject.org torproject.org pass softfail so254-57.mailgun.net. 1 torproject.org torproject.org pass pass outbound-ss-1298.hostmonster.com. 1 torproject.org torproject.org pass softfail mail.opsmail.ch. 1 torproject.org torproject.org pass pass mail-qt1-x82f.google.com. 1 torproject.org torproject.org pass none rude.torproject.org. 1 rt.torproject.org rude.torproject.org none mx1.riseup.net. 1 torproject.org torproject.org softfail shunt02.easymail.ca. 1 torproject.org torproject.org pass pass mx1.riseup.net. 1 torproject.org torproject.org pass softfail polyanthum.torproject.org. 1 polyanthum.torproject.org polyanthum.torproject.org none eugeni.torproject.org. 1 torproject.org torproject.org pass passit certainly looks like our SPF policy is going to need some work. it might also be that there are some reports that date from the transition period during which we were deploying the records so that some records were missing when some mail finally got through. maybe better to investigate again in a day or two after the dust settles.
i'm certainly not going to enable hard records at this point and will wait for next week for the next move.
-
so the reason this is happening, specifically:
IP Count From RFrom RDKIM RSPF eugeni.torproject.org. 8 gitlab.torproject.org gitlab.torproject.org softfail eugeni.torproject.org. 7 gitlab.torproject.org gitlab.torproject.org softfail... is because gitlab.torproject.org actually delivers mail to eugeni.torproject.org when the target is a @torproject.org, of course! from there, we get a failure because gitlab.tpo doesn't have eugeni in its allow list.
so i actually needed to add the MX on all of those hosts as well...
-
another extract from a google dmarc report:
IP Count From RFrom RDKIM RSPF polyanthum.torproject.org. 12 bridges.torproject.org torproject.org softfail polyanthum.torproject.org. 4 bridges.torproject.org torproject.org softfail... it seems we don't have a SPF record for bridges.tpo at all, so I added a soft one there as well.
-
We still have some issues with torproject.org hosts:
anarcat@angela:dmarc$ dmarc-cat -j 1 google.com\!torproject.org\!1670889600\!1670975999.zip | grep -e ^IP -e fail -e error| grep -e ^IP -e torproject.org\\. IP Count From RFrom RDKIM RSPF eugeni.torproject.org. 4 torproject.org gmail.com softfail eugeni.torproject.org. 4 torproject.org gmail.com softfail eugeni.torproject.org. 3 gitlab.torproject.org gitlab.torproject.org fail pass rude.torproject.org. 1 rt.torproject.org rt.torproject.org fail pass eugeni.torproject.org. 1 gitlab.torproject.org gitlab.torproject.org fail passi'm not sure what those first two are. The third and fifth ones seems to be a bad signature for mail relayed from gitlab through eugeni.. Maybe that was fixed recently by @lavamind? Not sure what the rt.torproject.org one is.
And of course there's all this mail that'S sent from outside our infrastructure... and there is a lot of those:
anarcat@angela:dmarc$ dmarc-cat -j 1 google.com\!torproject.org\!1670889600\!1670975999.zip | grep -e ^IP -e fail -e error| grep -v torproject.org\\. IP Count From RFrom RDKIM RSPF mail-sor-f69.google.com. 35 lists.torproject.org lists.torproject.org pass softfail mail-sor-f41.google.com. 11 lists.torproject.org lists.torproject.org pass softfail mail-sor-f69.google.com. 9 gitlab.torproject.org gitlab.torproject.org pass softfail mx0.riseup.net. 6 torproject.org torproject.org softfail mail-sor-f41.google.com. 4 lists.torproject.org lists.torproject.org pass fail 204.17.60.94 4 lists.torproject.org lists.torproject.org pass softfail foofus.com. 4 lists.torproject.org lists.torproject.org pass softfail mx1.riseup.net. 3 torproject.org torproject.org softfail mx0b-00007101.pphosted.com. 3 lists.torproject.org lists.torproject.org fail pass mailfwd.njal.la. 3 lists.torproject.org lists.torproject.org pass softfail mail-sor-f41.google.com. 3 lists.torproject.org lists.torproject.org pass permerror forward1-smtp.messagingengine.com. 3 lists.torproject.org lists.torproject.org pass softfail redirmail1.reg.bookmyname.com. 2 lists.torproject.org lists.torproject.org pass softfail relay9-d.mail.gandi.net. 2 lists.torproject.org lists.torproject.org pass softfail mailfwd.njal.la. 1 lists.torproject.org lists.torproject.org pass softfail relay4-d.mail.gandi.net. 1 lists.torproject.org lists.torproject.org pass softfail mail-sor-f69.google.com. 1 torproject.org torproject.org pass softfail del3.i.mail.ru. 1 lists.torproject.org lists.torproject.org pass softfail del9.i.mail.ru. 1 lists.torproject.org lists.torproject.org pass softfail relay9-d.mail.gandi.net. 1 lists.torproject.org lists.torproject.org pass softfail del13.i.mail.ru. 1 lists.torproject.org lists.torproject.org pass softfail mail-sor-f69.google.com. 1 torproject.org torproject.org pass softfail del17.i.mail.ru. 1 lists.torproject.org lists.torproject.org pass softfail relay1-d.mail.gandi.net. 1 lists.torproject.org lists.torproject.org pass softfail filtered.relay.mayfirst.org. 1 torproject.org torproject.org pass softfail redirmail2.reg.bookmyname.com. 1 lists.torproject.org lists.torproject.org pass softfail redirmail2.reg.bookmyname.com. 1 lists.torproject.org lists.torproject.org pass softfail mout-xforward.kundenserver.de. 1 metrics.torproject.org metrics.torproject.org fail pass relay10.mail.gandi.net. 1 lists.torproject.org lists.torproject.org pass softfail forward4-smtp.messagingengine.com. 1 lists.torproject.org lists.torproject.org pass softfail relay6-d.mail.gandi.net. 1 lists.torproject.org lists.torproject.org pass softfail mail-db3eur04lp2058.outbound.protection.outlook.com. 1 lists.torproject.org lists.torproject.org pass softfail mail-dbaeur03lp2175.outbound.protection.outlook.com. 1 lists.torproject.org lists.torproject.org pass softfail mx0a-00007101.pphosted.com. 1 lists.torproject.org lists.torproject.org fail pass mail-sor-f69.google.com. 1 cupani.torproject.org torproject.org pass softfail filtered.relay.mayfirst.org. 1 torproject.org torproject.org softfail mail-db8eur05lp2108.outbound.protection.outlook.com. 1 lists.torproject.org lists.torproject.org pass softfailIt's mostly google, of course, but there's also a good chunk from riseup, and also a long tail of random providers... all of those would be dropped to the ground whenever we setup "hard" records... so I'm not sure we're ready to do that switch just yet.
marked the checklist item DKIM records everywhere (#40989 (closed)) as completed
Please register or sign in to reply