service/lists.md

@@ -202,6 +202,8 @@ The frontend database needs to be rebuilt with:
 
     sudo -u www-data /usr/share/mailman3-web/manage.py migrate
 
+See also the [database documentation](https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/database.html).
+
 A site admin password was created by hand with:
 
     django-admin createsuperuser --pythonpath /usr/share/mailman3-web --settings settings --username admin --email postmaster@torproject.org
@@ -276,6 +278,8 @@ through the web interface.
 <!-- "architectural" document, which the final result might differ -->
 <!-- from, sometimes significantly -->
 
+TODO: explain a bit the mailman3 architecture, see https://docs.mailman3.org/en/latest/architecture.html
+
 ## Services
 
 <!-- open ports, daemons, cron jobs -->
@@ -532,3 +536,97 @@ label ~Foo.
 ## Other alternatives
 
 <!-- include benchmarks and procedure if relevant -->
+
+## Mailman 2 migration
+
+TODO: document https://gitlab.torproject.org/tpo/tpa/team/-/issues/40471, see also https://docs.mailman3.org/en/latest/migration.html
+ https://docs.mailman3.org/en/latest/faq-migration.html
+ https://docs.debops.org/en/stable-2.1/ansible/roles/mailman/mailman2-migration.html
+ 
+ not documented in debian https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=999861
+
+To check for anomalies in the mailing lists migrations, the following
+script was used:
+
+```
+def check_bounce_info(mlist):
+    print(mlist.bounce_info)
+
+def check_pending_reqs(mlist):
+    print(mlist.NumRequestsPending())
+
+def flush_digest_mbox(mlist):
+    mlist.send_digest_now()
+```
+
+and called with for example:
+
+    sudo -u list /var/lib/mailman/bin/withlist -l  -a -r check_list_mm3.check_pending_reqs
+
+The `bounce_info` check was done because of a comment found in [this
+post](https://docs.debops.org/en/stable-2.1/ansible/roles/mailman/mailman2-migration.html) saying the conversion script had problem with those, it
+wasn't checked.
+
+The `pending_reqs` check was done because those are not converted by
+the script.
+
+Similarly, we check for digest files with:
+
+    find /var/lib/mailman/lists -name digest.mbox 
+
+But it's simpler to just send the actual digest without checking with:
+
+    sudo -u list /usr/lib/mailman/cron/senddigests -l LISTNAME
+
+This essentially does a `mlist.send_digest_now` so perhaps it would be
+simpler to just add that to one script.
+
+### List migration procedure
+
+Procedure for migrating a single list, derived from [the upstream
+migration documentation](https://docs.mailman3.org/en/latest/migration.html#before-you-upgrade):
+
+1. flush digest mbox with:
+
+        sudo -u list /var/lib/mailman/bin/withlist -l LISTNAME -r mm3_cleanup.flush_digest_mbox
+
+2. check for pending requests with:
+
+        sudo -u list /var/lib/mailman/bin/withlist  -l -r mm3_cleanup.check_pending_reqs meeting-planners
+
+   Warn list operator one last time if matches.
+
+3. block mail traffic on the mm2 list
+4. resync the list data (archives and pickle file)
+5. create the list in mm3
+6. migrate the list pickle file to mm3
+7. migrate the archives to hyperkitty
+8. rebuild the archive index
+9. forward the list on eugeni
+
+The above assumes the presence of a `mm3_cleanup.py` script with the
+contents:
+
+```
+from __future__ import print_function
+
+def check_bounce_info(mlist):
+    print(mlist.bounce_info)
+
+def check_pending_reqs(mlist):
+    if mlist.NumRequestsPending() > 0:
+      print("list", mlist.internal_name(), "has", mlist.NumRequestsPending(), "pending requests")
+      if mlist.GetSubscriptionIds():
+        print("subscriptions:", len(mlist.GetSubscriptionIds()))
+      if mlist.GetUnsubscriptionIds():
+        print("unsubscriptions:", len(mlist.GetUnsubscriptionIds()))
+      if mlist.GetHeldMessageIds():
+        print("held:", len(mlist.GetHeldMessageIds()))
+
+def list_pending_reqs_owners(mlist):
+    if mlist.NumRequestsPending() > 0:
+      print(mlist.internal_name() + "-owner@lists.torproject.org")
+
+def flush_digest_mbox(mlist):
+    mlist.send_digest_now()
+```