Verified Commit 3093cb7b authored by anarcat's avatar anarcat
Browse files

meeting minutes

parent 742fad27

howto/upgrades/bookworm/data.csv

+3 −0
Original line number Diff line number Diff line
@@ -19,3 +19,6 @@ Date,release,count 
2023-09-26,bookworm,41

2023-09-26,bullseye,42

2023-09-26,buster,5

2023-10-02,buster,5

2023-10-02,bookworm,42

2023-10-02,bullseye,42

howto/upgrades/bookworm/predict.png

+177 B (48.5 KiB)
Loading image diff...

meeting.md

+1 −0
Original line number Diff line number Diff line
@@ -28,6 +28,7 @@ Those are just for TPA, there are broader notes on meetings in the 


# 2023



 * [2023-10-02](meeting/2023-10-02)

 * [2023-06-05](meeting/2023-06-05)

 * [2023-05-08](meeting/2023-05-08)

 * [2023-03-13](meeting/2023-03-13)

meeting/2023-10-02.md

0 → 100644
+79 −0
Original line number Diff line number Diff line 
# Roll call: who's there and emergencies



onionoo-backend running out of disk space ([tpo/tpa/team#41343][])



[tpo/tpa/team#41343]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/41343



# Dashboard cleanup



Normal per-user check-in:



 * https://gitlab.torproject.org/groups/tpo/-/boards?scope=all&utf8=%E2%9C%93&ssignee_username=anarcat

 * https://gitlab.torproject.org/groups/tpo/-/boards?scope=all&utf8=%E2%9C%93&assignee_username=kez

 * https://gitlab.torproject.org/groups/tpo/-/boards?scope=all&utf8=%E2%9C%93&assignee_username=lavamind



General dashboards:



 * https://gitlab.torproject.org/tpo/tpa/team/-/boards/117

 * https://gitlab.torproject.org/groups/tpo/web/-/boards

 * https://gitlab.torproject.org/groups/tpo/tpa/-/boards



Nextcloud roadmap / spreadsheet.



Overall, it seems we are as you would expect when returning from a

rather chaotic vacation. Backlog is large, but things seem to be under

control.



We added SVN back on the roadmap after one too many tickets asking for

setup.



# Metrics of the month



 * hosts in Puppet: 89, LDAP: 89, Prometheus exporters: 166

 * number of Apache servers monitored: 37, hits per second: 626

 * number of self-hosted nameservers: 6, mail servers: 10

 * pending upgrades: 1, reboots: 0

 * average load: 0.69, memory available: 3.58 TiB/4.98 TiB, running processes: 424

 * disk free/total: 53.19 TiB/126.72 TiB

 * bytes sent: 403.47 MB/s, received: 269.04 MB/s

 * planned bullseye upgrades completion date: 2024-08-02

 * [GitLab tickets][]: 196 tickets including...

   * open: 0

   * icebox: 163

   * needs information: 5

   * backlog: 13

   * next: 9

   * doing: 4

   * needs review: 2

   * (closed: 3301)



 [Gitlab tickets]: https://gitlab.torproject.org/tpo/tpa/team/-/boards



Upgrade prediction graph lives at:



https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/upgrades/bookworm/



Now also available as the main Grafana dashboard. Head to

<https://grafana.torproject.org/>, change the time period to 30 days,

and wait a while for results to render.



# Number of the month: 42



34 machines were upgraded from bullseye to bookworm in the two first

days of last week! We calculated this was an average of 20 minutes per

host to upgrade.



The trick, of course, is that things often break *after* the upgrade,

and that "fixing" time is not counted here. That said, last estimate

for this was one hour per machine, and we're doing a whole fleet

upgrade every 2-3 years, which means about ten hours of work saved per

year.



But the number of the month is, of course, 42, as we now have an equal

number of bookworm and bullseye machine, after the upgrade. And that

number is, naturally, [42][].



See also https://xkcd.com/1205/ which, interestingly, we fall out of

scope of.



[42]: https://en.wikipedia.org/wiki/42

meeting/monthly-report

+3 −3
Original line number Diff line number Diff line
@@ -140,7 +140,7 @@ def host_count_puppet(args): 


def host_count_ldap(args):

    ldap_data = subprocess.check_output(['ssh', args.ldap,

                                         'ldapsearch -ZZ -vLx -h db.torproject.org -b "ou=hosts,dc=torproject,dc=org" 2>/dev/null'])  # noqa: E501

                                         'ldapsearch -ZZ -vLx -H ldap://db.torproject.org -b "ou=hosts,dc=torproject,dc=org" 2>/dev/null'])  # noqa: E501

    return len(re.findall(r'^dn: host', ldap_data.decode('ascii'), re.M))
@@ -272,7 +272,7 @@ def main(): 
          (sizeof_fmt_decimal(prom.query(args, 'sum(rate(node_network_transmit_bytes_total[30d]))')),  # noqa: E501

           sizeof_fmt_decimal(prom.query(args, 'sum(rate(node_network_receive_bytes_total[30d]))'))))  # noqa: E501



    print(" * planned bullseye upgrades completion date: ???")

    print(" * planned bookworm upgrades completion date: ???")

    print(""" * [GitLab tickets][]: ? tickets including...

   * open: ?

   * icebox: ?
@@ -284,7 +284,7 @@ def main(): 
    print(" [Gitlab tickets]: https://gitlab.torproject.org/tpo/tpa/team/-/boards")  # noqa: E501



    print()

    print("Upgrade prediction graph lives at https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/upgrades/bullseye/")  # noqa: E501

    print("Upgrade prediction graph lives at https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/upgrades/bookworm/")  # noqa: E501

    print()

    print("Now also available as the main Grafana dashboard. Head to <https://grafana.torproject.org/>, change the time period to 30 days, and wait a while for results to render.")  # noqa: E501

    # TODO: talk with the gitlab API to extract those numbers