Changes

Page history
use checkmarks to track roadmap progress authored by anarcat's avatar anarcat
Show whitespace changes
Inline Side-by-side
roadmap/2021.md
View page @ 8d209b82
......@@ -22,94 +22,94 @@ attention. In general, the over-arching goals are therefore:
## Need to have
* email delivery improvements:
* handle bounces in CiviCRM ([issue 33037](https://gitlab.torproject.org/tpo/tpa/team/-/issues/33037))
* systematically followup on and respond to abuse complaints
* diagnose and resolve delivery issue (e.g. [yahoo delivery
* [ ] email delivery improvements:
* [ ] handle bounces in CiviCRM ([issue 33037](https://gitlab.torproject.org/tpo/tpa/team/-/issues/33037))
* [ ] systematically followup on and respond to abuse complaints
* [ ] diagnose and resolve delivery issue (e.g. [yahoo delivery
problems](https://gitlab.torproject.org/tpo/tpa/team/-/issues/34134))
* provide reliable delivery for users ("my email ends up in spam!")
* possible implementations:
* split mailing lists out of eugeni
* setup submit-01 to deliver people's emails ([issue 30608](https://gitlab.torproject.org/tpo/tpa/team/-/issues/30608)))
* split schleuder out of eugeni (or retire)
* stop using eugeni as a smart host (each host sends its own
* [ ] provide reliable delivery for users ("my email ends up in spam!")
* [ ] possible implementations:
* [ ] split mailing lists out of eugeni
* [ ] setup submit-01 to deliver people's emails ([issue 30608](https://gitlab.torproject.org/tpo/tpa/team/-/issues/30608)))
* [ ] split schleuder out of eugeni (or retire)
* [ ] stop using eugeni as a smart host (each host sends its own
email, particularly RT and CiviCRM)
* retire old services:
* SVN ([issue 17202](https://gitlab.torproject.org/tpo/tpa/team/-/issues/17202))
* fpcentral ([issue 40009](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40009))
* scale GitLab with ongoing and surely expanding usage
* possibly split in multiple server
* throw more hardware at it?
* monitoring?
* provide reliable and simple continuous integration services
* retire Jenkins
* replace with GitLab CI, with Windows, Mac and Linux runners
* fix the blog formatting and comment moderation, possible solutions:
* migrate to a static website and Discourse
* fix formatting and improve moderation within Drupal
* improve communications and monitoring:
* document "downtimes of 1 hour or longer", in a status page [issue
* [ ] retire old services:
* [ ] SVN ([issue 17202](https://gitlab.torproject.org/tpo/tpa/team/-/issues/17202))
* [ ] fpcentral ([issue 40009](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40009))
* [ ] scale GitLab with ongoing and surely expanding usage
* [ ] possibly split in multiple server
* [ ] throw more hardware at it?
* [ ] monitoring?
* [ ] provide reliable and simple continuous integration services
* [ ] retire Jenkins
* [ ] replace with GitLab CI, with Windows, Mac and Linux runners
* [ ] fix the blog formatting and comment moderation, possible solutions:
* [ ] migrate to a static website and Discourse
* [ ] fix formatting and improve moderation within Drupal
* [ ] improve communications and monitoring:
* [ ] document "downtimes of 1 hour or longer", in a status page [issue
40138](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40138)
* reduce alert fatigue in Nagios
* publicize debugging tools (Grafana, user-level logging in systemd
* [ ] reduce alert fatigue in Nagios
* [ ] publicize debugging tools (Grafana, user-level logging in systemd
services)
* encourage communication and ticket creation
* move root@ and tpa "noise" to RT ([ticket 31242]( https://gitlab.torproject.org/tpo/tpa/team/-/issues/31242)), make a real
* [ ] encourage communication and ticket creation
* [ ] move root@ and tpa "noise" to RT ([ticket 31242]( https://gitlab.torproject.org/tpo/tpa/team/-/issues/31242)), make a real
mailing list for admins so that gaba and non-tech can join
* be realistic:
* cover for the day-to-day routine tasks
* reserve time for the unexpected
* reduce expectations
* on budget: hosting expenses should rise outside of budget
* [ ] be realistic:
* [ ] cover for the day-to-day routine tasks
* [ ] reserve time for the unexpected
* [ ] reduce expectations
* [ ] on budget: hosting expenses should rise outside of budget
(January 2020: 1050EUR/mth, January 2021: 1150EUR/mth)
## Nice to have
* improve sysadmin code base
* avoid YOLO commits in Puppet (possibly: server-side linting, CI)
* publish our Puppet repository ([ticket 29387](https://gitlab.torproject.org/tpo/tpa/team/-/issues/29387))
* reduce dependency on Python 2 code (see [short term LDAP plan](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/ldap#short-term-merge-with-upstream-port-to-python-3-if-necessary))
* reduce dependency on LDAP (move hosts to Puppet? see [mid term
* [ ] improve sysadmin code base
* [ ] avoid YOLO commits in Puppet (possibly: server-side linting, CI)
* [ ] publish our Puppet repository ([ticket 29387](https://gitlab.torproject.org/tpo/tpa/team/-/issues/29387))
* [ ] reduce dependency on Python 2 code (see [short term LDAP plan](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/ldap#short-term-merge-with-upstream-port-to-python-3-if-necessary))
* [ ] reduce dependency on LDAP (move hosts to Puppet? see [mid term
LDAP plan](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/ldap#mid-term-move-hosts-to-puppet-possibly-replace-ud-ldap-with-simpler-dashboard))
* avoid duplicate git hosting infrastructure
* retire gitolite, gitweb ([issue 36](https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/36))
* retire more old services:
* testnet? talk to network team
* gitolite (replaced with GitLab, see above)
* gitweb (replaced with GitLab, see above)
* jenkins (replaced with GitLab, see above)
* provide secure, end-to-end authentication of Tor source code
* [ ] avoid duplicate git hosting infrastructure
* [ ] retire gitolite, gitweb ([issue 36](https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/36))
* [ ] retire more old services:
* [ ] testnet? talk to network team
* [ ] gitolite (replaced with GitLab, see above)
* [ ] gitweb (replaced with GitLab, see above)
* [ ] jenkins (replaced with GitLab, see above)
* [ ] provide secure, end-to-end authentication of Tor source code
([issue 81](https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/81))
* finish retiring old hardware (moly, [ticket 29974](https://gitlab.torproject.org/legacy/trac/-/issues/29974))
* varnish to nginx conversion (#32462)
* GitLab pages hosting
* experiment with containers/kubernetes for CI/CD
* upgrade to bullseye
* cover for some metrics services ([issue 40125](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40125))
* [ ] finish retiring old hardware (moly, [ticket 29974](https://gitlab.torproject.org/legacy/trac/-/issues/29974))
* [ ] varnish to nginx conversion (#32462)
* [ ] GitLab pages hosting
* [ ] experiment with containers/kubernetes for CI/CD
* [ ] upgrade to bullseye
* [ ] cover for some metrics services ([issue 40125](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40125))
## Non-goals
* complete email service: not enough time / budget (or delegate + pay Riseup)
* "provide development/experimental VMs": would be possible through
* [ ] complete email service: not enough time / budget (or delegate + pay Riseup)
* [ ] "provide development/experimental VMs": would be possible through
GitLab CD, to be investigated once we have GitLab CI solidly
running
* "improve interaction between TPA and devs when new services are
* [ ] "improve interaction between TPA and devs when new services are
setup": see "improve communications" above, and "experimental
VMs". The endgame here is people will be able to deploy their own
services through Docker, but this will likely not happen in 2021
* static mirror network retirement / rearchitecture: we want to test
* [ ] static mirror network retirement / rearchitecture: we want to test
out GitLab pages first and see if it can provide a decent
alternative
* web development stuff: goals like "finish main website transition", "broken links on website"... should be covered in the web team, but the capacity of TPA is affected by hiro working on the web stuff
* TODO: are service admins still a thing? should we cover for things
* [ ] web development stuff: goals like "finish main website transition", "broken links on website"... should be covered in the web team, but the capacity of TPA is affected by hiro working on the web stuff
* [ ] TODO: are service admins still a thing? should we cover for things
like the metrics team?
* complete puppetization: old legacy services are not in Puppet. that
* [ ] complete puppetization: old legacy services are not in Puppet. that
is fine: we keep maintaining them by hand when relevant, but new
services should all be built in Puppet
* replace Nagios with Prometheus: not a short term goal, no clear
* [ ] replace Nagios with Prometheus: not a short term goal, no clear
benefit. reduce the noise in Nagios instead
* solr/search.tpo deployment (#33106), postponed to 2022
* web metrics (#32996), postponed to 2022
* [ ] solr/search.tpo deployment (#33106), postponed to 2022
* [ ] web metrics (#32996), postponed to 2022
# Quarterly breakdown
......@@ -120,15 +120,15 @@ this roadmap is concerned. It should include items we are fairly
certain to be able to complete within the next few months or
so. Postponing those could cause problems.
* email delivery improvements:
* handle bounces in CiviCRM ([issue 33037](https://gitlab.torproject.org/tpo/tpa/team/-/issues/33037))
* followup on abuse complaints
* diagnose and resolve delivery issue (e.g. [yahoo delivery
* [ ] email delivery improvements:
* [ ] handle bounces in CiviCRM ([issue 33037](https://gitlab.torproject.org/tpo/tpa/team/-/issues/33037))
* [ ] followup on abuse complaints
* [ ] diagnose and resolve delivery issue (e.g. [yahoo delivery
problems](https://gitlab.torproject.org/tpo/tpa/team/-/issues/34134))
* GitLab CI deployment, plan for Jenkins retirement
* plan for blog replacement? not the migration!
* setup a long-term/sponsored discourse instance?
* document "downtimes of 1 hour or longer", in a status page [issue
* [ ] GitLab CI deployment, plan for Jenkins retirement
* [ ] plan for blog replacement? not the migration!
* [ ] setup a long-term/sponsored discourse instance?
* [ ] document "downtimes of 1 hour or longer", in a status page [issue
40138](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40138)
## Q2
......@@ -138,17 +138,17 @@ Second quarter is a little more vague, but should still be
wait a little longer or that are part of longer projects that will
take longer to complete.
* retire old services:
* SVN ([issue 17202](https://gitlab.torproject.org/tpo/tpa/team/-/issues/17202))
* fpcentral ([issue 40009](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40009))
* establish plan for gitolite/gitweb retirement
* improve sysadmin code base
* avoid YOLO commits in Puppet (possibly: server-side linting, CI)
* publish our Puppet repository ([ticket 29387](https://gitlab.torproject.org/tpo/tpa/team/-/issues/29387))
* reduce dependency on Python 2 code (see [short term LDAP plan](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/ldap#short-term-merge-with-upstream-port-to-python-3-if-necessary))
* reduce dependency on LDAP (move hosts to Puppet? see [mid term
* [ ] retire old services:
* [ ] SVN ([issue 17202](https://gitlab.torproject.org/tpo/tpa/team/-/issues/17202))
* [ ] fpcentral ([issue 40009](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40009))
* [ ] establish plan for gitolite/gitweb retirement
* [ ] improve sysadmin code base
* [ ] avoid YOLO commits in Puppet (possibly: server-side linting, CI)
* [ ] publish our Puppet repository ([ticket 29387](https://gitlab.torproject.org/tpo/tpa/team/-/issues/29387))
* [ ] reduce dependency on Python 2 code (see [short term LDAP plan](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/ldap#short-term-merge-with-upstream-port-to-python-3-if-necessary))
* [ ] reduce dependency on LDAP (move hosts to Puppet? see [mid term
LDAP plan](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/ldap#mid-term-move-hosts-to-puppet-possibly-replace-ud-ldap-with-simpler-dashboard))
* scale/split gitlab?
* [ ] scale/split gitlab?
## Q3
......@@ -162,16 +162,16 @@ everything in the air, so obviously plans kind of went out the
window. Hopefully this won't happen again and the pandemic will
somewhat subside, but we should plan for the worst.
* jenkins retirement?
* deprecate blog comments? replace with discourse?
* improve communications/monitoring
* [ ] jenkins retirement?
* [ ] deprecate blog comments? replace with discourse?
* [ ] improve communications/monitoring
## Q4
Obviously, the fourth quarter is sheer crystal balling at this stage,
but it should still be an interesting exercise to perform.
* gitolite/gitweb retirement?
* [ ] gitolite/gitweb retirement?
# 2020 roadmap evaluation
......
......