TPA issueshttps://gitlab.torproject.org/groups/tpo/tpa/-/issues2023-08-10T19:10:13Zhttps://gitlab.torproject.org/tpo/tpa/gitlab-lobby/-/issues/17Make separate settings.py file for production and development2023-08-10T19:10:13Zasu1996Make separate settings.py file for production and developmentWe can add separate settings.py file for production and development and use python-decouple to increase the security of certain elements.
Also, separate files can help in preventing really long settings file
This can be an extension to...We can add separate settings.py file for production and development and use python-decouple to increase the security of certain elements.
Also, separate files can help in preventing really long settings file
This can be an extension to #5
Opinions on this? @ahf @gabahttps://gitlab.torproject.org/tpo/tpa/gitlab-lobby/-/issues/18Bulk approval seems to have stopped working2023-08-10T19:10:13ZNick MathewsonBulk approval seems to have stopped workingI just tried to approve some accounts from the admin page, and nothing changed. I had to approve them one by one from the individual request pages instead.
This was roughly 1 minute ago, in case the timestamp is helpful. :)I just tried to approve some accounts from the admin page, and nothing changed. I had to approve them one by one from the individual request pages instead.
This was roughly 1 minute ago, in case the timestamp is helpful. :)https://gitlab.torproject.org/tpo/tpa/gitlab-lobby/-/issues/19Remember who approved a request, and when.2023-08-10T19:10:13ZNick MathewsonRemember who approved a request, and when.It would be good to display, along with each approved request, the admin who approved it, and when. That way, we can ask each other, "hey, why did you approve such-and-such"?
This would need a corresponding database change, unless we'r...It would be good to display, along with each approved request, the admin who approved it, and when. That way, we can ask each other, "hey, why did you approve such-and-such"?
This would need a corresponding database change, unless we're already storing this.https://gitlab.torproject.org/tpo/tpa/gitlab-lobby/-/issues/21Separate status for not-yet-approved-or-rejected requests2023-08-10T19:10:13ZNick MathewsonSeparate status for not-yet-approved-or-rejected requestsRight now, every request begins life in a "no" state and may become "yes" if anybody approves it.
Instead, I think requests should begin in a "pending" state, and require an admin to set them to "no" or "yes".
There should be no behavi...Right now, every request begins life in a "no" state and may become "yes" if anybody approves it.
Instead, I think requests should begin in a "pending" state, and require an admin to set them to "no" or "yes".
There should be no behavior difference between "pending" and "no" for now -- this feature would just be a way to distinguish between requests that nobody has looked at, and requests that somebody has looked at and rejected for now.https://gitlab.torproject.org/tpo/tpa/gitlab-lobby/-/issues/22Notify admins if any request is pending for more than N hours2023-08-10T19:10:13ZNick MathewsonNotify admins if any request is pending for more than N hoursOnce #21 is done, it would be good to have a feature to notify the admins if there are any requests that have sat around for a long time without being approved or rejected.Once #21 is done, it would be good to have a feature to notify the admins if there are any requests that have sat around for a long time without being approved or rejected.https://gitlab.torproject.org/tpo/tpa/gitlab-lobby/-/issues/23Automatically delete (or archive?) approved or denied requests after N days2023-08-10T19:10:13ZNick MathewsonAutomatically delete (or archive?) approved or denied requests after N daysOnce #21 is done, it would be handy to not have to delete requests by hand. Instead, there should be some mechanism that automatically deletes or archives requests after they have been approved or rejected for some number of days (say, ...Once #21 is done, it would be handy to not have to delete requests by hand. Instead, there should be some mechanism that automatically deletes or archives requests after they have been approved or rejected for some number of days (say, 7). The number should be configurable.https://gitlab.torproject.org/tpo/tpa/gitlab-lobby/-/issues/24Add the ability to ask questions about a request2023-08-10T19:10:12ZNick MathewsonAdd the ability to ask questions about a requestThis is a harder item than some of the other tickets I've created; it's more of a wishlist thing.
It might be nice if instead of rejecting or accepting a request, admins could ask, "huh"? For example, consider a request that just says ...This is a harder item than some of the other tickets I've created; it's more of a wishlist thing.
It might be nice if instead of rejecting or accepting a request, admins could ask, "huh"? For example, consider a request that just says "trying to log in" Is that a real request? Did the user not understand what the instructions said? Or is it just automated spam?
If we had a feature like this, it would probably need to go along with a feature for users to edit their requests or cancel them.https://gitlab.torproject.org/tpo/tpa/gitlab-lobby/-/issues/25Track average time between request and response, and tell users when they mak...2023-08-10T19:10:13ZNick MathewsonTrack average time between request and response, and tell users when they make their requestIf we remember how long it takes for the average approved request to _get_ approved (#19) we can give users a useful message when they make their request. Maybe something like, "M% of requests are approved within N hours."If we remember how long it takes for the average approved request to _get_ approved (#19) we can give users a useful message when they make their request. Maybe something like, "M% of requests are approved within N hours."https://gitlab.torproject.org/tpo/tpa/gitlab-lobby/-/issues/26Fix page layout bug2021-05-03T13:42:11ZmfonismFix page layout bugI stumbled upon this bug while working on !15.
Currently the body element is a 100% high flex container with its content vertically centred. This works perfectly when the content is totally contained within the body element.
When the c...I stumbled upon this bug while working on !15.
Currently the body element is a 100% high flex container with its content vertically centred. This works perfectly when the content is totally contained within the body element.
When the content grows taller than the body element, however, it leads to a situation where part of the content flows out of the top of the viewport and cannot be scrolled to, as demonstrated in this plunk https://plnkr.co/edit/tqAf08kTLTE2miz2?open=lib%2Fscript.js&preview.https://gitlab.torproject.org/tpo/tpa/gitlab-lobby/-/issues/27Prefer the in-built unique option to custom unique validator2021-05-03T13:42:16ZmfonismPrefer the in-built unique option to custom unique validatorWe're currently using a custom validator to enforce uniqueness of usernames in lobby requests. Django already provides the option `django.models.fields.Field.unique`, and in the spirit of DRY we should be using this.We're currently using a custom validator to enforce uniqueness of usernames in lobby requests. Django already provides the option `django.models.fields.Field.unique`, and in the spirit of DRY we should be using this.https://gitlab.torproject.org/tpo/tpa/team/-/issues/40077Nginx/GitLab Prometheus exporter should not be hardcoded2022-04-06T20:54:12ZanarcatNginx/GitLab Prometheus exporter should not be hardcodedHi!
I have noticed the services were "degraded" on `gitlab-02` this morning. After digging a little, it seemed the `prometheus-nginx-exporter` server was failing:
```
Nov 3 15:41:16 gitlab-02/gitlab-02 prometheus-nginx-exporter[2129]:...Hi!
I have noticed the services were "degraded" on `gitlab-02` this morning. After digging a little, it seemed the `prometheus-nginx-exporter` server was failing:
```
Nov 3 15:41:16 gitlab-02/gitlab-02 prometheus-nginx-exporter[2129]: 2020/11/03 15:41:16 Could not create Nginx Client: Failed to create NginxClient: failed to parse response body "<!DOCTYPE html>\n<html class=\"devise-layout-html\">\n<head prefix=\"og: http://ogp.me/ns#\">\n<meta charset=\"utf-8\">\n<link as=\"style\" href=\"http://127.0.0.1:8080/assets/application-bf1ba5d5d3395adc5bad6f17cc3cb21b3fb29d3e3471a5b260e0bc5ec7a57bc4.css\" rel=\"preload\">\n<link as=\"style\" href
```
... basically, it was hitting the GitLab frontpage instead of the `stub_status` page.
To clear the Nagios warning, I have added a Nginx `location` block in a new arbitrary `server` that does only that service. But then I realized that nothing was actually scraping the exporter: the main prometheus server does not have data on the nginx server running on Gitlab-02.
<del>Maybe I missed something, but it seems to me this data should be scraped on the prometheus server.</del> As @hiro pointed out in the comments, the target is being correctly scraped. But it's done by hardcoding the configuration on the Prometheus server, instead of using exported resources. We should still fix this and contribute our precious puppet code back upstream to the prometheus module, where it belongs...
The way this is typically done is by including a `profile` (e.g. `profile::prometheus::apache_exporter`) which configures a class from the 3rdparty Prometheus module (e.g. `prometheus::apache_exporter`). Except in this case, the third-party prometheus module doesn't support the nginx exporter we're using: it only supports the `vts` one.
So one of two things:
1. write our own prometheus exporter wrapper in Puppet for the exporter we're using (and, of course, ideally, contribute it back upstream), or;
2. just hack something together in `profile::prometheus::nginx_exporter` (which is basically the same, except we can't rely on sharing this with the community in the future)
We could start with hacking something together real quick (option 2) and move the code to the 3rdparty module (option 1) once we're happy with the result. In any case, we need something better than the current situation, because as things are, the prometheus nginx exporter just does nothing. It just sits there waiting for a prometheus scraper that never comes. It's pretty harmless, but then we don't get precious nginx metrics that could help us diagnose performance issues in the future.
assigned to @hiro because she built this, and it (working with prometheus puppet upstream) would be an important thing to learn. let me know if you want me to do it or need help with this!https://gitlab.torproject.org/tpo/tpa/team/-/issues/40092Audit gitolite for git repos that should notify tor-commits and irc2021-04-19T20:27:43ZRoger DingledineAudit gitolite for git repos that should notify tor-commits and ircWe have some recent git repos, like support, that ought to send commits to the tor-commits@ list and maybe also to the irc bots.
We should go through the gitolite conf and see which ones they are, and make them right.We have some recent git repos, like support, that ought to send commits to the tor-commits@ list and maybe also to the irc bots.
We should go through the gitolite conf and see which ones they are, and make them right.https://gitlab.torproject.org/tpo/tpa/team/-/issues/40095Build Windows/Mac CI infrastructure that is usable by all teams in the near f...2022-12-07T15:24:31ZAlexander Færøyahf@torproject.orgBuild Windows/Mac CI infrastructure that is usable by all teams in the near futureThe projects we have in Tor currently utilizes a mixture of different CI systems to ensure some form of quality assurance as part of the software development process:
- Jenkins (provided by TPA)
- Gitlab CI (currently Docker builders ki...The projects we have in Tor currently utilizes a mixture of different CI systems to ensure some form of quality assurance as part of the software development process:
- Jenkins (provided by TPA)
- Gitlab CI (currently Docker builders kindly provided by the FDroid project via Hans from The Guardian Project)
- Travis CI (used by some of our projects such as tpo/core/tor.git for Linux and MacOS builds)
- Appveyor (used by tpo/core/tor.git for Windows builds)
One big benefit that we have seen with Gitlab CI is how easy it is for each project to initially configure CI for their respective project and maintain it without sysadmin/CI-admin(?) involvement. This I believe is an important requirement here to distribute the workload of actually setting this up.
None of the goals of this ticket will solve the issue that Apple have recently announced the M1 processor and we have no way of virtualizing/emulating the ARM64 macOS builds, yet. This will have to be something we look into in the future. Other organizations will have this problem too, so we might be able to piggy-bag on them.
Jenkins have been hard for the network team to maintain and weasel have been a great help there. I am not sure how Jenkins is used by other teams right now, except that I know the web teams are utilizing it to publish changes to our websites to the production servers.
Travis CI recently announced a new scheme where MacOS builds will become a more scarce resource on their platform. This mixed with the wish to have faster builds for the network team is what triggered this post. We are already on some "free software beneficial plan" where they support us with more points, but it wont be enough for the network team to go through a month of MacOS builds for our needs, unfortunately.
Appveyor is very slow, and it often leads to frustrations amongst the network team members.
It would awesome if we could somehow reserve two (ideally) "fast" Debian-based machines on TPO infrastructure to build the following:
- Run Gitlab CI runners via KVM (initially with focus on Windows x86-64 and macOS x86-64). This will replace the need for Travis CI and Appveyor. This should allow both the network team, application team, and anti-censorship team to test software on these platforms (either by building in the VMs or by fetching cross-compiled binaries on the hosts via the Gitlab CI pipeline feature). Since none(?) of our engineering staff are working full-time on MacOS and Windows, we rely quite a bit on this for QA.
- Run Gitlab CI runners via KVM for the BSD's. Same argument as above, but is much less urgent.
- Spare capacity (once we have measured it) can be used a generic Gitlab CI Docker runner in addition to the FDroid builders.
- The faster the CPU the faster the builds.
- Lots of RAM allows us to do things such as having CoW filesystems in memory for the ephemeral builders and should speed up builds due to faster I/O.
I am by no means an expert on this, but I don't believe these machines can be virtual machines as we need to spawn other virtual machines using the "full virtualization" that is provided by "modern" x86-64 CPUs. It might be "recursive" virtualization works (some cloud providers have that), but I have no idea what the implications are for that, especially with the cluster management software we use for other physical hosts in TPO.
Please let me know if I need to add more details here :-)
I have no idea what label to put this in, so folks from TPA who organize these things are welcome to figure out where this belong the best.Alexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40096automate/puppetize (or replace) mandos installation2023-03-30T01:37:58Zanarcatautomate/puppetize (or replace) mandos installationwe use Mandos to unlock server's LUKS-encrypted partitions on boot, but the [setup is done manually](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/new-machine-mandos/). that is error-prone and slow, it's actually one of the sl...we use Mandos to unlock server's LUKS-encrypted partitions on boot, but the [setup is done manually](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/new-machine-mandos/). that is error-prone and slow, it's actually one of the slowest part of our install procedure.
in #31239, we identified the following steps to get this ball rolling:
* [x] export/import firewall rules (in `roles::fde`)
* [ ] generate and export new LUKS key in Puppet
* [ ] import new key on mandos server
* [ ] rebuild initramfs
We should also consider alternatives to Mandos, if this "Puppetization" is too complicated. On the top of my head, there is also:
* [arver](https://code.immerda.ch/immerda/apps/arver/)
* secure enclaves and secureboot, e.g. [this](https://blog.habets.se/2015/03/How-to-boot-an-encrypted-system-safely.html), [this](https://blog.dowhile0.org/2017/10/18/automatic-luks-volumes-unlocking-using-a-tpm2-chip/), or [this](https://safeboot.dev/), [this](https://github.com/xmikos/cryptboot), or [mortar](https://github.com/noahbliss/mortar)
* [clevis](https://github.com/latchset/clevis) (in Debian since buster), [introduced by RedHat as NDBE](https://www.redhat.com/en/blog/easier-way-manage-disk-decryption-boot-red-hat-enterprise-linux-75-using-nbde)cleanup and publish the sysadmin codebasehttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40116disable TLS 1.0 and 1.12024-03-25T20:05:50Zweasel (Peter Palfrader)disable TLS 1.0 and 1.1ssllabs now gives bad grades for servers that even offer TLS 1.0 and 1.1. Modern browsers deprecated TLS 1.0 or 1.1.
Re support see also:
* https://en.wikipedia.org/wiki/Transport_Layer_Security#Applications_and_adoption
* https://cani...ssllabs now gives bad grades for servers that even offer TLS 1.0 and 1.1. Modern browsers deprecated TLS 1.0 or 1.1.
Re support see also:
* https://en.wikipedia.org/wiki/Transport_Layer_Security#Applications_and_adoption
* https://caniuse.com/tls1-1
* https://caniuse.com/tls1-2
And it seems 1.2 has been around quite long. I propose we stop offering TLS 1.0 and 1.1 on our webservers.https://gitlab.torproject.org/tpo/tpa/team/-/issues/40124Move auth to Grafana instead of puppet2024-01-30T15:20:51ZHiroMove auth to Grafana instead of puppetThis ticket tracks moving authentication out of puppet for Grafana.
In #40102 we created a shared account for people accessing Grafana. This doesn't really scale and it would be nice if we could create accounts in Grafana directly.This ticket tracks moving authentication out of puppet for Grafana.
In #40102 we created a shared account for people accessing Grafana. This doesn't really scale and it would be nice if we could create accounts in Grafana directly.anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40129user management procedures are poorly documented2023-10-20T18:57:06Zanarcatuser management procedures are poorly documentedas identified by @arma in https://gitlab.torproject.org/tpo/tpa/team/-/issues/40126#note_2721379, it's not really clear how to actually create and remove accounts. we do have https://gitlab.torproject.org/tpo/tpa/team/-/issues/32519 whic...as identified by @arma in https://gitlab.torproject.org/tpo/tpa/team/-/issues/40126#note_2721379, it's not really clear how to actually create and remove accounts. we do have https://gitlab.torproject.org/tpo/tpa/team/-/issues/32519 which concerns the overall onboarding/offboarding process, but the actually nitty-gritty details of *how* to do things for sysadmins is really badly documented. in https://gitlab.torproject.org/tpo/tpa/team/-/issues/40126#note_2721468, i noted:
> This documentation seems to be a total mess. There is:
>
> * [howto/new-person](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/new-person) which you have found and seems to document how to get a new *sysadmin* on board
> * [doc/accounts](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/doc/accounts) which documents "accounts" in general, and is more targeted at users
> * [howto/create-a-new-user](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/create-a-new-user) actually documents how to create a new user
> * [howto/ldap](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/ldap) which documents "LDAP" in general and has a rather poor user-facing documentation and is mostly targeted about running the service
> * and then of course userdir-ldap-cgi has [its own inline documentation](https://db.torproject.org/) maintained as HTML/Perl templates shipped with the debian package and managed through git.
>
> Someone(tm) needs to sit down and make sense of this. I kind of made matters worse myself by creating howto/ldap and howto/new-person of course... :( so I guess i'm probably that someone.
So the task here is to merge or split or cleanup those pages so that one doesn't get lost like @arma did. Here it's not a matter of policy, it's just about creating a cohesive documentation. I suspect the following should happen, but this is just a first brainstorm and i'm open to suggestions:
- [ ] [howto/new-person](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/new-person) - should be merged into another page, a special section in create-new-user maybe? or renamed to "new-admin"?
- [ ] [doc/accounts](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/doc/accounts) - merge with create-a-new-user?
- [ ] [howto/create-a-new-user](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/create-a-new-user) - merge with howto/ldap? but keep in mind there are things about sudo in there
- [ ] [howto/ldap](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/ldap) - should this take over the userdir-ldap-cgi documentation below and cover *everything*?
- [ ] userdir-ldap-cgi has [its own inline documentation](https://db.torproject.org/) - maybe deprecate this and point to the wiki?
TBD.
Also note that our [retirement procedures](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/retire-a-user) are *also* fairly inadequate and would need much love. this was supposed to be covered by #32519 but was somehow overlooked... :(anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/anon_ticket/-/issues/8Feature: MR Links and View2021-01-28T20:56:13ZcypherpunksFeature: MR Links and ViewIssue detail view currently shows all notes and indicates when a merge request was made, but does not have a link to MR. Although it's possible to link this without adding any new views/urls, by linking directly to gitlab, this would be ...Issue detail view currently shows all notes and indicates when a merge request was made, but does not have a link to MR. Although it's possible to link this without adding any new views/urls, by linking directly to gitlab, this would be a lot of API calls.
Rec instead creating a new view showing the project MR details with a structure like /project/project_name/merge-requests/# and connect the project detail view to the merge-request-detail-view. That way, only a single gitlab API call is made at the moment that the link is clicked on to get the MR details, which will be used to populate the view.https://gitlab.torproject.org/tpo/tpa/anon_ticket/-/issues/14Create Issue when in a specific project doesn't prepopulate form2021-05-03T13:40:43ZcypherpunksCreate Issue when in a specific project doesn't prepopulate formCurrently, the "Create issue" link on a project detail view links to the generic "create issue" view, and the user then needs to select the appropriate project from the dropdown. It would be nice to have it auto-select the dropdown objec...Currently, the "Create issue" link on a project detail view links to the generic "create issue" view, and the user then needs to select the appropriate project from the dropdown. It would be nice to have it auto-select the dropdown object for you.https://gitlab.torproject.org/tpo/tpa/anon_ticket/-/issues/15Update Moderator View - Remove hard-coded links and update templates2021-05-03T13:40:44ZcypherpunksUpdate Moderator View - Remove hard-coded links and update templatesNeed to remove hard-coded links to admin on moderator view and replace template on admin actions for moderator view to custom templating so admin panel is not exposed. Should be able to use a method similar to the following: https://sta...Need to remove hard-coded links to admin on moderator view and replace template on admin actions for moderator view to custom templating so admin panel is not exposed. Should be able to use a method similar to the following: https://stackoverflow.com/questions/45176991/django-redirect-change-password-url