Skip to content
Snippets Groups Projects
Select Git revision
  • master default protected
  • tpa-rfc-80-merge-tor-and-tails-support-policies
  • puppet-merge-costs
  • rfc-77-improv
  • tails-merge-admin-mailing-lists
  • mkdocs
  • new-blog-doc
7 results
Blame Permalink
gitlab.md 41.58 KiB

GitLab is a web-based DevOps lifecycle tool that provides a Git-repository manager providing wiki, issue-tracking and continuous integration/continuous deployment pipeline features, using an open-source license, developed by GitLab Inc (Wikipedia). Tor uses GitLab mainly for issue tracking, wiki hosting and code review for now, at https://gitlab.torproject.org, after migrating from howto/trac.

Note that continuous integration is documented separately, in the CI page.

Tutorial

How to get an account?

You might already have an account! If you were active on Trac, your account was migrated with the same username and email address as Trac, unless you have an LDAP account, in which case that was used. So head over to the password reset page to get access to your account.

If your account was not migrated, send a mail to gitlab-admin@torproject.org to request a new one.

If you did not have an account in Trac and want a new account, you should request a new one at https://gitlab.onionize.space/.

How to report an issue in Tor software?

You first need to figure out which project the issue resides in. The project list is a good place to get started. Here are a few quick links for popular projects:

If you do not have a GitLab account or can't figure it out for any reason, you can also use the mailing lists. The tor-dev@lists.torproject.org mailing list is the best for now.

How to report an issue in the bugtracker itself?

If you have access to GitLab, you can file a new issue after you have searched the GitLab project for similar bugs.

If you do not have access to GitLab, you can email gitlab-admin@torproject.org.

Note about confidential issues

Note that you can mark issues as "confidentials" which will make them private to the members of the project the issue is reported on (the "developers" group and above, specifically).

Keep in mind, however, that it is still possible issue information gets leaked in cleartext, however. For example, GitLab sends email notifications in cleartext for private issue, an known upstream issue. (We have decided we cannot fix this ourselves in GitLab for now.) Some repositories might also have "web hooks" that notify IRC bots in clear text as well, although at the time of writing all projects are correctly configured.

How to contribute code?

As reporting an issue, you first need to figure out which project you are working on in the GitLab project list. Then, if you are not familiar with merge requests, you should read the merge requests introduction in the GitLab documentation. If you are unfamiliar with merge requests but familiar with GitHub's pull requests, those are similar.

Note that we do not necessarily use merge requests in all teams yet, and Gitolite still has the canonical version of the code. See issue 36 for a followup on this.

Also note that different teams might have different workflows. If a team has a special workflow that diverges from the one here, it should be documented here. Those are the workflows we know about:

If you do not have access to GitLab, please use one of the mailing lists: tor-dev@lists.torproject.org would be best.

How to quote a comment in a reply?

The "Reply" button only creates a new comment without any quoted text by default. It seems the solution to that is currently highlighting the text to quote and then pressing the r-key. See also the other keyboard shortcuts.

Alternatively, you can copy-paste the text in question in the comment form, select the pasted text, and hit the Insert a quote button which look like a styled, curly, and closing quotation mark ”.

How-to

Continuous Integration (CI)

All CI documentation resides in a different document see service/ci.

Email interactions

You can interact with GitLab by email too.

Creating a new issue

Clicking on the project issues gives a link at the bottom of the page, which says say "Email a new issue to this project".

That link should go into the "To" field of your email. The email subject becomes the title of the issue and the body the description. You can use shortcuts in the body, like /assign @foo, /estimate 1d, etc.

See the upstream docs for more details.

Commenting on an issue

If you just reply to the particular comment notification you received by email, as you would reply to an email in a thread, that comment will show up in the issue.

You need to have email notifications enabled for this to work, naturally.

You can also add a new comment to any issue by copy-pasting the issue-specific email address in the right sidebar (labeled "Issue email", introduced in GitLab 13.8).

This also works with shortcuts like /estimate 1d or /spend -1h. Note: for those you won't get notification emails back, though, while for others like /assign @foo you would.

See the upstream docs for more details.

Quick status updates by email

There are a bunch of quick actions available which are handy to update an issue. As mentioned above they can be sent by email as well, both within a comment (be it as a reply to a previous one or in a new one) or just instead of it. So, for example, if you want to update the amount of time spent on ticket $foo by one hour, find any notification email for that issue and reply to it by replacing any quoted text with /spend 1h.

How to migrate a Git repository from legacy to GitLab?

Important: this policy is still being debated. It is not clear if any or all repositories should be migrated to GitLab, see issue 36 for the discussion on this topic.

As an example of a repository migration, I have moved the wiki from gitolite to gitlab just now. I have followed the following procedure:

  1. create a project on gitlab (in tpo/tpa/wiki-archive in my case)

  2. push (manually) the latest git references present on git-rw to gitlab (git push --mirror...)

  3. if the repository is to be archived on GitLab, make it so in Settings -> General -> Advanced -> Archive project

  4. make an (executable) pre-receive hook in git-rw with an exit status of 1 warning about the new code location, example:

    $ cat /srv/git.torproject.org/repositories/project/help/wiki.git/hooks/pre-receive 
#!/bin/sh

cat <<EOF
This repository has been migrated to GitLab:

https://gitlab.torproject.org/tpo/tpa/services/-/wikis/home

Update your remotes to:

    git@gitlab.torproject.org:tpo/tpa/services.wiki.git

or:

    https://gitlab.torproject.org/tpo/tpa/services.wiki.git

See this issue for details:

https://gitlab.torproject.org/tpo/tpa/services/-/issues/34437
EOF

exit 1

    or in the case of a fully archived repository (non-writable):

    $ cat /srv/git.torproject.org/repositories/project/help/infra.git/hooks/pre-receive 
#!/bin/sh

cat <<EOF
This repository has been migrated to GitLab:

https://gitlab.torproject.org/tpo/tpa/wiki-infra-archive

We have migrated away from ikiwiki so it is not necessary anymore.

See this issue for details:

https://gitlab.torproject.org/tpo/tpa/services/-/issues/34437
EOF

exit 1

  5. in Gitolite, make the project part of the "Attic", for example

    @@ -328,13 +328,13 @@ admin/trac/TracAccountManager "The Tor Project" = "Tor specific changes to Matth
 
 repo project/help/infra
     RW+                                      = @torproject-admin
-    config gitweb.category                   = Infrastructure and Administration
-project/help/infra "The Tor Project" = "help.torproject.org infrastructure"
+    config gitweb.category                   = Attic
+project/help/infra "The Tor Project" = "help.torproject.org infrastructure (archived to GitLab: https://gitlab.torproject.org/tpo/tpa/wiki-infra-archive')"
 
 repo project/help/wiki
     RW                                       = anarcat
-    config gitweb.category                   = Infrastructure and Administration
-project/help/wiki "The Tor Project" = "help.torproject.org content"
+    config gitweb.category                   = Attic
+project/help/wiki "The Tor Project" = "help.torproject.org content (archived to GitLab: https://gitlab.torproject.org/tpo/tpa/wiki-archive')"
 
 repo project/jenkins/jobs
     RW                                       = @jenkins-admins

The only downside with that approach is that a git clone will not warn about the project redirection, but I am not sure there's a way to fix that.

See issue 36 for further discussion.

How to find the right emoji?

It's possible to add "reaction emojis" to comments and issues and merge requests in GitLab. Just hit the little smiley face and a dialog will pop up. You can then browse through the list and pick the right emoji for how you feel about the comment, but remember to be nice!

It's possible you get lost in the list. You can type the name of the emoji to restrict your search, but be warned that some emojis have particular, non-standard names that might not be immediately obvious. For example, 🎉, U+1F389 PARTY POPPER, is found as tada in the list! See this upstream issue for more details.

Hooking up a project with the bots

By default, new projects do not have notifications setup in #tor-bots like all the others. To do this, you need to configure a "Webhook", in the Settings -> Webhooks section of the project. The URL should be:

https://kgb-bot.torproject.org/webhook/

... and you should select the notifications you wish to see in #tor-bots. You can also enable notifications to other channels by adding more parameters to the URL, like (say) ?channel=%23tor-foo. The parameters are documented the KGB documentation.

Note that GitLab admins might be able to configure system-wide hooks in the admin section, although it's not entirely clear how does relate to the per-project hooks so those have not been enabled. Furthermore, it is possible for GitLab admins with root access to enable webhooks on all projects, with the webhook rake task. For example, running this on the GitLab server (currently gitlab-02) will enable the above hook on all repositories:

sudo gitlab-rake gitlab:web_hook:add URL='https://kgb-bot.torproject.org/webhook/'

Note that by default, the rake task only enables Push events. You need the following patch to enable others:

modified   lib/tasks/gitlab/web_hook.rake
@@ -10,7 +10,19 @@ namespace :gitlab do
       puts "Adding webhook '#{web_hook_url}' to:"
       projects.find_each(batch_size: 1000) do |project|
         print "- #{project.name} ... "
-        web_hook = project.hooks.new(url: web_hook_url)
+        web_hook = project.hooks.new(
+          url: web_hook_url,
+          push_events: true,
+          issues_events: true,
+          confidential_issues_events: false,
+          merge_requests_events: true,
+          tag_push_events: true,
+          note_events: true,
+          confidential_note_events: false,
+          job_events: true,
+          pipeline_events: true,
+          wiki_page_events: true,
+        )
         if web_hook.save
           puts "added".color(:green)
         else

See also the upstream issue and our GitLab issue 7 for details.

You can also remove a given hook from all repos with:

sudo gitlab-rake gitlab:web_hook:rm URL='https://kgb-bot.torproject.org/webhook/'

And, finally, list all hooks with:

sudo gitlab-rake gitlab:web_hook:list

Setting up two-factor authentication (2FA)

We strongly recommend you enable two-factor authentication on GitLab. This is well documented in the GitLab manual, but basically:

  1. first, pick a 2FA "app" (and optionally a hardware token) if you don't have one already

  2. head to your account settings

  3. register your 2FA app and save the recovery codes somewhere. if you need to enter a URL by hand, you can scan the qrcode with your phone or create one by following this format:

    otpauth://totp/$ACCOUNT?secret=$KEY&issuer=gitlab.torproject.org

    where...

    • $ACCOUNT is the Account field in the 2FA form
    • $KEY is the Key field in the 2FA form, without spaces

  4. register the 2FA hardware token if available

GitLab requires a 2FA "app" even if you intend to use a hardware token. The 2FA "app" must implement the TOTP protocol, for example the Google Authenticator or a free alternative (for example free OTP plus, see also this list from the Nextcloud project). The hardware token must implement the U2F protocol, which is supported by security tokens like the YubiKey, Nitrokey, or similar.

Deleting sensitive attachments

If a user uploaded a secret attachment by mistake, just deleting the issue is not sufficient: it turns out that doesn't remove the attachments from disk!

To fix this, ask a sysadmin to find the file in the /var/opt/gitlab/gitlab-rails/uploads/ directory. Assuming the attachment URL is:

https://gitlab.torproject.org/anarcat/test/uploads/7dca7746b5576f6c6ec34bb62200ba3a/openvpn_5.png

There should be a "hashed" directory and a hashed filename in there, which looks something like:

./@hashed/08/5b/085b2a38876eeddc33e3fbf612912d3d52a45c37cee95cf42cd3099d0a3fd8cb/7dca7746b5576f6c6ec34bb62200ba3a/openvpn_5.png

The second directory (7dca7746b5576f6c6ec34bb62200ba3a above) is the one visible in the attachment URL. The last part is the actual attachment filename, but since those can overlap between issues, it's safer to look for the hash. So to find the above attachement, you should use:

find /var/opt/gitlab/gitlab-rails/uploads/ -name 7dca7746b5576f6c6ec34bb62200ba3a

And delete the file in there. The following should do the trick:

find /var/opt/gitlab/gitlab-rails/uploads/ -name 7dca7746b5576f6c6ec34bb62200ba3a | sed 's/^/rm /' > delete.sh

Verify delete.sh and run it if happy.

Note that GitLab is working on an attachment manager that should allow web operators to delete old files, but it's unclear how or when this will be implemented, if ever.

Publishing GitLab pages

GitLab features a way to publish websites directly from the continuous integration pipelines, called GitLab pages. Complete documentation on how to publish such pages is better served by the official documentation, but creating a .gitlab-ci.yml should get you rolling. For example, this will publish a hugo site:

image: registry.gitlab.com/pages/hugo/hugo_extended:0.65.3
pages:
  script:
    - hugo
  artifacts:
    paths:
      - public
  only:
    - main

If this file is committed in a project called tpo/team/project, the pages will be published to https://tpo.pages.torproject.net/team/project/.

Accepting merge requests on wikis

It's possible to work around the limitation of Wiki permissions by creating a mirror of the git wiki backing the wikis. This way more users can suggest changes to the wiki by submitting merge requests. It's not as easy as editing the wiki, but at least provides a way for outside contributors to participate.

To do this, you'll need to create project access tokens in the Wiki and use the repository mirror feature to replicate the wiki into a separate project.

  1. in the Wiki project, head for the Settings: Access Tokens page and create a new token with write_repository access

  2. optionally, create a new project for the wiki, for example called wiki-replica. you can also use the same project as the wiki if you do not plan to host other source code specific to that project there. we'll call this the "wiki replica" in either case

  3. in the wiki replica, head for the Settings: Mirroring repositories section and fill in the details for the wiki HTTPS clone URL:

    • Git repository URL: the HTTPS URL of the Git repository (which you can find in the Clone repository page on the top-right of the wiki)
    • Mirror direction: push (only "free" option, pull is non-free)
    • Authentication method: Password (default)
    • Password: the Access token you created in the first step
    • Keep divergent refs: checked (optional, should make sure sync works in some edge cases)
    • Mirror only protected branches: checked (to keep merge requests from being needlessly mirrored to the wiki)

When you click the Mirror repository button, a sync will be triggered. Refresh the page to see status, you should see the Last successful update column updated. When you push to the replica, the wiki should be updated.

Naturally, because of limitations of GitLab, you cannot pull changes from the wiki to the replica. But considering only a limited set of users have access to the wiki in the first place, this shouldn't be a problem as long as everyone pushes to the replica.

Pager playbook

TODO: document how to handle common problems in GitLab

Troubleshooting

Upstream recommends running this command to self-test a GitLab instance:

sudo gitlab-rake gitlab:check SANITIZE=true

This command also shows general info about the GitLab instance:

sudo gitlab-rake gitlab:check SANITIZE=true

it is especially useful to find on-disk files and package versions.

Disaster recovery

In case the entire GitLab machine is destroyed, a new server should be provisionned in the howto/ganeti cluster (or elsewhere) and backups should be restored using the below procedure.

Running an emergency backup

A full backup can be ran as root with:

/usr/bin/gitlab-rake gitlab:backup:create

Backups are stored as a tar file in /srv/gitlab-backup and do not include secrets, which are backed up separately, for example with:

umask 0077 && tar -C /var/opt/gitlab -czf /srv/gitlab-backup/config_backup$(date +"\%Y\%m\%dT\%H\%M").tar.gz

See /etc/cron.d/gitlab-config-backup, and the gitlab::backup and profile::gitlab::app classes for the actual jobs that runs nightly.

baremetal recovery

Untested procedure extracted from the upstream docs:

  1. reinstall the same version you are restoring from

  2. restore the secrets backup:

    tar -C /opt/gitlab/etc/ -x -v -z -f config_backup20200627T0200.tar.gz

  3. configure and start gitlab:

    sudo gitlab-ctl reconfigure
sudo gitlab-ctl start

  4. restore the file:

    sudo cp 11493107454_2018_04_25_10.6.4-ce_gitlab_backup.tar /var/opt/gitlab/backups
sudo chown git.git /var/opt/gitlab/backups/11493107454_2018_04_25_10.6.4-ce_gitlab_backup.tar
sudo gitlab-ctl stop unicorn
sudo gitlab-ctl stop puma
sudo gitlab-ctl stop sidekiq
# Verify
sudo gitlab-ctl status
sudo gitlab-backup restore BACKUP=11493107454_2018_04_25_10.6.4-ce

Reference

Installation

Main GitLab installation

The current GitLab server was setup in the howto/ganeti cluster in a regular virtual machine. It was configured with howto/puppet with the roles::gitlab. That, in turn, relies on a series of profile elements which configure:

  • profile::gitlab::web: nginx vhost and TLS cert, depends on profile::nginx built for the howto/cache service and relying on the puppet/nginx module from the Forge
  • profile::gitlab::mail: dovecot and postfix configuration, for email replies
  • profile::gitlab::database: postgresql configuration, possibly not used by the Omnibus package, see issue 20
  • profile::gitlab::app: the core of the configuration of gitlab itself, uses the puppet/gitlab module from the Forge, with Prometheus, Grafana, and Nginx support disabled, but Redis, PostgreSQL, and Prometheus exporters enabled

This installs the GitLab Omnibus distribution which duplicates a lot of resources we would otherwise manage elsewhere in Puppet, including (but possibly not limited to):

This therefore leads to a "particular" situation regarding monitoring and PostgreSQL backups, in particular. See issue 20 for details.

Note that the first gitlab server (gitlab-01) was setup using the Ansible recipes used by the Debian.org project. That install was not working so well (e.g. 503 errors on merge requests) so we migrated to the omnibus package in March 2020, which seems to work better.

GitLab CI installation

See the CI documentation for documentation specific to GitLab CI.

GitLab pages installation

To setup GitLab pages, we followed the GitLab Pages administration manual. The steps taken were as follows:

  1. add pages.torproject.net to the public suffix list (issue 40121 and upstream PR) (although that takes months or years to propagate everywhere)
  2. add *.pages.torproject.net and pages.torproject.net to DNS (dns/domains.git repository), as A records so that LE DNS-01 challenges still work, along with a CAA record to allow the wildcard on pages.torproject.net
  3. get the wildcard cert from Let's Encrypt (in letsencrypt-domains.git)
  4. deploy the TLS certificate, some GitLab config and a nginx vhost to gitlab-02 with Puppet
  5. run the status-site pipeline to regenerate the pages

The GitLab pages configuration lives in the profile::gitlab::app Puppet class. The following GitLab settings were added:

gitlab_pages             => {
  ssl_certificate     => '/etc/ssl/torproject/certs/pages.torproject.net.crt-chained',
  ssl_certificate_key => '/etc/ssl/private/pages.torproject.net.key',
},
pages_external_url       => 'https://pages.torproject.net',

The virtual host for the pages.torproject.org domain was configured through the profile::gitlab::web class.

SLA

Migration from Trac

GitLab was put online as part of a migration from Trac, see the Trac documentation for details on the migration.

Design

GitLab is a fairly large program with multiple components. The upstream documentation has a good details of the architecture but this section aims at providing a shorter summary. Here's an overview diagram, first:

GitLab's architecture diagram

The web frontend is Nginx (which we incidentally also use in our howto/cache system) but GitLab wrote their own reverse proxy called GitLab Workhorse which in turn talks to the underlying GitLab Rails application, served by the Unicorn application server. The Rails app stores its data in a howto/postgresql database (although not our own deployment, for now, should be fixed). GitLab also offloads long-term background tasks to a tool called sidekiq.

Those all server HTTP(S) requests but GitLab is of course also accessible over SSH to push/pull git repositories. This is handled by a separate component called gitlab-shell which acts as a shell for the git user.

Workhorse, Rails, sidekiq and gitlab-shell all talk with Redis to store temporary information, caches and session information. They can also communicate with the Gitaly server which handles all communication with the git repositories themselves.

Continuous integration

GitLab also features Continuous Integration (CI). CI is handled by GitLab runners which can be deployed by anyone and registered in the Rails app to pull CI jobs. This is documented in the service/ci page.

Spam control

TODO: document lobby.

Discuss alternatives, e.g. this hackernews discussion about mediawiki moving to gitlab. Their gitlab migration documentation might give us hints on how to improve the spam situation on our end.

A few ideas on tools:

Scalability

We have not looked a lot into GitLab scalability. Upstream has reference architectures which explain how to scale for various user sizes. We have not yet looked into this, and so far have just thrown hardware at GitLab when performance issues come up.

GitLab pages

GitLab pages is "a simple HTTP server written in Go, made to serve GitLab Pages with CNAMEs and SNI using HTTP/HTTP2". In practice, the way this works is that artifacts from GitLab CI jobs get sent back to the central server.

GitLab pages is designed to scale horizontally: multiple pages servers can be deployed and fetch their content and configuration through NFS. They are rearchitecturing this with Object storage (ie. S3 through minio by default, or external existing providers) which might simplify running this but this actually adds complexity to a previously fairly simple design. Note that they have tried using CephFS instead of NFS but that did not work for some reason.

The new pages architecture also relies on the GitLab rails API for configuration (it was a set of JSON files before), which makes it dependent on the Rails API for availability, although that part of the design has exponential back-off time for unavailability of the rails API, so maybe it would survive a downtime of the rails API.

GitLab pages is not currently in use in our setup, but could be used as an alternative to the static mirroring system. See the discussion there for more information about how that compares with the static mirror system.

Update: some tests of GitLab pages were performed in January 2021, with moderate success. There are still concerns about the reliability and scalability of the service, but the service could be used for small sites at this stage. See the GitLab pages installation instructions for details on how this was setup.

Note that the pages are actually on disk, in /var/opt/gitlab/gitlab-rails/shared/pages/GROUP/.../PROJECT, for example the status site pipeline publishes to:

/var/opt/gitlab/gitlab-rails/shared/pages/tpo/tpa/status-site/

Maybe this could be abused to act as a static source in the static mirror system?

Issues

File or search for issues in the gitlab project.

Known issues:

Monitoring and testing

Monitoring right now is minimal: normal host-level metrics like disk space, CPU usage, web port and TLS certificates are monitored by Nagios with our normal infrastructure, as a black box.

Prometheus monitoring is built into the GitLab Omnibus package, so it is not configured through our Puppet like other Prometheus servers. It has still been (manually) integrated in our Prometheus setup and Grafana dashboards (see pager playbook) have been deployed.

More work is underway to improve monitoring in issue 33921.

Logs and metrics

Backups

There is a backup job (in the git user crontab) that makes sure to backup the content of /var/opt/gitlab/gitlab-rails/etc/ are backed up. We use this instead of the backup system provided by the GitLab Puppet module, because that is not covered by the gitlab-backup command. This is implemented with the tpo-gitlab-backup, a simple wrapper script which calls gitlab-backup and performs the configuration backup and rotation.

It is assumed that the existing howto/backup system will pick up those copies and store them for our normal rotation periods.

Ideally, this rather exotic backup system would be harmonized with our existing backup system, but this would require (for example) using our existing PostgreSQL infrastructure (issue 20).

Other documentation

Discussion

Meetings

Some meetings about tools discussed GitLab explicitly. Those are the minutes:

Overview

The GitLab project at Tor has been a long time coming. If you look at the history section above, you'll see it has been worked on since at least 2016, at which point an external server was setup for the "network team" to do code review. This server was ultimately retired.

The current server has been worked on since 2019, with the master ticket, issue 29400, created in the footsteps of the 2019 Brussels meeting. The service launched some time in June 2020, with a full migration of Trac tickets.

Goals

Must have

  • replacement of the Trac issue tracking server
  • rough equivalent of Trac features in GitLab

Nice to have

  • identical representation of Trac issues in GitLab, including proper issue numbering

Non-Goals

  • replacement of Gitolite (git hosting)
  • replacement of Gitweb (git hosting)
  • replacement of Jenkins (CI)
  • replacement of the static site hosting system

Those are not part of the first phase of the project, but it is understood that if one of those features gets used more heavily in GitLab, the original service MUST be eventually migrated into GitLab and turned off. We do not want to run multiple similar services at the same time (for example run both gitolite and gitaly on all git repositories, or run Jenkins and GitLab runners).

Approvals required

The GitLab migration was approved at the 2019 Brussels dev meeting.

Proposed Solution

The solution to the "code review" and "project management" problems are to deploy a GitLab instance which does not aim at managing all source code, in the first stage.

Cost

Staff not evaluated.

In terms of hardware, we start with a single virtual machine and agree that, in the worst case, we can throw a full Hetzner PX62-NVMe node at the problem (~70EUR/mth).

Alternatives considered

GitLab is such a broad project that multiple alternatives exist for different components:

  • GitHub
    • Pros:
      • widely used in the open source community
      • Good integration between ticketing system and code
    • Cons
      • It is hosted by a third party (Microsoft!)
      • Closed source
  • GitLab:
  • Pros:
    • Mostly free software
    • Feature-rich
  • Cons:
    • Complex software, high maintenance
    • "Opencore" - some interesting features are closed-source

GitLab command line clients

If you want to do batch operations or integrations with GitLab, you might want to use one of those tools, depending on your environment or prefered programming language:

GitLab upstream has a list of third-party commandline tools that is interesting as well.

Migration tools

ahf implemente the gitlab using his own home-made tools that talk to the GitLab and Trac API. but there's also tracboat which is designed to migrate from trac to GitLab.

We did not use Tracboat because it uses gitlab's DB directly and thus only works with some very specific version. Each time the database schema changes at GitLab, Tracboat needs to port to it. We prefered to use something that talked with the GitLab API.

We also didn't like the output entirely, so we modified it but still used some of its regular expressions and parser.

We also needed to implement the "ticket movement" hack (with the legacy project) which wasn't implemented in Tracboat.

Finally, we didn't want to do complete user migration, but lazily transfer only some users.

FAQ

Q: Do we have a way planned for external people to make accounts? To report bugs and to interact with them.

Answer: We tried to do it the same way as we have it in trac but we ended up having to spend a lot of time moderating out the abuse in the account.

For gitlab, accounts need to be approved manually. There is an application deployed in https://gitlab.onionize.space for people to request gitlab accounts. There are a few people at Tor periodically looking at the accounts and approving them.

Q: Do we have a process for people who will sign up to approve accounts, and documentation for how the process works?

Answer: We had some discussions among the service admin team, and they will help with documentation. So far it is ahf, gaba, nick, arma, geko. Documentation on this process needs to be created.

The end goal is that gitlab has features like user support, which allows us to create tickets from anybody who wants to submit user support requests.

Q: Does gitlab allow restricting users to certain functionality? Like, only modifying or commenting on tickets but not create repositories, etc.

Answer: It has a permission system. Also you can have security issues on the issue tracker. We don't have the same "GRP_x" approach as we had in trac, so there are some limitations.

Q: What happens to our wiki?

Answer: The wiki has been transfered and integrated. Gitlab has wikis. Specifically, the wiki will be converted to markdown, and put in a git repo. Some queries, like being able to list queries of tickets, will not be converted automatically.

Q: Will we have url-stability?

Answer: For tickets, bugs.torproject.org continue working. trac.torproject.org is read only right now and will disappear in July 2021.

Q: Did we migrated closed tickets?

Answer: Yes. And all the metadata is copied in the same way. Like, the keywords we used are converted into gitlab labels.

Q: Abuse handling. How does gitlab compare to trac in abuse handling?

Answer: We don't have the same kind of finegrained access control for individual users. So new users will have access to most things. We can't do a cypherpunks style account, because we can't stop people from changing their passwords. The idea is to build a front-end in front of gitlab, with a team of people who will moderate incoming user interactions.