Loading service/jenkins.md +27 −14 Original line number Diff line number Diff line Loading @@ -35,6 +35,10 @@ by Apache, which handles TLS. [Jenkins Job Builder][] is installed through the official [Debian package](https://tracker.debian.org/pkg/jenkins-job-builder). Slaves are installed through the `debian_build_box` Puppet class. TODO: how do slaves get hooked up to Jenkins? Is that automatic? ## SLA Jenkins is currently "low availability": it doesn't have any Loading @@ -47,11 +51,11 @@ TODO: overview and diagram ### Jobs configuration Jobs in Jenkins are configured using [Jenkins Job Builder][] which creates the jobs in Jenkins based on a set of YAML configuration files. TODO: mention Groovy and pipelines? Jenkins is configured using [Jenkins Job Builder][] which is based a set of YAML configuration files. In theory, job definitions are usually written in a Java-based [Apache Groovy](https://en.wikipedia.org/wiki/Apache_Groovy) domain-specific language, but in practice we only operate on the YAML files. Those define "pipelines" which run multiple "jobs". In our configuration, the YAML files are managed in the [jenkins/jobs.git](https://gitweb.torproject.org/project/jenkins/jobs.git) repository. When commits are pushed there, a Loading Loading @@ -80,7 +84,14 @@ Jenkins doesn't use a traditional (ie. SQL) database. Instead, data like jobs, logs and so on are stored on disk in `/var/lib/jenkins/`, inside XML, plain text logfiles, and other files. todo: chroots Builders also have copies of various Debian and Ubuntu "chroots", managed through the `schroot` program. Those chroots are managed through the `debian_build_box` Puppet class, which setup the Jenkins slave but also the various chroots. In practice, new chroots are managed in the `modules/debian_build_box/files/sbin/setup-all-dchroots` script, in `tor-puppet.git`. ### Authentication Loading Loading @@ -148,8 +159,13 @@ No Git consumers using SCM API plugin for: https://git.torproject.org/admin/tsa- Which comes straight out of the plain text output of the web hook. TODO: then what? how do yaml files execute jobs? through the tools thing? do we want to get into that? The actual job configuration defines what happens next. But in general, the `jenkins/tools.git` repository has a lot of common code that gets ran in jobs. In practice, we generally copy-paste a bunch of stuff until things work. TODO: this is obviously incomplete, but it might not be worth walking through the entire `jenkins/tools.git` repository... ### Interfaces Loading Loading @@ -197,12 +213,9 @@ TODO: document which jobs exist, broadly, and who uses Jenkins. ## Monitoring and testing <!-- describe how this service is monitored and how it can be tested --> <!-- after major changes like IP address changes or upgrades. describe --> <!-- CI, test suites, linting, how security issues and upgrades are --> <!-- tracked --> TODO: chroots monitoring? Chroots are monitored for freshness by Nagios (`dsa-check-dchroots-current`), but otherwise the service does not have special monitoring. ## Logs and metrics Loading Loading
service/jenkins.md +27 −14 Original line number Diff line number Diff line Loading @@ -35,6 +35,10 @@ by Apache, which handles TLS. [Jenkins Job Builder][] is installed through the official [Debian package](https://tracker.debian.org/pkg/jenkins-job-builder). Slaves are installed through the `debian_build_box` Puppet class. TODO: how do slaves get hooked up to Jenkins? Is that automatic? ## SLA Jenkins is currently "low availability": it doesn't have any Loading @@ -47,11 +51,11 @@ TODO: overview and diagram ### Jobs configuration Jobs in Jenkins are configured using [Jenkins Job Builder][] which creates the jobs in Jenkins based on a set of YAML configuration files. TODO: mention Groovy and pipelines? Jenkins is configured using [Jenkins Job Builder][] which is based a set of YAML configuration files. In theory, job definitions are usually written in a Java-based [Apache Groovy](https://en.wikipedia.org/wiki/Apache_Groovy) domain-specific language, but in practice we only operate on the YAML files. Those define "pipelines" which run multiple "jobs". In our configuration, the YAML files are managed in the [jenkins/jobs.git](https://gitweb.torproject.org/project/jenkins/jobs.git) repository. When commits are pushed there, a Loading Loading @@ -80,7 +84,14 @@ Jenkins doesn't use a traditional (ie. SQL) database. Instead, data like jobs, logs and so on are stored on disk in `/var/lib/jenkins/`, inside XML, plain text logfiles, and other files. todo: chroots Builders also have copies of various Debian and Ubuntu "chroots", managed through the `schroot` program. Those chroots are managed through the `debian_build_box` Puppet class, which setup the Jenkins slave but also the various chroots. In practice, new chroots are managed in the `modules/debian_build_box/files/sbin/setup-all-dchroots` script, in `tor-puppet.git`. ### Authentication Loading Loading @@ -148,8 +159,13 @@ No Git consumers using SCM API plugin for: https://git.torproject.org/admin/tsa- Which comes straight out of the plain text output of the web hook. TODO: then what? how do yaml files execute jobs? through the tools thing? do we want to get into that? The actual job configuration defines what happens next. But in general, the `jenkins/tools.git` repository has a lot of common code that gets ran in jobs. In practice, we generally copy-paste a bunch of stuff until things work. TODO: this is obviously incomplete, but it might not be worth walking through the entire `jenkins/tools.git` repository... ### Interfaces Loading Loading @@ -197,12 +213,9 @@ TODO: document which jobs exist, broadly, and who uses Jenkins. ## Monitoring and testing <!-- describe how this service is monitored and how it can be tested --> <!-- after major changes like IP address changes or upgrades. describe --> <!-- CI, test suites, linting, how security issues and upgrades are --> <!-- tracked --> TODO: chroots monitoring? Chroots are monitored for freshness by Nagios (`dsa-check-dchroots-current`), but otherwise the service does not have special monitoring. ## Logs and metrics Loading
