Commit f2297d01 authored by ViolanteCodes's avatar ViolanteCodes
Browse files

updated readme

parent e830fcdc
# Anon-Ticket: Anonymous Gitlab Reporting for Tor
A web application that allows web users to create
anonymous tickets on the Tor Browser's Gitlab instance by leveraging
the GitLab API (Python-Gitlab package.)<br>
A web application that allows users to create
anonymous tickets on the Tor Browser's GitLab instance by leveraging
the GitLab API (Python-Gitlab package). Created as an
[Outreachy](https://www.outreachy.org/) winter of 2021 internship project
by [Maria Violante](https://mviolante.com), who is also the
current maintainer.
For bug reports and feature requests, please consider submitting an
anonymous ticket ... to Anon-Ticket ... via [Anon-Ticket](https://anonticket.onionize.space/)! ;)
<br>
***
## Table of Contents:
1. Intro and Setup
- 1.1 Quickstart
- 1.2 Project Aim and Use
2. Notes for Admins
- 2.1 Fast Project Add From Gitlab
- 2.2 Programmatic Groups and Permissions
- 2.3 Adding Moderators and Account Approvers
3. Project Structure and Function:
- 3.1 Folder Layout
- 3.2 Anon-Ticket Request-Response in a Nutshell
- 3.3 Models
- 3.4 Views
- 3.5 URL Pathing
- 3.6 Templates
4. Packages
- 4.1 General notes
- 4.2 Python-Decouple
- 4.3 Python-Gitlab
- 4.4 Django-Markdownify
- 4.5 Django-Ratelimit
- 4.6 Django-Test-Plus
- 4.7 Python Coverage
5. Tests
- 5.0 Running Tests
- 5.1 Tagged Tests
- 5.2 Run with Coverage
***
## 1.0 Intro
***
## 1.0 Quickstart:
Clone the repo. Rename the env_sample.txt file to just “.env”. Open it
and delete first line that says "delete me", then set the SECRET_KEY to a string of
your choosing.
This project uses Django-Ratelimit.
### 1.1 QuickStart
Set the LIMIT_RATE value to determine how many form post requests
to allow per time unit, or choose '0/s' to block all requests or leave as 'None'
to allow all requests. Change the value of MAIN_RATE_GROUP to a group name
of your choosing, or you can leave blank to have views limit individually.
***
SETTINGS:
1. Clone the repo from a fork.
2. Rename the env_sample.txt file to just “.env”.
3. Open it and delete first line.
4. Set the secret key to one of your choosing.
5. Set GITLAB_SECRET_TOKEN as your Tor GitLab account token.
6. If you have the ability to make user accounts, you can set the
GITLAB_ACCOUNTS_SECRET_TOKEN variable.
7. Set the GITLAB_TIMEOUT to a number of seconds.
8. Enter a value in MAIN_RATE_GROUP - can be any string of your choosing.
9. Enter a value in LIMIT_RATE using an integer, a slash, and a unit of
time for the denominator, e.g., 1/s, 10/m, 100/hr, etc.
10. BLOCK_ALL should be left as false, setting as True will block all
POST requests from forms in Anon-Ticket.
11. If running on a server, add an extra host in ALLOWED_HOSTS.
INSTALLATION:
Run the following commands:
......@@ -26,39 +83,72 @@ Run the following commands:
$ source myvenv/bin/activate
3. Install required packages:
$ pip install -r requirements.txt
4. Make all migrations to set up database.
4. If you have not set up your .env file, do it now, as you will need some of these
settings in place to make migrations or runserver.
5. Make all migrations to set up database.
$ python manage.py makemigrations
5. Migrate the database:
(It may say nonothing to migrate, this is fine.)
6. Migrate the database:
$ python manage.py migrate
6. You can check or launch by using the "runserver command."
7. You can check or launch by using the "runserver command."
$ python manage.py runserver
7. Create a superuser
8. Stop runserver and create a superuser
$ python manage.py createsuperuser
(follow the prompts)
8. Create groups "Moderators" and "Account Approvers"(see explanation above):
9. Create groups "Moderators" and "Account Approvers"(see explanation above):
$ python manage.py create_groups
9. Add gitlab tokens and other variables in the .env file.
```
When `runserver` is running, you should be able to point your browser to
<http://127.0.0.1:8000/> and reach the local version of the application.<br>
<br>
<http://127.0.0.1:8000/> and reach the local version of the application.
***
### 1.2 Project Aim and Use
***
## Table of Contents:
1. Quickstart (above)
2. Notes for Admins
- 2.1 Fast Project Add From Gitlab
- 2.2 Programmatic Groups and Permissions
- 2.3 Adding Moderators and Account Approvers
3. Project Structure and Function:
- 3.1 Folder Layout
- 3.2 Anon-Ticket Request-Response in a Nutshell
- 3.3 Models
- 3.4 Views
- 3.5 URL Pathing
- 3.6 Templates
4. Packages
5. Tests
Anon-Ticket is Django/Bootstrap web portal designed to allow anonymous
users to submit tickets to supported GitLab repos without signing up
for an account with GitLab.
Users do not interact with the system via a username/password or
email combination. Instead, when they first arrive at the home screen,
they have the option to create a new user identifier, which is a
code phrase made via random dice rolls from the EFF's New Wordslist
for Random Passphrases.
Once a user identifier has been created, and the user has chosen to log
in with the identifier, they are taken to a landing page specific to
their user identifier, which is passed forward through the system at
each step as a kwarg in the URL path. Each time a new page is loaded
or an object is created, the user identifier prhase is checked by a
validator.
From the landing page, the user has the optiont to view supported
repos and their issues/notes, search for a specific issue, or create
a new issue or note of their own.
Once an object is created by the user, it's held in a pending status
for a moderator. The moderator view displays multiple simultaneous
formsets demonstrating all pending issues, notes, and (coming soon!)
GitLab account reqquests that the moderator has permissions to view.
(To that end, there are two different groups of moderator permissions:
"moderator", which can see/approve pending issues and notes, and
"account approvers", which can see/approve pending GitlLab account
requests. A moderator can belong to either or both groups.) Moderators
also have the option to edit a submitted ticket or write a comment
on a ticket for other moderators.
The final major view is a customized admin panel for the superuser,
which includes display filters for items by status, e.g., "all pending
issues", as well as the ability to add projects to the repo with only
the project's GitLab ID. Once the ID is filled in and the project is
saved, the rest of the project details automatically populate from GitLab.
These features are described in greater detail below in section 2.0:
Notes for admins.
<br>
***
......@@ -143,16 +233,24 @@ be turned “on” or “off” by changing the allowed_apps setting in
/ticketlobby/settings.py.
This project has been structured in such a way to maximize the potential
for later development and expansion. It is currently divided into three
for later development and expansion. It is currently divided into four
main folders:
1. ***/TicketLobby***: The main project folder, in which most
configuration files are found.
1. ***/ticketlobby***: The main project folder, which includes the
settings.py file.
2. ***/AnonTicket***: The folder for the AnonTicket app.
2. ***/anonticket***: The folder for the AnonTicket app.
3. ***/Shared***: This is a pseudo-app, primarily for static files and
3. ***/shared***: This is a pseudo-app, primarily for static files and
templates likely to be expanded or utilized in other parts of the project.
It also includes custom template tags, filters, and middleware.
4. ***/gl_bot***: This folder contains the python file and tests for
GitLabDownObject, a mock python-gitlab gitlab.GitLab object, which is
instantiated in case of a timeout error from GitLab. Instead of having
affected views reroute to a "GitLab is down right now" page, the
GitLabDownObject returns a mock project and issue detailing the problem
while allowing the view to render normally.
### 3.2 Anon-Ticket Request-Response in a Nutshell
......@@ -187,7 +285,9 @@ FBVs.
### 3.3 Models
The models for database objects are in anonticket/models.py.
The models for database objects are in anonticket/models.py, with the exception of
the GitLabDownObject, which is in gl_bot/gitlabdown.py. Additionally, the models for
most of the forms are in anonticket/forms.py
### 3.4 Views
......@@ -226,9 +326,13 @@ overall layout template, including side-bar menu, is here, as well as the css fi
and other static files (like images.) The CSS is based on Tor Project's style-guide
(styleguide.torproject.com), which uses bootstrap templates for layout.
***
## 4.0 Packages
### Notes on Packages:
***
### 4.1 Notes on Packages:
The packages in the requirements.txt file are necessary for the ticketlobby
to function.
......@@ -237,20 +341,44 @@ Environment variables are tracked using the python-decouple package, which
increases security by moving important keys, tokens, etc. to the .env
file located in the base folder. If you're unable to perform
manage.py runserver, make sure you have set something in the SECRET_KEY
field in the .env file, and that DEBUG is set to "True". If DEBUG is
set to "False", you will need to make sure something is filled in for
field in the .env file, and that DEBUG is set to "True" in the .env.
If DEBUG is False, you will need to make sure something is filled in for
the ALLOWED_HOSTS field.
### Python-Gitlab
### 4.2 Python-Decouple
The anonticket app uses the Python-Gitlab package to communicate with
the GitLab API.
This package is used to easily extract settings from the .env file.
Settings are called with a config() function, e.g.,
my_key = config(MY_KEY) would pull the line MY_KEY = (value) from the
.env file and assign it to the variable my_key.
By default, all values pulled from python-decouple are strings; in those
instances where another data type (such as boolean, int, etc) are needed,
call the variable with the "cast" parameter, e.g:
my_key = config(MY_KEY, cast=bool).
More information available at: [https://github.com/henriquebastos/python-decouple/]
### 4.3 Python-GitLab
Some sample reference files to demonstrate dictionaries returned by
get queries, including pretty-printed issue and note dictionaries,
The anonticket app uses the Python-Gitlab package to communicate with
the GitLab API. If GitLab is down, anonticket will instantiate a
mock python-gitlab object from gl_bot/gitlabdown.py that takes
the same calls as the python-gitlab object, but will instead
return a message stating that GitLab appears down.
The dictionary that describes an object (issue, note, etc) is
usually returned by getting object.attributes from python-gitlab,
e.g., my_issue = my_gitlab_object.issues.get(issue_iid), dict =
my_issue.attributes.
Some sample pretty-printed reference files to demonstrate dictionaries
returned by get queries, including project, isssue and note dictionaries,
are available in shared/reference_files.
### Django-Markdownify
Documentation at: [https://python-gitlab.readthedocs.io/en/stable/]
### 4.4 Django-Markdownify
Python-Django package that allows you to use a 'markdownify' filter
to render markdown as html, including safe functions. Automatically
......@@ -263,53 +391,116 @@ To load into a template, use the {% loadmarkdownify %} template tag.
Add '|markdownify|' as a filter where you want markdown rendered as
html.
Documentation here: https://django-markdownify.readthedocs.io/en/latest/index.html
Documentation here: [https://django-markdownify.readthedocs.io/en/latest/index.html]
### Python-Coverage
(See "Tests", below.)
### 4.5 Django-Ratelimit
### 5.0 Tests
Anon-Ticket uses the Django-Ratelimit package. Two custom decorators
have been written for Anon-Ticket, expanding on the standard ip and
key decorators included with the package: @custom_ratelimit_ip,
and @custom_ratelimit_post. These custom decorators default to settings
specified in the settings.py file, as well as taking a callable function,
"get_rate_limit()", which pulls the desired limit-rate out of the .env file.
If BLOCK_ALL is set to "True" in the .env file, all POST functions on
decorated views will immediately be disabled, protecting the site
in the event of an attack.
Tests currently live in the tests.py files that are automatically set up
when the apps are created (e.g., the anonticket folder has a tests.py file
specifically for functions in the anonticket app.)
Additionally, a custom MiddleWare has been included in shared.middleware
to faciliate rate-limiting with a reverse-proxy enabled.
Coverage is verified through a variety of tools, including coverage.py
[https://coverage.readthedocs.io/en/latest/]. Coverage includes a C
extension for speed (that is also required to execute some functions.)
Once requirements have all installed from requirements.txt, you can
verify the C extension is installed correctly with
### 4.6 Django-Test-Plus
$ coverage --version
Several of the tests use DjangoTestPlus, which adds extra functions to
TestCase, allowing for easier url/view/integration tests. Tests that
do not utilize Django-Test-Plus are slowly being rewritten to
incorporate it due to the ease of use.
which should return "with C extension" or "without C extension".
[https://coverage.readthedocs.io/en/latest/install.html]
More information is available here:
[https://django-test-plus.readthedocs.io/en/latest/]
Testing sets up a test_database and should not interact with the project's
actual database.
### 4.7 Python-Coverage
Test coverage tends to hover around 94 percent as measured by
python-coverage. More information about python-coverage is available
below in 5.0: Tests. Docs for python-coverage are also available
at [https://coverage.readthedocs.io/en/coverage-5.4/].
***
## 5.0 Tests
***
### 5.1 Running Tests
There are multiple tests.py files containing tests, including
anonticket/tests.py and gl_bot/tests.py. All tests can be run
at once from the command line via $ python manage.py test.
1. To run tests from the command line without coverage:
$ python manage.py test
Note that during the running of the tests, the console will likely
raise several exceptions; these can be disregarded as long as the
test itself passes. For example, rate-limit tests are designed to
raise a 403 error to ensure the rate-limiting is applied correctly.
2. If you would like a more verbose output from testing (e.g, with test_names),
you can add -v1, -v2, or -v3 as an argument:
$ python manage.py test -v2
3. To run tests with coverage (using Python-Coverage package):
3. Note that several of the tests have a tearDown method that
includes clearing the cache; this is necessary as the test battery includes
tests designed to exceed the limit-rate, which provokes a 403 forbidden response.
### 5.2 Tagged Tests
Many of the tests are also tagged using the @tag decorator. So, for
example, anonticket/tests.py has multiple test classes tagged with
@tag('shared-non-gitlab'), like class TestUserIdentifierInDatabase(TestCase).
To just run tests associated with a particular tag, use --tag:
$ python manage.py test --tag shared-non-gitlab
Note that if the terminal reports 0 tests run in 0.00 seconds, there
is likely an error in one of the tests. To diagnose the error, just
run the normal test suite ($ python manage.py test)
### 5.3 Run With Python-Coverage
Coverage is verified through python coverage.
[https://coverage.readthedocs.io/en/latest/]. Coverage includes a C
extension for speed (that is also required to execute some functions.)
Once requirements have all installed from requirements.txt, you can
verify the C extension is installed correctly with
$ coverage --version
which should return "with C extension" or "without C extension".
[https://coverage.readthedocs.io/en/latest/install.html]
Testing sets up a test_database and should not interact with the project's
actual database.
1. To run tests with coverage (using Python-Coverage package):
$ coverage erase
$ coverage run manage.py test
$ coverage report
4. After running coverage run, the coverage data is stored in the .coverage file.
Run this command to generate an HTML report from this file:
2. To access the coverage data, you can either type:
$ coverage html
$ coverage report
This creates a folder called htmlcov. Open up htmlcov/index.html and you'll
see an easier to parse version of the report.
to see the coverage report generated on-screen in the terminal, or
$ coverage html
to generate a folder called htmlcov in the base directory. If you
open htmlcov/index.html, you'll see an easy to parse, color-coded
version of the report with pretty tables.
Note that files with 100 percent coverage will not show up
in the report.
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment