CAPTCHA-Monitor issueshttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues2021-06-23T17:04:23Zhttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/63Check if the website returns Captcha2021-06-23T17:04:23ZhackhardCheck if the website returns CaptchaAn example would be google search in tor browser: https://www.google.com/search?&q=tor
(searching tor in google from tor browser)
Mostly returns a `429 status code`. (No Reason Phrase Error)An example would be google search in tor browser: https://www.google.com/search?&q=tor
(searching tor in google from tor browser)
Mostly returns a `429 status code`. (No Reason Phrase Error)https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/62Check if the website blocks Tor fully2021-06-23T17:04:46ZhackhardCheck if the website blocks Tor fullyThis issue is created to successfully check whether a website blocks tbb or any tor client completely. This rule takes in the advantages of the status codes and compares between the tbb and normal mainstream browsers.This issue is created to successfully check whether a website blocks tbb or any tor client completely. This rule takes in the advantages of the status codes and compares between the tbb and normal mainstream browsers.https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/61Rename target_fetcher column to fetcher_id in the database2021-05-21T06:23:20ZBarkin SimsekRename target_fetcher column to fetcher_id in the databaseBarkin SimsekBarkin Simsekhttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/60Run Captcha Monitor on sample websites2021-07-28T22:05:30ZhackhardRun Captcha Monitor on sample websitesOne of the first things would be running and checking to see how well the captcha monitor v2.0 could handle a few websites.
For eg: Just the [Top 50 sites](https://www.alexa.com/topsites) and then continue testing on the rest.One of the first things would be running and checking to see how well the captcha monitor v2.0 could handle a few websites.
For eg: Just the [Top 50 sites](https://www.alexa.com/topsites) and then continue testing on the rest.hackhardhackhard2021-07-19https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/59Working with the updated code of the Captcha Monitor2021-07-28T22:03:33ZhackhardWorking with the updated code of the Captcha MonitorLocally install the code of Captcha Monitor.Locally install the code of Captcha Monitor.https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/58Move environment variables declared in dockerfile to a separate file2021-05-31T10:13:48ZBarkin SimsekMove environment variables declared in dockerfile to a separate fileBarkin SimsekBarkin Simsekhttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/57Access Tor's geoip database to populate relay location information2021-06-03T13:33:53ZBarkin SimsekAccess Tor's geoip database to populate relay location informationBarkin SimsekBarkin Simsekhttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/56Create a new web API2021-06-07T17:21:18ZBarkin SimsekCreate a new web APICheck flask-swaggerCheck flask-swaggerBarkin SimsekBarkin Simsekhttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/55Add pylint and black to CI2021-05-31T13:49:13ZBarkin SimsekAdd pylint and black to CIBarkin SimsekBarkin Simsekhttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/54Use mypy library for static typing2021-05-31T13:48:42ZBarkin SimsekUse mypy library for static typingBarkin SimsekBarkin Simsekhttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/53Add the ability to run multiple workers in parallel2021-06-01T07:20:25ZBarkin SimsekAdd the ability to run multiple workers in parallelBarkin SimsekBarkin Simsekhttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/52Add persistent database setup to dockerfile2021-05-21T06:53:37ZBarkin SimsekAdd persistent database setup to dockerfileCurrently, the database runs on the docker volume and it gets wiped after shutdown. Make it persistent.Currently, the database runs on the docker volume and it gets wiped after shutdown. Make it persistent.Barkin SimsekBarkin Simsekhttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/51ModuleNotFoundError: No module named 'captchamonitor.utils.fetch'2021-05-18T05:35:59Zepcnt19ModuleNotFoundError: No module named 'captchamonitor.utils.fetch'Hi, I installed CAPTCHA-Monitor following README.md, and switched to gsoc2020 branch.
Installing process is completed successfull, but a first command (`captchamonitor run`) is failed.
- A reason of failing is `ModuleNotFoundError: No m...Hi, I installed CAPTCHA-Monitor following README.md, and switched to gsoc2020 branch.
Installing process is completed successfull, but a first command (`captchamonitor run`) is failed.
- A reason of failing is `ModuleNotFoundError: No module named 'captchamonitor.utils.fetch'`
```
test@ubuntu:~/tor$ captchamonitor run
Traceback (most recent call last):
File "/home/test/.local/bin/captchamonitor", line 11, in <module>
load_entry_point('CAPTCHA-Monitor', 'console_scripts', 'captchamonitor')()
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
return ep.load()
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
return self.resolve()
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/home/test/tor/CAPTCHA-Monitor/captchamonitor/__init__.py", line 6, in <module>
from .md5 import md5
File "/home/test/tor/CAPTCHA-Monitor/captchamonitor/md5.py", line 5, in <module>
from captchamonitor.utils.fetch import fetch_via_method
ModuleNotFoundError: No module named 'captchamonitor.utils.fetch'
```
I think fetch.py is not found in `CAPTCHA-Monitor/captchamonitor/utils`
- In the path, A `fetch_via.py` is existed instead of `fetch.py`.https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/50Failed to launch a container2021-04-01T22:04:31Zepcnt19Failed to launch a containerHi, I encountered a problem that is failed to launch a captcha-monitor container.
A reason of failing is refused connection by a postgres container.
How can I solved the problem ?
---
#### environments:
- Linux: Ubuntu 20.04.2 LTS
- d...Hi, I encountered a problem that is failed to launch a captcha-monitor container.
A reason of failing is refused connection by a postgres container.
How can I solved the problem ?
---
#### environments:
- Linux: Ubuntu 20.04.2 LTS
- docker-compose: 1.26.2
- docker: 20.10.5
---
#### trying process:
- make command is completed successfully
```
test@ubuntu:~/CAPTCHA-Monitor$ make
docker-compose down --remove-orphans
Stopping captcha-monitor_tor_1 ... done
Stopping captcha-monitor_postgres_1 ... done
Removing captcha-monitor_captchamonitor_1 ... done
Removing captcha-monitor_tor_1 ... done
Removing captcha-monitor_postgres_1 ... done
Removing network captcha-monitor_captchamonitor_network
docker-compose build
postgres uses an image, skipping
tor uses an image, skipping
WARNING: Native build is an experimental feature and could change at any time
Building captchamonitor
[+] Building 2.7s (13/13) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 38B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/python:3.9-buster 0.8s
=> [1/8] FROM docker.io/library/python:3.9-buster@sha256:56f1b4dbdebb3b6ec31126e256c0852d18e79909ed1df8b594e562ab31f9e562 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 55.80kB 0.0s
=> CACHED [2/8] COPY requirements.txt /tmp/ 0.0s
=> CACHED [3/8] RUN pip install -r /tmp/requirements.txt 0.0s
=> CACHED [4/8] RUN mkdir -p /src 0.0s
=> [5/8] COPY src/ /src/ 0.1s
=> [6/8] RUN pip install -e /src 1.5s
=> [7/8] COPY tests/ /tests/ 0.1s
=> [8/8] WORKDIR /src 0.1s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:95398a0573621a89ab0c7bb9a476df75516c3226818a2b761e36cfc51455829b 0.0s
=> => naming to docker.io/library/captcha-monitor_captchamonitor 0.0s
Successfully built 95398a0573621a89ab0c7bb9a476df75516c3226818a2b761e36cfc51455829b
docker-compose up -d
WARNING: Native build is an experimental feature and could change at any time
Creating network "captcha-monitor_captchamonitor_network" with driver "bridge"
Creating captcha-monitor_tor_1 ... done
Creating captcha-monitor_postgres_1 ... done
Creating captcha-monitor_captchamonitor_1 ... done
docker-compose run --rm --no-deps --entrypoint="pytest --cov=/src/captchamonitor/ --cov-report term-missing" captchamonitor /tests
============================================================================================================================ test session starts =============================================================================================================================
platform linux -- Python 3.9.2, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /tests, configfile: pytest.ini
plugins: icdiff-0.5, cov-2.11.1
collected 11 items
../tests/unit/test_cm.py . [ 9%]
../tests/unit/test_config.py .......... [100%]
----------- coverage: platform linux, python 3.9.2-final-0 -----------
Name Stmts Miss Cover Missing
------------------------------------------------------------------
captchamonitor/__init__.py 0 0 100%
captchamonitor/__main__.py 14 14 0% 1-24
captchamonitor/cm.py 23 6 74% 25, 39, 45, 52, 58, 64
captchamonitor/utils/__init__.py 0 0 100%
captchamonitor/utils/config.py 30 0 100%
captchamonitor/utils/database.py 21 13 38% 31-54
captchamonitor/utils/exceptions.py 3 0 100%
captchamonitor/utils/models.py 76 0 100%
------------------------------------------------------------------
TOTAL 167 33 80%
============================================================================================================================= 11 passed in 0.47s =============================================================================================================================
black --line-length 88 $(find * -name '*.py')
All done! ✨ 🍰 ✨
12 files left unchanged.
pylint -v --disable=C0301,R0903,R0913,C0114 --docstring-min-length=10 $(find * -name '*.py' -not -path "tests/*")
--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
```
- `captcha-monitor` container is failed to launch.
- it is refused connection by `postgres` container
```
test@ubuntu:~/CAPTCHA-Monitor$ docker ps -a --no-trunc
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9e98f8ac7a6eb0e701ef9fa1c05e65c4cf91e2f37889eef2e5a87229fb0e5f28 captcha-monitor_captchamonitor "python -m captchamonitor" 17 seconds ago Exited (1) 15 seconds ago captcha-monitor_captchamonitor_1
7f0469ad697834bd9432dc5e7b194fad2cf0de89d3a1a43317cfd29b69273495 osminogin/tor-simple:latest "tor" 18 seconds ago Up 16 seconds (health: starting) 9050/tcp captcha-monitor_tor_1
b4b65440f8bcb0c8cd982a7d9d9728f687e92db3587180730d8249eeda92d13a postgres:9.6 "docker-entrypoint.sh postgres" 18 seconds ago Up 16 seconds 0.0.0.0:5432->5432/tcp captcha-monitor_postgres_1
test@ubuntu:~/CAPTCHA-Monitor$ docker logs 9e98f8ac7a6eb0e701ef9fa1c05e65c4cf91e2f37889eef2e5a87229fb0e5f28
2021-04-01 16:15:39,935 __main__ [INFO] Intializing CAPTCHA Monitor
2021-04-01 16:15:39,946 database [WARNING] Could not connect to the database:
(psycopg2.OperationalError) could not connect to server: Connection refused
Is the server running on host "postgres" (172.26.0.2) and accepting
TCP/IP connections on port 5432?
(Background on this error at: http://sqlalche.me/e/13/e3q8)
2021-04-01 16:15:39,946 cm [WARNING] Could not initialize CAPTCHA Monitor, exitting
```https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/49Add screen shotting feature for web page fetches2021-04-09T13:22:56ZBarkin SimsekAdd screen shotting feature for web page fetcheshttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/48Automatically download browsers, don't ask users to download them2021-05-21T06:24:07ZBarkin SimsekAutomatically download browsers, don't ask users to download themBarkin SimsekBarkin Simsekhttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/47Increase number of measurements completed per hour2021-05-20T06:16:40ZBarkin SimsekIncrease number of measurements completed per hourhttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/46Add bootstrapping2021-05-20T06:18:21ZBarkin SimsekAdd bootstrappinghttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/45Add docstrings to code2021-05-21T06:23:52ZBarkin SimsekAdd docstrings to codeBarkin SimsekBarkin Simsekhttps://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/issues/44Add dynamic binning2021-05-20T06:18:14ZBarkin SimsekAdd dynamic binning