GitLab

Hi trac admins. I maintain the r2e feed that provides our tor-wiki-changes@ list [1]. Its been broken for a while and today I spent a few hours digging into why.

This list relies on trac's RSS feed [2]. Https looks to be broken on it. In particular...

• Firefox can view the feed, but unlike other trac pages indicates the site is insecure ('Page Info > Security' says the page is unencrypted and lacks any certificate).

• Curl responds with a 403...

% curl -s 'https://trac.torproject.org/projects/tor/timeline?wiki=on&format=rss' | grep 'Forbidden'
      Error: Forbidden – Tor Bug Tracker & Wiki
          <h1>Error: Forbidden</h1>

• Python's urllib (which is what r2e uses) fails with a 403 as well. It can access other trac pages...

import urllib.request
request = urllib.request.Request('https://trac.torproject.org/')

... but the feed fails with...

import urllib.request
request = urllib.request.Request('https://trac.torproject.org/projects/tor/timeline?wiki=on&format=rss')

% python3 scrap.py 
Traceback (most recent call last):
  File "scrap.py", line 4, in <module>
    urllib.request.urlopen(request)
  File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

At this point I'm pretty stumped. Something's broken with the feed and I'm unsure how to work around it. We don't care about encryption on these requests so if providing a plain http feed is easier than fixing this then happy to go with that too (presently http is a 302 to https).

Thanks!

[1] https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-wiki-changes [2] https://trac.torproject.org/projects/tor/timeline?wiki=on&format=rss