Fix up robots.txt

Kezrequested to merge
fix-robots-static into main
  • Remove Disallow /static/ from robots.txt

Disallowing /static means our images can't be indexed. It also breaks twitter's metacards (see donate-static#97 (closed))

  • Disallow entire directories instead of globbing

  • Simplify robots.txt

The robots.txt spec isn't as strict as I thought, so we don't need to worry about whitespace or capitalization

Merge request reports