service/donate.md

@@ -161,7 +161,85 @@ sandbox client id and app secret)."
 
 TODO: how do we deal with high latency and exceptions?
 
-TODO: card testing playbook. see [service/crm](service/crm) for now.
+### Stripe card testing
+
+A common problem for non-profits that accept donations via Stripe is
+"card testing". Card testing is the practice of making small
+transactions with stolen credit card information to check that the
+card information is correct and the card is still working. Card
+testing impacts organizations negatively in several ways: in addition
+to the bad publicity of taking money from the victims of credit card
+theft, Stripe will automatically block transactions they deem to be
+suspicious or fraudulent. Stripe's automated fraud-blocking costs a
+very small amount of money per blocked transaction, when tens of
+thousands of transactions start getting blocked, tens of thousands of
+dollars can suddenly disappear. It's important for the safety of
+credit card theft victims and for the safety of the organization to
+crush card testing as fast as possible.
+
+Most of the techniques used to stop card testing are also antithetical
+to Tor's mission. The general idea is that the more roadblocks you put
+in the way of a donation, the more likely it is that card testers will
+pick someone else to card test. These techniques usually result in
+blocking users of the tor network or tor browser, either as a primary
+or seide effect.
+
+- Using cloudflare
+- Forcing donors to create an account
+- Unusable captchas
+- Proof of work
+
+However, we have identified some techniques that do work, with minimal
+impact to our legitimate donors.
+
+- Rate limiting donations
+- pre-emptively blocking IP ranges in firewalls
+- Metrics
+
+An example of rate limiting looks something like this: Allow users to
+make no more than 10 donation attempts in a day. If a user makes 5
+failed attempts within 3 minutes, block them for a period of several
+days to a week. The trick here is to catch malicious users without
+losing donations from legitimate users who might just be bad at typing
+in their card details, or might be trying every card they have before
+they find one that works. This is where metrics and visualization
+comes in handy. If you can establish a pattern, you can find the
+culprits. For example: the IP range 123.256.0.0/24 is making one
+attempt per minute, with a 99% failure rate. Now you've established
+that there's a card testing attack, and you can go into EMERGENCY
+CARD-TESTING LOCKDOWN MODE, throttling or disabling donations, and
+blocking IP ranges.
+
+Blocking IP ranges is not a silver bullet. The standard is to block
+all non-residential Ip addresses; after all, why would a VPS IP
+address be donating to the Tor Project? It turns out that some people
+who like tor want to donate *over the tor network*, and their traffic
+will most likely be coming from VPS providers - not many people run
+exit nodes from their residential network. So while blocking all of
+Digital Ocean is a **bad idea**, it's less of a bad idea to block
+individual addresses. Card testers also occasionally use VPS providers
+that have lax abuse policies, but strict anti-tor/anti-exit policies;
+in these situations it's much more acceptable to block an entire AS,
+since it's extremely unlikely an exit node will get caught in the
+block.
+
+As mentioned above, metrics are the biggest tool in the fight against
+card testing. Before you can do anything or even realize that you're
+being card tested, you'll need metrics. Metrics will let you identify
+card testers, or even let you know it's time to turn off donations
+before you get hit with a $10,000 from Stripe. Even if your card
+testing opponents are smart, and use wildly varying IP ranges from
+different autonomous systems, metrics will show you that you're having
+abnormally large/expensive amounts of blocked donations.
+
+Sometimes, during attacks, log analysis is performed on the
+`ratelimit.og` file (below) to ban certain botnets. The block list is
+maintained in Puppet (`modules/profile/files/crm-blocklist.txt`) and
+deployed in `/srv/donate.torproject.org/blocklist.txt`. That file is
+hooked in the webserver which gives a 403 error when an entry is
+present. A possible improvement to this might be to proactively add
+IPs to the list once they cross a certain threshold and then redirect
+users to a 403 page instead of giving a plain error code like this.
 
 ## Disaster recovery