Skip to content

feat: Handle legacy Paypal IPN messages regarding donate-paleo subscriptions enough that we receive them and can develop against them

stephen requested to merge handle-legacy-ipns into main

Context for this MR begins here: https://gitlab.torproject.org/tpo/web/civicrm/-/issues/155#note_3082305

With the sudden revelation that Paypal will not send webhook information about subscriptions begun on donate-paleo, but will instead send legacy IPN messages, there is a pressing need for donate-neo to handle these legacy messages as well. This MR represents this effort to ensure that CiviCRM receives updates about legacy subscriptions in exactly the same manner that it would, were those subscriptions made on donate-neo.

  • The endpoint /paypal-ipn must be handled by donate-neo, as if we fail to return an HTTP 200 OK from this endpoint, Paypal will eventually cease sending IPN messages to us entirely, leaving us in the dark about the state of subscriptions. It is possible to change this endpoint in Paypal's settings, but this MR assumes we must accomodate it as is, so we tweak how Paypal URLs are defined to the main Django URL handler - removing the implicit "/paypal/" prependation - such that we can serve the URL /paypal-ipn directly from the Paypal app.
  • tordonate.paypal.views handles the incoming IPN message, which is formatted in the CGI-bin-compatible style of the time: with all of the data squashed into the URL parameters of the incoming message.

The next line item, were this a straightforward implementation, would concern tordonate.paypal.controller, and how it first validates IPN messages with Paypal, and then processes them such that their subscription data can be handed off cleanly to the webhook processing flow. However: First we need the materials necessary to implement IPN message validation.

You see, to test and implement IPN message validation, we need an IPN message generated by the Paypal sandbox. This can only be done in one specific way: By using a CGI-bin script hosted by, but removed from the sitemap of, and not linked to anywhere by, Paypal's logged-in developer section. (It might as well have an H1 which reads "Beware of the leopard.")

This script has a form front-end with exactly two fields: A textfield in which you type your destination IPN-handling URL, and a drop-down list of IPN types. You can choose which type to send to the endpoint, at which point Paypal generates a fake IPN which, crucially, can be validated back with Paypal's sandbox IPN endpoint.

To this end, this commit implements an IPN handler in tordonate.paypal.controller which does nothing but log the IPN's query string (which is how IPNs actually transmit their message data). The view calls this method and then immediately returns 200 OK. This is the purpose of this MR - to exist on staging, such that a handful of sandbox IPN messages can be generated and sent to it, so that we can log them, so that we can retrieve them from the logs, so that we can use them to complete development of the IPN endpoint. Following this endeavor, a subsequent MR will be spun up which completes the work described above, and eventually CiviCRM will receive Paypal subscription information from both IPN message data and webhook message data, and all will be well.


We've talked a lot about how useful this MR will be in staging. It is a little less useful in production - we log the event, but logging the IPN query-string itself is prevented in production because it's chock-full of PII, and we can't use production IPN messages for testing in the Paypal sandbox anyway. Importantly, however, because it serves 200 OK to POST requests, this MR will serve to convince Paypal that we'd like to keep receiving IPN messages and they shouldn't shut them off for us. (Recall that we only know of this issue because Paypal has been emailing us about the undeliverability of these IPN messages, and has let us know they'll shut those messages off for us if we don't start handling them. Well, 200 OK is what they'd like to receive instead of a 404 in order to consider their messages properly-handled.)

Merge request reports