Use fewer travis builders for typical commits?

We have accumulated a bunch of travis build processes. Maybe some of them should be "fast" ones that happen for every commit in every branch, and some should be "slow" ones that happen for pull requests and the mainline branch?

Or there might be some even better way to make this decision.