Replace much handwritten config code with use of derive_builder

My 2nd big MR for derive_builder (Support custom fields in the builder struct #245, which might have been controversial) was just merged \o/ and the 3rd and (I hope) final one (Nested builder support #253) has been submitted and I think we can hope for it to be accepted in some form.

So now we can get rid of these handwritten builder structs.

This is part of #371 (closed) and ultimately #285 (closed)