Orbot crashes when App-mode is enabled

When I enable App-mode after Orbot is connected, Orbot will crash when I open a network connection. When I look into the processes in Settings - Apps, Orbot is still running with 1 process and 1 service but the app itself shows that it is not connected anymore and is not able to connect anymore "Failed to parse validate config: Failed to bind one of the listener ports."

I see this behavior on a Moto X 2013 with CyanogenMod 12.1 and on a LG G4 with Android 5.1

Trac:
Username: nielsk