Check if inner conn has been set before closing

This prevents a crash if the BufferedConn is closed before the inner net.Conn has been set. #34 (closed)