Telegram Bridge Distributor Crash: fatal error: concurrent map read and map write
@nina has reported to us that @getbridgesbot does not work. I have verified this report and can confirm telegram bridge distributor is down, and cannot recover from error by just restart the service.
Feb 04 18:14:50 polyanthum rdsys-distributors[967517]: 2023/02/04 18:14:50 Attempting to load state from "storage/telegram/sr2.json".
Feb 04 18:14:50 polyanthum rdsys-distributors[967517]: 2023/02/04 18:14:50 Initialising resource stream.
Feb 04 18:14:50 polyanthum rdsys-distributors[967517]: 2023/02/04 18:14:50 Making HTTP request to initiate resource stream.
Feb 04 18:14:50 polyanthum rdsys-distributors[967517]: 2023/02/04 18:14:50 Adding 218 resources of type obfs4.
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: 2023/02/04 18:14:51 ***[Redacted]*** telegram: retry after 60 (429)
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: 2023/02/04 18:14:51 ***[Redacted]*** telegram: retry after 60 (429)
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: 2023/02/04 18:14:51 ***[Redacted]*** telegram: retry after 60 (429)
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: 2023/02/04 18:14:51 ***[Redacted]*** telegram: retry after 60 (429)
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: 2023/02/04 18:14:51 ***[Redacted]*** telegram: retry after 60 (429)
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: 2023/02/04 18:14:51 ***[Redacted]*** telegram: retry after 60 (429)
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: 2023/02/04 18:14:51 ***[Redacted]*** telegram: retry after 60 (429)
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: 2023/02/04 18:14:51 ***[Redacted]*** telegram: retry after 60 (429)
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: 2023/02/04 18:14:51 ***[Redacted]*** telegram: retry after 60 (429)
......***[Redacted]***......
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: fatal error: concurrent map read and map write
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: goroutine 1 [running]:
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: gopkg.in/telebot%2ev3.(*Bot).handle(...)
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: /home/user/go/pkg/mod/gopkg.in/telebot.v3@v3.1.2/update.go:285
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: gopkg.in/telebot%2ev3.(*Bot).ProcessUpdate(0xc0001b0ea0, {0x9bf696a, 0xc000264b40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...})
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: /home/user/go/pkg/mod/gopkg.in/telebot.v3@v3.1.2/update.go:55 +0x288
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: gopkg.in/telebot%2ev3.(*Bot).Start(0xc0001b0ea0)
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: /home/user/go/pkg/mod/gopkg.in/telebot.v3@v3.1.2/bot.go:219 +0x1cc
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: gitlab.torproject.org/tpo/anti-censorship/rdsys/pkg/presentation/distributors/telegram.(*TBot).Start(...)
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: /home/user/dev/rdsys/pkg/presentation/distributors/telegram/telegram.go:90
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: gitlab.torproject.org/tpo/anti-censorship/rdsys/pkg/presentation/distributors/telegram.InitFrontend(0xc0001a6000)
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: /home/user/dev/rdsys/pkg/presentation/distributors/telegram/telegram.go:68 +0x433
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: main.main()
Feb 04 18:14:51 polyanthum rdsys-distributors[967517]: /home/user/dev/rdsys/cmd/distributors/main.go:58 +0x4c3Hypothesis for the cause of this crash
There is read access of handlers variable after the bot is up and running. However, rdsys attempt to write this variable as a result of setting new handler when processing commands.
Suggested change under this hypothesis:
Add all handlers during the initialization period and before the Start() of the bot is called.