sometimes dir mirrors answer 503 busy when they should say 404

In dirserv_estimate_data_size() we estimate the size assuming we have a descriptor for every server we're asked about. This results in cases like

Jul 12 20:03:56.471 [info] connection_dir_client_reached_eof(): Received http status code 404 ("Servers unavailable") from server '161.53.160.104:9091' while fetching "/tor/server/d/15EB28E14A7983129CD78FC6DEA20FD21098A7EF+161342319749ADD4116CE0427DB328F5C4BE1629+165BAFDA7253DFEC1DE0A3C548F21DDEAEF52F9E+1692BCEEC6431120B7AEC3A1685C8FD0EEC5F5A9+16D6BEE79128246B32EE1C565B2C2DBCB9AB41D3+16E162BE70678D6BFD5CF63F86B9188FE771B31F+16E567D955496CA338D3432B097F157DF35DE8BC+1715CD5D98964C7E31B36B66C52E14DDA7816EA9+1723E44E2245391BEC1B483126B5FAA870444B7C+1731772EE54F2A51E6749F25F9EE6BFB65FB6858+1759016C497AC642B5A50130AFCCF1BB6B643A02+17AEAC0A87BB35E4F2EA041FDA24F5DBC3A792AB+17B881474EA0A2F64F989263A7AD30A0DCE6377D+183528CAC37D3FA3EF18281C948DD213CF08D087+184C7E375ACB1CC1AF908DD957A180C93AA4227E+1869C3BA669EF6264614A3888D5E9B43C5FE3884+1874E43C98993E8285DBC3752920D02C6C951663+18965516ED4CF01866788451421C9EDF1E6BD1AA+190A1DF5C543AC6BB9AB72FE8C4C7CDCAE86D2B0+197F9E91C8D2FED74C36FBD4DA00C5D632144468+19A126F8CC87F339F2B16A273F1CD0A247F2FA4A+19C704F44ECB6BF79170E09C634790B98C8D8544+19E487A6A3A272A04EF882FB07A2EE90895090CE+1A720FCD4A2849C83A2701CDD0D39D8FFD2E314D+1AA750EE5B7FED285F368B6FEB4428DF92C2BB31+1AB2DABF6007241131C5E443CDC60394761816AC+1AF2B7488D10BFDCFE6CA79D56FB83656F4ADC8D+1AF69AAD27DFB6AA63D1072FA5AED92B468540DE+1B6D306CA8E0C693FAD3FFAAFC59F653DEFB1491+1BA661163B8A14B9AFB98AA9D12B18A24CBE562A+1BC0D0980C5EB734AAD02B9E70CCFCC898CB315A+1BDDBDA5C4E6DB9B3443D979352A7BC9A9576FE1+1BE953E3BB58CE9B6EE261BF441C7F9B3BDA752D+1C0E414654093BA94EE84764C711449B0ED5704F+1C11365A76C3A583E708C040E6A3307F9043D02F+1C1CC7E04750FB2A0CADD6B2C70FFF50CC31C632+1C34CC4FEA875DBDDCF419DEB6A9DB48DDA652F9+1C7DFC2A6F38DC9EB1B3FCB6C2F865EBA6D19F74+1CB80E88CBE0573EE7E856797E0A7B32ECE7B4B7+1CF9F520154EA3C12A3AA42AE79C75F1DE4B6309+1D2BC778F1CEC8A37C33A6EE2897C7597B7CD526+1D878DC7566F10B8303138B36971023038F6F36C+1DB1A4D87A94E8CC2838C8E83F411A8CF5FBA0AC+1DC7FE119A441AFBD18E821B56CAB844278F8AF3+1DED6B4805370219D52E1C14C52092789C7EB8D7+1E33A33694596F3D6121C92FDA8882E2EC3A76C4+1E67F16C391DAFBD830049A379DD691EBC22E6B1+1ED62D8D7DA9F76501B7F95D9D7DD7E002F93640+1F35B4C9F94020AE9893C7358D9FA6D15A0B6807+1FCCEC0102ABE5A8BA18B4D34CBA3F3300FDE5AE+1FDE4FBC452C5D2BECEAF0B3CB15F644380D2A75+1FF2733FC1812FBAD51FEBBDE259B06BE0921A59+204697B4DF76664425ADB9F3F8D0B1A3591CA695+209F6EFEE6FDE553FEE8748FA5A7BA70964965F9+20AC0F4CFC447C4C9C18F42356104CABDFBD0D20+21036C9FF860815C8C043597B39BC80AE40A3560+212474B60CC275F91F716F89C5751E4BB7A78B5F+213E8B394E0590A9EC9360BB415867E00A1978E7+214AE65A8196F9C6EFC4765F0488A3CE7FBC433A+2172560E8EE391D29C5218446A242F822F1E1AE1+21743952EB0C848BE0D513926C989AA79EEAA0AC+21976CED8984D0099810A17793F49585C2BE5DD0+21A3E413C450399FD8AE5A7CEE3624DF7953DF54+21CE269692672D3FED7435B6F1F5D7BC22D334BC.z". I'll try again soon. Jul 12 20:03:58.430 [info] connection_dir_client_reached_eof(): Received server info (size 0) from server '213.239.207.67:9032' Jul 12 20:03:58.430 [info] connection_dir_client_reached_eof(): Received http status code 404 ("Servers unavailable") from server '213.239.207.67:9032' while fetching "/tor/server/d/9CE4717E88E096DF81C1C4FF04F9ECBEC7CD3A5B+9CF6B523BEDDAF0BB9E0365F235A30FE9B773481+9D1A6720FBD2A10956C0EC3F9906DB05F73BD0D4+9D6755D980C8A0FDE344C8D9DC075EFB55C1E65A+9D9674E83A649A5DB589467AE1BFC5D8C788344F+9DFE225679B7D8D1695E2F86EB9D5E25D8EBAB34+9E1B301AD61B971FE4014F90834D86167DB010CF+9E3C0B1343829809EEF2232B8F8A147DF40EB489+9E7D917BFA42EFA9A2FD520905662700FF39B6A9+9EFD73CEE0CF4DC558813599D6192FD493543894+9F0C194B70D97C3CDFFD8567054434BC58D241F4+9F37436BFE30729AE1C0F950F33BFC9111EA1C73+9F3E0FB041996920B40459961F195B5CD6399DC6+9FFCA011607CA2D04E5203DB9D0E81895F44C3F3+A0069CAD4ECA76DCB8A65665320DA7E17FFC93AD+A0133D0C1179A17FAC6C330012EAACE88CC611B9+A06385CA613FB0FCA712B7A5A2D2C5295AEBABAF+A09C82BE0551B262534A5E86343641B4D6773F85+A0AB774DE5543FBF9F9E79D6EA1EA7CF661E4461+A0AE08580B0169C82DB5D7D10FDC943BF8EEA7D3+A0C9E180FEAC7560B0560FAB88BF501C4714661A+A11B9D281FEA020597B0446248768F87D49EFE7A+A1841B8634E7766CD45119C907BF277C81C94FDB+A229B8C784C10932F5DD54F015E25D04730253EE+A23AB6FA0D08881A364E0E2CB881D7E3B805440F+A28055279C866549A1A0F51640D664D4A6FB3B41+A2BD80D6BD0E0805AEB05DF56178F0C158935A5C+A2C78532A2681EC56FCBE7E45C02353769BC1D95+A2D1067D7018B5EC457FFBF300F8829CB75031D2+A2E18660748CD4E2889655C13C515D2DDBC41256+A310436DD2C0BB21D0F1D821F25CF306C0BB8C37+A3329045F8D856CE1FC53318AF5078A36AD2012D+A3B99B3F0FD34C724249BD96D22B19402F3D028C+A419E8229A5DA157D2A08B76D29812B20E4F4433+A42CDE78731DB4732D4081737159F04945FE1AF2+A62B099009D02859F8B6CEAC9EC7593565B39E2A+A647325F16245859ABE97A91589550580719BEA1+A64CB0CF7D84039AC62B94727BF652204EC6816A+A695737A181822C144918FF5F4E9442EBB1A1249+A6CFCF877338AEF9423682989BE59A47910C9AFB+A6D5D4F300AEC3D2547A8087BECB2DC4BB7C3522+A705D5C1A04E02FEC2B9C65268B6066C53A95F3C+A723F2CCA05D6DBD26AD94C3A944E32BFDC78CDD+A72E0D5CA9E9EA457DA4FE59E94D0A9196F2C1E3+A74F5A31716382AA22CDBE96B5802527B5E4060E+A79F558B7A04CDCC76F3B9D7F57B12A49CEDC1E7+A7A114CFB2FD516205398920C8FC736305D8FE45+A828AB492F31A601136DD88EA4FAF313CB6B1A8A+A86240001CFD23E8501EB2CEE9B7E4FB0BE975DE+A8659A991509E1C2F26CB365184A106862A5060B+A878DCD037042319797E8017710138E835BE13BE+A886F2F8922803AD86242E20BDF17DA28A176791+A8DA069ABD97A98F4B842657D712E294648B6255+A8E6040879A14E6201D07DB10EE7AEBF1AC594C9+A9A306117B642E064D6065682325FAB355808904+A9B2472A36D800F37D31C7FCFDBA9625168CC6E1+AAEC2FDD441EC9AB9BF7C9F61B8E1C187A0643D7+AB03995964674450EC79D4A88E79898639780404+AB05341E510E4915874FF2C8A678B3FD0BE0EA7A+AB29C2B963427F98B18D4D8CA8927B671990AFDC+AB40DAFA0B6E91C114DBD9A6DEC69DF9A10C4D4A+ABBBE14C5B168675FE13B129B8B3076A0E1DFB59.z". I'll try again soon. Jul 12 20:03:59.267 [info] connection_dir_client_reached_eof(): Received http status code 503 ("Directory busy, try again later") from server '70.84.114.153:9030'. I'll try again soon.

where the client is inefficient at realizing that it's asking for descriptors that aren't available anywhere. If we estimated a bit better (by checking if we have an answer at all), we could help clients react quicker. This would require figuring out what descriptors we're being asked about, which means doing more work before we decide whether to 503.

[Automatically added by flyspray2trac: Operating System: All]