Recover from corrupted state or cache on startup.

When we start up, if we can't load our persistent state, we'll currently exit with an error. That's not good user experience: instead we should maybe move aside any broken/unreadable state files? Or perhaps we should ignore them and not use persistent state? There are multiple possibilities here, and it's not obvious which is best.

changed milestone to %Arti 1.0.0: Ready for production use

added UX label

added Backlog label

added Sponsor 119 label

added Roadmap label

changed time estimate to 16h

added Roadmap::Future label and removed Backlog label

added Backlog label and removed Roadmap::Future label

mentioned in issue #438 (closed)

added Q2 label

!511 (merged) improves this considerably for some dircache confusing-ness.

added Next label and removed Backlog label

assigned to @nickm

added Q3 label and removed Q2 label

added Doing label and removed Next label

tl;dr: I think we should just improve the error messages here, and take no other action for now.

---

Okay, so here are the principles I'd like to go with:

• We should never delete stuff that we don't understand.
• We should probably not move data aside just because we think it might be corrupted.
• Therefore, our best move is probably to make sure that all of our error messages say what is broken, and what you would need to do to fix it.
  • That's done partially by !614 (merged), where we made sure that each error message about persistent state says what file is at issue. As I keep making fixes for #323 (closed), I'll try to make sure more error messages still say what the problem files are (if any). But see #514 (closed): We might decide that logging filenames by default is problematic!
  • It might be a good idea to expose a list of maybe-broken files via an API on arti_client::Error.

We should revisit these principles if we find that any particular type of file corruption happens in practice. We ought to be relatively resistant to such corruption, since for now we've only got three types of files in our directories:

1. Files that we create with the "write, close, rename" pattern, which is "pretty atomic".
2. Sqlite3 databases, which are pretty robust.
3. Lock files, which have no content.

Therefore, I'd suspect that the likeliest causes of data corruption would be bugs in our code, or bugs in other code that's messing with our files when it shouldn't.

@Diziet @eta any thoughts about the above?

the "write, close, rename" pattern, which is "pretty atomic".

We can end up with corrupted files at least by

• Too low disk space, enough to write something but not all (?)
• Hardware problems
• "stuff that we don't understand" if user ran future version and downgraded.

I agree that these are sound principles.

I dare to suggest that suggesting the user blows away all of arti's state might be reasonable - assuming the user doesn't experience this often. OTOH maybe that option is obvious to the user. OTTH they may not know what to delete.

Those ideas look good to me! I'd second Diziet's opinion that ideally, it'd be nice to have some form of error help message that would explain how to get it going again (like help: to blow away all the state files, remove or move /path/to/directory or somesuch).

This is problematic on Android, where state directory is likely to be internal to app and even with advanced file manager capable of understanding the path you can't touch it, unless the phone is rooted. Erasing the app data would help, but it's too aggressive.

added Next label and removed Doing label

added Backlog label and removed Next label

added Roadmap::Future label and removed Backlog label

removed Q3 label

added Icebox label and removed Roadmap::Future label

removed Roadmap label

unassigned @nickm

mentioned in issue #878

added Backlog label and removed Icebox label

added Roadmap::Future label and removed Backlog label

mentioned in issue onionmasq#92 (closed)

added Sponsor 101 label

removed Project 119 label