Port from config-rs to figment
I am not sure we should actually do this. Having gotten the work done, I don't know whether it's an improvement or not. I think the source for figment is a bit nicer, and its approach to errors seems somewhat better (see #1267 (closed)), but there are still some issues where I wish figment did things differently.
(In particular, see https://github.com/SergioBenitez/Figment/issues/94 .)
Closes #1267 (closed).
Closes #1268 (closed).
If we choose not to merge this, we should IMO at least merge the earlier parts of this branch, which make config
no longer part of the public API for the tor-config
crate.
Merge request reports
Activity
requested review from @Diziet
assigned to @nickm
376 377 Ok(()) => {} 377 Err(fs_mistrust::Error::NotFound(_)) if !required => {} 378 Err(e) => return Err(foreign_err(e)), 378 Err(fs_mistrust::Error::NotFound(_)) if !required => { 379 continue; 380 } 381 Err(e) => return Err(ConfigError::Permissions(e)), 379 382 } 380 383 381 // Not going to use File::with_name here, since it doesn't 382 // quite do what we want. 383 let f: config::File<_, _> = file.into(); 384 builder = builder.add_source(f.format(config::FileFormat::Toml).required(required)); 384 // We need the path to be absolute here; since figment wants to look 385 // at all our parent directories. 386 // TODO Find a better way to disable this behavior? - Comment on lines +384 to +386
I've got an upstream PR for this:
Also see:
68 68 /// You can make one out of a `PathBuf`, examining its syntax like `arti` does, 69 69 /// using `ConfigurationSource::from_path`. 70 70 #[derive(Clone, Debug, Eq, PartialEq, Hash, Ord, PartialOrd)] 71 #[allow(clippy::exhaustive_enums)] // Callers will need to understand this 71 #[allow(clippy::exhaustive_enums)] 72 72 pub enum ConfigurationSource { 73 73 /// A plain file 74 74 File(PathBuf), 75 75 76 76 /// A directory 77 77 Dir(PathBuf), 78 79 /// A verbatim TOML file 80 Verbatim(Arc<String>), - Comment on lines +78 to +80
I think this is a good change.
I wondered whether the existing command line override handling
--option KEY=VALUE
etc. ought to be using this but on inspection it wants to handle all of those afterwards (so that the precedence is right). So probably it is right that it continues to be an ad-hocVec<String>
.
131 /// An error that occurs while trying to read and process our configuration. 132 #[derive(Debug, Clone, thiserror::Error)] 133 #[non_exhaustive] 134 pub enum ConfigError { 135 /// We encoundered a problem with file permissions. 136 #[error("Bad permissions on configuration file")] 137 Permissions(#[source] fs_mistrust::Error), 138 /// Our underlying configuration library gave an error. 139 #[error("Invalid configuration data")] 140 Parse(#[source] ConfigParseError), 141 /// Encountered an IO error while reading (or preparing to read) a file. 142 #[error("Unable to read configuration file {0:?}")] 143 Io( 144 Option<std::path::PathBuf>, 145 #[source] std::sync::Arc<std::io::Error>, 146 ), changed this line in version 2 of the diff
77 77 #[doc(hidden)] 78 78 pub use flatten::flattenable_extract_fields; 79 79 80 /// A set of configuration fields, represented as a set of nested K=V 81 /// mappings. 82 /// 83 /// (This is a wrapper for an underlying type provided by the library that 84 /// actually does our configuration.) 85 #[derive(Clone, Debug)] 86 pub struct ConfigurationFields(config::Config); I think we might want to do this even if we don't switch to figment.
I think that whether or not we switch, we should take everything in this branch up to the Figment conversion. (That is, everything that makes the
config
dependency hidden.)I don't like the name, though. "Fields" isn't quite what's going on here. "ConfigurationTree"?
I like
ConfigurationTree
.
I'm not sure.
When I ported Otter to Rocket 0.5 it had switched to Figment (Sergio is the author of Rocket). I'm definitely a fan of Rocket, and Figment is kind of part of Rocket. (That's where the "profiles" thing comes from - webapps generally seem to want something like that, although I often find it confusing.)
It's disappointing to find ourselves still doing the ignored key tracking ourselves!
I'm inclined to say "yes" but maybe after upstream !95/#94 is fixed? You've had some time to sleep on it - what do you think now?
(I had some quibbles about details too.)
It's disappointing to find ourselves still doing the ignored key tracking ourselves!
I think that this is a side-effect of the way that we superimpose multiple configuration objects. Other tools seem to put all of their configuration into one
Deserialize
object, which lets them tell serde to reject unrecognized fields.I'm inclined to say "yes" but maybe after upstream !95 (closed)/#94 is fixed? You've had some time to sleep on it - what do you think now?
I like how much better the figment errors are than the config errors, and I like how the maintainer was more responsive so far—compare response time on figment#94 to lack of response on config-rs#532.
That said, I'm fine until we see how the upstream issue is resolved.
FWIW, with respect to the example in #1267 (closed), the error with config-rs is:
./target/quicktest/arti: error: read configuration: Config contents not as expected: Invalid listen specification: failed to parse string: invalid socket address syntax
And the error with this branch is
./target/quicktest/arti: error: read configuration: Config contents not as expected: Invalid configuration data: Invalid listen specification: failed to parse string: invalid socket address syntax for key "default.proxy.socks_listen" in fred.cfg TOML file
So I think that that's a point in favor of figment.
350 343 351 344 /// Add every file and commandline source to `builder`, returning a new 352 345 /// builder. 353 fn add_sources(self, mut builder: ConfigBuilder) -> Result<ConfigBuilder, ConfigError> { 346 fn add_sources(self, mut builder: Figment) -> Result<Figment, ConfigError> { 347 use figment::providers::Format; 348 349 // Note that we're using `admerge` here. It causes later sources' options 350 // to replace those in earlier sources, and causes arrays to be extended 351 // rather than replaces. 352 // 353 // TODO: Maybe we should use "merge" instead? That does the same, 354 // except for the array behavior. We probably don't want to make this depend on which TOML syntax was used -
[[array]]
vsarray = [...]
.We need to be able to override arrays on the command line. But ideally we'd like to be able to extend them too. That might mean different command line options...
I don't think we need to be able to override arrays in subsequent config files, although that might be nice.
What's the current behaviour?
admerge
is documented at https://docs.rs/figment/latest/figment/struct.Figment.html#method.admerge . AFAICT it isn't syntax-aware, and can't easily be made syntax-aware.One option might be to avoid suggesting
[[arrays]]
in our configuration.In any case, I think that
config
probably just replaces arrays, though that's something we should test.
mentioned in merge request !2040 (merged)
mentioned in merge request !2041 (merged)
I have split this MR into !2040 (merged) (encapsulating config-rs) and !2041 (merged) (porting to figment). I've referenced this MR from !2041 (merged), with a note that all the comments here are still relevant, and need to be addressed.
mentioned in issue #1337