Skip to content
Snippets Groups Projects

Port from config-rs to figment

Closed Nick Mathewson requested to merge nickm/arti:figment into main
5 unresolved threads

I am not sure we should actually do this. Having gotten the work done, I don't know whether it's an improvement or not. I think the source for figment is a bit nicer, and its approach to errors seems somewhat better (see #1267 (closed)), but there are still some issues where I wish figment did things differently.

(In particular, see https://github.com/SergioBenitez/Figment/issues/94 .)

Closes #1267 (closed).

Closes #1268 (closed).

If we choose not to merge this, we should IMO at least merge the earlier parts of this branch, which make config no longer part of the public API for the tor-config crate.

Merge request reports

Approval is optional

Closed by Nick MathewsonNick Mathewson 1 year ago (Mar 13, 2024 2:02pm UTC)

Merge details

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
376 377 Ok(()) => {}
377 Err(fs_mistrust::Error::NotFound(_)) if !required => {}
378 Err(e) => return Err(foreign_err(e)),
378 Err(fs_mistrust::Error::NotFound(_)) if !required => {
379 continue;
380 }
381 Err(e) => return Err(ConfigError::Permissions(e)),
379 382 }
380 383
381 // Not going to use File::with_name here, since it doesn't
382 // quite do what we want.
383 let f: config::File<_, _> = file.into();
384 builder = builder.add_source(f.format(config::FileFormat::Toml).required(required));
384 // We need the path to be absolute here; since figment wants to look
385 // at all our parent directories.
386 // TODO Find a better way to disable this behavior?
  • Ian Jackson
    Ian Jackson @Diziet started a thread on commit 4c340178
  • 68 68 /// You can make one out of a `PathBuf`, examining its syntax like `arti` does,
    69 69 /// using `ConfigurationSource::from_path`.
    70 70 #[derive(Clone, Debug, Eq, PartialEq, Hash, Ord, PartialOrd)]
    71 #[allow(clippy::exhaustive_enums)] // Callers will need to understand this
    71 #[allow(clippy::exhaustive_enums)]
    72 72 pub enum ConfigurationSource {
    73 73 /// A plain file
    74 74 File(PathBuf),
    75 75
    76 76 /// A directory
    77 77 Dir(PathBuf),
    78
    79 /// A verbatim TOML file
    80 Verbatim(Arc<String>),
    • Comment on lines +78 to +80

      I think this is a good change.

      I wondered whether the existing command line override handling --option KEY=VALUE etc. ought to be using this but on inspection it wants to handle all of those afterwards (so that the precedence is right). So probably it is right that it continues to be an ad-hoc Vec<String>.

    • Please register or sign in to reply
  • Ian Jackson
    Ian Jackson @Diziet started a thread on an outdated change in commit bd3f8a14
  • 131 /// An error that occurs while trying to read and process our configuration.
    132 #[derive(Debug, Clone, thiserror::Error)]
    133 #[non_exhaustive]
    134 pub enum ConfigError {
    135 /// We encoundered a problem with file permissions.
    136 #[error("Bad permissions on configuration file")]
    137 Permissions(#[source] fs_mistrust::Error),
    138 /// Our underlying configuration library gave an error.
    139 #[error("Invalid configuration data")]
    140 Parse(#[source] ConfigParseError),
    141 /// Encountered an IO error while reading (or preparing to read) a file.
    142 #[error("Unable to read configuration file {0:?}")]
    143 Io(
    144 Option<std::path::PathBuf>,
    145 #[source] std::sync::Arc<std::io::Error>,
    146 ),
  • Ian Jackson
    Ian Jackson @Diziet started a thread on an outdated change in commit e9fb6b6b
  • 77 77 #[doc(hidden)]
    78 78 pub use flatten::flattenable_extract_fields;
    79 79
    80 /// A set of configuration fields, represented as a set of nested K=V
    81 /// mappings.
    82 ///
    83 /// (This is a wrapper for an underlying type provided by the library that
    84 /// actually does our configuration.)
    85 #[derive(Clone, Debug)]
    86 pub struct ConfigurationFields(config::Config);
    • Comment on lines +80 to +86

      I think we might want to do this even if we don't switch to figment.

      I don't like the name, though. "Fields" isn't quite what's going on here. "ConfigurationTree"?

    • I think we might want to do this even if we don't switch to figment.

      I think that whether or not we switch, we should take everything in this branch up to the Figment conversion. (That is, everything that makes the config dependency hidden.)

      I don't like the name, though. "Fields" isn't quite what's going on here. "ConfigurationTree"?

      I like ConfigurationTree.

    • Please register or sign in to reply
  • I'm not sure.

    When I ported Otter to Rocket 0.5 it had switched to Figment (Sergio is the author of Rocket). I'm definitely a fan of Rocket, and Figment is kind of part of Rocket. (That's where the "profiles" thing comes from - webapps generally seem to want something like that, although I often find it confusing.)

    It's disappointing to find ourselves still doing the ignored key tracking ourselves!

    I'm inclined to say "yes" but maybe after upstream !95/#94 is fixed? You've had some time to sleep on it - what do you think now?

    (I had some quibbles about details too.)

  • It's disappointing to find ourselves still doing the ignored key tracking ourselves!

    I think that this is a side-effect of the way that we superimpose multiple configuration objects. Other tools seem to put all of their configuration into one Deserialize object, which lets them tell serde to reject unrecognized fields.

    I'm inclined to say "yes" but maybe after upstream !95 (closed)/#94 is fixed? You've had some time to sleep on it - what do you think now?

    I like how much better the figment errors are than the config errors, and I like how the maintainer was more responsive so far—compare response time on figment#94 to lack of response on config-rs#532.

    That said, I'm fine until we see how the upstream issue is resolved.

  • Nick Mathewson added 2 commits

    added 2 commits

    • 42f784b5 - Rename ConfigurationFields to ConfigurationTree.
    • d42119c6 - Add an action to Io error in tor-config.

    Compare with previous version

  • FWIW, with respect to the example in #1267 (closed), the error with config-rs is:

    ./target/quicktest/arti: error: read configuration: Config contents not as expected: Invalid listen specification: failed to parse string: invalid socket address syntax

    And the error with this branch is

    ./target/quicktest/arti: error: read configuration: Config contents not as expected: Invalid configuration data: Invalid listen specification: failed to parse string: invalid socket address syntax for key "default.proxy.socks_listen" in fred.cfg TOML file

    So I think that that's a point in favor of figment.

  • Nick Mathewson
    Nick Mathewson @nickm started a thread on commit bd3f8a14
  • 350 343
    351 344 /// Add every file and commandline source to `builder`, returning a new
    352 345 /// builder.
    353 fn add_sources(self, mut builder: ConfigBuilder) -> Result<ConfigBuilder, ConfigError> {
    346 fn add_sources(self, mut builder: Figment) -> Result<Figment, ConfigError> {
    347 use figment::providers::Format;
    348
    349 // Note that we're using `admerge` here. It causes later sources' options
    350 // to replace those in earlier sources, and causes arrays to be extended
    351 // rather than replaces.
    352 //
    353 // TODO: Maybe we should use "merge" instead? That does the same,
    354 // except for the array behavior.
    • Comment on lines +349 to +354

      BTW, @Diziet, any opinion on this?

    • We probably don't want to make this depend on which TOML syntax was used - [[array]] vs array = [...].

      We need to be able to override arrays on the command line. But ideally we'd like to be able to extend them too. That might mean different command line options...

      I don't think we need to be able to override arrays in subsequent config files, although that might be nice.

      What's the current behaviour?

    • admerge is documented at https://docs.rs/figment/latest/figment/struct.Figment.html#method.admerge . AFAICT it isn't syntax-aware, and can't easily be made syntax-aware.

      One option might be to avoid suggesting [[arrays]] in our configuration.

      In any case, I think that config probably just replaces arrays, though that's something we should test.

    • Please register or sign in to reply
  • I think I'm sold on switching to figment. I'm not sure about array merging.

  • Nick Mathewson mentioned in merge request !2040 (merged)

    mentioned in merge request !2040 (merged)

  • Nick Mathewson mentioned in merge request !2041 (merged)

    mentioned in merge request !2041 (merged)

  • I have split this MR into !2040 (merged) (encapsulating config-rs) and !2041 (merged) (porting to figment). I've referenced this MR from !2041 (merged), with a note that all the comments here are still relevant, and need to be addressed.

  • closed

  • Ian Jackson mentioned in issue #1337

    mentioned in issue #1337

  • Please register or sign in to reply
    Loading