Production-ready microservice in Rust: 3. Configuration

How to load configuration from files or environment variables

To work with configuration data we will need a few more crates, add them to the dependencies section of shelter_main/Cargo.toml:

serde = { version = "1.0", features = ["derive"] }
serde_json = "1"
config = "0.13"

Now go to the shelter_main/src directory and create a new file named settings.rs. We will create a simple schema for our configuration data:

pub struct Database {
    pub url: String,
}

pub struct Logging {
    pub log_level: String,
}

pub struct Settings {
    pub database: Database,
    pub logging: Logging,
}

The Database struct will store a url to our database, the Logging struct will store the log level configuration and the main Settings struct will include both a Database and a Logging configuration.

To be able to deserialize these structs from various formats we have to add the Deserialize macro to the structs. I also add the Default macro to be able to instantiate these structs without specifing a value for all the fields. The Debug macro is also handy, so we can easily log the contents of the configuration later.

use serde::Deserialize;

#[derive(Debug, Deserialize, Default)]
#[allow(unused)]
pub struct Database {
    pub url: String,
}

#[derive(Debug, Deserialize, Default)]
#[allow(unused)]
pub struct Logging {
    pub log_level: String,
}

#[derive(Debug, Deserialize, Default)]
#[allow(unused)]
pub struct Settings {
    pub database: Database,
    pub logging: Logging,
}

I also added the allow(unused) marker, to silence compiler warnings about unused fields.

There is one more problem with this schema: all the fields are required, so you cannot import an empty json for example and use the defaults for all configuration options.

There are two possible ways to solve this problem:

• make fields optional with Option<>
• or use default values

I usually prefer Option<> for basic values like strings, numbers, boolean flags. This way we can easily replace missing values with defaults:

pub struct Logging {
    pub log_level: Option<String>
}

let log_level = settings.logging.log_level.unwrap_or("info");

For structures, I prefer to go with default values, so an empty structure of the configuration is always built for us:

#[derive(Debug, Deserialize, Default)]
#[allow(unused)]
pub struct Settings {
    #[serde(default)]
    pub database: Database,
    #[serde(default)]
    pub logging: Logging,
}

The config-rs crate can load configuration from both configuration files and environment variables. A config file is handy for the bulk of the configuration, enviroment variables are preferable for sensitive values like passwords and settings that usually deviate in different environments. In a kubernetes-based deployment probably the whole configuration will be built from environment variables.

To use both a config file and enviroment variables, we use a layered configuration:

impl Settings {
    pub fn new(location: &str, env_prefix: &str) -> anyhow::Result<Self> {
        let s = Config::builder()
            .add_source(File::with_name(location))
            .add_source(
                Environment::with_prefix(env_prefix)
                    .separator("__")
                    .prefix_separator("__"),
            .build()?;

        let settings = s.try_deserialize()?;

        Ok(settings)
    }
}

First we load a configuration file from location then override these settings with values found in environment variables.

Assuming an env_prefix value of SHELTER the enviroment variable names will look like these:

• SHELTER__DATABASE__URL
• SHELTER__LOGGING__LOG_LEVEL

I also prefer to store the config file location and other parameters required to be able to reload the configuration later:

#[derive(Debug, Deserialize, Default)]
#[allow(unused)]
pub struct ConfigInfo {
    pub location: Option<String>,
    pub env_prefix: Option<String>,
}

#[derive(Debug, Deserialize, Default)]
#[allow(unused)]
pub struct Settings {
    #[serde(default)]
    pub config: ConfigInfo,
    #[serde(default)]
    pub database: Database,
    #[serde(default)]
    pub logging: Logging,
}

impl Settings {
    pub fn new(location: &str, env_prefix: &str) -> anyhow::Result<Self> {
        let s = Config::builder()
            .add_source(File::with_name(location))
            .add_source(
                Environment::with_prefix(env_prefix)
                    .separator("__")
                    .prefix_separator("__"),
            )
            .set_override("config.location", location)?
            .set_override("config.env_prefix", env_prefix)?
            .build()?;

        let settings = s.try_deserialize()?;

        Ok(settings)
    }
}

I save these values with the set_override calls.

Now our settings.rs is ready, include it in lib.rs:

pub mod commands;
pub mod settings;

Let's see how to use the new settings from main.rs:

use shelter_main::settings;

pub fn main() -> anyhow::Result<()> {
    dotenv().ok();

    let mut command = Command::new("Dog Shelter sample application")
        .version("1.0")
        .author("Sandor Apati <[email protected]>")
        .about("A sample application to experiment with Rust-based microservices")
        .arg(
            Arg::new("config")
                .short('c')
                .long("config")
                .help("Configuration file location")
                .default_value("config.json"),
        );

    command = commands::configure(command);

    let matches = command.get_matches();

    let config_location = matches
        .get_one::<String>("config")
        .map(|s| s.as_str())
        .unwrap_or("");

    let settings = settings::Settings::new(config_location, "SHELTER")?;

    println!(
        "db url: {}",
        settings
            .database
            .url
            .unwrap_or("missing database url".to_string())
    );

    println!(
        "log level: {}",
        settings.logging.log_level.unwrap_or("info".to_string())
    );

    Ok(())
}

Go to the project root directory, compile and test our code:

$ cargo build
...
$ ./target/debug/shelter_main 
Error: configuration file "config.json" not found

Well, our config.json is missing. Create a simple one:

{
   "database": {
       "url": "pgsql://"
   }
}

And run again:

$ ./target/debug/shelter_main
db url: pgsql://
log level: info

We can see the db url configured in config.json and the default log level. Now define an environment variable to override the db url:

$ export SHELTER__DATABASE__URL="mysql://"
$ ./target/debug/shelter_main
db url: mysql://
log level: info

Thanks to the dotenv crate loaded at the start of main we can also utilize a .env file:

SHELTER__LOGGING__LOG_LEVEL="warn"

Run again:

$ ./target/debug/shelter_main 
db url: mysql://
log level: warn

The configuration file format is not limited to JSON, config-rs can use TOML, YAML, INI and others. See here

Finally, we have to pass our settings to all the subcommands implemented in commands. Modify our commands modules like this:

use clap::{ArgMatches, Command};
use crate::settings::Settings;

pub fn configure() -> Command {
    Command::new("hello").about("Hello World!")
}

pub fn handle(matches: &ArgMatches, _settings: &Settings) -> anyhow::Result<()> {
    if let Some(_matches) = matches.subcommand_matches("hello") {
        println!("Hello World!");
    }

    Ok(())
}

and propage the settings in commands/mod.rs:

mod hello;
mod serve;

use clap::{ArgMatches, Command};
use crate::settings::Settings;

pub fn configure(command: Command) -> Command {
    command
        .subcommand(hello::configure())
        .subcommand(serve::configure())
}

pub fn handle(matches: &ArgMatches, settings: &Settings) -> anyhow::Result<()> {
    hello::handle(matches, settings)?;
    serve::handle(matches, settings)?;

    Ok(())
}

In main.rs pass the settings to the handle method:

let settings = settings::Settings::new(config_location, "SHELTER")?;
commands::handle(&matches, &settings)?;

You can find the sample code on GitHub

Next article »