Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: supports reading config values from CLI #605

Merged
merged 20 commits into from
Oct 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
7b5bc27
feat: supports reading config values from CLI
anthonykimani Sep 25, 2024
9d0ac1f
chore: adds config lib to cargo.toml, uses server default config valu…
anthonykimani Sep 26, 2024
ea7c31b
chore: tries to load YAML file
anthonykimani Sep 26, 2024
f2bf609
chore: tries to load YAML file
anthonykimani Sep 26, 2024
ec9ede0
Merge branch 'dev' of github.com:tlsnotary/tlsn into dev
anthonykimani Sep 26, 2024
15c5bd6
fix: loads config.yaml properly, refactors code
anthonykimani Sep 27, 2024
9d04e31
fix: removes .idea folder and moves config lib to notary-server cargo…
anthonykimani Sep 27, 2024
4f962ae
feat: uses serde-aux to deserialize env vars port and tls-enabled fro…
anthonykimani Oct 1, 2024
a78a417
Merge branch 'dev' into dev
anthonykimani Oct 1, 2024
ac1a24a
fix: parses int and bool using try-parsing instead of serde-aux and r…
anthonykimani Oct 4, 2024
04c7976
chore: converts config to snake_case for consistency
anthonykimani Oct 4, 2024
224257b
Merge branch 'dev' into dev
yuroitaki Oct 8, 2024
a75e60f
doc: adds configuration documentation, code comments and fixes lintin…
anthonykimani Oct 8, 2024
9fc3f32
doc: adds configuration documentation, code comments and fixes lintin…
anthonykimani Oct 8, 2024
7855a46
Merge branch 'dev' into dev
yuroitaki Oct 9, 2024
9c68a0c
fix: fixes ci linting formatting
anthonykimani Oct 9, 2024
1f23f0c
fix: fixes ci linting formatting
anthonykimani Oct 9, 2024
0c3a02f
Merge branch 'dev' of github.com:anthonykimani/tlsn into dev
anthonykimani Oct 9, 2024
44cd324
fix: adjusts formatting for settings.rs and minor adjustments to docu…
anthonykimani Oct 10, 2024
20f33c4
fix: uses cargo nightly to format correctly
anthonykimani Oct 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions crates/notary/server/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ axum-core = { version = "0.4" }
axum-macros = { version = "0.4" }
base64 = { version = "0.21" }
chrono = { version = "0.4" }
config = { version = "0.14", features = ["yaml"] }
csv = { version = "1.3" }
eyre = { version = "0.6" }
futures = { workspace = true }
Expand Down
28 changes: 28 additions & 0 deletions crates/notary/server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,34 @@ docker run --init -p 127.0.0.1:7047:7047 -v <your folder path>:/root/.notary-ser
```bash
docker run --init -p 127.0.0.1:7047:7047 -v <your folder path>:/root/.notary-server/fixture/notary notary-server:local
```

### Configuration

The notary server can be configured using three methods: a configuration file, command-line interface (CLI) arguments, and environment variables. These methods provide flexibility in how you set up and run the server.

1. Configuration File - By default, the server looks for a config.yaml file in the `notary/server/config/` directory. This file contains all the configurable settings for the server, e.g.
```yaml
server:
name: "notary-server"
host: "0.0.0.0"
port: 7047

notarization:
max_sent_data: 4096
max_recv_data: 16384
anthonykimani marked this conversation as resolved.
Show resolved Hide resolved

...
```

2. Command-Line Interface (CLI) Arguments - You can override *some* configuration file settings using CLI arguments when starting the server. This also takes precedence over the environment variable method below. E.g.
```shell
cargo run -- --port 8080 --tls-enabled false --log-level INFO
```

3. Environment Variables - This can be used to configure all the server settings, where it will override the config file. It uses the prefix `NOTARY_SERVER__` followed by the configuration key(s) in uppercase. Double underscores are used in nested configuration keys, e.g. `tls.enabled` in the config file will be `NOTARY_SERVER__TLS__ENABLED`. E.g.
```shell
NOTARY_SERVER__SERVER__PORT=8080 NOTARY_SERVER__NOTARIZATION__MAX_SENT_DATA=2048 NOTARY_SERVER__TLS__ENABLED=false cargo run
```
---
## API
All APIs are TLS-protected, hence please use `https://` or `wss://`.
Expand Down
18 changes: 9 additions & 9 deletions crates/notary/server/config/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ server:
name: "notary-server"
host: "0.0.0.0"
port: 7047
html-info: |
html_info: |
anthonykimani marked this conversation as resolved.
Show resolved Hide resolved
<h1>Notary Server {version}!</h1>
<ul>
<li>git commit hash: <a href="https://github.com/tlsnotary/tlsn/commit/{git_commit_hash}">{git_commit_hash}</a></li>
Expand All @@ -12,21 +12,21 @@ server:
<a href="/healthcheck">health check</a> - <a href="/info">info</a><br/>

notarization:
max-sent-data: 4096
max-recv-data: 16384
max_sent_data: 4096
max_recv_data: 16384

tls:
enabled: true
private-key-pem-path: "./fixture/tls/notary.key"
certificate-pem-path: "./fixture/tls/notary.crt"
private_key_pem_path: "./fixture/tls/notary.key"
certificate_pem_path: "./fixture/tls/notary.crt"

notary-key:
private-key-pem-path: "./fixture/notary/notary.key"
public-key-pem-path: "./fixture/notary/notary.pub"
notary_key:
private_key_pem_path: "./fixture/notary/notary.key"
public_key_pem_path: "./fixture/notary/notary.pub"

logging:
level: DEBUG

authorization:
enabled: false
whitelist-csv-path: "./fixture/auth/whitelist.csv"
whitelist_csv_path: "./fixture/auth/whitelist.csv"
7 changes: 0 additions & 7 deletions crates/notary/server/src/config.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
use serde::Deserialize;

#[derive(Clone, Debug, Deserialize, Default)]
#[serde(rename_all = "kebab-case")]
pub struct NotaryServerProperties {
/// Name and address of the notary server
pub server: ServerProperties,
Expand All @@ -18,7 +17,6 @@ pub struct NotaryServerProperties {
}

#[derive(Clone, Debug, Deserialize, Default)]
#[serde(rename_all = "kebab-case")]
pub struct AuthorizationProperties {
/// Switch to turn on or off auth middleware
pub enabled: bool,
Expand All @@ -27,7 +25,6 @@ pub struct AuthorizationProperties {
}

#[derive(Clone, Debug, Deserialize, Default)]
#[serde(rename_all = "kebab-case")]
pub struct NotarizationProperties {
/// Global limit for maximum number of bytes that can be sent
pub max_sent_data: usize,
Expand All @@ -36,7 +33,6 @@ pub struct NotarizationProperties {
}

#[derive(Clone, Debug, Deserialize, Default)]
#[serde(rename_all = "kebab-case")]
pub struct ServerProperties {
/// Used for testing purpose
pub name: String,
Expand All @@ -49,7 +45,6 @@ pub struct ServerProperties {
}

#[derive(Clone, Debug, Deserialize, Default)]
#[serde(rename_all = "kebab-case")]
pub struct TLSProperties {
/// Flag to turn on/off TLS between prover and notary (should always be
/// turned on unless TLS is handled by external setup e.g. reverse proxy,
Expand All @@ -60,14 +55,12 @@ pub struct TLSProperties {
}

#[derive(Clone, Debug, Deserialize, Default)]
#[serde(rename_all = "kebab-case")]
pub struct NotarySigningKeyProperties {
pub private_key_pem_path: String,
pub public_key_pem_path: String,
}

#[derive(Clone, Debug, Deserialize, Default)]
#[serde(rename_all = "kebab-case")]
pub struct LoggingProperties {
/// Log verbosity level of the default filtering logic, which is
/// notary_server=<level>,tlsn_verifier=<level>,tls_mpc=<level> Must be either of <https://docs.rs/tracing/latest/tracing/struct.Level.html#implementations>
Expand Down
12 changes: 12 additions & 0 deletions crates/notary/server/src/domain/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,16 @@
/// Configuration file location
#[structopt(long, default_value = "./config/config.yaml")]
pub config_file: String,

/// Port of notary server

Check warning on line 11 in crates/notary/server/src/domain/cli.rs

View check run for this annotation

Codecov / codecov/patch

crates/notary/server/src/domain/cli.rs#L11

Added line #L11 was not covered by tests
#[structopt(long)]
anthonykimani marked this conversation as resolved.
Show resolved Hide resolved
pub port: Option<u16>,

/// Flag to turn on/off TLS when connecting to prover

Check warning on line 15 in crates/notary/server/src/domain/cli.rs

View check run for this annotation

Codecov / codecov/patch

crates/notary/server/src/domain/cli.rs#L15

Added line #L15 was not covered by tests
#[structopt(long)]
anthonykimani marked this conversation as resolved.
Show resolved Hide resolved
pub tls_enabled: Option<bool>,

/// Level of logging

Check warning on line 19 in crates/notary/server/src/domain/cli.rs

View check run for this annotation

Codecov / codecov/patch

crates/notary/server/src/domain/cli.rs#L19

Added line #L19 was not covered by tests
#[structopt(long)]
anthonykimani marked this conversation as resolved.
Show resolved Hide resolved
pub log_level: Option<String>,
}
2 changes: 2 additions & 0 deletions crates/notary/server/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ mod middleware;
mod server;
mod server_tracing;
mod service;
mod settings;
mod signing;
mod util;

Expand All @@ -19,4 +20,5 @@ pub use domain::{
pub use error::NotaryServerError;
pub use server::{read_pem_file, run_server};
pub use server_tracing::init_tracing;
pub use settings::Settings;
pub use util::parse_config_file;
18 changes: 8 additions & 10 deletions crates/notary/server/src/main.rs
Original file line number Diff line number Diff line change
@@ -1,25 +1,23 @@
use eyre::{eyre, Result};
use notary_server::{init_tracing, run_server, CliFields, NotaryServerError, Settings};
use structopt::StructOpt;
use tracing::debug;

use notary_server::{
init_tracing, parse_config_file, run_server, CliFields, NotaryServerError,
NotaryServerProperties,
};

#[tokio::main]
async fn main() -> Result<(), NotaryServerError> {
// Load command line arguments which contains the config file location
// Load command line arguments

Check warning on line 8 in crates/notary/server/src/main.rs

View check run for this annotation

Codecov / codecov/patch

crates/notary/server/src/main.rs#L8

Added line #L8 was not covered by tests
let cli_fields: CliFields = CliFields::from_args();
let config: NotaryServerProperties = parse_config_file(&cli_fields.config_file)?;

let settings =
Settings::new(&cli_fields).map_err(|err| eyre!("Failed to load settings: {}", err))?;

Check warning on line 12 in crates/notary/server/src/main.rs

View check run for this annotation

Codecov / codecov/patch

crates/notary/server/src/main.rs#L10-L12

Added lines #L10 - L12 were not covered by tests

// Set up tracing for logging
init_tracing(&config).map_err(|err| eyre!("Failed to set up tracing: {err}"))?;
init_tracing(&settings.config).map_err(|err| eyre!("Failed to set up tracing: {err}"))?;

Check warning on line 15 in crates/notary/server/src/main.rs

View check run for this annotation

Codecov / codecov/patch

crates/notary/server/src/main.rs#L15

Added line #L15 was not covered by tests

debug!(?config, "Server config loaded");
anthonykimani marked this conversation as resolved.
Show resolved Hide resolved
debug!(?settings.config, "Server config loaded");

Check warning on line 17 in crates/notary/server/src/main.rs

View check run for this annotation

Codecov / codecov/patch

crates/notary/server/src/main.rs#L17

Added line #L17 was not covered by tests

// Run the server
run_server(&config).await?;
run_server(&settings.config).await?;

Check warning on line 20 in crates/notary/server/src/main.rs

View check run for this annotation

Codecov / codecov/patch

crates/notary/server/src/main.rs#L20

Added line #L20 was not covered by tests

Ok(())
}
45 changes: 45 additions & 0 deletions crates/notary/server/src/settings.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
use crate::{CliFields, NotaryServerProperties};
use config::{Config, ConfigError, Environment, File};
use serde::Deserialize;
use std::path::Path;

#[derive(Debug, Deserialize)]

Check warning on line 6 in crates/notary/server/src/settings.rs

View check run for this annotation

Codecov / codecov/patch

crates/notary/server/src/settings.rs#L6

Added line #L6 was not covered by tests
pub struct Settings {
#[serde(flatten)]
pub config: NotaryServerProperties,
}

impl Settings {
pub fn new(cli_fields: &CliFields) -> Result<Self, ConfigError> {
let config_path = Path::new(&cli_fields.config_file);

let mut builder = Config::builder()
// Load base configuration
.add_source(File::from(config_path))
// Add in settings from environment variables (with a prefix of NOTARY_SERVER and '__'
// as separator).
.add_source(
Environment::with_prefix("NOTARY_SERVER")
.try_parsing(true)
.prefix_separator("__")
.separator("__"),
);

Check warning on line 26 in crates/notary/server/src/settings.rs

View check run for this annotation

Codecov / codecov/patch

crates/notary/server/src/settings.rs#L13-L26

Added lines #L13 - L26 were not covered by tests

// Apply CLI argument overrides
if let Some(port) = cli_fields.port {
builder = builder.set_override("server.port", port)?;
}
if let Some(tls_enabled) = cli_fields.tls_enabled {
builder = builder.set_override("tls.enabled", tls_enabled)?;
}
if let Some(log_level) = &cli_fields.log_level {
builder = builder.set_override("logging.level", log_level.clone())?;
}

Check warning on line 37 in crates/notary/server/src/settings.rs

View check run for this annotation

Codecov / codecov/patch

crates/notary/server/src/settings.rs#L29-L37

Added lines #L29 - L37 were not covered by tests

let config = builder.build()?;

Check warning on line 39 in crates/notary/server/src/settings.rs

View check run for this annotation

Codecov / codecov/patch

crates/notary/server/src/settings.rs#L39

Added line #L39 was not covered by tests

let settings: Settings = config.try_deserialize()?;

Check warning on line 41 in crates/notary/server/src/settings.rs

View check run for this annotation

Codecov / codecov/patch

crates/notary/server/src/settings.rs#L41

Added line #L41 was not covered by tests

Ok(settings)
}

Check warning on line 44 in crates/notary/server/src/settings.rs

View check run for this annotation

Codecov / codecov/patch

crates/notary/server/src/settings.rs#L43-L44

Added lines #L43 - L44 were not covered by tests
}