Configuration

Setting Configuration Values

Fides can be configured in two different ways: either via a toml file or via environment variables. Both methods can be used simultaneously, with environment variables taking precedence over the toml file values.

Using a Configuration File

Fides will use the first config file it can read from the following locations. Listed in order of precedence they are:

At the path specified using the config file argument passed through the CLI, i.e. fides -f <config_path>
At the path specified by the FIDES__CONFIG_PATH environment variable
In the current working directory it will check for a subdir .fides and a file within named fides.toml, i.e. ./.fides/fides.toml

Generating a Config File

If you'd like to generate a new config file automatically using default values, run fides init. This will create the file at the default location of ./.fides/fides.toml

Setting Values Via Environment Variables

Fides follows a set pattern for configuration via environment variables. It looks for environment variables that start with FIDES followed by the config subsection name, followed by the key name, all separated with double underscores. In practice this would look like FIDES__<SUBSECTION>__<KEY>

As a toml configuration value:

[database]
host = config_example

As an environment variable:

EXPORT FIDES__DATABASE__HOST=config_example

Viewing your configuration

You can view the current configuration of your application via either the CLI or API.

CLI

To view your application configuration via the CLI, run:

1	`fides view config`

This will show all configuration variables, including sensitive ones. It is printed to the console as valid toml, so this can be copy/pasted as needed.

API

To view your application configuration in the API, run:

1	`GET /api/v1/config`

For security reasons, sensitive configuration values will not be shown here.

Special Sections

There are a few "special" configuration sections that behave in unique ways compared to the other sections. These sections will be addressed in the following documentation.

Celery

Fides uses Celery for asynchronous task management.

To simplify deployments and remove the need for two different toml configuration files, it is possible to configure Celery via the Fides configuration file. Any valid configuration key/value pair for Celery can instead be added to the Fides toml configuration file and will automatically be passed through to the Celery deployment. Note that Fides will not validate any of these key/value pairs. See the above configuration file reference for an example of using celery configuration pass-through.

For a full list of possible variable overrides, see the Celery configuration documentation.

Example Celery Section
[celery]
event_queue_prefix = "fides_worker"
task_default_queue = "fides"
task_always_eager = true

Credentials

The credentials section uses custom keys which can be referenced in specific commands that take the --credentials-id option. For example, a command that uses a credential might look like fides scan dataset db --credentials-id app_postgres. The credential object itself will be validated at the time of use depending on what type of credential is required. For instance if fides scan system okta is used, it will expect the object to contain orgUrl and token key/value pairs. In the case of a typical database like postgres, it will only expect a connection_string. The following is an example of what a credentials section might look like in a given deployment with various applications:

Example Credentials Section
[credentials]
app_postgres = {connection_string="postgresql+psycopg2://postgres:fides@fides-db:5432/fides"}

Configuration File Reference

This following file is an autogenerated configuration reference file. It shows application defaults and is a valid toml file that can be used for configuration of Fides.

fides.toml
# Fides Configuration File
# Additional Documentation at : https://ethyca.github.io/fides/stable/config/

#--------------#
#-- ADMIN_UI --#
#--------------------------------------------------------------------#
[admin_ui] # Configuration settings for the Admin UI.

# Toggle whether the Admin UI is served.
enabled = true # boolean

#------------#
#-- CELERY --#
#--------------------------------------------------------------------#
[celery] # Configuration settings for Celery.  Only a small subset can be configured through Fides env vars

# The prefix to use for event receiver queue names
event_queue_prefix = "fides_worker" # string

# The name of the default queue if a message has no route or no custom
# queue has been specified
task_default_queue = "fides" # string

# If true, tasks are executed locally instead of being sent to the
# queue.  If False, tasks are sent to the queue.
task_always_eager = true # boolean

#---------#
#-- CLI --#
#--------------------------------------------------------------------#
[cli] # Configuration settings for the command-line application.

# A fully anonymized unique identifier that is automatically generated
# by the application. Used for anonymous analytics when opted-in.
analytics_id = "9eaf16cd30d15d9b0c3ff1ee309899f4" # string

# When set to True, disables functionality that requires making calls
# to a Fides webserver.
local_mode = false # boolean

# The protocol used by the Fides webserver.
server_protocol = "http" # string

# The hostname of the Fides webserver.
server_host = "localhost" # string

# The port of the Fides webserver
server_port = "8080" # string

# The path of the Fides webserver
server_path = "/" # string

#-------------#
#-- CONSENT --#
#--------------------------------------------------------------------#
[consent] # Configuration settings for Consent.

# Toggle whether TCF is enabled.
tcf_enabled = false # boolean

# Toggle whether Google AC Mode is enabled.
ac_enabled = false # boolean

# Whether or not vendor purposes can be globally overridden.
override_vendor_purposes = false # boolean

#--------------#
#-- DATABASE --#
#--------------------------------------------------------------------#
[database] # Configuration settings for the application database.

# Automatically runs migrations on webserver startup. If set to
# `false`, will require the user to run migrations manually via the CLI
# or API. WARNING: Must be set to `true` for first-time startup.
automigrate = true # boolean

# Number of concurrent database connections Fides will use for API
# requests. Note that the pool begins with no connections, but as they
# are requested the connections are maintained and reused up to this
# limit.
api_engine_pool_size = 50 # integer

# Number of additional 'overflow' concurrent database connections Fides
# will use for API requests if the pool reaches the limit. These
# overflow connections are discarded afterwards and not maintained.
api_engine_max_overflow = 50 # integer

# Number of seconds of inactivity before the client sends a TCP
# keepalive packet to verify the database connection is still alive.
api_engine_keepalives_idle = 30 # integer

# Number of seconds between TCP keepalive retries if the initial
# keepalive packet receives no response. These are client-side retries.
api_engine_keepalives_interval = 10 # integer

# Maximum number of TCP keepalive retries before the client considers
# the connection dead and closes it.
api_engine_keepalives_count = 5 # integer

# The name of the application database.
db = "default_db" # string

# When set to True, initializes the database with sample data for
# testing (Systems, Datasets, Connectors, etc.) Used by 'fides deploy'
# to configure the sample project.
load_samples = false # boolean

# The password with which to login to the application database.
password = "defaultpassword" # string

# The port at which the application database will be accessible.
port = "5432" # string

# The hostname of the application database server.
server = "default-db" # string

# Number of concurrent database connections Fides will use for
# executing privacy request tasks, either locally or on each worker.
# Note that the pool begins with no connections, but as they are
# requested the connections are maintained and reused up to this limit.
task_engine_pool_size = 50 # integer

# Number of additional 'overflow' concurrent database connections Fides
# will use for executing privacy request tasks, either locally or on
# each worker, if the pool reaches the limit. These overflow
# connections are discarded afterwards and not maintained.
task_engine_max_overflow = 50 # integer

# Number of seconds of inactivity before the client sends a TCP
# keepalive packet to verify the database connection is still alive.
task_engine_keepalives_idle = 30 # integer

# Number of seconds between TCP keepalive retries if the initial
# keepalive packet receives no response. These are client-side retries.
task_engine_keepalives_interval = 10 # integer

# Maximum number of TCP keepalive retries before the client considers
# the connection dead and closes it.
task_engine_keepalives_count = 5 # integer

# The database user with which to login to the application database.
user = "defaultuser" # string

# Additional connection parameters used when connecting to the
# application database.
params = {} # object

#---------------#
#-- EXECUTION --#
#--------------------------------------------------------------------#
[execution] # Configuration settings for DSR execution.

# If set to True, only use UPDATE requests to mask data. If False,
# Fides will use any defined DELETE or GDPR DELETE endpoints to remove
# PII, which may extend beyond the specific data categories that
# configured in your execution policy.
masking_strict = true # boolean

# The amount of time to wait for actions which delay privacy requests
# (e.g., pre- and post-processing webhooks).
privacy_request_delay_timeout = 3600 # integer

# Whether privacy requests require explicit approval to execute.
require_manual_request_approval = false # boolean

# Whether privacy requests require user identity verification.
subject_identity_verification_required = false # boolean

# The backoff factor for retries, to space out repeated retries.
task_retry_backoff = 1 # integer

# The number of times a failed request will be retried.
task_retry_count = 0 # integer

# The delays between retries in seconds.
task_retry_delay = 1 # integer

# Allows the collection of custom privacy request fields from incoming
# privacy requests.
allow_custom_privacy_request_field_collection = false # boolean

# Allows custom privacy request fields to be used in request execution.
allow_custom_privacy_request_fields_in_request_execution = false # boolean

# The number of seconds a request task should live.
request_task_ttl = 604800 # integer

# Seconds between polling for Privacy Requests that should change state
state_polling_interval = 30 # integer

# Temporary flag to switch to using DSR 3.0 to process your tasks.
use_dsr_3_0 = false # boolean

#-------------#
#-- LOGGING --#
#--------------------------------------------------------------------#
[logging] # Configuration settings for application logging.

# The output location for log files. Accepts any valid file path. If
# left unset, log entries are printed to stdout and log files are not
# produced.
destination = "" # string

# Force colored logs. Any value set via environment variables is
# considered 'True'.
colorize = false # boolean

# The minimum log entry level to produce. Also accepts TRACE, DEBUG,
# WARNING, ERROR, or CRITICAL (case insensitive).
level = "INFO" # string

# The format with which to produce log entries. If left unset, produces
# log entries formatted using the internal custom formatter. Also
# accepts 'JSON' (case insensitive).
serialization = "" # string

# If True, PII values will display unmasked in log output. This
# variable should always be set to 'False' in production systems.
log_pii = false # boolean

#-------------------#
#-- NOTIFICATIONS --#
#--------------------------------------------------------------------#
[notifications] # Configuration settings for Data Subject and/or Data Processor notifications.

# When set to True, enables subject notifications upon privacy request
# completion.
send_request_completion_notification = false # boolean

# When set to True, enables subject notifications upon privacy request
# receipt.
send_request_receipt_notification = false # boolean

# When set to True, enables subject notifications upon privacy request
# review.
send_request_review_notification = false # boolean

# When set to True, enables property specific messaging feature,
# otherwise fall back on the messaging template type env flags set
# above.
enable_property_specific_messaging = false # boolean

#-----------#
#-- REDIS --#
#--------------------------------------------------------------------#
[redis] # Configuration settings for Redis.

# Character set to use for Redis, defaults to 'utf8'. Not recommended
# to change.
charset = "utf8" # string

# The application will use this index in the Redis cache to cache data.
db_index = 0 # integer

# Whether or not to automatically decode the values fetched from Redis.
# Decodes using the `charset` configuration value.
decode_responses = true # boolean

# The number of seconds for which data will live in Redis before
# automatically expiring.
default_ttl_seconds = 604800 # integer

# Whether the application's Redis cache should be enabled. Only set to
# false for certain narrow uses of the application.
enabled = true # boolean

# The network address for the application Redis cache.
host = "redis" # string

# Sets TTL for cached identity verification code as part of subject
# requests.
identity_verification_code_ttl_seconds = 600 # integer

# The password with which to login to the Redis cache.
password = "testpassword" # string

# The port at which the application cache will be accessible.
port = 6379 # integer

# Whether the application's connections to the cache should be
# encrypted using TLS.
ssl = false # boolean

# If using TLS encryption, set this to 'required' if you wish to
# enforce the Redis cache to provide a certificate. Note that not all
# cache providers support this without setting ssl_ca_certs (e.g. AWS
# Elasticache).
ssl_cert_reqs = "required" # string

# If using TLS encryption rooted with a custom Certificate Authority,
# set this to the path of the CA certificate.
ssl_ca_certs = "" # string

# The user with which to login to the Redis cache.
user = "" # string

#--------------#
#-- SECURITY --#
#--------------------------------------------------------------------#
[security] # Configuration settings for application security.

# Length of desired encryption key when using Fides to generate a
# random secure string used for AES encryption.
aes_encryption_key_length = 16 # integer

# Length of desired random byte str for the AES GCM encryption used
# throughout Fides.
aes_gcm_nonce_length = 12 # integer

# The key used to sign Fides API access tokens.
app_encryption_key = "" # string

# Text encoding to use for the application.
encoding = "UTF-8" # string

# The default, `dev`, does not apply authentication to endpoints
# typically used by the CLI. The other option, `prod`, requires
# authentication for _all_ endpoints that may contain sensitive
# information.
env = "prod" # string

# The number of times identity verification will be attempted before
# raising an error.
identity_verification_attempt_limit = 3 # integer

# The value used to identify the Fides application root API client.
oauth_root_client_id = "" # string

# The secret value used to authenticate the Fides application root API
# client.
oauth_root_client_secret = "" # string

# The time in minutes for which Fides API tokens will be valid. Default
# value is equal to 8 days.
oauth_access_token_expire_minutes = 11520 # integer

# Sets desired length in bytes of generated client id used for oauth.
oauth_client_id_length_bytes = 16 # integer

# Sets desired length in bytes of generated client secret used for
# oauth.
oauth_client_secret_length_bytes = 16 # integer

# The number of requests from a single IP address allowed to hit a
# public endpoint within the specified time period
public_request_rate_limit = "2000/minute" # string

# The prefix given to keys in the Redis cache used by the rate limiter.
rate_limit_prefix = "fides-" # string

# The number of requests from a single IP address allowed to hit an
# endpoint within a rolling 60 second period.
request_rate_limit = "1000/minute" # string

# The list of scopes that are given to the root user.
root_user_scopes = [] # array

# The list of roles that are given to the root user.
root_user_roles = [] # array

# If set to True, the user interface will display a download button for
# subject requests.
subject_request_download_ui_enabled = false # boolean

# If set to True, contributor and owner roles will be able to run test
# privacy requests.
dsr_testing_tools_enabled = false # boolean

# The number of seconds that a pre-signed download URL when using S3
# storage will be valid. The default is equal to 5 days.
subject_request_download_link_ttl_seconds = 432000 # integer

# Either enables the collection of audit log resource data or bypasses
# the middleware
enable_audit_log_resource_middleware = false # boolean

# The timeout in seconds for the transport socket
# (``socket.settimeout``)
bastion_server_ssh_timeout = 0.1 # number

# The timeout in seconds for tunnel connection (open_channel timeout)
bastion_server_ssh_tunnel_timeout = 10 # number

#----------#
#-- USER --#
#--------------------------------------------------------------------#
[user] # Configuration settings that apply to the current user as opposed to the entire application instance.

# When set to true, prevents sending privacy-respecting anonymous
# analytics data to Ethyca.
analytics_opt_out = true # boolean

# An arbitrary string used to encrypt the user data stored in the
# database. Encryption is implemented using PGP.
encryption_key = "test_encryption_key" # string

# The username used to log into the Fides webserver.
username = "" # string

# The password used to log into the Fides webserver.
password = "" # string

#-----------------#
#-- CREDENTIALS --#
#--------------------------------------------------------------------#
[credentials] # This is a special section that is used to store arbitrary key/value pairs to be used as credentials.
# For more info, please visit: https://ethyca.github.io/fides/stable/config/#credentials