patroni.config module¶

Facilities related to Patroni configuration.

class patroni.config.Config(configfile: str, validator: Optional[Callable[[Dict[str, Any]], List[str]]] = <function default_validator>)¶

Bases: object

Handle Patroni configuration.

This class is responsible for:

Building and giving access to effective_configuration from:

Config.__DEFAULT_CONFIG – some sane default values;

dynamic_configuration – configuration stored in DCS;

local_configuration – configuration from config.yml or environment.

Saving and loading dynamic_configuration into ‘patroni.dynamic.json’ file located in local_configuration[‘postgresql’][‘data_dir’] directory. This is necessary to be able to restore dynamic_configuration if DCS was accidentally wiped.

Loading of configuration file in the old format and converting it into new format.

Mimicking some dict interfaces to make it possible to work with it as with the old config object.

Variables

PATRONI_CONFIG_VARIABLE – name of the environment variable that can be used to load Patroni configuration from.
__CACHE_FILENAME – name of the file used to cache dynamic configuration under Postgres data directory.
__DEFAULT_CONFIG – default configuration values for some Patroni settings.

PATRONI_CONFIG_VARIABLE = 'PATRONI_CONFIGURATION'¶

__CACHE_FILENAME = 'patroni.dynamic.json'¶

__DEFAULT_CONFIG: Dict[str, Any] = {'loop_wait': 10, 'postgresql': {'parameters': <CaseInsensitiveDict{'wal_level': 'hot_standby', 'hot_standby': 'on', 'max_connections': 100, 'max_wal_senders': 10, 'max_prepared_transactions': 0, 'max_locks_per_transaction': 64, 'track_commit_timestamp': 'off', 'max_replication_slots': 10, 'max_worker_processes': 8, 'wal_log_hints': 'on'} at 7efca4388880>, 'use_slots': True}, 'retry_timeout': 10, 'standby_cluster': {'archive_cleanup_command': '', 'create_replica_methods': '', 'host': '', 'port': '', 'primary_slot_name': '', 'recovery_min_apply_delay': '', 'restore_command': ''}, 'ttl': 30}¶

__get_and_maybe_adjust_int_value(config: Dict[str, Any], param: str, min_value: int) → int¶

Get, validate and maybe adjust a param integer value from the config dict.

Parameters

config – dict object with new global configuration.
param – name of the configuration parameter we want to read/validate/adjust.
min_value – the minimum possible value that a given param could have.

Returns

an integer value which corresponds to a provided param.

__init__(configfile: str, validator: Optional[Callable[[Dict[str, Any]], List[str]]] = <function default_validator>) → None¶

Create a new instance of Config and validate the loaded configuration using validator.

Note

Patroni will read configuration from these locations in this order:

file or directory path passed as command-line argument (configfile), if it exists and the file or files found in the directory can be parsed (see _load_config_path()), otherwise

YAML file passed via the environment variable (see PATRONI_CONFIG_VARIABLE), if the referenced file exists and can be parsed, otherwise

from configuration values defined as environment variables, see _build_environment_configuration().

Parameters

configfile – path to Patroni configuration file.
validator – function used to validate Patroni configuration. It should receive a dictionary which represents Patroni configuration, and return a list of zero or more error messages based on validation.

Raises

ConfigParseError: if any issue is reported by validator.

_build_effective_configuration(dynamic_configuration: Dict[str, Any], local_configuration: Dict[str, Union[Dict[str, Any], Any]]) → Dict[str, Any]¶

Build effective configuration by merging dynamic_configuration and local_configuration.

Note

local_configuration takes precedence over dynamic_configuration if a setting is defined in both.

Parameters

dynamic_configuration – Patroni dynamic configuration.
local_configuration – Patroni local configuration.

Returns

_description_

static _build_environment_configuration() → Dict[str, Any]¶

Get local configuration settings that were specified through environment variables.

Returns: dictionary containing the found environment variables and their values, respecting the expected structure of Patroni configuration.

_load_cache() → None¶: Load dynamic configuration from patroni.dynamic.json.

_load_config_file() → Dict[str, Any]¶

Load configuration file(s) from filesystem and apply values which were set via environment variables.

Returns: final configuration after merging configuration file(s) and environment variables.

_load_config_path(path: str) → Dict[str, Any]¶

Load Patroni configuration file(s) from path.

If path is a file, load the yml file pointed to by path. If path is a directory, load all yml files in that directory in alphabetical order.

Parameters: path – path to either an YAML configuration file, or to a folder containing YAML configuration files.
Returns: configuration after reading the configuration file(s) from path.
Raises: ConfigParseError: if path is invalid.

static _process_postgresql_parameters(parameters: Dict[str, Any], is_local: bool = False) → Dict[str, Any]¶

Process Postgres parameters.

Note

If is_local configuration discard any setting from parameters that is listed under CMDLINE_OPTIONS as those are supposed to be set only through dynamic configuration.

When setting parameters from CMDLINE_OPTIONS through dynamic configuration their value will be validated as per the validator defined in that very same attribute entry. If the given value cannot be validated, a warning will be logged and the default value of the GUC will be used instead.

Some parameters from CMDLINE_OPTIONS cannot be set even if not is_local configuration:

listen_addresses: inferred from postgresql.listen local configuration or from
PATRONI_POSTGRESQL_LISTEN environment variable;

port: inferred from postgresql.listen local configuration or from
PATRONI_POSTGRESQL_LISTEN environment variable;

cluster_name: set through scope local configuration or through PATRONI_SCOPE environment
variable;

hot_standby: always enabled;

wal_log_hints: always enabled.

Parameters

parameters – Postgres parameters to be processed. Should be the parsed YAML value of postgresql.parameters configuration, either from local or from dynamic configuration.
is_local – should be True if parameters refers to local configuration, or False if parameters refers to dynamic configuration.

Returns

new value for postgresql.parameters after processing and validating parameters.

_safe_copy_dynamic_configuration(dynamic_configuration: Dict[str, Any]) → Dict[str, Any]¶

Create a copy of dynamic_configuration.

Merge dynamic_configuration with __DEFAULT_CONFIG (dynamic_configuration takes precedence), and process postgresql.parameters from dynamic_configuration through _process_postgresql_parameters(), if present.

Note

The following settings are not allowed in postgresql section as they are intended to be local configuration, and are removed if present:

connect_address;

proxy_address;

listen;

config_dir;

data_dir;

pgpass;

authentication;

Besides that any setting present in dynamic_configuration but absent from __DEFAULT_CONFIG is discarded.

Parameters: dynamic_configuration – Patroni dynamic configuration.
Returns: copy of dynamic_configuration, merged with default dynamic configuration and with some sanity checks performed over it.

_validate_and_adjust_timeouts(config: Dict[str, Any]) → None¶

Validate and adjust loop_wait, retry_timeout, and ttl values if necessary.

Minimum values:

loop_wait: 1 second;

retry_timeout: 3 seconds.

ttl: 20 seconds;

Maximum values: In case if values don’t fulfill the following rule, retry_timeout and loop_wait are reduced so that the rule is fulfilled:

loop_wait + 2 * retry_timeout <= ttl

Parameters: config – dict object with new global configuration.

_validate_failover_tags() → None¶: Check nofailover/failover_priority config and warn user if it’s contradictory.

Note

To preserve sanity (and backwards compatibility) the nofailover tag will still exist. A contradictory configuration is one where nofailover is True but failover_priority > 0, or where nofailover is False, but failover_priority <= 0. Essentially, nofailover and failover_priority are communicating different things. This checks for this edge case (which is a misconfiguration on the part of the user) and warns them. The behaviour is as if failover_priority were not provided (i.e nofailover is the bedrock source of truth)

property config_file: Optional[str]¶: Path to Patroni configuration file, if any, else None.

copy() → Dict[str, Any]¶

Get a deep copy of effective Patroni configuration.

Returns: a deep copy of the Patroni configuration.

property dynamic_configuration: Dict[str, Any]¶: Deep copy of cached Patroni dynamic configuration.

get(key: str, default: Optional[Any] = None) → Any¶

Get effective value of key setting from Patroni configuration root.

Designed to work the same way as dict.get().

Parameters

key – name of the setting.
default – default value if key is not present in the effective configuration.

Returns

value of key, if present in the effective configuration, otherwise default.

classmethod get_default_config() → Dict[str, Any]¶

Deep copy default configuration.

Returns: copy of __DEFAULT_CONFIG

get_global_config(cluster: Optional[patroni.dcs.Cluster]) → patroni.config.GlobalConfig¶

Instantiate GlobalConfig based on input.

Use the configuration from provided cluster (the most up-to-date) or from the local cache if cluster.config is not initialized or doesn’t have a valid config.

Parameters: cluster – the currently known cluster state from DCS.
Returns: GlobalConfig object.

property local_configuration: Dict[str, Any]¶

Deep copy of cached Patroni local configuration.

Returns: copy of _local_configuration

reload_local_configuration() → Optional[bool]¶

Reload configuration values from the configuration file(s).

Note

Designed to be used when user applies changes to configuration file(s), so Patroni can use the new values with a reload instead of a restart.

Returns: True if changes have been detected between current local configuration

save_cache() → None¶: Save dynamic configuration to patroni.dynamic.json under Postgres data directory.

Note

patroni.dynamic.jsonXXXXXX is created as a temporary file and than renamed to patroni.dynamic.json, where XXXXXX is a random suffix.

set_dynamic_configuration(configuration: Union[patroni.dcs.ClusterConfig, Dict[str, Any]]) → bool¶

Set dynamic configuration values with given configuration.

Parameters: configuration – new dynamic configuration values. Supports dict for backward compatibility.
Returns: True if changes have been detected between current dynamic configuration and the new dynamic configuration, False otherwise.

class patroni.config.GlobalConfig(config: Dict[str, Any])¶

Bases: object

A class that wraps global configuration and provides convenient methods to access/check values.

It is instantiated either by calling get_global_config() or Config.get_global_config(), which picks either a configuration from provided Cluster object (the most up-to-date) or from the local cache if ClusterConfig is not initialized or doesn’t have a valid config.

__init__(config: Dict[str, Any]) → None¶

Initialize GlobalConfig object with given config.

Parameters: config – current configuration either from ClusterConfig or from Config.dynamic_configuration().

check_mode(mode: str) → bool¶

Checks whether the certain parameter is enabled.

Parameters: mode – parameter name, e.g. synchronous_mode, failsafe_mode, pause, check_timeline, and so on.
Returns: True if parameter mode is enabled in the global configuration.

get(name: str) → Any¶

Gets global configuration value by name.

Parameters: name – parameter name.
Returns: configuration value or None if it is missing.

get_int(name: str, default: int = 0) → int¶

Gets current value of name from the global configuration and try to return it as int.

Parameters

name – name of the parameter.
default – default value if name is not in the configuration or invalid.

Returns

currently configured value of name from the global configuration or default if it is not set or invalid.

get_standby_cluster_config() → Union[Dict[str, Any], Any]¶

Get standby_cluster configuration.

Returns: a copy of standby_cluster configuration.

property is_paused: bool¶: True if cluster is in maintenance mode.

property is_standby_cluster: bool¶: True if global configuration has a valid standby_cluster section.

property is_synchronous_mode: bool¶: True if synchronous replication is requested and it is not a standby cluster config.

property is_synchronous_mode_strict: bool¶: True if at least one synchronous node is required.

property maximum_lag_on_failover: int¶

Currently configured value of maximum_lag_on_failover from the global configuration.

Assume 1048576 if it is not set or invalid.

property maximum_lag_on_syncnode: int¶

Currently configured value of maximum_lag_on_syncnode from the global configuration.

Assume -1 if it is not set or invalid.

property min_synchronous_nodes: int¶: The minimal number of synchronous nodes based on whether synchronous_mode_strict is enabled or not.

property primary_start_timeout: int¶

Currently configured value of primary_start_timeout from the global configuration.

Assume 300 if it is not set or invalid.

Note

master_start_timeout is still supported to keep backward compatibility.

property primary_stop_timeout: int¶

Currently configured value of primary_stop_timeout from the global configuration.

Assume 0 if it is not set or invalid.

Note

master_stop_timeout is still supported to keep backward compatibility.

property synchronous_node_count: int¶

Currently configured value of synchronous_node_count from the global configuration.

Assume 1 if it is not set or invalid.

patroni.config.default_validator(conf: Dict[str, Any]) → List[str]¶

Ensure conf is not empty.

Designed to be used as default validator for Config objects, if no specific validator is provided.

Parameters: conf – configuration to be validated.
Returns: an empty list – Config expects the validator to return a list of 0 or more issues found while validating the configuration.
Raises: ConfigParseError: if conf is empty.

patroni.config.get_global_config(cluster: Optional[patroni.dcs.Cluster], default: Optional[Dict[str, Any]] = None) → patroni.config.GlobalConfig¶

Instantiates GlobalConfig based on the input.

Parameters

cluster – the currently known cluster state from DCS.
default – default configuration, which will be used if there is no valid cluster.config.

Returns

GlobalConfig object.