Replica¶
Usage example¶
Basically, when you want a simple replica configuration without any hubs, you can use create_topology function. In more complex cases you have to use our Replica API to build your own topology exactly the way you want it. Still, it is better if you’ll use the ‘create_topology’ method for basic initial setup and then you can continue to expand it.
from lib389.topologies import create_topology topology = create_topology({ReplicaRole.MASTER: 2, ReplicaRole.CONSUMER: 2})
For basic Replica operations (the rest in the docs bellow):
from lib389.replica import Replicas replicas = Replicas(standalone) # Enable replication # - changelog will be created # - replica manager will be with the defaults # - replica.create() will be executed replica = replicas.enable(suffix=DEFAULT_SUFFIX, role=ReplicaRole.MASTER, replicaID=REPLICAID_MASTER_1) # Or you can get it as usual DSLdapObject replica = replicas.list()[0] # Roles - ReplicaRole.MASTER, ReplicaRole.HUB, and ReplicaRole.CONSUMER # For masters and hubs you can use the constants REPLICAID_MASTER_X and REPLICAID_HUB_X # Change X for a number from 1 to 100 - for role ReplicaRole.MASTER only # Disable replication # - agreements and replica entry will be deleted # - changelog is not deleted (but should?) replicas.disable(suffix=DEFAULT_SUFFIX) # Get RUV entry replicas.get_ruv_entry() # Get DN replicas.get_dn(suffix) # Promote replicas.promote(suffix=DEFAULT_SUFFIX, newrole=ReplicaRole.MASTER, binddn=REPL_BINDDN, rid=REPLICAID_MASTER_1) # Demote replicas.demote(suffix=DEFAULT_SUFFIX, newrole=ReplicaRole.CONSUMER) # Test, that replication works replicas.test(master2) # Additional replica object methods # Get role replica.get_role() replica.deleteAgreements()
Module documentation¶
-
class
lib389.replica.ReplicationManager(suffix, logger=None)[source]¶ The lib389 replication manager. This is used to coordinate replicas and agreements between servers.
Unlike the raw replicas / agreement types that manipulate the servers configuration, this is a “high level” coordination type. It’s capable of taking multiple instances and joining them. It consumes many lib389 types like Replicas, Agreements and more.
It is capable of creating the first master in a topoolgy, joining masters and consumers to that topology, populating per-server replication credentials, dynamic rid allocation, and more.
Unlike hand management of agreements, this is able to take simpler steps to agreement creation. For example:
repl = ReplicationManager(<suffix>) repl.create_first_master(master1) repl.join_master(master1, master2)
Contrast to previous implementations of replication which required much more knowledge and parameters, this is able to securely add masters.
- Parameters
suffix (str) – The suffix to replicate.
logger (python logging) – A logging interface
-
create_first_master(instance)[source]¶ In a topology, this creates the “first” master that has the database and content. A number of bootstrap tasks are performed on this master, as well as creating it’s replica type.
Once the first master is created, all other masters can be joined to it via “join_master”.
- Parameters
instance (lib389.DirSrv) – An instance
-
disable_to_master(to_instance, from_instances=[])[source]¶ For all masters “from” disable all agreements “to” instance.
- Parameters
to_instance (lib389.DirSrv) – The instance to stop recieving data.
from_instances (list[lib389.DirSrv]) – The instances to stop sending data.
-
enable_to_master(to_instance, from_instances=[])[source]¶ For all masters “from” enable all agreements “to” instance.
- Parameters
to_instance (lib389.DirSrv) – The instance to start recieving data.
from_instances (list[lib389.DirSrv]) – The instances to start sending data.
-
ensure_agreement(from_instance, to_instance, init=False)[source]¶ Guarantee that a replication agreement exists ‘from_instance’ send data ‘to_instance’. This can be for any instance, master, hub, or consumer.
Both instances must have been added to the topology with create first master, join_master, join_consumer or join_hub.
- Parameters
from_instance (lib389.DirSrv) – An instance already in the topology.
to_instance (lib389.DirSrv) – An instance to replicate to.
-
get_rid(instance)[source]¶ For a given master, retrieve it’s RID for this suffix.
- Parameters
instance (lib389.DirSrv) – The instance
- Returns
str
-
join_consumer(from_instance, to_instance)[source]¶ Join a new consumer to this instance. This will complete a total init of the data “from instance” to “to instance”.
This can be conducted from any master or hub in the topology as “from” master.
- Parameters
from_instance (lib389.DirSrv) – An instance already in the topology.
to_instance (lib389.DirSrv) – An instance to join to the topology.
-
join_hub(from_instance, to_instance)[source]¶ Join a new hub to this instance. This will complete a total init of the data “from instance” to “to instance”.
This can be conducted from any master or hub in the topology as “from” master.
Not implement yet.
- Parameters
from_instance (lib389.DirSrv) – An instance already in the topology.
to_instance (lib389.DirSrv) – An instance to join to the topology.
-
join_master(from_instance, to_instance)[source]¶ Join a new master in MMR to this instance. This will complete a total init of the data “from instance” to “to instance”.
This can be conducted from any master in the topology as “from” master.
- Parameters
from_instance (lib389.DirSrv) – An instance already in the topology.
to_instance (lib389.DirSrv) – An instance to join to the topology.
-
remove_master(instance, remaining_instances=[], purge_sa=True)[source]¶ Remove an instance from the replication topology.
If purge service accounts is true, remove the instances service account.
The purge_sa must be conducted on a remaining master to guarantee the result.
We recommend remaining instances contains all masters that have an agreement to instance, to ensure no dangling agreements exist. Masters with no agreement are skipped.
-
test_replication(from_instance, to_instance, timeout=20)[source]¶ Wait for a replication event to occur from instance to instance. This shows some point of synchronisation has occured.
- Parameters
from_instance (lib389.DirSrv) – The instance whos state we we want to check from
to_instance (lib389.DirSrv) – The instance whos state we want to check matches from.
timeout (int) – Fail after timeout seconds.
-
test_replication_topology(instances, timeout=20)[source]¶ Confirm replication works between all permutations of masters in the topology.
-
wait_for_replication(from_instance, to_instance, timeout=20)[source]¶ Wait for a replication event to occur from instance to instance. This shows some point of synchronisation has occured.
- Parameters
from_instance (lib389.DirSrv) – The instance whos state we we want to check from
to_instance (lib389.DirSrv) – The instance whos state we want to check matches from.
timeout (int) – Fail after timeout seconds.
-
wait_for_ruv(from_instance, to_instance, timeout=20)[source]¶ Wait for the in-memory ruv ‘from_instance’ to be advanced past on ‘to_instance’. Note this does not mean the ruvs are “exact matches” only that some set of CSN states has been advanced past. Topics like fractional replication may or may not interfer in this process.
In essence this is a rough check that to_instance is at least at the replication state of from_instance. You should consider using wait_for_replication instead for a guarantee.
- Parameters
from_instance (lib389.DirSrv) – The instance whos state we we want to check from
to_instance (lib389.DirSrv) – The instance whos state we want to check matches from.
-
class
lib389.replica.RUV(ruvs=[], logger=None)[source]¶ Represents the server in memory RUV object. The RUV contains each update vector the server knows of, along with knowledge of CSN state of the replica we have sent data to.
- Parameters
-
alloc_rid()[source]¶ Based on the RUV, determine an available RID for the replication topology that is unique.
- Returns
str
-
is_synced(other_ruv)[source]¶ Compare two server ruvs to determine if they are synced. This does not mean that replication is in sync (due to things like fractional repl), but in some cases can show that “at least some known point” has been achieved in the replication process.
- Parameters
other_ruv (RUV object) – The other ruv object
- Returns
bool
-
class
lib389.replica.Replicas(instance)[source]¶ Replica DSLdapObjects for all replicas
- Parameters
instance (lib389.DirSrv) – A instance
-
ensure_state(rdn=None, properties=None)[source]¶ Create an object under base DN of our entry, or assert it exists and update it’s properties.
-
get(selector=[], dn=None)[source]¶ Get a child entry (DSLdapObject, Replica, etc.) with dn or selector using a base DN and objectClasses of our object (DSLdapObjects, Replicas, etc.) After getting the replica, update the replica._suffix parameter.
- Parameters
dn (str) – DN of wanted entry
selector – An additional filter to objectClasses, i.e. ‘backend_name’
- Returns
A child entry
-
lint(spec: Union[str, None, Type[List]] = None) → Generator[Any, None, None][source]¶ Lint the objects returned by list method according to the spec.
-
lint_list(spec: Optional[str] = None) → Generator[Tuple[str, Callable], None, None][source]¶ Yield specs the objects returned by list method provide.
-
list(paged_search=None, paged_critical=True)[source]¶ Get a list of children entries (DSLdapObject, Replica, etc.) using a base DN and objectClasses of our object (DSLdapObjects, Replicas, etc.)
- Parameters
paged_search – None for no paged search, or an int of page size to use.
- Returns
A list of children entries
-
process_and_dump_changelog(replica_root, output_file, csn_only=False, preserve_ldif_done=False, decode=False)[source]¶ Dump and decode Directory Server replication changelog
- Parameters
replica_root (str) – Replica suffix that needs to be processed
output_file – The file name for the exported changelog LDIF file
csn_only (bool) – Grep only the CSNs from the file
preserve_ldif_done (bool) – Preserve the result LDIF and rename it to [old_name].done
decode – Decode any base64 values from the changelog
-
class
lib389.replica.Replica(instance, dn=None)[source]¶ Replica DSLdapObject with: - must attributes = [‘cn’, ‘nsDS5ReplicaType’, ‘nsDS5ReplicaRoot’,
‘nsDS5ReplicaBindDN’, ‘nsDS5ReplicaId’]
RDN attribute is ‘cn’
There is one “replica” per backend
- Parameters
instance (lib389.DirSrv) – A instance
dn (str) – Entry DN
-
apply_mods(mods)[source]¶ Perform modification operation using several mods at once
- Parameters
mods (list of tuples) – [(action, key, value),] or [(ldap.MOD_DELETE, key),]
- Raises
ValueError - if a provided mod op is invalid
-
cleanRUV(rid)[source]¶ Run a cleanallruv task, only on a master, after deleting or demoting it. It is okay if it fails.
-
classmethod
compare(obj1, obj2)[source]¶ Compare if two RDN objects have same attributes and values.
This comparison is a loose comparison, not a strict one i.e. “this object is this other object” It will just check if the attributes are same. ‘nsUniqueId’ attribute is not checked intentionally because we want to compare arbitrary objects i.e they may have different ‘nsUniqueId’ but same attributes.
Example:
cn=user1,ou=a cn=user1,ou=b
Comparision of these two objects should result in same, even though their ‘nsUniqueId’ attribute differs.
- Parameters
obj1 (lib389._mapped_object.DSLdapObject) – An entry to check
obj2 (lib389._mapped_object.DSLdapObject) – An entry to check
- Returns
True if objects have same attributes else returns False
- Raises
ValueError - if obj1 or obj2 don’t inherit DSLdapObject
-
delete()[source]¶ Delete a replica related to the provided suffix.
If this replica role was ReplicaRole.HUB or ReplicaRole.MASTER, it also deletes the changelog associated to that replica. If it exists some replication agreement below that replica, they are deleted. If this is a master we also clean the database ruv.
- Returns
None
- Raises
InvalidArgumentError - if suffix is missing
ldap.LDAPError - for all other update failures
-
demote(newrole)[source]¶ Demote a replica to a hub or consumer
- Parameters
newrole (ReplicaRole) – The new replication role for the replica: CONSUMER and HUB
- Returns
None
- Raises
ValueError - If replica is not demoted
-
display(attrlist=['*'])[source]¶ Get an entry but represent it as a string LDIF
- Returns
LDIF formatted string
-
display_attr(attr)[source]¶ Get all values of given attribute - ‘attr: value’
- Returns
Formatted string
-
property
dn¶ Get an object DN
- Returns
DN
-
ensure_attr_state(state)[source]¶ Given a dict of attr-values, ensure they are in the same state on the entry. This is a stateful assertion, generally used by things like PATCH in a REST api.
- The format is:
- {
‘attr_1’: [‘value’, ‘value’], ‘attr_2’: [],
}
If a value is present in the list, but not in the entry it is ADDED. If a value is NOT present in the list, and is on the entry, it is REMOVED. If a value is an empty list [], the attr is REMOVED from the entry. If an attr is not named in the dictionary, it is not altered.
This function is atomic - all changes are applied or none are. There are no partial updates.
This function is idempotent - submitting the same request twice will cause no action to be taken as we are ensuring a state, not listing actions to take.
- Parameters
state (dict) – The entry ava state
-
ensure_present(attr, value)[source]¶ Ensure that an attribute and value are present in a state, or add it.
-
ensure_removed(attr, value)[source]¶ Ensure that a attribute and value has been removed and not present or remove it.
-
ensure_state(rdn=None, properties=None, basedn=None)[source]¶ Ensure an entry exists with the following state, created if necessary.
-
get_agreements(winsync=False)[source]¶ Return the set of agreements related to this suffix replica :param: winsync: If True then return winsync replication agreements,
otherwise return teh standard replication agreements.
- Returns
A list Replicas objects
-
get_all_attrs(use_json=False)[source]¶ Get a dictionary having all the attributes of the entry
- Returns
Dict with real attributes and operational attributes
-
get_all_attrs_utf8(use_json=False)[source]¶ Get a dictionary having all the attributes of the entry
- Returns
Dict with real attributes and operational attributes
-
get_attr_val_bytes(key, use_json=False)[source]¶ Get a single attribute value from the entry in bytes type
- Parameters
key (str) – An attribute name
- Returns
A single bytes value
- Raises
ValueError - if instance is offline
-
get_attr_val_int(key, use_json=False)[source]¶ Get a single attribute value from the entry in int type
- Parameters
key (str) – An attribute name
- Returns
A single bytes value
- Raises
ValueError - if instance is offline
-
get_attr_val_utf8(key, use_json=False)[source]¶ Get a single attribute value from the entry in utf8 type
- Parameters
key (str) – An attribute name
- Returns
A single bytes value
- Raises
ValueError - if instance is offline
-
get_attr_val_utf8_l(key, use_json=False)[source]¶ Get a single attribute value from the entry in utf8 type
- Parameters
key (str) – An attribute name
- Returns
A single bytes value
- Raises
ValueError - if instance is offline
-
get_attr_vals_bytes(key, use_json=False)[source]¶ Get attribute values from the entry in bytes type
- Parameters
key (str) – An attribute name
- Returns
A single bytes value
- Raises
ValueError - if instance is offline
-
get_attr_vals_int(key, use_json=False)[source]¶ Get attribute values from the entry in int type
- Parameters
key (str) – An attribute name
- Returns
A single bytes value
- Raises
ValueError - if instance is offline
-
get_attr_vals_utf8(key, use_json=False)[source]¶ Get attribute values from the entry in utf8 type
- Parameters
key (str) – An attribute name
- Returns
A single bytes value
- Raises
ValueError - if instance is offline
-
get_attr_vals_utf8_l(key, use_json=False)[source]¶ Get attribute values from the entry in utf8 type and lowercase
- Parameters
key (str) – An attribute name
- Returns
A single bytes value
- Raises
ValueError - if instance is offline
-
get_compare_attrs(use_json=False)[source]¶ Get a dictionary having attributes to be compared i.e. excluding self._compare_exclude
-
get_consumer_replicas(get_credentials)[source]¶ Return the set of consumer replicas related to this suffix replica through its agreements
- Parameters
get_credentials – A user-defined callback function which returns the binding credentials using given host and port data - {“binddn”: “cn=Directory Manager”, “bindpw”: “password”}
- Returns
Replicas object
-
get_role()[source]¶ Return the replica role
- Returns
ReplicaRole.MASTER, ReplicaRole.HUB, ReplicaRole.CONSUMER
-
get_ruv()[source]¶ Return the in memory ruv of this replica suffix.
- Returns
RUV object
- Raises
LDAPError
-
get_ruv_agmt_maxcsns()[source]¶ Return the in memory ruv of this replica suffix.
- Returns
RUV object
- Raises
LDAPError
-
lint(spec: Union[str, None, Type[List]] = None) → Generator[Any, None, None][source]¶ Lint the object according to the spec.
-
lint_list(spec: Optional[str] = None) → Generator[Tuple[str, Callable], None, None][source]¶ Yield specs the object provides.
This yields from each lint method yielding all specs it can provide.
-
promote(newrole, binddn=None, binddn_group=None, rid=None)[source]¶ Promote the replica to hub or master
- Parameters
- Returns
None
- Raises
ValueError - If replica is not promoted
-
property
rdn¶ Get an object RDN
- Returns
RDN
-
remove_all(key)[source]¶ Remove all values defined by key (if possible).
If an attribute is multi-valued AND required all values except one will be deleted.
- Parameters
key (str) – an attribute name
-
rename(new_rdn, newsuperior=None, deloldrdn=True)[source]¶ Renames the object within the tree.
If you provide a newsuperior, this will move the object in the tree. If you only provide a new_rdn, it stays in the same branch, but just changes the rdn.
Note, if you use newsuperior, you may move this object outside of the scope of the related DSLdapObjects manager, which may cause it not to appear in .get() requests.
-
replace_many(*args)[source]¶ Replace many key, value pairs in a single operation. This is useful for configuration changes that require atomic operation, and ease of use.
An example of usage is replace_many((key, value), (key, value))
No wrapping list is needed for the arguments.
- Parameters
*args –
tuples of key,value to replace.
-
status(binddn=None, bindpw=None, winsync=False)[source]¶ Get a list of the status for every agreement
-
test_replication(replica_dirsrvs)[source]¶ Make a “dummy” update on the the replicated suffix, and check all the provided replicas to see if they received the update.
- Parameters
*replica_dirsrvs –
DirSrv instance, DirSrv instance, …
- Returns
True - if all servers have recevioed the update by this replica, otherwise return False
- Raises
LDAPError - when failing to update/search database