Class FileChangelogDB
- java.lang.Object
-
- org.opends.server.replication.server.changelog.file.FileChangelogDB
-
- All Implemented Interfaces:
ChangelogDB
,ReplicationDomainDB
public class FileChangelogDB extends Object implements ChangelogDB, ReplicationDomainDB
Log file implementation of the ChangelogDB interface.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface org.opends.server.replication.server.changelog.api.ReplicationDomainDB
ReplicationDomainDB.ReplicaDbDescription, ReplicationDomainDB.ReplicaDbFileContent
-
-
Constructor Summary
Constructors Constructor Description FileChangelogDB(ReplicationServer replicationServer, String dbDirectoryPath, LogCryptoCfg cryptoCfg)
Creates a new changelog DB.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description void
clearDB()
Clears all records from the changelog (does not remove the changelog itself).ChangeNumberIndexDB
getChangeNumberIndexDB()
Returns theChangeNumberIndexDB
object.DBCursor<UpdateMsg>
getCursorFrom(Dn baseDN, ServerState startState, DBCursor.CursorOptions options)
Generates aDBCursor
across all the replicaDBs for the specified replication domain starting before, at or after the providedServerState
for each replicaDB, depending on the provided matching and positioning strategies.MultiDomainDBCursor
getCursorFrom(MultiDomainServerState startState, DBCursor.CursorOptions options)
Generates aDBCursor
across all the domains starting before, at or after the providedMultiDomainServerState
for each domain, depending on the provided matching and positioning strategies.MultiDomainDBCursor
getCursorFrom(MultiDomainServerState startState, DBCursor.CursorOptions options, Set<Dn> excludedDomainDns)
Generates aDBCursor
across all the domains starting before, at or after the providedMultiDomainServerState
for each domain, excluding a provided set of domain DNs.DBCursor<UpdateMsg>
getCursorFrom(DomainReplicaId replica, CSN startCsn, DBCursor.CursorOptions options)
ServerState
getDomainNewestCSNs(Dn baseDN)
Returns the newestCSN
s from the replicaDBs for each replica id in the specified replication domain.ServerState
getDomainOldestCSNs(Dn baseDN)
Returns the oldestCSN
s from the replicaDBs for each replica id in the specified replication domain.UpdateMsg
getNewestMessage(DomainReplicaId replica)
Returns the newest message from the replica DB for the provided replica.Flowable<UpdateRecord>
getPublisherForDomain(String name, Dn baseDn, ServerState startState, ServerState endState)
Returns a publisher of changes from the provided domain, using the provided states to determine the starting and ending points of each replica of the domain.Flowable<UpdateRecord>
getPublisherFromCookie(String name, MultiDomainServerState cookie, Set<Dn> excludedDomains)
Returns a publisher of all changes starting from the provided cookie, excluding the changes from the provided set of excluded domains.RemovedReplicasStates
getRemovedReplicasCsns(Dn baseDn)
Returns the latest known CSNs for removed replicas.ReplicationDomainDB.ReplicaDbDescription
getReplicaDbDescription(DomainReplicaId replica)
Returns a textual description of the contents of the changelog of the provided replica and domain.ReplicationDomainDB
getReplicationDomainDB()
Returns theReplicationDomainDB
object.boolean
hasReplicaDb(DomainReplicaId replica)
Returnstrue
if a changelog already exists for the replica in the domain.void
initializeDB()
Initializes the replication database by reading its previous state and building the relevant ReplicaDBs according to the previous state.static void
migrateEncryptedHeadLogFile(Path headLogFilePath, LogCryptoCfg cryptoCfg)
Used by upgrade only: Rewrite the head log file to take confidentiality settings into account.static void
migrateOfflineStateToChangelog(Path replicationDbDir, Path replicaIdDir, LogCryptoCfg cryptoCfg, CSN offlineCsn)
Used by upgrade only: Append the CSN at which the server/domains was considered offline if this CSN appears to be more recent than the last message already contained in the log, otherwise the changelog is not modified.boolean
publishUpdateMsg(Dn baseDN, UpdateMsg updateMsg)
Publishes the provided change to the changelog DB for the specified replica id and replication domain.void
removeDB()
Removes the changelog database directory.void
removeDomain(Dn baseDN)
Removes all the data relating to the specified replication domain and shutdown all its replica databases.static void
renameHeadLogFileToUseOldestKey(Path headLogFilePath, LogCryptoCfg cryptoCfg)
Renames thehead.log
files to a name using the oldest key in the file.void
replicaHeartbeat(Dn baseDN, CSN heartbeatCsn)
Let the DB know this replica is alive.void
setComputeChangeNumber(boolean computeChangeNumber)
Sets whether the replication database must compute change numbers for replicated changes.void
setPurgeDelay(Duration purgeDelay)
Sets the purge delay for the replication database.void
shutdownDB()
Shutdown the replication database.void
unregisterCursor(DBCursor<?> cursor)
Unregisters the provided cursor from this replication domain.
-
-
-
Constructor Detail
-
FileChangelogDB
public FileChangelogDB(ReplicationServer replicationServer, String dbDirectoryPath, LogCryptoCfg cryptoCfg) throws ConfigException
Creates a new changelog DB.- Parameters:
replicationServer
- the local replication server.dbDirectoryPath
- the path where the changelog files reside.cryptoCfg
- the cryptoCfg to use for encryption- Throws:
ConfigException
- if a problem occurs opening the supplied directory
-
-
Method Detail
-
initializeDB
public void initializeDB()
Description copied from interface:ChangelogDB
Initializes the replication database by reading its previous state and building the relevant ReplicaDBs according to the previous state. This method must be called once before using the ChangelogDB.- Specified by:
initializeDB
in interfaceChangelogDB
-
shutdownDB
public void shutdownDB()
Description copied from interface:ChangelogDB
Shutdown the replication database.- Specified by:
shutdownDB
in interfaceChangelogDB
-
clearDB
public void clearDB() throws ChangelogException
Clears all records from the changelog (does not remove the changelog itself).- Throws:
ChangelogException
- If an error occurs when clearing the changelog.
-
removeDB
public void removeDB()
Description copied from interface:ChangelogDB
Removes the changelog database directory.- Specified by:
removeDB
in interfaceChangelogDB
-
getDomainOldestCSNs
public ServerState getDomainOldestCSNs(Dn baseDN)
Description copied from interface:ReplicationDomainDB
Returns the oldestCSN
s from the replicaDBs for each replica id in the specified replication domain.- Specified by:
getDomainOldestCSNs
in interfaceReplicationDomainDB
- Parameters:
baseDN
- the replication domain baseDN- Returns:
- a new ServerState object holding the {replicaId to oldest CSN} mapping. If a replica DB is empty or closed, the oldest CSN will be null for that replica. The caller owns the generated ServerState.
-
getDomainNewestCSNs
public ServerState getDomainNewestCSNs(Dn baseDN)
Description copied from interface:ReplicationDomainDB
Returns the newestCSN
s from the replicaDBs for each replica id in the specified replication domain.- Specified by:
getDomainNewestCSNs
in interfaceReplicationDomainDB
- Parameters:
baseDN
- the replication domain baseDN- Returns:
- a new ServerState object holding the {replicaId to newest CSN} Map. If a replica DB is empty or closed, the newest CSN will be null for that replica. The caller owns the generated ServerState.
-
getNewestMessage
public UpdateMsg getNewestMessage(DomainReplicaId replica)
Description copied from interface:ReplicationDomainDB
Returns the newest message from the replica DB for the provided replica.- Specified by:
getNewestMessage
in interfaceReplicationDomainDB
- Parameters:
replica
- the base dn and replica id of the replica- Returns:
- the newest message from the replica DB
-
removeDomain
public void removeDomain(Dn baseDN) throws ChangelogException
Description copied from interface:ReplicationDomainDB
Removes all the data relating to the specified replication domain and shutdown all its replica databases. In particular, it will:- remove all the changes from the replica databases
- remove all knowledge of the replicaIds in this domain
- remove any knowledge of the current generationId for this domain
- Specified by:
removeDomain
in interfaceReplicationDomainDB
- Parameters:
baseDN
- the replication domain baseDN- Throws:
ChangelogException
- If a database problem happened
-
setPurgeDelay
public void setPurgeDelay(Duration purgeDelay)
Description copied from interface:ChangelogDB
Sets the purge delay for the replication database. Can be called while the database is running.Purging happens on a best effort basis, i.e. the purge delay is used by the replication database to know which data can be purged, but there are no guarantees on when the purging will actually happen.
- Specified by:
setPurgeDelay
in interfaceChangelogDB
- Parameters:
purgeDelay
- the purge delay.Duration.ZERO
disables purging.
-
setComputeChangeNumber
public void setComputeChangeNumber(boolean computeChangeNumber)
Description copied from interface:ChangelogDB
Sets whether the replication database must compute change numbers for replicated changes. Change numbers are computed using a separate new thread.- Specified by:
setComputeChangeNumber
in interfaceChangelogDB
- Parameters:
computeChangeNumber
- whether to compute change numbers for replicated changes
-
getChangeNumberIndexDB
public ChangeNumberIndexDB getChangeNumberIndexDB()
Description copied from interface:ChangelogDB
Returns theChangeNumberIndexDB
object.- Specified by:
getChangeNumberIndexDB
in interfaceChangelogDB
- Returns:
- the
ChangeNumberIndexDB
object
-
getReplicationDomainDB
public ReplicationDomainDB getReplicationDomainDB()
Description copied from interface:ChangelogDB
Returns theReplicationDomainDB
object.- Specified by:
getReplicationDomainDB
in interfaceChangelogDB
- Returns:
- the
ReplicationDomainDB
object
-
getPublisherFromCookie
public Flowable<UpdateRecord> getPublisherFromCookie(String name, MultiDomainServerState cookie, Set<Dn> excludedDomains)
Description copied from interface:ReplicationDomainDB
Returns a publisher of all changes starting from the provided cookie, excluding the changes from the provided set of excluded domains.More precisely, it publishes all records with a CSN which is greater than the corresponding CSN in the cookie or are from a domain or replica that is not present in the cookie, except those that are present in the set of excluded domains.
The subscription to this publisher must be cancelled when it is not used anymore.
The publisher is automatically updated if some base DN or replica is either added or removed to/from the changelog database, except those that are present in the set of excluded domains.
Reading will start only when the returned
Flowable
has been subscribed to.- Specified by:
getPublisherFromCookie
in interfaceReplicationDomainDB
- Parameters:
name
- The name of this publishercookie
- Provides the starting point for each replica. If a replica is not present in the cookie, then the starting point is the oldest record of the replica.excludedDomains
- The domains which are excluded from the publisher. All replicas belonging to these domains are excluded even if they are present in the cookie.- Returns:
- a publisher of all changes starting from the provided cookie, excluding the changes from the provided set of excluded domains.
-
getPublisherForDomain
public Flowable<UpdateRecord> getPublisherForDomain(String name, Dn baseDn, ServerState startState, ServerState endState)
Description copied from interface:ReplicationDomainDB
Returns a publisher of changes from the provided domain, using the provided states to determine the starting and ending points of each replica of the domain.More precisely, it publishes all records with a CSN which is greater than the corresponding CSN in the
startState
or are from a replica that is not present in thestartState
, and with a CSN that is lower or equal than the corresponding CSN in theendState
.Thus, provided a couple (startCsn, endCns), there are four possibilities for a given replica of the domain:
(null, null)
: publishes ]-infinite, +infinite[(startCsn, null)
: publishes ]startCsn, +infinite[(startCsn, endCsn) where startCsn <= endCsn
: publishes ]startCsn, endCsn]. Automatically removes the replica from the publisher once the endState is reached.(startCsn, endCsn) where startCsn > endCsn
: : publishes no record In other words, excludes this replica from the publisher.
The publisher returns an error immediately after subscription if there is a change with a CSN greater than the start CSN in a replica changelog and no change with the start CSN (which indicates that the corresponding change has been purged). If the most recent change in a replica changelog is older than the start CSN, then it is not an error and the changes will be eventually published just after the start CSN is reached.
The publisher must be cancelled at the end of use.
Reading will start only when the returned
Flowable
has been subscribed to.- Specified by:
getPublisherForDomain
in interfaceReplicationDomainDB
- Parameters:
name
- The name of this publisherbaseDn
- The base DN of the domain to be published.startState
- Provides the starting point for each replica. If a replica is not present in this state, then the starting point is the oldest record of the replica.endState
- Provides the ending point for each replica. If a replica is not present in this state, then it has no ending point and its records will be published until the publisher is stopped or cancelled.- Returns:
- a publisher of changes from the provided domain, using the provided states to determine the starting and ending points of each replica of the domain.
-
getCursorFrom
public MultiDomainDBCursor getCursorFrom(MultiDomainServerState startState, DBCursor.CursorOptions options)
Description copied from interface:ReplicationDomainDB
Generates aDBCursor
across all the domains starting before, at or after the providedMultiDomainServerState
for each domain, depending on the provided matching and positioning strategies.When the cursor is not used anymore, client code MUST call the
DBCursor.close()
method to free the resources and locks used by the cursor.If the positioning strategy defined in
options
isGREATER_THAN_KEY
orGREATER_THAN_OR_EQUAL_TO_KEY
, positioning may happen at a change from a replica added after the matching point, otherwise it will position at a change from one of the known replicas.- Specified by:
getCursorFrom
in interfaceReplicationDomainDB
- Parameters:
startState
- Starting point for each domain cursor. If anyServerState
for a domain is null, then start from the oldest CSN for each replicaDBsoptions
- The cursor options- Returns:
- a non null
DBCursor
- See Also:
ReplicationDomainDB.getCursorFrom(Dn, ServerState, CursorOptions)
-
getCursorFrom
public MultiDomainDBCursor getCursorFrom(MultiDomainServerState startState, DBCursor.CursorOptions options, Set<Dn> excludedDomainDns)
Description copied from interface:ReplicationDomainDB
Generates aDBCursor
across all the domains starting before, at or after the providedMultiDomainServerState
for each domain, excluding a provided set of domain DNs.When the cursor is not used anymore, client code MUST call the
DBCursor.close()
method to free the resources and locks used by the cursor.If the positioning strategy defined in
options
isGREATER_THAN_KEY
orGREATER_THAN_OR_EQUAL_TO_KEY
, positioning may happen at a change from a replica added after the matching point, otherwise it will position at a change from one of the known replicas.- Specified by:
getCursorFrom
in interfaceReplicationDomainDB
- Parameters:
startState
- Starting point for each domain cursor. If anyServerState
for a domain is null, then start from the oldest CSN for each replicaDBsoptions
- The cursor optionsexcludedDomainDns
- Every domain appearing in this set is excluded from the cursor- Returns:
- a non null
DBCursor
- See Also:
ReplicationDomainDB.getCursorFrom(Dn, ServerState, CursorOptions)
-
getCursorFrom
public DBCursor<UpdateMsg> getCursorFrom(Dn baseDN, ServerState startState, DBCursor.CursorOptions options)
Description copied from interface:ReplicationDomainDB
Generates aDBCursor
across all the replicaDBs for the specified replication domain starting before, at or after the providedServerState
for each replicaDB, depending on the provided matching and positioning strategies.When the cursor is not used anymore, client code MUST call the
DBCursor.close()
method to free the resources and locks used by the cursor.If the positioning strategy defined in
options
isGREATER_THAN_KEY
orGREATER_THAN_OR_EQUAL_TO_KEY
, positioning may happen at a change from a replica added after the matching point, otherwise it will position at a change from one of the known replicas.- Specified by:
getCursorFrom
in interfaceReplicationDomainDB
- Parameters:
baseDN
- the replication domain baseDNstartState
- Starting point for each ReplicaDB cursor. If any CSN for a replicaDB is null, then start from the oldest CSN for this replicaDBoptions
- The cursor options- Returns:
- a non null
DBCursor
- See Also:
ReplicationDomainDB.getCursorFrom(DomainReplicaId, CSN, CursorOptions)
-
getCursorFrom
public DBCursor<UpdateMsg> getCursorFrom(DomainReplicaId replica, CSN startCsn, DBCursor.CursorOptions options)
Description copied from interface:ReplicationDomainDB
Generates aDBCursor
for one replicaDB for the specified replication domain and replica id starting before, at or after the providedCSN
, depending on the provided matching and positioning strategies.When the cursor is not used anymore, client code MUST call the
DBCursor.close()
method to free the resources and locks used by the cursor.- Specified by:
getCursorFrom
in interfaceReplicationDomainDB
- Parameters:
replica
- the domain replicaId of the replicaDBstartCsn
- Starting point for the ReplicaDB cursor. If the CSN is null, then start from the oldest CSN for this replicaDBoptions
- The cursor options- Returns:
- a non null
DBCursor
-
unregisterCursor
public void unregisterCursor(DBCursor<?> cursor)
Description copied from interface:ReplicationDomainDB
Unregisters the provided cursor from this replication domain.- Specified by:
unregisterCursor
in interfaceReplicationDomainDB
- Parameters:
cursor
- the cursor to unregister.
-
publishUpdateMsg
public boolean publishUpdateMsg(Dn baseDN, UpdateMsg updateMsg) throws ChangelogException
Description copied from interface:ReplicationDomainDB
Publishes the provided change to the changelog DB for the specified replica id and replication domain. After a change has been successfully published, it becomes available to be returned by the External ChangeLog.- Specified by:
publishUpdateMsg
in interfaceReplicationDomainDB
- Parameters:
baseDN
- the replication domain baseDNupdateMsg
- the update message to publish to the replicaDB- Returns:
- true if a db had to be created to publish this message
- Throws:
ChangelogException
- If a database problem happened
-
replicaHeartbeat
public void replicaHeartbeat(Dn baseDN, CSN heartbeatCsn)
Description copied from interface:ReplicationDomainDB
Let the DB know this replica is alive.This method allows the medium consistency point to move forward in case this replica did not publish new changes.
- Specified by:
replicaHeartbeat
in interfaceReplicationDomainDB
- Parameters:
baseDN
- the replication domain baseDNheartbeatCsn
- The CSN heartbeat sent by this replica (contains the replica id and timestamp of the heartbeat)
-
getRemovedReplicasCsns
public RemovedReplicasStates getRemovedReplicasCsns(Dn baseDn)
Description copied from interface:ReplicationDomainDB
Returns the latest known CSNs for removed replicas.A removed replica needs to remember both the CSN of the last sent message, which can be a
ReplicaOffline
or anLDAPUpdateMsg
depending on the version it was running, and the lastLDAPUpdateMsg
message to support old servers talking to it.- Specified by:
getRemovedReplicasCsns
in interfaceReplicationDomainDB
- Parameters:
baseDn
- the baseDn of the replicated domain- Returns:
- the latest known CSN for removed replicas
-
renameHeadLogFileToUseOldestKey
public static void renameHeadLogFileToUseOldestKey(Path headLogFilePath, LogCryptoCfg cryptoCfg) throws IOException, ChangelogException
Renames thehead.log
files to a name using the oldest key in the file. If the file is empty, it will be removed.Note: This is used by upgrade only. It was decided to put this method here to avoid changing the visibility of a lot of APIs to public.
- Parameters:
headLogFilePath
- path to the head log filecryptoCfg
- the crypto configuration to be used to read the changelog files- Throws:
ChangelogException
- if any problem occurs with the changelogIOException
- if any I/O problem occurs
-
migrateOfflineStateToChangelog
public static void migrateOfflineStateToChangelog(Path replicationDbDir, Path replicaIdDir, LogCryptoCfg cryptoCfg, CSN offlineCsn) throws ChangelogException
Used by upgrade only: Append the CSN at which the server/domains was considered offline if this CSN appears to be more recent than the last message already contained in the log, otherwise the changelog is not modified.- Parameters:
replicationDbDir
- Root path of replication files.replicaIdDir
- Path to the folder were the change log for a specific server/domain is stored.cryptoCfg
- the crypto configuration to use for optionally encrypting/decrypting the content.offlineCsn
- CSN at which the server has been considered offline.- Throws:
ChangelogException
- If any problem occurs.
-
migrateEncryptedHeadLogFile
public static void migrateEncryptedHeadLogFile(Path headLogFilePath, LogCryptoCfg cryptoCfg) throws ChangelogException, IOException
Used by upgrade only: Rewrite the head log file to take confidentiality settings into account.Previous versions relied on secret keys being global to the server. To keep confidentiality working, the log file should now contain the secret key in the header, so we create a new one and copy all records re-encrypting them with the new secret key.
- Parameters:
headLogFilePath
- path to the head log filecryptoCfg
- the crypto configuration to be used to read the changelog files- Throws:
ChangelogException
- if a problem occurs while modifying the changelog fileIOException
- if a problem occurs when renaming the changelog file
-
getReplicaDbDescription
public ReplicationDomainDB.ReplicaDbDescription getReplicaDbDescription(DomainReplicaId replica) throws ChangelogException
Description copied from interface:ReplicationDomainDB
Returns a textual description of the contents of the changelog of the provided replica and domain.- Specified by:
getReplicaDbDescription
in interfaceReplicationDomainDB
- Parameters:
replica
- theDomainReplicaId
to describe- Returns:
- a textual description of the contents of the changelog of the provided replica and domain.
- Throws:
ChangelogException
- if an error occurs while accessing the changelog
-
hasReplicaDb
public boolean hasReplicaDb(DomainReplicaId replica)
Description copied from interface:ReplicationDomainDB
Returnstrue
if a changelog already exists for the replica in the domain.- Specified by:
hasReplicaDb
in interfaceReplicationDomainDB
- Parameters:
replica
- theDomainReplicaId
to describe- Returns:
true
if a changelog already exists for the replica in the domain
-
-