Oracle Cloud Infrastructure Documentation

Troubleshooting Cross-Region Replication

Use troubleshooting information to identify and address common issues with cross-region replication.

Primary Cluster Isn't Available for Selection

When creating or converting a secondary cluster, the primary cluster you want to use doesn't appear for selection.

The cluster isn't eligible

The cluster might not meet one or more cross-region replication requirements.

Verify the cluster requirements

Verify that the cluster meets the following requirements:

  1. The cluster is in another OCI region.
  2. The cluster has the standalone role.
  3. The cluster is in the Active state.
  4. The cluster is non-sharded.
  5. The cluster isn't already part of another cross-region replication relationship.
  6. The cluster doesn't have another operation in progress.
  7. When converting an existing cluster, the cluster being converted has the same engine version, cluster mode, and memory per node as the cluster that you want to select as the primary cluster.

Creating or Converting a Secondary Cluster Fails

The work request for creating or converting a secondary cluster fails.

The clusters don't meet the requirements

The primary or secondary cluster might have an unsupported role, incompatible settings, or an incompatible lifecycle state.

Review the work request and cluster settings

Review the failed operation and verify the cluster requirements:

  1. Open the failed work request and review the error message.
  2. Confirm that the primary cluster has the standalone role and doesn't already have an associated secondary cluster.
  3. When converting a cluster, confirm that it has the standalone role.
  4. Confirm that the primary and secondary clusters are in different OCI regions.
  5. Confirm that both clusters are non-sharded.
  6. When converting a cluster, confirm that its engine version, cluster mode, and memory per node match the primary cluster.
  7. Confirm that both clusters are in the Active state.
  8. Verify that both clusters use custom OCI Cache configuration sets that meet the cross-region replication requirements, including matching reserved-memory-percentage and maxmemory-policy values. If databases is configured to a value other than 16, verify that it also matches. The reserved-memory-percentage must be at least 35, and maxmemory-policy must not be set to noeviction. When you convert an existing cluster to a secondary cluster, ensure that you apply the required custom configuration set before you start the conversion.

Another operation is in progress

The primary or secondary cluster might be busy with another operation.

Wait for the current operation to finish

Wait until both clusters return to the Active state, if another operation is in progress, then retry the operation.

Cross-Region Replication Lag Is High

The CrossRegionReplicationLag metric is higher than expected or continues to increase.

Network latency or congestion has increased

Cross-region replication is asynchronous. Geographic distance, inter-region latency, and transient network congestion can increase replication lag.

Review the cross-region replication metrics

Use the Monitoring service to review the following metrics:

  1. Review CrossRegionReplicationLag and CrossRegionMasterLinkStatus.
  2. Compare metric values with the cluster's typical workload pattern. See Available Metrics.

Memory utilization or write activity is high

High memory utilization or a sustained increase in write activity can prevent the secondary cluster from keeping up with the primary cluster.

Verify that both clusters use custom configuration sets that meet the cross-region replication requirements, including matching values for the reserved-memory-percentage and maxmemory-policy parameters. If you set databases to a value other than 16, verify that both clusters use the same value. The reserved-memory-percentage must be at least 35, and maxmemory-policy must not be set to noeviction.

Review cluster resource usage

Review resource usage for both clusters:

  1. Check the MemoryUtilization metric for the primary and secondary clusters.
  2. Review NetworkBytesOut for the primary cluster and NetworkBytesIn for the secondary cluster.
  3. Reduce unnecessary write activity if possible.

Switchover Fails

The work request for a switchover fails.

The primary cluster is unavailable

A switchover requires both clusters to be available.

Use the disaster recovery procedure

If the primary cluster is unavailable, convert the secondary cluster to standalone instead of performing a switchover. See Performing a Switchover.

A cluster is busy or unhealthy

One of the clusters might have another operation in progress or might not be in a state that supports switchover.

Review both clusters

Verify the state of the cross-region replication relationship before retrying:

  1. Confirm that the switchover was initiated from the secondary cluster.
  2. Confirm that both clusters are in the Active state.
  3. Confirm that no conflicting work request is in progress.
  4. Review CrossRegionMasterLinkStatus and CrossRegionReplicationLag.
  5. Stop write traffic to the current primary cluster before retrying the switchover.

Primary Cluster Can't Be Deleted

You can't delete a primary cluster that has an associated secondary cluster.

The cross-region replication relationship still exists

A primary cluster can't be deleted while a secondary cluster is associated with it.

Remove the relationship

Use one of the following methods:

  1. Delete the secondary cluster, and then delete the primary cluster. See Deleting a Secondary Cluster.
  2. Alternatively, convert the secondary cluster to standalone, and then delete the clusters in any order. See Converting a Secondary Cluster to Standalone.

Cluster Remains in Updating or Enters Failed

A cluster doesn't return to the active state after a cross-region replication operation.

The work request failed or didn't complete

The cross-region replication operation might have encountered an error.

Review the work request

Review the operation before taking further action:

  1. Open the work request for the operation and review its status, logs, and error messages.
  2. Don't start another cluster operation while the current issue is unresolved.
  3. Contact Oracle Support if the cluster remains in the Updating or Failed state.