Cross-Region Message Replication
Download
フォーカスモード
フォントサイズ
TDMQ for Apache Pulsar pro clusters implement cross-region cluster replication based on the GEO Replication solution, supporting use cases such as cross-region disaster recovery, cross-region data archiving, and cross-region consumption.
After cross-region message replication is enabled in Pulsar, a Replicator is started in the cluster. For details, refer to Understanding Pulsar Cross-Region Message Replication. The following example illustrates replication from the Shanghai region to the Beijing region:
The Replicator component runs in the Pulsar cluster in the Shanghai region. Within this component, a Producer-R is started and bound to Topic1 of the peer cluster in the Beijing region, sending message data to the Beijing IDC in the form of a producer.
Note:
Messages produced from the cluster in the Shanghai region are first persisted in the local cluster and then asynchronously forwarded to the cluster in the Beijing region.
The Producer-R in the Replicator component of the cluster in the Shanghai region is independent of the client Producer1 in the same cluster. The cluster address configured for Producer-R is the address of the cluster in the Beijing region.
Replication Process Description
1. Producer1 produces a message to Topic1 in the cluster in the Shanghai region;
2. The cluster in the Shanghai region persists the messages to BookKeeper;
3. Upon successful write, BookKeeper triggers delivery of the message to the Cursor within the Replicator component;
4. The Replicator's Cursor sends the message to Topic1 in the cluster in the Beijing region through Producer-R;
5. After Topic1 in the cluster in the Beijing region successfully writes the message to BookKeeper, it sends an acknowledgment message to the Cursor of the Replicator in the cluster in the Shanghai region. Upon receiving this acknowledgment message, the Cursor in the Shanghai IDC triggers Producer-R to push the next message.
Operation Guide
1. Log in to the TDMQ for Apache Pulsar console.
2. Create a pro cluster in the target replication region. On the cluster purchase page, enable the Cross-Region ReplicationOn switch and select the source data cluster.
3. In the left sidebar of the console, select Cross-Region Replication, click New Link, and configure the message data synchronization link for the cluster:
Copy Link Name: Define a name for the synchronization link. It cannot be empty, can contain digits, letters, and characters "-_=:.", and its length cannot exceed 128 characters.
Link Type: Select Message Level.
Source Cluster: Select the source Pulsar cluster for replication.
Select Target Cluster: Select the target cluster that has been created in the desired region. Only clusters with the same Cluster ID are displayed here.
Replication Level: Three levels are supported: Cluster Level, Namespace Level, and Topic Level.
Cluster Level is applicable to cluster-level replication.
Namespace Level is suitable for scenarios where clusters in both regions are active on a daily basis, with different namespaces distributed across regions.
Topic Level is the finest granularity for cross-region message replication.
Terms of Service: Select I have read and agree to the TDMQ Service Terms.
4. Click Save to complete the payment.
5. After creation, the message-level replication task will appear on the Monitoring page where you can view real-time metrics such as real-time replication rates and message backlog data.
Billing Information
For detailed billing information about message-level cross-region message replication, see Billing Rules for Message-Level Cross-Region Message Replication.
フィードバック