tencent cloud

Stream Compute Service

ドキュメントStream Compute ServiceSQL Developer GuideReferencing Multiple External Catalogs Simultaneously

Referencing Multiple External Catalogs Simultaneously

Download
フォーカスモード
フォントサイズ
最終更新日: 2026-08-11 14:58:16
AI翻訳
Note:
Newly created clusters now support a single job using multiple external catalogs concurrently. If your current cluster edition does not yet support this feature, please submit a ticket to apply for a cluster upgrade.

Use Cases

This is designed for scenarios where a single job needs to reference multiple external catalogs concurrently. However, because multiple Hive/HDFS configuration files can cause conflicts, using multiple catalogs that require this configuration file support (for example, Hive Catalog or Paimon(hive) Catalog) simultaneously is not supported.

Usage

For the method to create an external Catalog, refer to Table Management.
For using multiple catalogs in an SQL job, refer to the following code:

-- 1. Define the Paimon destination table
CREATE TABLE IF NOT EXISTS `116-paimon-sink1`.`default`.`HiveSink` (
id INT PRIMARY KEY NOT ENFORCED, -- Paimon requires a primary key, and `id` serves as the primary key.
name STRING
) WITH (
);
-- 2. Define the Hive source table
CREATE TABLE IF NOT EXISTS `hive116-1`.`default`.`source` (
id INT,
name STRING,
dt STRING,
hr STRING
) PARTITIONED BY (dt, hr)
WITH
'connector' = 'hive',
'hive-version' = '3.1.1',
'hive-database' = 'default',
'partition.time-extractor.timestamp-pattern'='$dt $hr:00:00',
'sink.partition-commit.trigger'='partition-time',
'sink.partition-commit.delay'='10 min',
'sink.partition-commit.policy.kind'='metastore,success-file'
);


INSERT INTO `116-paimon-sink1`.`default`.`HiveSink`
SELECT
id,
`name`
FROM `hive116-1`.`default`.`source`;


ヘルプとサポート

この記事はお役に立ちましたか?

フィードバック