tencent cloud

TencentDB for PostgreSQL

DocumentaçãoTencentDB for PostgreSQLUser GuideExtension Managementtencentdb_ai 1.6 Auto Embedding Feature Description

tencentdb_ai 1.6 Auto Embedding Feature Description

Download
Modo Foco
Tamanho da Fonte
Última atualização: 2026-08-14 18:02:38
Traduzido por IA
Note:
Environment: This document uses PostgreSQL 17.10 as the database version and tencentdb_ai 1.6 as the plugin.

Overview

tencentdb_ai 1.6 supports registering embedding models in two ways to achieve automatic vectorization:

Method 1: Direct Registration of Hunyuan Model
Method 2: Registration via TokenHub
Authentication Method
SecretId + SecretKey (TC3 signature)
api_key(Bearer Token)
Backend Type
backend_type = 'hunyuan'
backend_type = 'tokenhub'
Use Cases
Already have the SecretId/SecretKey for the Hunyuan model.
Already migrated to the TokenHub platform for unified API Key management.
Model Example
hunyuan-embedding (1024 dimensions)
kinfra-text-embedding-0.6b (1024 dimensions)
Credential Required or Not
Requires Hunyuan key.
Requires a TokenHub API Key.
Tested in Current Environment
Passed.
Passed.
Attention:
The auto model (backend_type=tokenhub) pre-installed in the environment is used for ChatCompletions (LLM conversation) and does not support Embedding. To use TokenHub for automatic vectorization, you must register an embedding model separately using Method 2.

Method 1: Directly Registering the Hunyuan Embedding Model

Checking the Environment

Checking Whether the Extension Is Available

SELECT name, default_version, comment
FROM pg_available_extensions
WHERE name IN ('tencentdb_ai', 'pgvector', 'pgmq', 'vector');
Database Raw Return:
name | default_version | installed_version | comment
--------------+-----------------+-------------------+----------------------------------------------------------------
tencentdb_ai | 1.6 | | tencentdb_ai is an ai extension that allows your database to integrate AI capabilities, such as promt, embedding, etc.
pgmq | 1.11.1 | | A lightweight message queue. Like AWS SQS and RSMQ but on Postgres.
vector | 0.8.2 | | vector data type and ivfflat and hnsw access methods
(3 rows)
Result Description: tencentdb_ai 1.6 / pgmq 1.11.1 / vector 0.8.2 are all available.

Checking shared_preload_libraries

SHOW shared_preload_libraries;
Database Raw Return:
shared_preload_libraries
-----------------------------------------------------------------------------------------------------------------------------------------
pg_stat_statements,pg_stat_log,wal2json,decoderbufs,decoder_raw,pg_hint_plan,rds_server_handler,tencentdb_pwdcheck,auto_explain,pgaudit,tencentdb_ai
(1 row)
Result Description: tencentdb_ai is already in shared_preload_libraries, at the last position.

Checking GUC Parameters Related to autoembedding

SELECT name, setting, context
FROM pg_settings
WHERE name LIKE 'tencentdb_ai.autoembedding%'
ORDER BY name;
Database Raw Return:
name | setting | context
--------------------------------------------------+----------+---------
tencentdb_ai.autoembedding_batch_size | 32 | sighup
tencentdb_ai.autoembedding_database | postgres | sighup
tencentdb_ai.autoembedding_max_input_bytes | 65536 | sighup
tencentdb_ai.autoembedding_max_retry | 5 | sighup
tencentdb_ai.autoembedding_naptime_ms | 5000 | sighup
tencentdb_ai.autoembedding_retry_base_ms | 1000 | sighup
tencentdb_ai.autoembedding_task_launch_jitter_ms | 10 | sighup
tencentdb_ai.autoembedding_worker | on | sighup
(8 rows)
Result Description: The default values for autoembedding_database = postgres and autoembedding_worker = on are all reasonable.

Installing the Extension

-- pgcrypto, vector, and pgmq are installed automatically.
CREATE EXTENSION IF NOT EXISTS tencentdb_ai CASCADE;
Database Raw Return:
NOTICE: installing required extension "pgcrypto"
NOTICE: installing required extension "vector"
NOTICE: installing required extension "pgmq"
WARNING: change unlogged table to logged table, If you want to use unlogged tables, please set tencentdb_log_unlogged_table to false.
CREATE EXTENSION
Result Description: The extension was installed successfully. CASCADE automatically installed the three dependent extensions: pgcrypto, vector, and pgmq.

Checking the pgmq Queue

SELECT * FROM pgmq.list_queues();
Database Raw Return:
queue_name | is_partitioned | is_unlogged | created_at
-------------------------------------+----------------+-------------+-------------------------------
tencentdb_ai_autoembedding_incr | f | f | 2026-08-05 22:36:41.689961+08
tencentdb_ai_autoembedding_backfill | f | f | 2026-08-05 22:36:41.689961+08
(2 rows)
Result Description: pgmq automatically created two queues: tencentdb_ai_autoembedding_incr (incremental) and tencentdb_ai_autoembedding_backfill (backfill).

Registering the Hunyuan Embedding Model

Note:
Enter the SecretId and SecretKey here.

Viewing Registered Models

-- View the structure of the model_list table.
SELECT column_name, data_type FROM information_schema.columns
WHERE table_schema = 'tencentdb_ai' AND table_name = 'model_list'
ORDER BY ordinal_position;

-- View existing models.
SELECT * FROM tencentdb_ai.model_list;
Database Raw Return:
column_name | data_type
-----------------+-----------
model_name | name
json_path | jsonpath
secretid | bytea
secretkey | bytea
version | text
region | text
id_random | integer
key_random | integer
backend_type | text
real_model_name | name
api_key | bytea
api_key_random | integer
embedding_dim | integer
(13 rows)

model_name | json_path | secretid | secretkey | version | region | id_random | key_random | backend_type | real_model_name | api_key | api_key_random | embedding_dim
------------+------------------------------------+----------+-----------+---------+--------+-----------+------------+--------------+-----------------+---------+----------------+---------------
auto | $."choices"[0]."message"."content" | | | | | | | tokenhub | auto | | |
(1 row)
Result Description: Initially, only the auto model (tokenhub backend, for ChatCompletions, not for Embedding) is available. You need to register the Hunyuan Embedding-specific model.

Registering the hunyuan-embedding Model

-- Register a model. (The last parameter, backend_type, has a default value of 'hunyuan' and can be omitted.)
SELECT tencentdb_ai.add_model('hunyuan-embedding', '2023-09-01', NULL, NULL);

-- [Enter the SecretId here]
SELECT tencentdb_ai.update_model_attr(
'hunyuan-embedding',
'SecretId',
'<your_SecretId>' -- Replace with your SecretId
);

-- [Enter the SecretKey here]
SELECT tencentdb_ai.update_model_attr(
'hunyuan-embedding',
'SecretKey',
'<your_SecretKey>' -- Replace with your SecretKey
);

-- Set the embedding vector dimension.
UPDATE tencentdb_ai.model_list
SET embedding_dim = 1024
WHERE model_name = 'hunyuan-embedding';

-- Verify the model configuration.
SELECT model_name, backend_type, embedding_dim
FROM tencentdb_ai.model_list
WHERE model_name = 'hunyuan-embedding';
Database Raw Return:
add_model
-----------

(1 row)

update_model_attr
-------------------

(1 row)

update_model_attr
-------------------

(1 row)

UPDATE 1

model_name | backend_type | embedding_dim
-------------------+--------------+---------------
hunyuan-embedding | hunyuan | 1024
(1 row)
Result Description:The Hunyuan embedding model has been registered successfully, with backend_type = hunyuan and embedding_dim = 1024.

Creating Test Tables and Data

-- Create a table. (A single-column primary key is required.)
DROP TABLE IF EXISTS kb_articles CASCADE;
CREATE TABLE kb_articles (
id bigserial PRIMARY KEY,
title text,
content text
);

-- Insert test data.
INSERT INTO kb_articles (title, content) VALUES
('PostgreSQL Introduction',
'PostgreSQL is a powerful, open-source, object-relational database system. Through over 30 years of active development, it has earned a strong reputation for reliability, feature robustness, and performance.'),
('Tencent Cloud VectorDB',
'Tencent Cloud VectorDB is a type of database system specifically designed for storing and searching high-dimensional vectors. It enables AI applications such as semantic search, recommendation systems, and RAG through vector similarity calculations.'),
('RAG Technology',
'Retrieval-Augmented Generation (RAG) is an AI technology that combines retrieval and generation capabilities. It enhances the response quality of large language models by retrieving relevant information from a knowledge base.'),

-- Verify the write operation.
SELECT 'Test data inserted: ' || count(*)::text FROM kb_articles;
Database Raw Return:
NOTICE: table "kb_articles" does not exist, skipping
DROP TABLE
CREATE TABLE
INSERT 0 3
?column?
-----------------------
Test data inserted: 3
(1 row)
Result Description: The table was created successfully, and three rows of test data were written.

Creating an Automatic Vectorization Task

-- 5.1 Create an incremental task.
-- Automatically create a content_embedding vector(1024) column.
-- Automatically create INSERT AFTER + UPDATE BEFORE triggers.
SELECT tencentdb_ai.add_incr_autoembedding_task(
'public', 'kb_articles',
ARRAY['content'],
'hunyuan-embedding'
) AS incr_task_id;

-- 5.2 Create a stock task (backfill historical data).
SELECT tencentdb_ai.add_backfill_autoembedding_task(
'public', 'kb_articles',
ARRAY['content'],
'hunyuan-embedding'
) AS backfill_task_id;
Database Raw Return:
incr_task_id
--------------
1
(1 row)

backfill_task_id
------------------
1
(1 row)
Result Description: Both the incremental and stock tasks were created successfully, and their task_id is 1.

Verifying Table Structures and Triggers

View the table structure

\\d kb_articles
Database Raw Return:
Table "public.kb_articles"
Column | Type | Collation | Nullable | Default
-------------------+--------------+-----------+----------+-----------------------------------------
id | bigint | | not null | nextval('kb_articles_id_seq'::regclass)
title | text | | |
content | text | | |
content_embedding | vector(1024) | | |
Indexes:
"kb_articles_pkey" PRIMARY KEY, btree (id)
Triggers:
tencentdb_ai_autoemb_ins_1 AFTER INSERT ON kb_articles FOR EACH ROW
EXECUTE FUNCTION tencentdb_ai._autoembedding_enqueue_trigger(...)
tencentdb_ai_autoemb_upd_1 BEFORE UPDATE OF content ON kb_articles FOR EACH ROW
EXECUTE FUNCTION tencentdb_ai._autoembedding_enqueue_trigger(...)
Result Description: add_incr_autoembedding_task automatically created a content_embedding vector(1024) column and two triggers.

Viewing Trigger Details

SELECT trigger_name, event_manipulation, action_statement
FROM information_schema.triggers
WHERE event_object_table = 'kb_articles';
Database Raw Return:
trigger_name | event_manipulation | action_statement
----------------------------+--------------------+------------------------------------------------------------------------------------------------------------------------
tencentdb_ai_autoemb_ins_1 | INSERT | EXECUTE FUNCTION tencentdb_ai._autoembedding_enqueue_trigger('1', '17136', 'content_embedding', 'content', '{}', 'id')
tencentdb_ai_autoemb_upd_1 | UPDATE | EXECUTE FUNCTION tencentdb_ai._autoembedding_enqueue_trigger('1', '17136', 'content_embedding', 'content', '{}', 'id')
(2 rows)
Result Description: Both the INSERT AFTER trigger + UPDATE BEFORE OF content trigger have been activated.

View the task status

SELECT task_kind, task_id, table_name, target_column, status, backfill_state, pending, failed_count
FROM tencentdb_ai.autoembedding_status
ORDER BY task_id;
Database Raw Return:
task_kind | task_id | table_name | target_column | status | backfill_state | pending | failed_count
-----------+---------+-------------+-------------------+---------+----------------+---------+--------------
incr | 1 | kb_articles | content_embedding | enabled | | 0 | 0
backfill | 1 | kb_articles | content_embedding | | not_started | 0 | 0
(2 rows)
Result Description: The view can be queried normally. ⚠️ The stock task backfill_state = not_started was not started automatically (see the Bug section).

Viewing Incremental Task Details

SELECT table_name, relid, status FROM tencentdb_ai.autoembedding_incr_task;
Database Raw Return:
table_name | relid | status
-------------+-------+---------
kb_articles | 17136 | enabled
(1 row)
Result Description: The incremental task status is enabled, and relid = 17136.

Testing the Incremental Path

INSERT Trigger

-- Insert new data (insert after the incremental task is created).
INSERT INTO kb_articles (title, content) VALUES
('Embedding Model', 'An embedding model is a machine learning model that converts text into vector representations. Common embedding models include BERT, GPT, and Hunyuan Embedding, among others.');
Database Raw Return:
INSERT 0 1
Verification Steps:
-- Check the pending status immediately. (You should see pending = 1, indicating that it has been enqueued.)
SELECT task_kind, pending FROM tencentdb_ai.autoembedding_status;
Database Raw Return:
task_kind | pending
-----------+---------
incr | 1
backfill | 0
(2 rows)
Result Description: After the INSERT trigger fires, pending = 1 and the message has been enqueued.
-- Wait for the background worker to process.
SELECT pg_sleep(5);
Database Raw Return:
pg_sleep
----------

(1 row)
-- Check whether vectors have been generated. (Note: The first 3 rows existed before the task was created and will not be backfilled automatically.)
SELECT id, title,
content_embedding IS NOT NULL AS has_embedding,
CASE WHEN content_embedding IS NOT NULL
THEN vector_dims(content_embedding) ELSE NULL END AS dims
FROM kb_articles
ORDER BY id;
Database Raw Return:
id | title | has_embedding | dims
----+-----------------+---------------+------
1 | PostgreSQL Introduction | f |
2 | Tencent Cloud VectorDB | f |
3 | RAG Technology | f |
4 | Embedding Model | t |
(4 rows)
Result Description: Vectors for id=4 (the row INSERTed after task creation) were generated successfully. id=1~3 existed before the task was created, were not captured by the incremental trigger, and require stock backfilling.

Verifying Continuous Enqueuing with Another INSERT

INSERT INTO kb_articles (title, content) VALUES
('Hunyuan Large Model', 'Tencent Hunyuan is a general-purpose large language model self-developed by Tencent, featuring powerful natural language understanding and generation capabilities.');
Database Raw Return:
INSERT 0 1
-- Check the pending status immediately.
SELECT task_kind, pending FROM tencentdb_ai.autoembedding_status;
Database Raw Return:
task_kind | pending
-----------+---------
incr | 1
backfill | 0
(2 rows)
The second INSERT was also successfully enqueued.
SELECT pg_sleep(5);

SELECT task_kind, pending, failed_count FROM tencentdb_ai.autoembedding_status;

SELECT id, title, content_embedding IS NOT NULL AS has_embedding
FROM kb_articles ORDER BY id;

-- Error Table
SELECT * FROM tencentdb_ai.autoembedding_error ORDER BY created_at DESC LIMIT 5;
Database Raw Return:
pg_sleep
----------

(1 row)

task_kind | pending | failed_count
-----------+---------+--------------
incr | 0 | 0
backfill | 0 | 0
(2 rows)

id | title | has_embedding
----+-----------------+---------------
1 | PostgreSQL Introduction | f
2 | Tencent Cloud VectorDB | f
3 | RAG Technology | f
4 | Embedding Model | t
5 | Hunyuan Large Model | t
(5 rows)

error_id | msg_id | task_kind | task_id | row_id | error_code | error_message | detail | created_at
----------+--------+-----------+---------+--------+------------+---------------+--------+------------
(0 rows)
Result Description: All INSERT paths for id=4 and 5 succeeded, with pending = 0, failed_count = 0, and no error records. id=1~3 remain null (due to a lack of stock backfilling).

Vector Dimension Verification

SELECT id, title,
vector_dims(content_embedding) AS dims,
length(content_embedding::text) AS text_len
FROM kb_articles
WHERE content_embedding IS NOT NULL;
Database Raw Return:
id | title | dims | text_len
----+------------+------+----------
4 | Embedding Model | 1024 | 12727
5 | Hunyuan Large Model | 1024 | 12779
(2 rows)
Result Description:All vectors are 1024-dimensional, consistent with the model configuration.

UPDATE Trigger (Bypassing the Backfill Bug for Existing Data and Generating Vectors for id=1~3)

-- Updating existing rows triggers the UPDATE trigger even if the content remains unchanged.
UPDATE kb_articles SET content = content WHERE id IN (1,2,3);
Database Raw Return:
UPDATE 3
-- Check the pending status immediately (should be 3).
SELECT task_kind, pending FROM tencentdb_ai.autoembedding_status;
Database Raw Return:
task_kind | pending
-----------+---------
incr | 3
backfill | 0
(2 rows)
pending = 3. All three rows have been enqueued.
SELECT pg_sleep(5);

-- Perform a full validation.
SELECT id, title, content_embedding IS NOT NULL AS has_embedding
FROM kb_articles ORDER BY id;
Database Raw Return:
pg_sleep
----------

(1 row)

id | title | has_embedding
----+-----------------+---------------
1 | PostgreSQL Introduction | t
2 | Tencent Cloud VectorDB | t
3 | RAG Technology | t
4 | Embedding Model | t
5 | Hunyuan Large Model | t
(5 rows)
Result Description: The UPDATE trigger path is functioning normally. Vectors for id=1~3 were successfully generated via UPDATE. All 5 rows have has_embedding = t.

Vector Similarity Search

Similarity with Only Two Rows

SELECT id, title,
content_embedding <=> (
SELECT content_embedding FROM kb_articles WHERE id = 4
) AS distance
FROM kb_articles
WHERE content_embedding IS NOT NULL
ORDER BY distance;
Database Raw Return:
id | title | distance
----+------------+---------------------
4 | Embedding Model | 0
5 | Hunyuan Large Model | 0.28689392595353047
(2 rows)
Result Description: The cosine distance from the Embedding Model to itself is 0, and the distance to the Hunyuan Large Model is approximately 0.287. The two models are semantically similar (both relate to AI models), which aligns with expectations.

Full Similarity Search

SELECT id, title,
content_embedding <=> (
SELECT content_embedding FROM kb_articles WHERE id = 1
) AS distance
FROM kb_articles
WHERE content_embedding IS NOT NULL
ORDER BY distance;
Database Raw Return:
id | title | distance
----+-----------------+---------------------
1 | PostgreSQL Introduction | 0
2 | Tencent Cloud VectorDB | 0.2789466068930575
3 | RAG Technology | 0.3563481692751316
4 | Embedding Model | 0.3840361738420558
5 | Hunyuan Large Model | 0.40611747180705204
(5 rows)
Result Description: The semantic distance between "PostgreSQL Introduction" and "Tencent Cloud VectorDB" is the closest (0.279, both are in the database domain), while the distance to "Hunyuan Large Model" is the farthest (0.406). This ranking is reasonable.

Method 2: Registering an Embedding Model via TokenHub

What Is TokenHub

TokenHub is Tencent Cloud's unified AI model gateway platform, providing unified API Key management and billing. In the tencentdb_ai 1.6 version, TokenHub now supports the Embedding API (previously it only supported ChatCompletions).
Advantages of Using TokenHub:
Unified Credential Management: A single API Key manages all models (chat + embedding), eliminating the need to manage SecretId/SecretKey pairs separately.
Simple Authentication: Bearer Token authentication, simply pass the api_key.

TokenHub Embedding Model Overview

TokenHub provides a total of 4 Embedding models, which fall into two main categories: text vector and multimodal vector. All models are compatible with the OpenAI Embeddings API format. The API endpoint is:
Text Vector: POST https://tokenhub.tencentmaas.com/v1/embeddings
Multimodal Vector: POST https://tokenhub.tencentmaas.com/v1/embeddings/multimodal
Authentication Method: Authorization: Bearer <api_key>

Text Embedding Model

Attribute
kinfra-text-embedding-0.6b
kinfra-text-embedding-4b
Parameter Volume
0.6B (600 million)
4B (4 billion)
Output Dimension
1024 dimensions
2560 dimensions
Context Length
32k tokens
32k tokens
Supported Languages
30+ languages: Chinese, English, Japanese, Korean, French, German, Russian, Portuguese, Spanish, and so on.
Same as left
Recommended Scenarios
Large-scale text recall, latency-sensitive, cost-sensitive
High-quality text search and deep semantic understanding
Typical Tasks
Semantic search, similarity calculation, text clustering, text classification, FAQ matching, intelligent Q&A, knowledge base search
Same as left, with higher precision
CMTEB Benchmark Comparison (Higher values indicate better performance):
Evaluation Metric
0.6b
4b
Improved by
Mean(Task) Overall Score
66.64
72.63
+9.0%
Retrieval
71.01
77.02
+8.5%
Clustering
68.60
78.15
+13.9%
Semantic Textual Similarity (STS)
54.88
61.41
+11.9%
Classification
71.46
75.55
+5.7%
Reranking
64.16
68.26
+6.4%

Multimodal Embedding Model

Supports vectorization for three modalities: text, image_url, and video_url.
Attribute
kinfra-vl-embedding-2b
kinfra-vl-embedding-8b
Parameter Volume
2B (2 billion)
8B (8 billion)
Output Dimension
2048 dimensions
4096 dimensions
Context Length
32k tokens
32k tokens
Supported Modalities
Text + Image + Video
Text + Image + Video
Supported Languages
30+ mainstream languages
Same as left
Recommended Scenarios
Multimodal online search, video search, response speed prioritized
High-precision multimodal search, precision prioritized
Typical Tasks
Image-text search, video search, multimodal semantic matching, cross-modal image-text search
Same as left, with higher precision
Multimodal Benchmark Comparison (Higher values indicate better performance):
Evaluation Task
2b
8b
Improved by
MSCOCO Image-to-Text Retrieval
0.70
0.76
+8.6%
VisualNews Image-to-Text Retrieval
0.60
0.67
+11.7%
WebQA
0.87
0.90
+3.4%
VisDial
0.69
0.87
+26.1%
Image-Text Retrieval mean
0.698
0.795
+13.9%
MMEB-V2 Comprehensive
69.82
75.26
+7.8%
Multimodal Input Limitations:
Limit
Description
Image format.
JPEG,PNG,WEBP,BMP,TIFF
Image pixel range.
4,096(~64×64)– 1,843,200(~1280×1440)
Video format.
MP4,AVI,MOV
Maximum total video pixels.
7,864,320
Maximum video sampling frames.
64 frames
Default video fps.
1.0
Vector normalization.
Default L2 normalization
Encoding format.
Only float is supported.

Model Selection Quick Reference: Choosing the Right Model for Your Scenario

Your Requirements
Recommended Model
Reason
Chinese knowledge base / FAQ search, cost-sensitive, high concurrency
kinfra-text-embedding-0.6b
1024 dimensions, lightweight and fast
Multilingual search, requiring deep semantic understanding, with high precision requirements
kinfra-text-embedding-4b
2560 dimensions, CMTEB comprehensive score 72.63
Cross-modal image-text search (search images by text, search text by images)
kinfra-vl-embedding-2b
2048 dimensions, fast response speed
Video search, high-precision multimodal matching
kinfra-vl-embedding-8b
4096 dimensions, MMEB-V2 comprehensive score 75.26
Plain text + potential multimodal extension in the future
First register -0.6b or -4b, then add a multimodal model later.
One table can be bound to one model, and multiple tables can use different models.

Registering a TokenHub Embedding Model

Note:
Key Differences from Method 1:
Explicitly specify backend_type = 'tokenhub' using the fifth parameter of add_model() (the default value is 'hunyuan').
For authentication, use the api_key (Bearer Token), not SecretId / SecretKey.
real_model_name points to the actual model name on TokenHub (for example, kinfra-text-embedding-0.6b).

Viewing the model_list Table Structure

SELECT column_name, data_type FROM information_schema.columns
WHERE table_schema = 'tencentdb_ai' AND table_name = 'model_list'
ORDER BY ordinal_position;
Database Raw Return:
column_name | data_type
-----------------+-----------
model_name | name
json_path | jsonpath
secretid | bytea
secretkey | bytea
version | text
region | text
id_random | integer
key_random | integer
backend_type | text
real_model_name | name
api_key | bytea
api_key_random | integer
embedding_dim | integer
(13 rows)
Result Description: All required TokenHub columns exist: the api_key column (encrypted and stored as bytea), the backend_type column, and the real_model_name column.

Registering a Text Embedding Model

-- ===== Text Vector Model Registration Template =====
-- Key Parameter: You must explicitly pass 'tokenhub' for the fifth parameter, p_backend_type (the default value is 'hunyuan').
-- The sixth parameter, real_model_name, is the actual model name on TokenHub.
-- Select one of the following based on your actual requirements.

-- Model A: Lightweight Text Vector (1024 dimensions, low cost, fast speed)
-- Scenarios: Chinese knowledge base search, FAQ matching, text clustering and classification
SELECT tencentdb_ai.add_model(
'my-text-embed-0.6b', -- Model alias (customizable)
NULL, NULL, NULL, -- version, region, json_path (not required)
'tokenhub', -- backend_type: must be explicitly specified as 'tokenhub'
'kinfra-text-embedding-0.6b' -- real_model_name
);
SELECT tencentdb_ai.update_model_attr('my-text-embed-0.6b', 'api_key', 'sk-tp-yourAPIKey');
UPDATE tencentdb_ai.model_list SET embedding_dim = 1024 WHERE model_name = 'my-text-embed-0.6b';

-- Model B: High-Precision Text Vector (2560 dimensions, high precision)
-- Scenarios: Multilingual deep semantic search, cross-language matching, high-quality semantic understanding
SELECT tencentdb_ai.add_model(
'my-text-embed-4b', -- Model alias (customizable)
NULL, NULL, NULL,
'tokenhub',
'kinfra-text-embedding-4b'
);
SELECT tencentdb_ai.update_model_attr('my-text-embed-4b', 'api_key', 'sk-tp-yourAPIKey');
UPDATE tencentdb_ai.model_list SET embedding_dim = 2560 WHERE model_name = 'my-text-embed-4b';

Registering a Multimodal Embedding Model

-- ===== Multimodal Vector Model Registration Template =====
-- The multimodal model supports three types of input: text, image_url, and video_url.
-- Used for scenarios such as cross-modal image-text search and video semantic search.

-- Model C: Lightweight Multimodal (2048 dimensions, fast response speed)
-- Scenarios: Cross-modal image-text search, online multimodal search
SELECT tencentdb_ai.add_model(
'my-vl-embed-2b', -- Model alias (customizable)
NULL, NULL, NULL,
'tokenhub',
'kinfra-vl-embedding-2b' -- VL = Vision-Language (visual-language)
);
SELECT tencentdb_ai.update_model_attr('my-vl-embed-2b', 'api_key', 'sk-tp-yourAPIKey');
UPDATE tencentdb_ai.model_list SET embedding_dim = 2048 WHERE model_name = 'my-vl-embed-2b';

-- Model D: High-Precision Multimodal (4096 dimensions, highest precision)
-- Scenarios: Video search, high-precision multimodal semantic matching
SELECT tencentdb_ai.add_model(
'my-vl-embed-8b', -- Model alias (customizable)
NULL, NULL, NULL,
'tokenhub',
'kinfra-vl-embedding-8b'
);
SELECT tencentdb_ai.update_model_attr('my-vl-embed-8b', 'api_key', 'sk-tp-yourAPIKey');
UPDATE tencentdb_ai.model_list SET embedding_dim = 4096 WHERE model_name = 'my-vl-embed-8b';

Registration Verification (Using the 0.6b Text Model as an Example)

-- Verify the registration result.
SELECT model_name, backend_type, real_model_name, embedding_dim FROM tencentdb_ai.model_list;
Database raw return (registered tokenhub-embedding, alias pointing to kinfra-text-embedding-0.6b):
model_name | backend_type | real_model_name | embedding_dim
---------------------+--------------+----------------------------+---------------
auto | tokenhub | | -1
hunyuan-embedding | hunyuan | | 1024
tokenhub-embedding | tokenhub | kinfra-text-embedding-0.6b | 1024
(3 rows)
Result Description:The TokenHub embedding model has been registered successfully:
backend_type = tokenhub (which is distinctly different from the hunyuan in Method 1)
real_model_name = kinfra-text-embedding-0.6b (the actual model on TokenHub)
embedding_dim = 1024
Credential Acquisition Method: TokenHub Console → API Key Management
Quick Reference for Four Model Registration Methods:
Model Alias (Custom)
real_model_name
embedding_dim
Type
my-text-embed-0.6b
kinfra-text-embedding-0.6b
1024
Lightweight Text
my-text-embed-4b
kinfra-text-embedding-4b
2560
High-precision Text
my-vl-embed-2b
kinfra-vl-embedding-2b
2048
Lightweight Multimodal
my-vl-embed-8b
kinfra-vl-embedding-8b
4096
High-precision Multimodal

Creating Test Tables and Automatic Vectorization Tasks

Creating Tables and Inserting Data

DROP TABLE IF EXISTS kb_articles_tokenhub CASCADE;
CREATE TABLE kb_articles_tokenhub (
id bigserial PRIMARY KEY,
title text,
content text
);

INSERT INTO kb_articles_tokenhub (title, content) VALUES
('Introduction to PostgreSQL', 'PostgreSQL is a powerful, open-source, object-relational database system, developed actively for over 30 years.'),
('Tencent Cloud VectorDB', 'Tencent Cloud VectorDB is a type of database system specifically designed for storing and searching high-dimensional vectors.'),
('AI Technology Trends', 'In 2026, artificial intelligence technology witnessed a new wave of development, with multimodal large models becoming mainstream.');
Database Raw Return:
DROP TABLE
CREATE TABLE
INSERT 0 3

Creating an Incremental Task

-- Note: The model_name parameter uses the alias 'tokenhub-embedding' from the registration.
SELECT tencentdb_ai.add_incr_autoembedding_task(
'public', 'kb_articles_tokenhub',
ARRAY['content'],
'tokenhub-embedding' -- The model registered via TokenHub
) AS incr_task_id;
Database Raw Return:
incr_task_id
--------------
4
(1 row)
Result Description:The incremental task was created successfully, with task_id = 4 (using the same API as Method 1).

Verifying Table Structures and Triggers

\\d kb_articles_tokenhub
Database Raw Return:
Table "public.kb_articles_tokenhub"
Column | Type | Collation | Nullable | Default
-------------------+--------------+-----------+----------+--------------------------------------------------
id | bigint | | not null | nextval('kb_articles_tokenhub_id_seq'::regclass)
title | text | | |
content | text | | |
content_embedding | vector(1024) | | |
Indexes:
"kb_articles_tokenhub_pkey" PRIMARY KEY, btree (id)
Triggers:
tencentdb_ai_autoemb_ins_4 AFTER INSERT ON kb_articles_tokenhub FOR EACH ROW
EXECUTE FUNCTION tencentdb_ai._autoembedding_enqueue_trigger('4', '17167', ...)
tencentdb_ai_autoemb_upd_4 BEFORE UPDATE OF content ON kb_articles_tokenhub FOR EACH ROW
EXECUTE FUNCTION tencentdb_ai._autoembedding_enqueue_trigger('4', '17167', ...)
Result Description:The table structure and triggers are identical to those in Method 1. The content_embedding vector(1024) column + INSERT/UPDATE triggers are automatically created.

View the task status

SELECT task_kind, task_id, table_name, model_name, status
FROM tencentdb_ai.autoembedding_status
ORDER BY task_id;
Database Raw Return:
task_kind | task_id | table_name | model_name | status
-----------+---------+----------------------+---------------------+---------
incr | 1 | kb_articles | hunyuan-embedding | enabled
backfill | 1 | kb_articles | hunyuan-embedding |
incr | 4 | kb_articles_tokenhub | tokenhub-embedding | enabled
(3 rows)
Result Description:The TokenHub task (task_id=4) and the Hunyuan task (task_id=1) can coexist, and both methods can be used simultaneously.

Testing the Incremental Path

INSERT Trigger Enqueuing

-- Insert new data, and the trigger is automatically enqueued.
INSERT INTO kb_articles_tokenhub (title, content) VALUES
('Hunyuan Large Model', 'Tencent Hunyuan is a general-purpose large language model self-developed by Tencent, featuring powerful natural language understanding and generation capabilities.');
Database Raw Return:
INSERT 0 1
-- Check pending
SELECT task_kind, task_id, table_name, model_name, pending
FROM tencentdb_ai.autoembedding_status ORDER BY task_id;
Database Raw Return:
task_kind | task_id | table_name | model_name | pending
-----------+---------+----------------------+-------------------+---------
incr | 1 | kb_articles | hunyuan-embedding | 0
backfill | 1 | kb_articles | hunyuan-embedding | 0
incr | 5 | kb_articles_tokenhub | tokenhub-embed | 1
(3 rows)
Result Description:The INSERT operation was triggered successfully, pending = 1, and the message has been enqueued.

Worker Consuming and Generating Vectors

-- Wait for the worker to process.
SELECT pg_sleep(10);

-- pending status
SELECT task_kind, task_id, pending, failed_count
FROM tencentdb_ai.autoembedding_status ORDER BY task_id;

-- Vector result
SELECT id, title,
content_embedding IS NOT NULL AS has_embedding,
CASE WHEN content_embedding IS NOT NULL THEN vector_dims(content_embedding)::text ELSE 'NULL' END AS dims
FROM kb_articles_tokenhub ORDER BY id;

-- Error Record
SELECT * FROM tencentdb_ai.autoembedding_error WHERE task_id = 5 ORDER BY created_at DESC LIMIT 5;
Database Raw Return:
pg_sleep
----------

(1 row)

task_kind | task_id | pending | failed_count
-----------+---------+---------+--------------
incr | 1 | 0 | 0
backfill | 1 | 0 | 0
incr | 5 | 0 | 0
(3 rows)

id | title | has_embedding | dims
----+-----------------+---------------+------
1 | PostgreSQL Introduction | f | NULL
2 | Tencent Cloud VectorDB | f | NULL
3 | RAG Technology | f | NULL
4 | Hunyuan Large Model | t | 1024
(4 rows)

error_id | msg_id | task_kind | task_id | row_id | error_code | error_message | detail | created_at
----------+--------+-----------+---------+--------+------------+---------------+--------+------------
(0 rows)
Result Description:The TokenHub Embedding vector has been generated successfully. pending = 0, failed_count = 0, with no errors. The vector dimension for id=4 is 1024.

UPDATE Backfilling Historical Data

-- Updating existing rows triggers the UPDATE trigger to backfill vectors.
UPDATE kb_articles_tokenhub SET content = content WHERE id IN (1,2,3);
Database Raw Return:
UPDATE 3
-- Check the pending status (should be 3).
SELECT task_kind, pending FROM tencentdb_ai.autoembedding_status WHERE task_id = 5;
Database Raw Return:
task_kind | pending
-----------+---------
incr | 3
(1 row)
pending = 3. All three rows have been enqueued.
SELECT pg_sleep(8);

-- Full vectors + dimension
SELECT id, title,
content_embedding IS NOT NULL AS has_embedding,
CASE WHEN content_embedding IS NOT NULL THEN vector_dims(content_embedding)::text ELSE 'NULL' END AS dims
FROM kb_articles_tokenhub ORDER BY id;
Database Raw Return:
pg_sleep
----------

(1 row)

id | title | has_embedding | dims
----+-----------------+---------------+------
1 | PostgreSQL Introduction | t | 1024
2 | Tencent Cloud VectorDB | t | 1024
3 | RAG Technology | t | 1024
4 | Hunyuan Large Model | t | 1024
(4 rows)
Result Description:All four rows of vectors have been generated, each with a dimension of 1024.

Vector Similarity Search

SELECT id, title,
content_embedding <=> (
SELECT content_embedding FROM kb_articles_tokenhub WHERE id = 1
) AS distance
FROM kb_articles_tokenhub
WHERE content_embedding IS NOT NULL
ORDER BY distance;
Database Raw Return:
id | title | distance
----+-----------------+--------------------
1 | PostgreSQL Introduction | 0
2 | Tencent Cloud VectorDB | 0.5327960213665903
3 | RAG Technology | 0.6819517300768105
4 | Hunyuan Large Model | 0.710405861090913
(4 rows)
Result Description:The vector similarity search is functioning normally. "PostgreSQL Introduction" and "Tencent Cloud VectorDB" have the closest distance (0.533, indicating a database domain association), and the search results are reasonable.
Selection Suggestion:
If you already have a Hunyuan API key, use Method 1, which is the simplest and most direct approach.
If you have already migrated to the TokenHub platform / want to centrally manage credentials for all AI models, use Method 2.
If you require multimodal capabilities (such as cross-modal image-text search or video retrieval), you must use Method 2 (TokenHub), as the Hunyuan model does not support them.
The two methods can coexist, and different tables can use different models.
TokenHub Model Selection Quick Reference:
Your Requirements
Model Selection
embedding_dim
Chinese knowledge base search, FAQ matching, cost-sensitive
kinfra-text-embedding-0.6b
1024
Multilingual deep semantic search, with high precision requirements
kinfra-text-embedding-4b
2560
Cross-modal image-text search, with response speed prioritized
kinfra-vl-embedding-2b
2048
Video search, high-precision multimodal matching
kinfra-vl-embedding-8b
4096

Monitoring and Troubleshooting

Worker Process Status

SELECT pid, backend_type, state, query, query_start, state_change
FROM pg_stat_activity
WHERE backend_type LIKE '%tencentdb_ai%';
Database Raw Return:
pid | backend_type | state | query | query_start | state_change
------+-----------------------------------+--------+-------------------------------------------------------------+-------------------------------+------------------------------
67832 | tencentdb_ai scheduler | | | |
67834 | tencentdb_ai autoembedding worker | active | SELECT tencentdb_ai._autoembedding_process_backfill_batch() | 2026-08-05 22:40:37.409162+08 | 2026-08-05 22:40:37.41009+08
(2 rows)
Result Description: Both background processes, tencentdb_ai scheduler and tencentdb_ai autoembedding worker, are running.

Error Table

SELECT * FROM tencentdb_ai.autoembedding_error
ORDER BY created_at DESC LIMIT 10;
Database Raw Return:
error_id | msg_id | task_kind | task_id | row_id | error_code | error_message | detail | created_at
----------+--------+-----------+---------+--------+------------+---------------+--------+------------
(0 rows)
Result Description:No error records were generated during the entire test, and all embedding API calls were successful.

Ajuda e Suporte

Esta página foi útil?

comentários