Event Time
In Event Time mode, the timestamp is provided by a specific field in the input data. The WATERMARK FOR statement can be used to specify this field and enable Event Time mode. It is suitable for scenarios where the data source contains precise timestamps.
Computing Resources
The minimum unit of computing resources provided by stream computing is 1 CU. The specific meaning of 1 CU is: 1 CPU core and 4 GB of memory. The billing standard is the user's actual CU usage.
Stream Computing
Stream computing is a type of computation oriented towards streaming data. It continuously reads data generated from one or more streaming data sources, performs efficient computation through the combination of multiple operators within a job, and then outputs the results to various downstream data destinations, such as message queues, databases, data warehouses, and storage services, based on actual requirements.
Stream Connection Partition
A stream join Partition is the smallest unit for storing data in a Topic. For topics with high throughput, you can create multiple partitions.
Stream Connection Data Integration Task
A Stream Join Data Integration Task (Integrator) refers to a task that imports data from a Topic to a cloud product. For a single Topic, multiple Integrators can be started to import data to different cloud products.
Processing Time
In Processing Time mode, timestamps are automatically generated by the system and added to the data source (named PROCTIME, invisible in SELECT *, and must be explicitly specified when used). The timestamp for each piece of data is based on the time it is processed by the system, which introduces a degree of unpredictability. This mode is suitable for scenarios where high time precision is not required.
Schema
It represents the structural information of a table, such as column names, column types, and so on.
Source Time
In Source Time mode, you can use the timestamp contained in the metadata of each Kafka record as the timestamp for stream computing processing (named SOURCETIME, invisible in SELECT *, and must be explicitly specified when the timestamp is used). This avoids the unpredictability introduced by using Processing Time mode when the input data lacks a timestamp field.
Time Pattern
Guides the system on how to obtain timestamps when processing data. Currently, three time modes are supported: Event Time, Processing Time, and Source Time.
Data Destination
A data Sink refers to the destination where the stream computing system outputs its processing results, such as Tencent Cloud CKafka, TencentDB for MySQL, and so on.
Tuple Stream
Tuple (also known as Append) is a type of data stream that can store streaming data without a primary key. Users can continuously append new data to this type of data stream. It does not involve updating previously emitted data. Currently, various data sources and sinks support the input and output of Append streams.
Upsert Stream
An Upsert stream (Update OR Insert, Upsert) is generated by queries such as DISTINCT, GROUP BY statements without time windows, and JOIN statements without time ranges. It has a primary key definition. If subsequently emitted data shares the same primary key as a previously emitted record, that record is updated with the new value. Otherwise, a new row of data is inserted. This mechanism ensures that previously emitted data can be updated to reflect the latest values.
Watermark
It represents a specific point in time, before which all data has been properly processed. The Watermark is automatically generated by the system. Users can specify the maximum tolerance for timestamps using the WATERMARK FOR BOUNDED statement.
Topic
In Stream Compute Service, a topic is the smallest unit of stream connection for subscription and publishing. Each topic represents a category of stream data. Topics are similar to tables in a database.
In Internet of Things Hub, a topic refers to a subject for message communication. In the Pub/Sub model, a topic serves as an intermediary for message communication. Topics are required in the Pub/Sub model, and communication between devices is based on specific topics.
In message queues such as TDMQ, CKafka, and CMQ, a Topic is a logical concept for storing messages. It can be viewed as a collection of messages. Multiple producers can push messages to a Topic, and any number of consumers can consume messages from it.
In CLS, a Topic is the basic management unit provided by CLS. Each log Topic corresponds to an application or service. A log Topic is the smallest management unit in CLS. Configurations for collection, indexing, and delivery are centered around topics. A logset can contain multiple log topics.