Performance Description
ダウンロード
フォーカスモード
フォントサイズ
Overview
This document describes the overall performance and stability of the new architecture (version 3.4.0 and above) of Tencent Cloud CLS LogListener in log collection.
Collection rate significantly improved: In compute-intensive scenarios such as regex and multiline parsing, the new architecture achieves multiple times higher collection rates.
Low-load resource optimization: Under conventional write rates, CPU resource consumption is reduced by approximately 30% compared to the legacy architecture.
Enhanced isolation stability: By systematically optimizing critical paths, the new architecture effectively resolves collection bottlenecks in high-load scenarios, ensuring the continuity of data reporting.
The following will explain the performance of the new LogListener architecture from dimensions such as file collection rate and resource consumption, comparing it with the old LogListener architecture (version 2.x) and a collector from a major competitor.
Testing Environment
Instance Specifications: Standard S6 | 4-core CPU | 16GB memory.
Public Network Bandwidth: 200Mbps.
Storage: Enhanced SSD CBS.
Write Rate: 10MB/s, 20MB/s, 60MB/s, 100MB/s.
Comparison Objects: LogListener old architecture (version 2.9.20), collector from a major competitor.
Performance Comparison (File Collection Scenario)
File collection rate
In typical log collection scenarios such as single-line, multiline, regex, delimiter, and JSON formats, comparative tests were conducted on the new LogListener architecture, legacy architecture, and mainstream competitor collectors. The overall results are as follows.
Single-line full text mode
At various write rates, the performance of both the new and legacy architectures, as well as the competitor's collector, remains comparable, with all three solutions demonstrating similar performance.
Multiline full-text parsing mode
At low write rates (10MB/s - 20MB/s), the collection performance of both the new and legacy architectures, as well as the competitor's collector, remains comparable. In high-load scenarios with write rates exceeding 60MB/s, the new architecture demonstrates significantly superior collection performance, achieving approximately 1.5 to 2 times higher throughput compared to the legacy architecture and mainstream competitor collectors.
Single-line regex parsing mode
At low write rates (10MB/s - 20MB/s), there is no significant performance difference between the new/legacy architectures and the competitor's solution. In high-load scenarios exceeding 60MB/s, the new architecture demonstrates approximately 2.5 times higher collection performance than the legacy architecture. At 100MB/s write rates, the new architecture achieves about 2.6 times the collection speed of the competitor's collector.
Multiline regex parsing mode
At low write rates (10MB/s - 20MB/s), the new architecture performs on par with the legacy architecture and slightly outperforms the competitor's collector. When write rates exceed 60MB/s, the new architecture demonstrates approximately 40% improvement in collection performance compared to both the legacy architecture and the competitor's solution.
Delimiter parsing mode
At low write rates (10MB/s - 20MB/s), the performance across all solutions remains comparable. When write rates reach 60MB/s and above, the new architecture demonstrates approximately 3 - 3.5 times higher collection performance than the legacy architecture, performing on par with mainstream competitor products.
JSON parsing mode
At various write rates, the collection performance of both the new and legacy architectures, as well as the competitor's collector, remains comparable. At 100MB/s write rates, the new architecture performs slightly lower than the competitor's collector but slightly higher than the legacy architecture.
Resource Consumption
CPU Resource Usage Instructions
Taking the single-line full-text log scenario as an example, it demonstrates the CPU resource consumption of each collector at different write rates.
At low write rates (10MB/s - 20MB/s), the new architecture demonstrates significantly lower CPU consumption than the legacy architecture (approximately 30% reduction). At high write rates, all three solutions show comparable performance, with the new architecture maintaining stable overall performance.
Write rate | new architecture | legacy architecture | competitor |
10MB/s | 2.18% | 3.11% | 2.64% |
20MB/s | 3.33% | 4.67% | 3.51% |
60MB/s | 7.57% | 7.24% | 4.31% |
100MB/s | 8.23% | 7.80% | 6.40% |
Memory Resources Usage Instructions
Taking the single-line full-text log scenario as an example, it demonstrates the resource consumption of each collector at different write rates.
Overall, across various write rate levels, the new architecture exhibits increased memory footprint compared to the legacy architecture. This is attributed to its implementation of a more robust memory buffering mechanism designed to handle high-volume writes, network jitter, and downstream processing fluctuations. While memory usage is higher, this trade-off delivers more stable continuous reporting capabilities, enhanced processing performance, and effectively prevents bottleneck propagation in critical paths, thereby ensuring resource availability for smooth business process continuity.
Write rate | new architecture | legacy architecture | competitor |
10MB/s | 0.44% | 0.19% | 0.20% |
20MB/s | 0.70% | 0.18% | 0.19% |
60MB/s | 2.35% | 0.16% | 0.17% |
100MB/s | 3.87% | 0.16% | 0.19% |
Summary
The new LogListener architecture significantly enhances log collection throughput and completeness compared to the legacy version, particularly excelling in high-load scenarios involving multi-line logs, regex parsing, and delimiter processing. While memory usage increases, this trade-off delivers superior CPU efficiency, more stable continuous reporting capabilities, and higher processing performance.
フィードバック