In the era of rapid AI development, ES is widely used for vector search, multimodal search, and knowledge base building. Its unique hybrid text-vector search capability, which balances recall and search accuracy, is favored by a broad range of developers. This document focuses on tuning techniques drawn from practical experience to help you better leverage the performance advantages of ES vector search.
Configuration Planning
1. Reasonable Cluster Configuration Assessment
To ensure the performance of ES vector search, a reasonable cluster configuration is essential. The key is to ensure sufficient memory. If memory is inadequate and searches frequently hit the disk, performance can degrade by more than 10 times. For cluster configuration estimation for vector search, see ES Vector Cluster Configuration Assessment. 2. Reasonable Index Planning
You need to evaluate your business data volume and future growth in advance, and plan your indexing and sharding accordingly. If the data volume is large, avoid putting everything in a single index, which would cause every search to scan the entire index. It is recommended to split indexes by category, such as department or product type. Both oversized shards and an excessive number of shards can affect read and write performance. The following are practical recommendations for shard settings:
A single shard is recommended to be 20 GB–50 GB in size. You can use this to initially determine the number of shards for your index.
The number of shards should be as close as possible to the number of data nodes. If you have a large number of shards, it is recommended that you set the number of shards to an integer multiple of the number of data nodes to facilitate even distribution of shards across the data nodes.
The total number of shards for all indexes on a single node should not exceed 1,000. The total number of shards in the cluster should be kept below 30,000.
Increasing replicas can improve query throughput (QPS), as searches can run in parallel on both primary and replica shards.
Recommendation: In scenarios with low write pressure and high query concurrency, increase the number of replicas appropriately.
PUT /my_index
{
"settings": {
"index": {
"number_of_shards": 10,
"number_of_replicas": 1
}
},
"mappings": {
"properties": {
...
}
}
3. Test Set Preparation and Iterative Tuning
Vector search tuning is an iterative process of trial and optimization. You can prepare a high-quality test set containing query terms, known relevant documents (positive samples), and irrelevant documents (negative samples), define target requirements for recall (the proportion of positive samples retrieved out of all positive samples) and precision (the proportion of positive samples in the query results), and use a "hypothesis–verification–iteration" approach to gradually find the optimal configuration for your business.
Vector Index Mapping Settings
The following is an example of typical vector index mapping settings:
PUT /my-index
{
"mappings": {
"properties": {
"category": {
"type": "keyword"
},
"title": {
"type": "text"
},
"title_vector": {
"type": "dense_vector",
"dims": 768,
"similarity": "cosine",
"index_options": {
"type": "hnsw",
"m": 16,
"ef_construction": 100
}
}
}
}
}
dims - Vector dimensions: Higher dimensions contain richer information and yield higher search accuracy, but also incur higher storage and computation costs. You can start by testing recall/precision at 384 or 768 dimensions. If the results are unsatisfactory, increase the dimensions; if satisfactory, try reducing them.
index setting: The default value is true. If set to false, k-nearest neighbor (kNN) search will not be performed, and only brute-force scanning (script_score) can be performed.
type: Default values vary by version (hnsw in 8.13, int8_hnsw in 8.16, and bbq_hnsw in 9.1.3). You can configure this parameter by referring to the following information based on your vector scale:
hnsw: It is recommended for scenarios where the number of vectors is less than 100 million.
int8_hnsw: It is recommended for scenarios where the number of vectors is between 100 million and 2 billion.
bbq_hnsw: It is recommended for scenarios where the number of vectors is between 1 billion and 10 billion.
bbq_disk: It is recommended for scenarios where the number of vectors is between 1 billion and 100 billion. (diskbbq is supported in ES 9.2 and later, and the cloud edition is expected to be released in Q1.)
m: The default value is 16, which is the maximum number of connections per node in the HNSW graph. A larger m improves recall and query speed but increases indexing time and memory usage. Start with 16 for most scenarios. If you require extremely high recall and can accept longer indexing time and larger index size, you can try increasing it to 32 or higher.
efConstruction: The default value is 100. It represents the number of candidate neighbors examined when connections are established for each new node during HNSW index creation. Increasing this value improves index quality and recall but extends the index creation time. You can start with 100. If your data distribution is complex or you require high precision, you can set it to 200 or higher.
Bulk Vector Writing
1. Rationale
For vector search scenarios, bulk writing is strongly recommended. Bulk writing is a core optimization technique for high-throughput ES scenarios. By reducing network interactions, optimizing disk I/O, and lowering indexing overhead, it can significantly improve data writing efficiency and ensure cluster stability.
2. Bulk Write Settings
When using bulk writes, you can start testing with a batch size of about 500–1,000 documents, monitor cluster CPU and memory load, and gradually increase the batch size until performance no longer improves or requests start being rejected.
POST /_bulk
{ "index" : { "_index" : "my-index", "_id" : "1" } }
{ "category":"cat", "title": "Lazy cat", "title_vector": [0.1, 0.2, ... ] }
{ "index" : { "_index" : "my-index", "_id" : "2" } }
{ "category":"dog", "title": "Walking dog", "title_vector": [0.3, 0.4, ... ] }
3. Other Adjustable Parameters (Optional)
Adjust the refresh interval
refresh_interval defaults to 1s, making newly written data visible for search. However, frequent refreshing generates a large number of small segments, which affects vector index creation and query performance. Therefore, it is recommended to set refresh_interval to a larger value (such as 30s or 60s) or -1 during bulk data import, and restore it after the import is complete. Note: When refresh_interval is -1, newly written data will not be searchable until a manual refresh is performed or the next automatic refresh occurs after restoration.
Temporarily disable replicas
When a large volume of data is written, you can also set number_of_replicas to 0 first, and restore the number of replicas after the write is complete.
PUT /my_index
{
"settings": {
"index": {
"number_of_shards": 10,
"number_of_replicas": 0,
"refresh_interval": -1
}
},
"mappings": {
"properties": {
...
}
}
Vector Search Parameter Settings
The following is a simple vector search query statement:
GET /my-index/_search
{
"size" : 3,
"knn": {
"field": "title_vector",
"query_vector": [-5, 9, -12, ...],
"k": 10,
"num_candidates": 100
}
"fields": ["category"],
"_source": false
}
GET /my-index/_search
{
"size" : 3,
"knn": {
"field": "title_vector",
"query_vector": [-5, 9, -12, ...],
"k": 10,
"num_candidates": 100
}
"_source": {
"excludes": ["title_vector"]
}
}
k
Number of top results to recall at query time. Set it as needed.
num_candidates
Specifies the number of candidate vectors to search on each shard. This value should be greater than or equal to k. Increasing num_candidates improves recall (especially with quantization or filters), but increases query latency. It is generally recommended to set it to 5 to 10 times the value of k to balance recall and performance.
Avoid returning vector fields
Vector fields are typically large. If you do not need to return vector data, refer to the following recommendations:
If you need to recall only a small number of non-text fields, such as IDs (such as keyword, numeric, or other field types with doc_values enabled) or explicitly stored fields (store: true), you can use "fields" to explicitly specify the fields to recall. ES will query doc values or stored values directly, avoiding the overhead of parsing _source for better performance.
If you need to recall a large number of fields or text fields, you can use _source: false, _source: exclude, or _source: ["field1", "field2"] to exclude vector fields, thereby reducing network transfer and serialization overhead.
Vector Quantization and Oversampling
When ES vector search scales to hundreds of millions or even billions of vectors, the challenges of high memory usage and slow computation speed need to be addressed. Striking a balance between cost, performance, and accuracy requires an important optimization policy at this stage: vector quantization. For a detailed introduction to vector quantization and oversampling policies, see Vector Quantization. The following is a code example of vector quantization and oversampling:
PUT /my-quantized-index
{
"mappings": {
"properties": {
"title": {
"type": "text",
},
"title_vector": {
"type": "dense_vector",
"dims": 768,
"index": true,
"similarity": "cosine",
"index_options": {
"type": "int8_hnsw",
"m": 16,
"ef_construction": 100
}
}
}
}
}
GET my-quantized-index/_search
{
"knn": {
"field": "title_vector",
"query_vector": [0.15, 0.50, ..., 0.05],
"k": 10,
"num_candidates": 100,
"rescore_vector": {
"oversample": 2.0
}
}
}
File System Cache Warm-Up
1. Background
ES relies on the operating system's page cache to accelerate reading of on-disk index files. By default, index files are loaded into the cache only when accessed. This can cause a problem: when the host operating system restarts, the page cache is cleared, and search performance may degrade significantly while the cache is being "warmed up" again, because the system needs to frequently read data from disk.
2. Solution
By setting index.store.preload, you can proactively preload specified key data files into memory when the index is opened, thereby avoiding costly disk I/O during subsequent searches. This is especially useful for frequently searched "hot" indexes.
Note:
1. If the file system cache capacity is insufficient to hold all data, preloading data into the page cache too early on too many indexes or files can slow down searches. Use with caution.
2. If you use a quantized index, only preload the relevant quantized values and index structures, such as the HNSW graph. Preloading raw vectors (vec) is unnecessary and may be counterproductive, as preloading raw vectors can cause the operating system to evict important index structures from the cache.
3. File Types Related to Vector Search
For which index files index.store.preload actually loads, refer to the file extension descriptions below:
File Extension | File Purpose |
vex | Stores HNSW graph structure files. |
vec | All non-quantized vector values. Includes all element types: float, byte, and bit. |
veq | Quantized vectors for quantized indexes: int4 or int8. |
veb | Binary vectors for quantized indexes: bbq. |
vem, vemf, vemq, vemb | Metadata, usually very small and does not require preloading. |
4. Preloading Settings
index.store.preload is a static setting, meaning it can only be specified at index creation time or in the configuration file, and cannot be dynamically modified after index creation. For non-quantized scenarios, we generally recommend simply setting it to ve*. For quantized scenarios, you can specify which file types to preload individually (for example, by doing so, .vex and .vec are excluded from preloading).
We generally recommend also setting mmapfs, which maps index files directly into the process's virtual memory address space. Thereafter, ES can read and write files as if accessing ordinary memory (actual data loading is handled by the operating system via page fault interrupts). Using mmapfs together with preload achieves efficient queries and a smooth experience without cold starts, but increases node restart time.
PUT /my-preloaded-index
{
"settings": {
"index.store.preload": ["ve*"],
"index.store.type": "mmapfs"
},
"mappings": {
...
}
}
If you want this to apply to all new indexes created on the cluster, you can set it in the global configuration file config/elasticsearch.yml:
index.store.preload: ["ve*"] // Preload specific data files into the page cache.
index.store.type: mmapfs // Map index files to the process's virtual memory address space.
For existing indexes, you should close them before modifying this setting:
POST /my-existing-index/_close
PUT /my-existing-index/_settings
{
"index.store.preload": ["ve*"]
"index.store.type": "mmapfs"
}
POST /my-existing-index/_open
Reduction of the Number of Index Segments
ES shards are composed of segments, which are internal storage elements within an index. For approximate kNN search, ES stores the vector values of each segment as a separate HNSW graph, so kNN search should examine each segment. Although kNN search parallelization significantly speeds up searches across multiple segments, kNN search can still be several times faster when the number of segments is small. By default, ES periodically merges smaller segments into larger ones through background merging. If this is insufficient, you can take the following explicit steps to reduce the number of index segments.
1. Increasing the Maximum Segment Size
ES provides many adjustable parameters to control the merge process. An important one is index.merge.policy.max_merged_segment, which controls the maximum size of segments created during merging. Increasing this value can reduce the number of segments in the index. This value defaults to 5 GB, which may be too small for high-dimensional vectors. It is recommended to increase it to 10 GB or 20 GB to help reduce the number of segments. This is a static setting, so for existing indexes, you need to close the index before configuring it:
POST /my-index/_close
PUT /my-index/_settings
{
"merge.policy.floor_segment": "300mb",
"merge.policy.max_merged_segment": "10g"
}
POST my-index/_open
2. Creating Large Segments During Bulk Indexing
A common practice is to perform the initial bulk writing and index creation first. In addition to relying on periodic background segment merging, you can also adjust index settings to encourage ES to create larger initial segments:
Ensure that no searches are performed during the bulk upload, and set index.refresh_interval to -1. This prevents refresh operations and avoids generating additional segments.
Allocate a large index buffer to ES so it can receive more documents before flushing. You can set indices.memory.index_buffer_size to 10% of the heap size, which is usually sufficient for larger heap sizes such as 32 GB. To allow the full index buffer to be used, you should also set index.translog.flush_threshold_size to be smaller than indices.memory.index_buffer_size.
3. Forcing a Segment Merge (Force Merge)
As mentioned earlier, vector indexes are created on each segment. The more segments there are, the more graphs need to be searched at query time, resulting in worse performance. It is recommended to perform a force merge after bulk writing data, or on indexes where data is no longer frequently updated. For a read-only index, merge to one segment: POST /my-index/_forcemerge?max_num_segments=1. Note: Force merge is a resource-intensive operation and should be performed during off-peak hours.
Pre-Filter Optimization for Improved Hybrid Search Performance
The following is a typical pre-filter search:
GET /my-index/_search
{
"knn":{
"field":"title_vector",
"query_vector": [-5, 9, -12, ...],
"k": 10,
"num_candidates": 100,
"filter":{
"term":{
"category":"cat"
}
}
}
}
In open-source ES, pre-filtering performs scalar filtering first, obtaining all qualifying DocIDs via the inverted index chain. It then performs a kNN query, checking each neighboring vector against the DocID set generated by scalar filtering until the top N results are obtained. In this case, if the scalar filtering result set is very large, for example, a query for gender "male" could yield hundreds of millions of results, query performance will degrade significantly.
To address the performance issue of pre-filtering in hybrid search, Tencent Cloud ES has developed proprietary pre-filter optimization, which can be enabled via optimize_prefilter_enable. This is a dynamic parameter and does not require reopening the index.
PUT /my-index/_settings
{
"index.query.knn.optimize_prefilter_enable": "true"
}
GPU-Powered Model Inference
Vector generation (embedding) requires substantial computation. The community edition of ES supports machine learning nodes dedicated to vector model inference, but does not yet support GPU inference.
The good news is that Tencent Cloud ES has supported GPU inference on machine learning nodes since version 8.16, which can dramatically accelerate vector inference speed. Inference performance is 30 times higher than that of CPU, and the price-performance ratio is over 10 times better than that of CPU. Therefore, for large-scale vector generation, it is recommended to use Tencent Cloud ES machine learning nodes for GPU inference. This is done by enabling machine learning nodes when purchasing an ES cluster and selecting a GPU instance type. For existing clusters, you can enable machine learning nodes by adjusting the configuration.
Special Note on Using script_score with Caution for Vector Search
ES's script_score query is a powerful advanced feature popular among ES developers in text search scenarios. It allows you to use scripts to customize document scoring, combining text relevance with business metrics for complex sorting logic. However, when script_score is used for vector search, it performs brute-force search (even if the index type is set to hnsw in the index mapping). Unless working with small datasets (< 100,000) or scenarios requiring exact results, we generally do not recommend using script_score for vector search. For details, see Special Note on Using script_score Flexibly and with Caution for Brute-Force Vector Search.
GET my-index/_search
{
"query": {
"script_score": {
"query" : {
"bool" : {
"filter" : {
"term" : {
"status" : "published"
}
}
}
},
"script": {
"source": "cosineSimilarity(params.query_vector, 'title_vector') + 1.0",
"params": {
"query_vector": [4, 3.4, -0.2]
}
}
}
}
}