The ES Inference Service is a core AI capability introduced in Elasticsearch 8 and later versions. It allows you to directly deploy models for inference within an ES cluster or call pre-deployed online inference services through a unified inference API, providing native support for semantic search, vector search, and Retrieval-Augmented Generation (RAG) applications.
Key Strengths
Flexible options: It enables you to deploy your own models through machine learning nodes, directly call out-of-the-box atomic services, or directly call third-party inference services.
Native integration: It seamlessly integrates with ES ingest pipelines, vector fields, and search APIs.
Preparations
Environment Requirements
Elasticsearch version: 8.16 or later. It is recommended to use the highest stable version whenever possible.
Machine learning node: If you deploy models using machine learning nodes, ensure that at least one machine learning node (dedicated to model inference) is configured in the cluster.
Model Preparation
Select an appropriate model type based on your business requirements. For example:
Text embedding model: used to generate text vectors (such as bge-m3 and .multilingual-e5-small)
Reranking model: used for fine ranking of search results.
Large language model: used for text generation and Q&A.
Note:
When selecting a model, you should focus on its parameters. For example, different embedding models determine the vector dimensions, supported languages, and other aspects.