Calling Third-Party Inference Services
Download
포커스 모드
폰트 크기
ES can integrate third-party inference services through its inference capabilities, enabling flexible and efficient calls of various required capabilities. For more details, refer to the related document.
Step 1: Enabling Node Outbound Access
To enable ES to access third-party services on the public network, you need to enable Node Outbound Access first.
Step 2: Creating an Inference Endpoint
The following is a reference template for creating a custom inference:
PUT _inference/text_embedding/test-text-embedding{"service": "custom","service_settings": {"secret_parameters": {"api_key": "<some api key>"},"url": "...endpoints.huggingface.cloud/v1/embeddings","headers": {"Authorization": "Bearer ${api_key}","Content-Type": "application/json"},"request": "{\\"input\\": ${input}}","response": {"json_parser": {"text_embeddings":"$.data[*].embedding[*]"}}}}
For example, the following is an example of using the SiliconFlow embedding service. You can refer to the template and fill in your API key and model name:
PUT _inference/text_embedding/test-text-embedding{"service": "custom","service_settings": {"secret_parameters": {"api_key": "sk-your_actual_API_key"},"url": "https://api.siliconflow.cn/v1/embeddings","headers": {"Authorization": "Bearer sk-your_actual_API_key","Content-Type": "application/json;charset=utf-8"},"request": "{\\"input\\": ${input}, \\"model\\": \\"BAAI/bge-large-zh-v1.5\\"}","response": {"json_parser": {"text_embeddings": "$.data[*].embedding[*]"}}}}
After execution, it indicates that the creation is successful if the following information is returned:
Step 3: Verifying the Inference Service
You can verify the inference service using the following statement. If successful, the embedding result is returned.
POST _inference/text_embedding/test-text-embedding{"input": "hello, this is an inference service"}
피드백