Embeddings and retrieval

Generate text vectors, preserve input mapping, and build a small retrieval example.

Embeddings turn text into vectors for semantic retrieval, clustering, and related tasks. The embeddings endpoint returns vectors; it does not create an index or store documents. A model in the model list may not support embeddings. Confirm an available embedding model in your deployment.

Python: batch vectors and a small retrieval example

Set API_BASE_URL (including /v1), API_KEY, and an embedding model’s API_MODEL using the Python SDK guide. Save as embeddings.py and run python embeddings.py. It sends one query and two candidate texts together, restores the input mapping using each response index, normalizes vectors, computes cosine similarity, and prints ranked source text.

This is a small in-memory demonstration for models using the same encoding mode for queries and documents. Models requiring query/document prefixes or task types must encode each according to their instructions; do not apply this example unchanged. Actual ranking depends on the model; no document is guaranteed to rank first.

import json
import math
import os
from openai import OpenAI, APIConnectionError, APIStatusError

query = "How can I reset my password?"
documents = [
    "Invoices are available in the billing dashboard.",
    "Use the password reset link on the sign-in page.",
]
inputs = [query, *documents]

with OpenAI(base_url=os.environ["API_BASE_URL"].rstrip("/"),
            api_key=os.environ["API_KEY"], timeout=60.0, max_retries=0) as client:
    try:
        result = client.embeddings.create(
            model=os.environ["API_MODEL"], input=inputs, encoding_format="float",
        )
        vectors = {}
        dimensions = None
        if not isinstance(result.data, list) or len(result.data) != len(inputs):
            raise ValueError("Expected one vector per input.")
        for item in result.data:
            index = getattr(item, "index", None)
            vector = getattr(item, "embedding", None)
            if type(index) is not int or not 0 <= index < len(inputs) or index in vectors:
                raise ValueError("Invalid or duplicate input index.")
            if not isinstance(vector, list) or not vector:
                raise ValueError("Expected a nonempty float vector.")
            if any(type(value) not in (int, float) or not math.isfinite(value) for value in vector):
                raise ValueError("Invalid vector value.")
            if dimensions is not None and len(vector) != dimensions:
                raise ValueError("Inconsistent vector dimensions.")
            dimensions = len(vector)
            norm = math.hypot(*vector)
            if not math.isfinite(norm) or norm == 0:
                raise ValueError("Cannot normalize this vector.")
            vectors[index] = [value / norm for value in vector]
        matches = []
        for index, document in enumerate(documents):
            score = math.fsum(a * b for a, b in zip(vectors[0], vectors[index + 1]))
            matches.append({"index": index, "score": score, "text": document})
        matches.sort(key=lambda item: item["score"], reverse=True)
        print(json.dumps(matches, ensure_ascii=False, allow_nan=False))
    except APIStatusError as error:
        raise SystemExit(f"Request failed (HTTP {error.status_code}). Check model access and request limits.")
    except APIConnectionError:
        raise SystemExit("Connection failed. Check the API address and network.")
    except (ValueError, TypeError, OverflowError):
        raise SystemExit("Invalid embedding response; check indices, dimensions, and numeric values.")

Inputs, encoding, and dimensions

• This guide uses a string or string array. Upstream OpenAI also accepts token-ID arrays, but the gateway’s text extraction and some channel conversions only handle strings. Token arrays are not a portable cross-channel input.
• The example explicitly requests encoding_format: float. It checks nonempty finite numeric vectors, nonzero norms, consistent dimensions, and complete unique indices. Array position in data does not replace index.
• Set dimensions only with model and channel support; do not truncate vectors arbitrarily. Base64 output is also capability-dependent. For example, the current Jina adaptor removes encoding_format, so inspect the actual response type.
• Text length, batch size, and total-token limits depend on the deployment and model. Character counts are not token counts. Chunk long documents and batch within limits. Both endpoints use synchronous JSON for the requests described here, not SSE.

From the demo to a retrieval application

Usually generate and store document vectors offline, then encode only the query at request time. Record document/chunk IDs, model version, dimensions, and preprocessing. Queries and documents must use compatible vector spaces. Equal dimensions do not make different models interchangeable; changing the model or encoding usually requires reindexing. Confirm normalization and the index distance metric; this example normalizes vectors itself for cosine similarity.

Use your application or a vector database to retrieve candidates, then send their text to Rerank. Limit retrieval and reranking to documents accessible to the current user; do not delegate access control to a model. Rerank cannot see documents you did not send. Connecting a production vector database is outside this demo.

Troubleshooting

For 400 errors, inspect input types, length, dimensions, and encoding options. For 401/403, check credentials and model access; back off on 429 as described in error handling. Reject indexing when HTTP 200 contains missing vectors, duplicate indices, or inconsistent dimensions, and investigate using the request identifier. Automatic retries are disabled; examples do not print keys or complete error bodies.

Field semantics: OpenAI Embeddings API. Cross-channel behavior depends on this project’s adaptors and the deployment.

Next: combine retrieval, optional reranking, and cited answers in the RAG guide.

简体中文