> ## Documentation Index
> Fetch the complete documentation index at: https://docs.inputapi.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Image understanding

> Send an image as conversation input and receive a text description.

This guide covers image input with text output, not image generation. Select a model supporting both image input and Chat Completions. A model ID in the list alone does not establish vision support.

## Two image sources

* HTTPS image URL: use an image address directly accessible to the gateway or upstream. Browser access does not prove server access; login cookies, temporary authorization, or redirects may cause failures.
* Base64 data URL: read a local image and encode it with its actual MIME type, such as `data:image/png;base64,...`. Public image hosting is unnecessary, but the request body grows.

Do not use a local path or `file://` as a remote image URL. Remote access, formats, dimensions, and image counts depend on the deployment and model.

## Runnable example: send a local image

Follow the [Python SDK guide](/en/guides/python.md), then set one additional environment variable:

```bash theme={null}
export API_IMAGE_PATH='/absolute/path/to/image.png'
```

Replace the path with an image you are authorized to submit. Save as `vision.py` and run `python vision.py`. The command sends the image to the configured API service, which may forward it upstream. This example accepts PNG/JPEG up to 5 MiB; that is an example-specific limit, not a gateway or model allowance.

```python theme={null}
import base64
import os
from pathlib import Path
from openai import OpenAI, APIConnectionError, APIStatusError

# This example's local limit is not the gateway or model's documented limit.
image_limit = 5 * 1024 * 1024
with Path(os.environ["API_IMAGE_PATH"]).open("rb") as image_file:
    image = image_file.read(image_limit + 1)
if not image or len(image) > image_limit:
    raise SystemExit("Choose a nonempty image up to 5 MiB for this example.")
if image.startswith(b"\x89PNG\r\n\x1a\n"):
    mime_type = "image/png"
elif image.startswith(b"\xff\xd8\xff"):
    mime_type = "image/jpeg"
else:
    raise SystemExit("This example accepts PNG or JPEG images only.")
image_url = f"data:{mime_type};base64,{base64.b64encode(image).decode('ascii')}"

with OpenAI(
    base_url=os.environ["API_BASE_URL"].rstrip("/"),
    api_key=os.environ["API_KEY"],
    timeout=60.0,
    max_retries=0,
) as client:
    try:
        result = client.chat.completions.create(
            model=os.environ["API_MODEL"],
            messages=[{
                "role": "user",
                "content": [
                    {"type": "text", "text": "Describe this image in one sentence."},
                    {"type": "image_url", "image_url": {"url": image_url}},
                ],
            }],
        )
        choice = result.choices[0] if result.choices else None
        if (choice is None or choice.finish_reason != "stop"
                or choice.message.refusal or choice.message.tool_calls
                or not choice.message.content):
            raise SystemExit("Image request did not produce a complete text reply.")
        print(choice.message.content)
    except APIStatusError as error:
        raise SystemExit(f"Request failed (HTTP {error.status_code}). Check the request logs.")
    except APIConnectionError:
        raise SystemExit("Connection failed. Check the API address and network.")
```

The example identifies MIME from the file header; it does not validate full image decodability. Services can additionally limit pixels, encoding, and total request size. Base64 payloads are larger than the original file. Avoid logging complete data URLs.

## Protocol field differences

| Item | Chat Completions | Responses |
| - | - | - |
| Text block | `type: text`, text in `text` | `type: input_text`, text in `text` |
| Image block | `type: image_url`, `image_url` is an object containing `url` | `type: input_image`, `image_url` is a string |

Switching to Responses also requires changing the input structure. Optional fields such as `detail` require model support; validate a request without extra options first.

## Python: vision with Responses

Save as `responses-vision.py`, use the Python environment and `API_IMAGE_PATH` from above, and run `python responses-vision.py`. This submits the image to your configured service; choose a model and channel supporting Responses image input. The example keeps the same local 5 MiB file limit, uses `input_text` and `input_image`, and does not require a separate Files API upload.

```python theme={null}
import base64
import os
from pathlib import Path
from openai import OpenAI, APIConnectionError, APIStatusError

# This example's local limit is not the gateway or model's documented limit.
image_limit = 5 * 1024 * 1024
with Path(os.environ["API_IMAGE_PATH"]).open("rb") as image_file:
    image = image_file.read(image_limit + 1)
if not image or len(image) > image_limit:
    raise SystemExit("Choose a nonempty image up to 5 MiB for this example.")
if image.startswith(b"\x89PNG\r\n\x1a\n"):
    mime_type = "image/png"
elif image.startswith(b"\xff\xd8\xff"):
    mime_type = "image/jpeg"
else:
    raise SystemExit("This example accepts PNG or JPEG images only.")
image_url = f"data:{mime_type};base64,{base64.b64encode(image).decode('ascii')}"

with OpenAI(
    base_url=os.environ["API_BASE_URL"].rstrip("/"),
    api_key=os.environ["API_KEY"],
    timeout=60.0,
    max_retries=0,
) as client:
    try:
        result = client.responses.create(
            model=os.environ["API_MODEL"],
            input=[{
                "role": "user",
                "content": [
                    {"type": "input_text", "text": "Describe this image in one sentence."},
                    {"type": "input_image", "image_url": image_url},
                ],
            }],
        )
        if result.status != "completed" or not result.output_text.strip():
            raise SystemExit("Image request did not produce a complete text reply.")
        for item in result.output:
            if item.type == "reasoning":
                continue
            if (item.type != "message" or item.status != "completed"
                    or any(part.type != "output_text" for part in item.content)):
                raise SystemExit("Image request did not produce a complete text reply.")
        print(result.output_text)
    except APIStatusError as error:
        raise SystemExit(f"Request failed (HTTP {error.status_code}). Check the request logs.")
    except APIConnectionError:
        raise SystemExit("Connection failed. Check the API address and network.")
```

As in the Chat example, a file signature check is not full image decoding. The response must be complete with nonempty text; refusals, incomplete messages, and tool items are not published as image descriptions.

## Troubleshooting

| Symptom | Check |
| - | - |
| Rejected content type or parameter | Model vision support, endpoint protocol, and channel conversion support |
| Unreadable URL | Server reachability, authorization, URL expiry, and image content type |
| 413 or oversized request | Proxy, gateway, and upstream limits; resize or compress appropriately in the application |
| Empty or truncated text | Completion reason, refusal information, and model response format |
| Inaccurate interpretation | Image clarity, text size, and question specificity; verify manually when needed |

[HTTP integration](/en/guides/http.md) · [Errors and retries](/en/guides/errors.md)

Reference: [OpenAI image inputs](https://developers.openai.com/api/docs/guides/images-vision).

[For generating or editing images, see image generation](/en/guides/image-generation.md).

[简体中文](/zh/guides/vision.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.