Structured Outputs with OpenAI-Compatible APIs: A JSON Schema Guide

A field-level guide to strict JSON Schema outputs across Responses and Chat Completions, including validation layers, failure handling, and route compatibility tests.

Structured Outputs let an application request JSON that follows a defined schema instead of merely asking the model to “return JSON.” With an OpenAI-compatible API, the same schema can be used through Responses or Chat Completions, but the request fields are different—and compatibility must be verified for the selected model and route.

The safe production pattern has three layers: use strict schema output when the model supports it, parse and validate the returned JSON in the application, and then apply business rules to the values. Schema compliance removes many formatting failures; it does not prove that the model’s answer is factually or semantically correct.

Structured Outputs are not the same as JSON mode

There are three common ways to ask a model for JSON-like output.

Method Produces valid JSON Enforces your schema Typical use
Prompt only Not guaranteed No Prototypes where parsing failures are acceptable
JSON mode Yes, when supported and completed No Flexible JSON where the application validates the shape
Structured Outputs with strict: true Yes, subject to completion and refusal handling Yes, within the supported JSON Schema subset Typed extraction and application workflows

The OpenAI Structured Outputs guide recommends Structured Outputs instead of JSON mode when the selected model and endpoint support it. The same guide also distinguishes two different jobs:

  • Use a structured response format when the model should answer the user with a predictable data object.
  • Use function calling when the model should request an action from your application.

This article focuses on the first job. A later tool loop can reuse strict argument schemas, but its call IDs and result messages follow a different contract.

Define one schema before choosing the endpoint

Consider a support-triage workflow. The application needs four fields:

  • category: one of billing, technical, or account;
  • priority: an integer from 1 through 3;
  • requires_human: a boolean;
  • summary: a short description for the support queue.

The JSON Schema is:

{
  "type": "object",
  "properties": {
    "category": {
      "type": "string",
      "enum": ["billing", "technical", "account"]
    },
    "priority": {
      "type": "integer",
      "minimum": 1,
      "maximum": 3
    },
    "requires_human": {
      "type": "boolean"
    },
    "summary": {
      "type": "string"
    }
  },
  "required": ["category", "priority", "requires_human", "summary"],
  "additionalProperties": false
}

Making every property required and setting additionalProperties to false produces a stable object for downstream code. It does not make every possible JSON Schema keyword portable. OpenAI supports a documented subset, and another OpenAI-compatible provider may support a different subset or no strict schema mode at all.

Send the schema with the Responses API

The Responses API places the response schema under text.format. Set a Modelflare API key and choose a model that is currently listed for your API key and has been verified for Structured Outputs.

export MODELFLARE_API_KEY="<YOUR_API_KEY>"
export MODEL_ID="<MODEL_WITH_VERIFIED_STRUCTURED_OUTPUT_SUPPORT>"

curl -sS https://modelflare.dev/v1/responses \
  -H "Authorization: Bearer $MODELFLARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"$MODEL_ID\",
    \"input\": \"The customer was charged twice and wants a refund.\",
    \"text\": {
      \"format\": {
        \"type\": \"json_schema\",
        \"name\": \"support_ticket\",
        \"schema\": {
          \"type\": \"object\",
          \"properties\": {
            \"category\": {
              \"type\": \"string\",
              \"enum\": [\"billing\", \"technical\", \"account\"]
            },
            \"priority\": {
              \"type\": \"integer\",
              \"minimum\": 1,
              \"maximum\": 3
            },
            \"requires_human\": { \"type\": \"boolean\" },
            \"summary\": { \"type\": \"string\" }
          },
          \"required\": [\"category\", \"priority\", \"requires_human\", \"summary\"],
          \"additionalProperties\": false
        },
        \"strict\": true
      }
    }
  }"

In a Responses response, generated content appears in typed output items. SDKs may expose convenience helpers such as aggregated output text or parsed output, but an HTTP client should not assume that the structured object is a top-level field. Read the completed response object, locate its text output, then parse that text as JSON.

Send the same schema with Chat Completions

Chat Completions places the response schema under response_format.json_schema. The schema body is unchanged; only the wrapper differs.

curl -sS https://modelflare.dev/v1/chat/completions \
  -H "Authorization: Bearer $MODELFLARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"$MODEL_ID\",
    \"messages\": [
      {
        \"role\": \"user\",
        \"content\": \"The customer was charged twice and wants a refund.\"
      }
    ],
    \"response_format\": {
      \"type\": \"json_schema\",
      \"json_schema\": {
        \"name\": \"support_ticket\",
        \"schema\": {
          \"type\": \"object\",
          \"properties\": {
            \"category\": {
              \"type\": \"string\",
              \"enum\": [\"billing\", \"technical\", \"account\"]
            },
            \"priority\": {
              \"type\": \"integer\",
              \"minimum\": 1,
              \"maximum\": 3
            },
            \"requires_human\": { \"type\": \"boolean\" },
            \"summary\": { \"type\": \"string\" }
          },
          \"required\": [\"category\", \"priority\", \"requires_human\", \"summary\"],
          \"additionalProperties\": false
        },
        \"strict\": true
      }
    }
  }"

For Chat Completions, the JSON text is normally returned in choices[0].message.content. Parse the content rather than treating the string as an already typed object.

The two wrappers can be summarized as follows:

Purpose Responses API Chat Completions
Schema container text.format response_format.json_schema
Format type text.format.type response_format.type
Schema name text.format.name response_format.json_schema.name
Schema body text.format.schema response_format.json_schema.schema
Strict flag text.format.strict response_format.json_schema.strict
Result location Typed output items / output text helper choices[0].message.content

Modelflare’s OpenAI compatibility layer has explicit conversion logic for these two wrappers on supported OpenAI/Codex routes. That conversion does not create Structured Outputs support in an upstream model that lacks it. For other model families served as raw Chat Completions pass-through, the upstream provider’s exact request contract remains the source of truth.

Validate structure and meaning separately

Suppose the model returns:

{
  "category": "billing",
  "priority": 2,
  "requires_human": true,
  "summary": "Customer reports a duplicate charge and requests a refund."
}

This object satisfies the schema, but the application should still perform two distinct checks.

Structural validation

Use a JSON Schema validator or a typed SDK helper to confirm:

  • the object contains every required property;
  • no unexpected property is present;
  • each value has the required type;
  • the category is in the allowed enum;
  • the priority is within the permitted range.

Even when the provider promises strict adherence, application-side validation protects the rest of the system from unsupported routes, integration mistakes, truncated content, and future contract changes.

Semantic and business validation

The schema cannot determine whether the customer was actually charged twice, whether priority 2 is correct, or whether a human must approve a refund. Those decisions require source data and application policy.

For high-impact workflows, treat the model object as a proposed classification. Compare it with authoritative records, enforce permissions and financial limits in deterministic code, and retain a safe audit reference. Never let a structurally valid model object bypass an authorization, billing, or security boundary.

Handle incomplete and exceptional output

A production parser needs more than a happy-path JSON.parse call.

Refusals

A model may refuse a request for safety reasons. OpenAI’s structured-output contract makes refusals distinguishable from normal schema output. Check the response status and refusal representation before looking for the JSON object; do not report a refusal as a schema parsing error.

Truncation and output limits

If generation stops before the object is complete, no schema can repair the missing suffix. Check the endpoint’s completion status and stop reason. Set an output limit that is sufficient for the largest valid object, while keeping the schema and requested content bounded.

Unsupported schema keywords

Strict output implementations commonly support only a subset of JSON Schema. Start with objects, arrays, primitive types, enums, required fields, and explicit additional-property behavior. Check the current provider documentation before using references, recursive shapes, complex unions, or advanced validation keywords.

First-use schema latency

Some providers preprocess and cache a new schema. The first request for a new schema can therefore take longer than later requests. Reuse stable, versioned schemas instead of generating a unique schema for every request, and separate first-use measurements from steady-state latency comparisons.

Model or route incompatibility

An OpenAI-compatible endpoint can accept ordinary Chat Completions while rejecting json_schema, ignoring strict, or forwarding the fields to a model that does not support them. A 200 response containing JSON is still not proof of strict adherence. Test invalid and edge-case inputs and validate the resulting object.

Run a compatibility test before production

Use a small matrix for every model and route you intend to enable.

Test Evidence to record
Minimal required object Status, model, route, parsed object, validation result
Every enum value Whether each allowed value can be produced and parsed
Missing information Whether the model uses a bounded fallback value or asks for clarification
Safety-triggering input Refusal representation and application handling
Low output limit Completion status and truncation handling
Unsupported schema feature Explicit error versus silently ignored constraint
Responses and Chat wrappers Whether both produce equivalent application objects
Repeated stable schema First-use versus steady-state timing

Do not compare models with different schemas, prompts, regions, streaming modes, or output limits and then attribute the result only to model quality. Record the exact test date because provider behavior and model availability can change.

Choose the endpoint after testing the workflow

Use Responses when the application already works with typed output items, Responses streaming events, or a broader Responses tool workflow. Use Chat Completions when the application has a stable message-based integration and the selected route supports its response_format contract. The broader decision is covered in Responses API vs Chat Completions.

Whichever endpoint you select:

  1. keep one versioned source for the JSON Schema;
  2. translate that schema into the endpoint’s wrapper without changing its meaning;
  3. validate the completed response before using it;
  4. apply deterministic business rules after structural validation;
  5. monitor parsing, refusal, truncation, and compatibility failures separately.

For initial client migration, start with the OpenAI-Compatible API Guide. For event parsing and incomplete streams, use the AI API Streaming Guide. Check Models & Pricing for the current model and group surface, then verify Structured Outputs on the exact route your API key will use.

Structured Outputs are most useful when they narrow a model response into a dependable application interface. They are not a substitute for factual verification, authorization, or billing logic—and “OpenAI-compatible” remains a claim to test feature by feature, not a guarantee to infer from the Base URL.