Slow When Invoking Large Language Model API in China? A Practical Guide to 4-Step Speed Testing and Locating Cross-Border Latency

Author’s note: Use curl and the SDK to time DNS, TLS, first token, and total latency, then run through a 4-step method to figure out whether a slow Large Language Model API call from mainland China is bottlenecked by the “cross-border link” or by “model inference.”

“Why is this API so slow?” That’s the line developers in China say most often when they first integrate a Large Language Model API. The thing is, “slow” is only a symptom. It might come from DNS resolution, a cross-border TLS handshake, or model queuing — or it might just be the model thinking hard. This article walks you through a 4-step latency profiling method that splits a single request into its component parts, so you can pin down exactly why calling a Large Language Model API from mainland China is slow.

This isn’t a mystical “just try a different node” kind of guide. It’s a reproducible troubleshooting process assembled from real integration support tickets. We get asked all the time: if my service is deployed on a US server, what should I put in as the base URL? Will access from mainland China stutter because of the cross-border hop? Can large results like images and videos make it back in one piece? We’ll answer each of these below.

What you’ll get out of this: By the end, you’ll know how to time each phase with curl and the Python SDK, tell network-link cost apart from model-inference cost, and pick the right base_url for your deployment region.

Key Points: Why Large Language Model API Calls Are Slow from China

Before you start benchmarking, get one basic idea straight: the total time for a single Large Language Model API call is the sum of network time and model time. Cross-border latency only affects the former, and much of the “slowness” people perceive actually comes from the latter.

What Makes Up Cross-Border Latency

When an HTTPS request leaves your server, it goes through DNS resolution, the TCP three-way handshake, and the TLS handshake before the request body is even sent. In a cross-border scenario, both the TCP and TLS handshakes require multiple round trips, and the round-trip delay caused by physical distance gets multiplied several times over — that’s where “slow connection setup” comes from. The good news: connection setup usually happens only on the first request, and once the connection is reused, this cost drops sharply.

Once the request reaches the server, the model inference phase begins. For chat models, the wait before the first token appears depends on model size, input length, and current load; after the first token, output speed depends on the model’s generation rate. Image and video generation work differently — they typically return large results synchronously, so the bandwidth and stability of the return path directly determine whether the result arrives intact.

How to Tell Whether the Network or the Model Is Slow

The rule of thumb is simple: slow connection setup points to the network; slow time to first token points to the model and your input; slow and erratic output points to the return path and response body size. Measure these three sets of numbers, and most “slow” problems can be diagnosed in a matter of minutes.

4-Step Speed Test to Locate Cross-Border Latency and Get Started Fast

The examples below all use an OpenAI-compatible interface. Whether your service is deployed domestically or overseas, you only need to change the base_url to connect — the rest of the code stays the same.

Step 1: Break Down Cross-Border Connection Setup Time with curl

curl’s built-in -w parameter can output timing for each stage, making it the fastest tool for troubleshooting cross-border latency. I recommend hitting a lightweight endpoint like /v1/models to avoid model inference time interfering with network measurements.

export API_KEY="YOUR_API_KEY"

curl -o /dev/null -s \
  -H "Authorization: Bearer $API_KEY" \
  -w "DNS lookup:   %{time_namelookup}s\nTCP connect:   %{time_connect}s\nTLS handshake:   %{time_appconnect}s\nFirst byte:    %{time_starttransfer}s\nTotal time:    %{time_total}s\n" \
  https://api.apiyi.com/v1/models

One thing to watch out for: each value curl outputs is a cumulative timestamp measured from the start of the request, not the standalone duration of a single stage. Here’s what each field means and how to calculate it:

I recommend running it 5 to 10 times in a row and checking whether the values are stable. If the TCP and TLS stage timings fluctuate a lot, the link quality is unstable. If the values are stable and within an acceptable range, the network is probably not the main bottleneck.

Step 2: Measure First Token Latency with SDK Streaming Calls

Once the connection is solid, the next thing to check is how fast the model starts responding. Time to first token (TTFT) is the most perceptible metric for chat-based applications—users stare at the screen waiting for that first character, and every second feels like forever.

import time
import openai

client = openai.OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.apiyi.com/v1"  # Works for both domestic and overseas deployments
)

start = time.perf_counter()
stream = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Introduce yourself in one sentence"}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(f"First token latency: {time.perf_counter() - start:.2f}s")
        break

Once you have TTFT, subtract the connection time from Step 1. What’s left is basically the model’s queueing and inference time. If that part accounts for the vast majority, then network optimization won’t help much. You’re better off switching to a faster model, shortening the input context, or enabling streaming output.

Step 3: Multi-Round Statistics for Overall Latency and Large Response Body Delivery

A single measurement is easily thrown off by occasional fluctuations. For a production evaluation, you should run multiple rounds and pay attention to the median and max values. The complete script below automatically measures DNS, TCP, TLS, TTFT, and overall latency, then prints a summary.

import socket
import ssl
import statistics
import time
from urllib.parse import urlparse

import openai

BASE_URL = "https://api.apiyi.com/v1"  # Unified access endpoint
API_KEY = "YOUR_API_KEY"
MODEL = "gpt-4o-mini"
ROUNDS = 5


def measure_handshake(url: str) -> dict:
    """Measure DNS, TCP, and TLS latency in segments"""
    host = urlparse(url).hostname
    t0 = time.perf_counter()
    ip = socket.gethostbyname(host)
    t1 = time.perf_counter()
    sock = socket.create_connection((ip, 443), timeout=10)
    t2 = time.perf_counter()
    ctx = ssl.create_default_context()
    tls = ctx.wrap_socket(sock, server_hostname=host)
    t3 = time.perf_counter()
    tls.close()
    return {"dns": t1 - t0, "tcp": t2 - t1, "tls": t3 - t2}


def measure_llm(client: openai.OpenAI) -> dict:
    """Measure first token latency and overall latency"""
    start = time.perf_counter()
    ttft = None
    chars = 0
    stream = client.chat.completions.create(
        model=MODEL,
        messages=[{"role": "user", "content": "Write a ~200-word product introduction"}],
        stream=True,
    )
    for chunk in stream:
        if chunk.choices and chunk.choices[0].delta.content:
            if ttft is None:
                ttft = time.perf_counter() - start
            chars += len(chunk.choices[0].delta.content)
    total = time.perf_counter() - start
    return {"ttft": ttft or total, "total": total, "chars": chars}


def summarize(name: str, values: list) -> None:
    print(f"{name:<8} median {statistics.median(values):.3f}s  "
          f"max {max(values):.3f}s  min {min(values):.3f}s")


def main():
    client = openai.OpenAI(api_key=API_KEY, base_url=BASE_URL, timeout=120)
    records = {"dns": [], "tcp": [], "tls": [], "ttft": [], "total": []}

    for i in range(ROUNDS):
        hs = measure_handshake(BASE_URL)
        llm = measure_llm(client)
        for k in ("dns", "tcp", "tls"):
            records[k].append(hs[k])
        records["ttft"].append(llm["ttft"])
        records["total"].append(llm["total"])
        print(f"Round {i + 1} complete, output {llm['chars']} chars")

    print("\n===== Speed Test Summary =====")
    for k, v in records.items():
        summarize(k.upper(), v)


if __name__ == "__main__":
    main()

For image endpoints, there’s one extra thing to watch: image generation is typically a synchronous call, and the response body it returns is much larger than for text. If the total time is far longer than the time to first byte, or you get sporadic connection drops, the problem is usually in the return path rather than slow model generation. In that case, bump up the client timeout and add idempotent retries for interruptions.

Step 4: Use the Diagnostic Matrix to Identify the Bottleneck

Once you have the data from the first three steps, map it against the matrix below, and you’ll usually be able to pinpoint the problem.

🚀 Quick start: If you want to run the speed test script above right away, we recommend signing up at APIYI apiyi.com to get an API key. New users get free test credits—enough to run multiple rounds of speed comparisons without deploying any proxy.

How to Choose a base_url Across Different Deployment Regions

Lots of developers get hung up on this: if part of the business runs overseas and part runs in mainland China, do you need different endpoint addresses? The answer is no. APIYI’s servers are themselves located in the US West, so overseas services naturally get a shorter path. At the same time, the platform has added return-route acceleration for traffic from mainland China, specifically to keep large-response workloads like images and video stable on the way back. So no matter where you deploy, one shared base_url does the job.

The nice part of this design is that the same code moves seamlessly between a test environment (in China) and production (overseas) — no juggling multiple configs. What you actually need to do is run the four-step benchmark covered earlier inside each deployment environment and confirm the real-world numbers match what the business expects.

Optimization Tips for Large-Response Cross-Border Calls

Image and video endpoints are the most sensitive to cross-border links. Here’s what’s worth doing at the code level:

• Reuse the client globally: don’t re-establish a connection on every request — in a cross-border setup this saves a huge amount of handshake time.
• Set timeouts by workload type: text chat can be short, but image generation and video result retrieval need generous headroom.
• Retry idempotently on interruptions: use exponential backoff so you’re not resubmitting generation tasks in rapid succession.
• Poll video tasks asynchronously: submit the task, check status periodically, download once the result is ready — this lowers the risk of long-lived connection drops.
• Log segmented timings: feed TTFT and total duration into your monitoring so you can trace issues quickly when something goes wrong.

Frequently Asked Questions (FAQ)

Key Takeaways: Speed Testing Large Language Model APIs from China

• Split first, optimize second: Total time equals network time plus model time, so you have to measure each segment separately.
• Use curl for connection setup: The -w flag gives you cumulative timings for DNS, TCP, and TLS.
• Use the SDK for TTFT: Record the first-token time during streaming calls to gauge how quickly the model responds.
• Check the median across multiple rounds: A single run is easily thrown off by fluctuations, so gather at least 5 rounds.
• Watch the return trip for large response bodies: For images and video, focus on the gap between total time and time-to-first-byte.
• One unified endpoint: Connect through APIYI apiyi.com, using the same base_url for both domestic and overseas traffic.

Summary

Core points for troubleshooting slow Large Language Model API calls from China:

1. The 4-step localization method: Connection speed test, first-token speed test, total time measurement, and diagnostic matrix comparison—narrowing the scope layer by layer.
2. Distinguish two types of latency: Cross-border latency is solved with connection reuse and route optimization; model latency is solved with model selection and leaner inputs.
3. Treat large response bodies separately: Image and video workloads need sensible timeouts, idempotent retries, and async polling.
4. No need to split base_url by region: Overseas traffic connects directly to the US West data center, while domestic traffic benefits from return-route acceleration—one config does it all.

Rather than guessing that something is “slow,” spend a few minutes running a speed test script and let the data show you where the bottleneck is. Get a free test quota through APIYI apiyi.com, run real measurements in your own deployment environment, and then decide on your model selection and timeout strategy.

📚 References

1. APIYI Documentation Center: Connection endpoint and three-step onboarding guide
2. Link: docs.apiyi.com

3. Details: Covers OpenAI-compatible mode configuration, image API invocation notes, and connection-interruption troubleshooting

4. curl Official Manual: Full documentation of -w write-out variables

5. Link: curl.se/docs/manpage.html

6. Details: Authoritative definitions of timing fields like time_namelookup and time_appconnect

7. OpenAI Python SDK: Official open-source repository

8. Link: github.com/openai/openai-python
9. Details: How to use streaming calls, timeouts, and retry parameters

Author: APIYI Technical Team
Technical Support: If you run into any issues during speed testing, feel free to reach out to us on the APIYI homepage. For more integration docs, visit the docs.apiyi.com documentation center.

Related Articles

• Troubleshooting Image API Connection Interruptions: 5 Common Causes and Fixes for Cross-Border Image Generation: A systematic overview of five common causes of image API connection interruptions—timeout settings, proxy gateways, cross-border network links, retry strategies, and runtime environments—to help you reliably retrieve results when generating images across borders.