API Clients: How to Make Multiple API Calls Concurrently 

Here is a scenario every data engineer knows too well: You have 10,000 items to fetch from a third-party API. Your first instinct is to loop through them one by one.

Two hours later, your script is still running. Your boss is asking why the data is late. And you are staring at a terminal wondering if there is a better way.

There is. Concurrent API calls let you send multiple requests simultaneously, slashing execution time from hours to minutes. But here is the catch: do it wrong, and you will get rate-limited, blocked, or even banned.

In this guide, I will show you exactly how to build a robust, concurrent API client that respects rate limits, handles failures gracefully, and scales to tens of thousands of requests.

Why Make API Calls Concurrently?

Let us start with the obvious: speed.

Imagine you need to fetch data from 100 different endpoints. Sequential execution means:

• Request 1 → Wait for response → Request 2 → Wait → Request 3...

If each request takes 200ms, that is 20 seconds total.

Concurrent execution means:

• Request 1, 2, 3, ... 100 all fire off at roughly the same time.

If the API can handle 10 concurrent requests, your total time drops to around 2 seconds—a 10x improvement.

The Core Tools for Concurrent API Calls in Python

For Python developers, the standard stack for concurrent API calls is:

1. asyncio + aiohttp

This is the most common approach for I/O-bound tasks like API calls. asyncio manages the event loop, while aiohttp handles the HTTP requests asynchronously.

Here is the basic pattern:

import asyncio
import aiohttp

async def fetch(session, url):
    async with session.get(url) as resp:
        return await resp.json()

async def main(urls):
    async with aiohttp.ClientSession() as session:
        tasks = [fetch(session, url) for url in urls]
        results = await asyncio.gather(*tasks)
    return results

urls = [f"https://api.example.com/items/{i}" for i in range(1000)]
data = asyncio.run(main(urls))

What is happening here?

• asyncio.gather(*tasks) runs all the fetch tasks concurrently.

• The ClientSession is reused across requests for efficiency.

The problem: This naive approach works for small sets. But for thousands of requests, you will hit rate limits and get HTTP 429 errors.

The Two Biggest Pitfalls (And How to Avoid Them)

Pitfall 1: Rate Limiting (HTTP 429 Errors)

Most APIs enforce limits—e.g., 100 requests per minute. If you send 1,000 requests at once, the API will reject most of them.

The Fix: Use a rate limiter to spread requests evenly over time.

from aiolimiter import AsyncLimiter
from tenacity import retry, stop_after_attempt, wait_exponential

# Allow up to 100 requests per minute
rate_limiter = AsyncLimiter(100, 60)

@retry(wait=wait_exponential(multiplier=1, min=2, max=60), stop=stop_after_attempt(5))
async def fetch_with_limits(session, url):
    async with rate_limiter:  # This ensures we don't exceed 100/min
        async with session.get(url) as resp:
            if resp.status == 429:
                raise Exception("Rate limited")  # Triggers retry
            return await resp.json()

Why this works: AsyncLimiter controls the request rate, while tenacity handles retries with exponential backoff—so you do not hammer the API again immediately after a failure.

Pitfall 2: Concurrent Request Limits (Bulkheads)

Here is a subtle one: even if you respect the per-minute rate limit, you can still get 429s if you send too many requests at the exact same time.

For example, Phrase's API enforces both 6,000 requests per minute AND a concurrent request cap. Bursting 100 requests at once can trigger a 429 even if your RPM is well under the limit.

The Fix: Use a semaphore to limit how many requests run in parallel.

semaphore = asyncio.Semaphore(10)  # Max 10 concurrent requests

async def fetch_with_semaphore(session, url):
    async with semaphore:
        async with session.get(url) as resp:
            return await resp.json()

Think of it like having only 10 "passes" to make requests at the same time. If all 10 are in use, new requests wait until one finishes.

Building a Production-Ready Concurrent API Client

Here is a complete, robust implementation that handles:

• Concurrency control (semaphore)

• Rate limiting (per-minute quotas)

• Retries with exponential backoff (tenacity)

• Worker queue pattern for large batches

import asyncio
import aiohttp
from aiolimiter import AsyncLimiter
from tenacity import retry, stop_after_attempt, wait_exponential

class ConcurrentAPIClient:
    def __init__(self, max_concurrent=10, requests_per_minute=100):
        self.semaphore = asyncio.Semaphore(max_concurrent)
        self.rate_limiter = AsyncLimiter(requests_per_minute, 60)

    @retry(wait=wait_exponential(multiplier=1, min=2, max=60), stop=stop_after_attempt(3))
    async def _fetch(self, session, url):
        async with self.semaphore:  # Limit concurrency
            async with self.rate_limiter:  # Limit rate
                async with session.get(url) as resp:
                    if resp.status == 429:
                        raise Exception("Rate limit exceeded")
                    return await resp.json()

    async def fetch_all(self, urls):
        async with aiohttp.ClientSession() as session:
            tasks = [self._fetch(session, url) for url in urls]
            # return_exceptions=True prevents one failure from crashing everything
            results = await asyncio.gather(*tasks, return_exceptions=True)
            return results

Worker Queue Pattern (For 10,000+ Requests)

If you are dealing with tens of thousands of requests, launching that many coroutines at once can overwhelm your system. Use a worker queue pattern instead:

async def worker(session, queue):
    while True:
        url = await queue.get()
        if url is None:
            break
        try:
            await fetch(session, url)
        finally:
            queue.task_done()

async def main(urls, n_workers=10):
    queue = asyncio.Queue()
    for url in urls:
        await queue.put(url)

    async with aiohttp.ClientSession() as session:
        workers = [asyncio.create_task(worker(session, queue)) for _ in range(n_workers)]
        await queue.join()  # Wait for all tasks to complete
        for _ in range(n_workers):
            await queue.put(None)  # Signal workers to stop
        await asyncio.gather(*workers)

Why this works: Instead of 10,000 concurrent tasks, you run a fixed number of workers that pull URLs from a queue. This is far more memory-efficient and easier to debug.

Language-Specific Solutions

JavaScript/Node.js

The @4i4/multi-request package (built on Axios) allows you to define concurrent requests with dependencies and sub-requests.

Go

The go-ai library provides rate limiting middleware with token bucket, sliding window, and concurrent request limiting strategies.

Pre-built Python Client Libraries

The llm-api-client library handles concurrency, rate limiting (RPM and TPM), retries, and usage tracking out of the box.

Common Pitfalls to Avoid

1. Shared State Across Coroutines

Do not modify shared counters or lists directly from multiple async tasks without coordination. This creates race conditions where counts become inaccurate.

2. Ignoring Retry-After Headers

When you get a 429, the API often sends a Retry-After header telling you how long to wait. Respect it.

3. Launching Too Many Coroutines

Asyncio can handle thousands of tasks, but each has overhead. For very large batches, the worker queue pattern is better.

The Bottom Line

Concurrent API calls are essential for building fast, efficient data pipelines. But speed without control is a recipe for disaster.

Your Action Plan:

1. Start with asyncio + aiohttp for basic concurrency.

2. Add a rate limiter (like aiolimiter) to stay under API quotas.

3. Use a semaphore to limit concurrent requests.

4. Implement retries with exponential backoff (using tenacity).

5. For large batches, switch to a worker queue pattern.

Build it right, and you will go from waiting hours for data to finishing in minutes—without getting banned by the API provider.

Contact Us

Phone: +91 9667708830
Email: info@codingnow.in
Website: https://codingnowai.in/

Address:
2nd Floor, Kapil Vihar (Opp. Metro Pillar No.354)
Pitampura, New Delhi – 110034

Backlink to main website: Explore Python and AI courses at Coding Now – Gurukul of Ai