Research agents need
room to keep moving.
A research agent may wait on search providers, fetch several pages and extract document text before it can answer. Keeping that work responsive requires attention to the execution path around every model call.
Discuss your systemBy Sourish Sarmah · . Informed by Sourish’s prior work at Agentic Private Markets. This was employment experience, not a Meld Origins client engagement or endorsement.
An async function
can still block.
Imagine an assistant researching several companies. Network requests run asynchronously, but each response is immediately passed to a synchronous document extractor. While that extractor runs on the event-loop thread, other tasks sharing the loop have to wait.
Adding more concurrent requests does not remove that bottleneck. It can create a larger backlog of responses waiting to be processed. The function’s async declaration describes how it can cooperate with other tasks; it does not automatically move every operation onto a separate worker.
Implementation work
behind the lesson.
In his prior work at Agentic Private Markets, Sourish contributed changes that moved synchronous content extraction off the event-loop thread, reused HTTP clients and bounded external work with concurrency controls and timeouts.
This note explains the general decisions behind that kind of work using a fictional research workflow. It does not publish the employer’s source code, provider configuration, request limits or performance measurements.
Treat fetching and
extraction separately.
Reuse network connections
A scoped async HTTP client can reuse pooled connections across requests. Creating a new client for every item loses that opportunity and complicates cleanup. Give the client a clear owner and close it when that owner’s work ends.
Move blocking extraction off the loop
A worker executor can prevent a synchronous library call from monopolizing the event-loop thread. This is about keeping the loop responsive; it does not promise a speedup for CPU-heavy Python code. Profile the extractor and consider processes or a separate worker service when the workload warrants them.
Bound the whole pipeline
Limit active fetches and extraction work, and avoid scheduling an unbounded number of tasks that simply wait for a semaphore. A bounded queue can make pending work explicit. Connection-pool limits, provider quotas, memory and executor capacity all constrain a sensible concurrency setting.
A slow source should
have a clear outcome.
Give each source a deadline within an overall research budget. Distinguish an unavailable page, unsupported content, extraction failure and a successful source with no relevant evidence. Those outcomes help the synthesis step describe what it actually found.
Independent sources can often return partial results when one fails. Dependent work needs a different policy: a missing prerequisite may make downstream analysis invalid. Decide this from the research task instead of treating every exception as permission to continue.
Retries should respect provider guidance and consume the same overall budget. Cancel work when its result is no longer needed, and release network responses and other resources in cleanup paths. Cancelling an awaiting task does not necessarily stop a synchronous function already running in a thread; bound that work and design its lifetime explicitly.
Keep the answer tied
to the evidence.
Faster collection is useful only when the final answer remains traceable. Carry source URLs and extraction outcomes into synthesis, distinguish retrieved facts from inference, and make missing evidence visible. A failed fetch is not evidence that something does not exist.
Evaluate the pipeline with representative short pages, large documents, slow sources and failed responses. Observe queue wait, fetch time, extraction time, event-loop responsiveness, timeouts and memory use. Compare the same workload before and after a change rather than inferring a speedup from the presence of concurrency.
A useful failure test combines one slow source, one extraction failure and one successful source. Verify that the budget is respected, successful evidence remains available when partial results are allowed, and the answer does not imply that every source was checked. These are recommended evaluation cases, not reported results from an employer’s system.
Related work
and references.
For another recovery boundary, read what happens after an AI-generated database query fails.
- Python: task scheduling, cancellation and running blocking work in threads
- HTTPX: async clients and connection reuse
- HTTPX: connection, read, write and pool timeouts
Planning a research assistant or connecting an agent to external systems? Explore AI agent implementation or discuss your workflow.