Skip to content
Truto
Back
AI & Agents

Connect Coviu to AI Agents: Orchestrate Call Queues and Submissions

Learn how to connect Coviu to ai agents using Truto. Step-by-step guide to tool calling, API quirks, and autonomous workflows.

Uday Gajavalli Uday Gajavalli · · 10 min read
Connect Coviu to AI Agents: Orchestrate Call Queues and Submissions

You want to connect Coviu to an AI agent so your telehealth or virtual clinic system can independently manage waiting queues, provision patient sessions, and audit clinical recordings. Here is exactly how to do it using Truto's /tools endpoint and SDK, bypassing the need to write and maintain custom API wrappers for your language models.

Giving a Large Language Model (LLM) read and write access to a live telehealth platform is an engineering headache. You either spend weeks building, hosting, and maintaining a custom connector, or you use a managed infrastructure layer that handles the boilerplate for you. If your team uses ChatGPT, check out our guide on connecting Coviu to ChatGPT, or if you are building on Anthropic's models, read our guide on connecting Coviu to Claude. For developers building custom autonomous workflows, you need a programmatic way to fetch these tools and bind them to your agent framework.

This guide breaks down exactly how to fetch AI-ready tools for Coviu, bind them natively to an LLM using LangChain (or any framework like LangGraph, CrewAI, or the Vercel AI SDK), and execute complex clinical orchestration workflows. For a deeper look at the architecture behind this approach across different platforms, refer to our research on architecting AI agents and the SaaS integration bottleneck.

The Engineering Reality of Custom Coviu Connectors

Building AI agents is conceptually straightforward. Connecting them to external SaaS APIs safely is hard. Giving an LLM access to external data sounds simple in a prototype - you write a Node.js function that makes a fetch request and wrap it in an @tool decorator. In production, this approach collapses entirely under the weight of API-specific quirks and edge cases.

If you decide to build a custom integration for Coviu, you own the entire API lifecycle. Coviu's API introduces several specific integration challenges that break standard LLM assumptions.

Temporal State Constraints on Resources

Unlike a typical CRM where you can update or delete a record at any time, Coviu enforces strict temporal constraints tied to real-world events. For example, you cannot add a participant to a session that has already finished. Similarly, you cannot cancel or delete a participant if their associated session has already started.

When hand-coding tools, you must write complex validation logic and prompt instructions to force the LLM to query the state or actual_end_time of a session before attempting a write operation. If you fail to do this, the LLM will repeatedly attempt invalid operations, hallucinate success, or get stuck in an error loop when the API rejects the request.

Binary Streams vs. JSON Metadata

LLMs are designed to consume text. When an agent audits past sessions, it typically wants to review transcripts or recordings. In Coviu, fetching a submission via get_single_coviu_submission_by_id returns a JSON object containing the metadata and file IDs. However, actually retrieving the recording (get_single_coviu_submission_file_by_id or get_single_coviu_audio_recording_by_id) returns a binary video/webm or audio/webm stream.

If you blindly expose these endpoints to an LLM, the model will attempt to read a binary stream as a string, instantly blowing up your context window and crashing the agent. You must build an architectural divide where the LLM tool only returns the metadata and download URL, while a deterministic background worker handles the actual file stream and passes it to a transcription service (like Whisper or AssemblyAI).

Deeply Nested Queue Topologies

Coviu's waiting areas are not stored in a flat list. To interact with waiting calls, an agent cannot simply ask for "all waiting calls." It must understand a deeply nested hierarchy. To list waiting calls in a specific queue, the agent must supply both the team_id and the waiting_queue_id. Single call lookups require both team_id and the call id.

If the LLM loses context of the team_id across a long multi-step conversation, its subsequent API calls will fail. Truto mitigates this by enforcing strict JSON schemas for every proxy tool, rejecting invalid tool calls before they hit the network, and forcing the LLM to provide required parameters.

Auto-Generating Tools with Truto

Rather than hand-coding every endpoint, Truto maps Coviu's endpoints into REST-based Proxy APIs. Truto handles all authentication and query parameter processing, returning data in a predefined format.

Truto provides a set of tools for your LLM frameworks by offering a description and schema for all the methods defined on a Coviu integration. By calling the GET /integrated-account/<id>/tools endpoint on the Truto API, you receive an array of fully-formed tool definitions that your LLM can natively understand.

If the LLM needs to know exactly what parameters are required to create a session, Truto provides a strict JSON schema dictating that session_name, start_time, and end_time are mandatory, while participants and feature_flags are optional. You can edit these descriptions directly in the Truto UI to give your agent highly specific instructions tailored to your clinic's business logic.

Coviu Hero Tools

Below are the highest-leverage tools available for orchestrating Coviu workflows. We have focused on the endpoints that drive the most value for autonomous orchestration, rather than simple CRUD operations.

Create a Coviu Session

This tool provisions a new telehealth room. It is the foundational step for any booking workflow. The agent must provide a session name, start time, and end time. It can optionally inject feature flags to customize the clinical interface, such as disabling menus or enforcing participant uniqueness.

"A new patient, Sarah Jenkins, needs an intake appointment tomorrow at 10:00 AM for 45 minutes. Create a new Coviu session for this, name it 'Intake - Jenkins', and ensure the session enforces unique participant entry."

List All Coviu Waiting Queue Calls

This tool allows the agent to monitor incoming traffic for a specific waiting area. It returns detailed timestamps for the call lifecycle, including joinedAt and lastActivityAt. This is critical for building triage agents that monitor wait times and alert staff.

"Check the primary clinical waiting queue for team ID 'team_883' and queue ID 'q_912'. Are there any patients who have been waiting longer than 15 minutes?"

Create a Coviu Participant

Once a session is established, this tool generates the specific entry URLs for attendees. The agent must pass the session_id. Because of the temporal constraints mentioned earlier, the agent must ensure the session is active and not finished.

"Take the session ID you just created for the Jenkins intake, and generate two participant links - one for the patient, and one for Dr. Smith acting as the host."

Get Single Coviu Session Summary By ID

After a call concludes, this tool pulls the definitive summary. It returns the actual_end_time alongside the scheduled times, providing exact data on how long the consultation ran. This is highly useful for automated billing and invoicing agents.

"Pull the session summary for session ID 'sess_9942'. Compare the actual end time against the scheduled end time to calculate the total billable minutes for this consultation."

List All Coviu Submissions

Coviu allows clinics to capture forms, images, and documents during a call via collections. This tool lists all submissions within a specific collection, returning metadata and action statuses. It is essential for medical records reconciliation.

"Check the intake collection for team 'team_883' over the last 24 hours. Give me a list of all item submission IDs that are currently locked and ready for processing."

Delete a Coviu Session By ID

This tool cancels an existing session. Agents can use this to handle schedule conflicts or patient cancellations autonomously. It returns a boolean confirming success.

"The patient for session ID 'sess_7721' just texted to cancel their appointment. Please cancel the Coviu session and free up the calendar block."

For the complete tool inventory, including detailed schemas for managing participants, collections, and binary file streams, review the full Coviu integration page.

Workflows in Action

To understand how these tools map to real-world operations, here are two concrete examples of multi-step autonomous workflows.

Scenario 1: Automated Patient Triage and Provisioning

A healthcare administrator configures an agent to monitor high-volume triage channels and provision secure rooms when a specialist is required.

"A priority patient has just cleared the asynchronous triage chat. They require a live consultation with a duty nurse immediately. Create a secure session for them, generate the participant link, and verify no other urgent calls are currently holding in the main queue."

  1. The agent calls list_all_coviu_waiting_queue_calls to check the current load on the main triage queue.
  2. Seeing an acceptable wait time, the agent calls create_a_coviu_session with the current timestamp to spin up an immediate room.
  3. The agent calls create_a_coviu_participant to generate the secure entry link.
  4. The agent formulates a response containing the entry URL to be passed back to the patient via the chat interface.

Scenario 2: Post-Consultation Audit and Reconciliation

A compliance officer relies on an agent to audit daily operations, ensuring that all scheduled telehealth calls have actual end times logged and related intake forms filed.

"Run the end-of-day audit for yesterday. Find all completed sessions, calculate the actual duration of each, and verify if there are any pending document submissions in the intake collection for those sessions."

  1. The agent calls list_all_coviu_sessions, applying an end_time filter for the previous 24 hours.
  2. For each returned session, the agent calls get_single_coviu_session_summary_by_id to retrieve the actual_end_time and calculate the true duration.
  3. The agent calls list_all_coviu_submissions against the team's primary collection ID to cross-reference which sessions have completed intake forms.
  4. The agent generates a structured markdown report highlighting any sessions that lack associated forms or ran significantly over schedule.

Building Multi-Step Workflows

To orchestrate these multi-step processes, you must build a resilient execution loop. The core architectural pattern involves binding Truto's dynamically generated schemas directly to an LLM, allowing the model to dictate when to pause, fetch data, and resume.

This approach works universally across agent frameworks. Whether you are using LangChain's .bindTools(), Vercel AI SDK's generateText, or building custom loops with standard OpenAI SDKs, the underlying mechanics remain the same.

Architecture of an Agent Loop

When a user issues a prompt, the framework feeds the system prompt, user input, and the array of Coviu tool schemas into the LLM. The LLM responds with a tool_calls array instead of text. The framework intercepts this, executes the network request against Truto's proxy API, and feeds the resulting JSON back to the LLM as a new message with the role of tool.

sequenceDiagram
    participant User
    participant Framework as Agent Framework
    participant LLM
    participant Truto as Truto Proxy
    participant Coviu as Coviu API

    User->>Framework: "Create a session and add a participant"
    Framework->>Truto: GET /integrated-account/{id}/tools
    Truto-->>Framework: Return Coviu Tool Schemas
    Framework->>LLM: Send Prompt + Tools
    LLM-->>Framework: return tool_call: create_a_coviu_session
    Framework->>Truto: POST /proxy/coviu/sessions
    Truto->>Coviu: Map to Coviu API
    Coviu-->>Truto: 200 OK (Session Data)
    Truto-->>Framework: Return JSON response
    Framework->>LLM: Send tool_result (Session ID)
    LLM-->>Framework: return tool_call: create_a_coviu_participant
    Framework->>Truto: POST /proxy/coviu/sessions/{id}/participants
    Truto->>Coviu: Map to Coviu API
    Coviu-->>Truto: 200 OK (Participant Data)
    Truto-->>Framework: Return JSON response
    Framework->>LLM: Send tool_result
    LLM-->>User: "Session created. Here is the link."

Implementation with LangChain.js

To implement this in Node.js, we recommend using the @trutohq/truto-langchainjs-toolset. This SDK handles the boilerplate of fetching the tools and structuring them for LangChain.

import { ChatOpenAI } from "@langchain/openai";
import { AgentExecutor, createOpenAIToolsAgent } from "langchain/agents";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { TrutoToolManager } from "@trutohq/truto-langchainjs-toolset";
 
async function runCoviuAgent() {
  // 1. Initialize the Truto Tool Manager with your tenant credentials
  const toolManager = new TrutoToolManager({
    apiKey: process.env.TRUTO_API_KEY,
  });
 
  // 2. Fetch the tools dynamically for the specific Coviu account
  const tools = await toolManager.getTools(process.env.COVIU_ACCOUNT_ID);
 
  // 3. Initialize the LLM (e.g., GPT-4o)
  const llm = new ChatOpenAI({
    modelName: "gpt-4o",
    temperature: 0,
  });
 
  // 4. Define system instructions for temporal constraints
  const prompt = ChatPromptTemplate.fromMessages([
    ["system", "You are a telehealth orchestration assistant. Always check the actual_end_time before adding participants to an existing session. Do not attempt to read binary streams directly."],
    ["human", "{input}"],
    ["placeholder", "{agent_scratchpad}"],
  ]);
 
  // 5. Bind the tools and construct the executor
  const agent = createOpenAIToolsAgent({
    llm,
    tools,
    prompt,
  });
 
  const executor = new AgentExecutor({
    agent,
    tools,
    maxIterations: 10,
    tools,
  });
 
  // 6. Execute the workflow
  const result = await executor.invoke({
    input: "List the currently waiting calls for team 'team_883' in queue 'q_912'."
  });
 
  console.log(result.output);
}
 
runCoviuAgent().catch(console.error);

Error Handling and Rate Limits

When orchestrating multi-step workflows, hitting rate limit errors is inevitable - especially if an agent is looping through hundreds of sessions to compile a compliance report. It is critical to understand how this is handled at the network layer.

Truto operates as a transparent proxy for errors. Truto does not retry, throttle, or apply backoff on rate limit errors. If your agent hits an endpoint too aggressively and the upstream Coviu API returns an HTTP 429 Too Many Requests, Truto passes that 429 error directly back to the caller.

What Truto does do is normalize the rate limit metadata. Different APIs expose their rate limits via completely different HTTP headers (e.g., X-RateLimit-Remaining vs X-Rate-Limit-Remaining). Truto normalizes upstream rate limit info into standardized headers per the IETF specification:

  • ratelimit-limit: The total request quota.
  • ratelimit-remaining: The number of requests left in the current window.
  • ratelimit-reset: The timestamp indicating when the quota refreshes.

Your application code (or your agent framework) is strictly responsible for inspecting these headers, interpreting the 429 status code, and implementing the necessary retry and exponential backoff logic. Do not build agents assuming the infrastructure layer will absorb request spikes; your execution loop must gracefully catch 429 errors, pause execution based on the ratelimit-reset value, and attempt the tool call again.

Orchestrating Telehealth Without the Overhead

Building agentic workflows for clinical operations shouldn't require your engineering team to become experts in Coviu's temporal state logic or nested queue architecture. By utilizing dynamic tool schemas, you can separate the business logic of your AI agent from the operational drudgery of API maintenance.

Your LLMs focus on reasoning about patient wait times and resource allocation, while Truto handles the translation layer, ensuring every tool call respects the precise shape of the underlying API.

More from our Blog