Skip to content

Archive SaaS API Data to Azure Blob: SAS Tokens & Sync Jobs

Archive multi-tenant SaaS API data in Azure Blob Storage with Truto sync jobs, a container SAS token, and uploadObject block blob writes.

Sidharth Verma Sidharth Verma · · 9 min read
Archive SaaS API Data to Azure Blob: SAS Tokens & Sync Jobs

To archive raw SaaS API data to Azure Blob Storage with Truto sync jobs, you configure an azure_blob_storage datastore with a tightly scoped container SAS token, design a predictable uploadObject path, and let a declarative sync job write each batch of records to a block blob.

If you are building a B2B SaaS product, your application's value is directly tied to the external data it can ingest. Your customers expect you to pull millions of records from their existing CRMs, HRIS platforms, ticketing systems, and accounting tools. You need a reliable, scalable architecture to extract this data and load it into an Azure data lake. You do not want to build and maintain dozens of custom API polling scripts, nor do you want to force customer-facing integrations through heavy, consumption-based internal ETL tools that were designed for offline analytics.

This guide breaks down how to architect a multi-tenant data sync using Truto. We will cover why teams land raw data in a lake first, the security differences between Shared Key and SAS token authentication, how to configure Azure Blob Storage as a first-class datastore destination, and how the files you write can feed downstream tools such as Microsoft Fabric or Synapse Analytics, which can read from Blob Storage.

The Shift to Azure Data Lakes for B2B SaaS

For teams already on Azure, Blob Storage is the natural place to land raw SaaS data.

For B2B SaaS companies, the architectural pattern has shifted from writing third-party API data directly into operational databases (like Postgres or MongoDB) to landing raw payloads into a data lake first. This ELT (Extract, Load, Transform) approach provides several distinct advantages:

  1. Schema Resilience: SaaS APIs change constantly. Custom fields are added, data types shift, and endpoints are deprecated. By landing raw JSON or Parquet files into Azure Blob Storage, you decouple the extraction phase from the transformation phase. If a downstream schema breaks, you can replay the raw data from the blob.
  2. AI and Analytics Readiness: Raw data lakes serve as a good foundation for downstream AI workflows. Whether you are training models, generating embeddings for RAG, or feeding Microsoft Fabric pipelines, having the raw, historical API data archived in block blobs is valuable.
  3. Cost Efficiency: Storing large volumes of historical sync data in Azure Blob Storage typically costs far less than storing it in a managed relational database.

For a broader look at the strategic options for managing this data, review our guide on Syncing Data from SaaS Apps.

The Challenge of Archiving Multi-Tenant SaaS Data

Building custom pipelines to extract bulk data from 50+ different SaaS APIs is a massive engineering drain. When you pull data for internal business intelligence, you control the source credentials and the destination. When you extract data on behalf of your customers, you control neither. You are operating in a multi-tenant environment where every customer brings their own API limits, data volumes, and edge cases.

Engineers attempting to build this in-house run into three immediate walls:

  • Pagination Hell: Every API paginates differently. Some use opaque cursors, some return next-page URLs or link headers, and legacy APIs use strict offset/limit parameters. Writing custom logic to traverse these pages for millions of records is brittle.
  • State Management: When a sync job fails halfway through a 10-gigabyte extraction, you need to know exactly where it left off to resume gracefully. Building durable state management for cursor tracking is a distributed systems problem.
  • Authentication Management: Securely storing and refreshing OAuth tokens for thousands of tenants, while simultaneously managing the credentials required to write to their respective Azure storage containers, introduces significant security overhead.

For a deeper dive into solving the extraction problem, see our breakdown on Handling Bulk Data Extraction and ETL Workflows Through Unified APIs.

Azure Blob Storage Authentication: SAS Tokens vs. Account Keys

When configuring Azure Blob Storage as a destination for your multi-tenant data pipelines, authentication is the most critical security decision you will make. Truto's azure_blob_storage datastore supports both of Azure's key-based options through its auth_type field: sas_token (a Shared Access Signature) and account_key (Shared Key).

The Danger of Shared Keys

A Shared Key (or Account Key) provides full access to your entire Azure Storage account. If an application possesses this key, it can read, write, delete, and modify any blob in any container across the entire account.

Using long-lived account keys for external integrations is a real security risk. Leaked keys often stay valid long after the leak, and if an account key is compromised, the blast radius is every container in the account. Reserve account keys for cases where a SAS token is not an option.

Why SAS Tokens Are the Better Default

A Shared Access Signature (SAS) provides delegated access to resources in your storage account. SAS tokens let you limit the blast radius by defining:

  • Scope: Access can be limited to a specific container or even a specific blob.
  • Permissions: You grant only the operations the integration needs and leave out the rest, such as Delete.
  • Time Bounds: Tokens can be configured with strict expiration times.
  • Network Restrictions: Access can be limited to specific IP ranges.
Tip

Best Practice: Use a container-level SAS token with only write and list permissions (wl). Truto's Azure Blob datastore lists blobs and uploads blobs, and needs nothing else. Set a fixed expiry, and rotate the token before it lapses. A leaked token is then limited to one container. But a token with w can still overwrite any blob in that container. To protect the archive, turn on blob versioning, soft delete, or an immutability policy on the container.

Generating a scoped container SAS token via the Azure CLI is straightforward:

az storage container generate-sas \
  --account-name mystorageaccount \
  --name my-tenant-container \
  --permissions wl \
  --expiry 2027-05-01T00:00:00Z \
  --https-only \
  --output tsv

By using SAS tokens, you follow the principle of least privilege and keep your multi-tenant data lake contained.

Architecting the Sync: From SaaS API to Block Blobs

Truto handles the multi-tenant extraction problem through its Sync Job runtime, which separates the extraction logic from the destination routing. This guide uses a V4 sync job. A sync job is a graph of nodes: request nodes pull from Truto's unified APIs, optional transform nodes reshape the records with JSONata, and destination nodes write the result somewhere. A destination with destination_type: "datastore" writes to a datastore such as Azure Blob Storage.

Truto writes every file to Azure as a block blob, with a single Put Blob request per uploadObject call. Block blobs are Azure's general-purpose blob type for text and binary files such as JSON or Parquet.

Here is how the architecture flows:

flowchart TD
  A["SaaS API (Salesforce, Zendesk)"] -->|"Extract (Pagination)"| B["Truto Sync Runtime"]
  B -->|"Unified records"| C["Transform node (JSON or Parquet)"]
  C -->|"uploadObject (path + file_name)"| D["Azure Blob Storage"]
  subgraph Azure ["Azure Data Lake"]
    D --> E["Downstream readers (e.g. Microsoft Fabric, Synapse)"]
  end

The uploadObject Path Strategy

The uploadObject method takes four parameters: content and file_name, which you must send, and path and headers, which are optional. Truto joins path and file_name into the blob name inside your container. Truto drops empty segments without an error. So if a placeholder in path resolves to an empty string, the blob lands one level up and the write still succeeds. Only a segment that is whitespace, ., or .. fails the write. Check that every value in your path is present before the destination runs.

If you send no Content-Type header, Truto writes the blob as application/octet-stream.

For multi-tenant architectures, a consistent path layout matters for two reasons. Downstream analytics tools can prune partitions, and you can call listObjects with a prefix to see one tenant's files. A standard partition strategy looks like the pseudo-layout below. The single braces are not Truto placeholder syntax. They only name the parts. The sync job example later in this section shows the real {{...}} placeholders.

{tenant_id}/{integration_name}/{resource_name}/ingest_dt={YYYY-MM-DD}/ingest_hr={HH}/{resource_name}-{uuid}.json

Tools that read Hive-style partitions from Blob Storage, such as Azure Synapse Analytics or Databricks, can use partition pruning to skip the files a query does not need.

What you write is decided by your sync job config. content may be a string, bytes, or a JSON value, so a transform node can emit JSON documents or Parquet bytes. Truto's published Parquet templates target S3: a transform calls $jsonToParquet to build one Parquet blob per updated_at hour, and an uploadObject destination writes each blob. You can point the same pattern at an Azure Blob datastore:

{
  "name": "azure-<source>-<r>-parquet",
  "type": "destination",
  "destination_type": "datastore",
  "method": "uploadObject",
  "loop_on": "payload.records",
  "run_if": "$exists(payload.records) and $type(payload.records) = \"array\" and $count(payload.records) > 0",
  "config": {
    "id": "{{args.azure_datastore_id}}",
    "config": {
      "path": "tenant={{args.company_id}}/source=<source>/entity=<r>/ingest_dt={{payload.records.ingest_dt}}/ingest_hr={{payload.records.ingest_hr}}",
      "file_name": "{{payload.records.file_name}}",
      "content": "{{payload.records.parquet:any}}",
      "headers": { "Content-Type": "application/vnd.apache.parquet" }
    }
  },
  "resources_to_persist": ["<r>-to-parquet"]
}

loop_on iterates the transform's output array, so inside each iteration payload.records is the current element. The :any suffix keeps the Parquet bytes binary instead of coercing them to a string. The <source> and <r> values are template slots that you fill in. args.azure_datastore_id and args.company_id are run arguments, so declare them in the sync job's args_schema, and set "default_runtime_version": 4 on the job.

A failed uploadObject call does not fail the run. The run still ends completed, the watermark moves forward, and Truto does not replay the failed records. The only signal is a sync_job_run:record_error webhook event, and that event does not carry the failed rows. Watch the webhook and backfill by resetting the watermark.

For more details on declarative sync configurations, see our guide on Declarative Data Sync Pipelines: Ship Integrations as Config, Not Code.

Info

Inside a V4 sync job, Truto handles upstream HTTP 429 responses for you. The run pauses, sends a sync_job_run:rate_limited webhook event, and resumes by itself. Only when your own application calls Truto's Unified API directly, outside a sync job, does Truto pass the 429 through to you with IETF ratelimit-limit, ratelimit-remaining, and ratelimit-reset headers. Then your code owns the retry and backoff.

Configuring Azure Blob Storage as a Truto Datastore

To write data to Azure, you must first configure it as a Datastore within Truto. Datastores in Truto are first-class entities that represent connections to external storage systems.

This system relies on a strict separation of concerns:

  • The Datastore holds the how to connect (the storage account name, the container, the SAS token or account key, and an optional base_url).
  • The Sync Job Destination holds the what to do (the method to call, such as uploadObject, and its path, file_name, content, and headers).

This modularity means a single Azure Blob Datastore configuration can be reused across multiple sync jobs without duplicating credentials.

Security and Encryption at Rest

Datastores are scoped to an environment (Truto's multi-tenancy unit). When you provide your SAS token or account key to Truto, it is not stored in plaintext. Truto encrypts these fields at rest and strips them from every API response. If you set base_url, it must be a public https URL.

Example Datastore Configuration

Creating the datastore with POST /datastore takes the connection details specific to Azure Blob Storage:

{
  "type": "azure_blob_storage",
  "label": "Primary Azure Data Lake",
  "config": {
    "account_name": "prodstorageaccount",
    "container": "tenant-archives",
    "auth_type": "sas_token",
    "sas_token": "sp=wl&st=2026-11-01T...&se=2027-05-01T...&spr=https&sv=...&sr=c&sig=..."
  }
}

Without base_url, Truto sends requests to https://{account_name}.blob.core.windows.net. To switch to Shared Key auth, set auth_type to account_key and send account_key instead of sas_token. A PATCH keeps only the secret that matches auth_type, so switching the auth type drops the other secret.

Once the Datastore is created, reference its ID in the config.id field of a sync job's datastore destination node, as in the example above.

If you are evaluating other destinations alongside Azure Blob, such as Snowflake, the architectural approach is the same. Read our deep dive on Load SaaS Data to Snowflake: Key-Pair Auth, SQL API & VARIANT Columns to understand how this separation of concerns applies across different storage backends.

Building a Scalable Data Lake on Azure

Archiving raw SaaS API data into Azure Blob Storage is a resilient, cost-effective way to build a multi-tenant data foundation. By landing data in block blobs first, you protect your application from upstream schema drift, enable downstream AI workflows, and reduce the cost of storing historical records.

However, building the extraction infrastructure from scratch is a trap. Managing pagination, state, and authentication across dozens of unique SaaS APIs will consume your engineering team and distract from your core product.

With Truto's Sync Jobs and Datastores, you define declarative data pipelines that extract this data and write it to Azure with a scoped SAS token. Truto handles pagination, normalization, and the writes to Blob Storage, while your sync job config decides the file format and path layout.

The result is a scalable, secure data lake on Azure.

FAQ

How do I sync SaaS data to Azure Blob Storage?
Create a Truto datastore of type azure_blob_storage with your account name, container, and a SAS token. Then add a datastore destination node to a sync job that calls uploadObject with a path, file_name, and content built from the synced records.
Should I use an Account Key or a SAS token for Azure Blob integrations?
Truto supports both through the auth_type field. A container SAS token is the safer choice because it is limited to one container, a set of permissions, and an expiry. Truto needs only write and list (wl) on that container. An account key gives full access to the whole storage account.
What happens when a write to Azure Blob Storage fails during a sync job?
A failed uploadObject call does not fail the run. The run still ends completed, the watermark moves forward, and Truto does not replay the failed records. The only signal is a sync_job_run:record_error webhook event, which does not carry the failed rows. Watch the webhook and backfill by resetting the watermark.
Can Truto write data in Parquet format to Azure Blob Storage?
The Parquet templates Truto ships target S3. You can build the same pattern for Azure yourself - a transform node that calls $jsonToParquet, then an uploadObject destination that points at your Azure Blob datastore with a Parquet Content-Type header.

More from our Blog