Use this page as a quick reference for creating a new pipeline. For a more detailed, step-by-step tutorial, please see the Quickstart guide.

Workflow Diagram 

flowchart TD
    SelectSource([1. Select Source Integration])
    SelectTarget([2. Select Target Integration])
    Schedule([3. Configure Pipeline Settings])
    Save([4. Save & Run])
    SelectSource --> SelectTarget --> Schedule --> Save

Quick Steps

  1. Choose a Source: Navigate to the Pipelines page and click + Pipeline. Select the source integration (e.g., Aeroqual, NOAA) that will provide the data.

  2. Choose a Target: Select the target integration (e.g., iTwin IoT, Sensemetrics) that will receive the data.

  3. Configure the Pipeline:

    • Polling Rate: Define how often the pipeline runs (e.g., every 15 minutes, every hour).
    • Backtrack: Set the initial backtrack window (up to seven days) to fetch historical data on the first run.
    • Nightly Resync: For supported pipelines, optionally request the previous 24 hours of data each night to help recover delayed readings.
    • Name: Give the pipeline a descriptive name for easy identification (e.g., Aeroqual Site A → iTwin IoT Project B).
    • Flag Errored: Choose how many consecutive failed executions mark the pipeline Errored, or disable this condition.
    • Flag Stale: Choose how long the pipeline can go without a successful execution before it is marked Stale, or disable this condition.
    • Pause Stale Timer on Weekends: Optionally exclude weekend time based on the organization’s configured timezone.
    • Source or Target Settings: Configure any options specific to the selected integrations. These settings belong to this pipeline rather than the reusable integration. Some options cannot be changed after the pipeline is saved; see the selected integration’s documentation for details.

  4. Save and Run: Save the pipeline. The first execution will be queued to run immediately to pull your initial historical data. Subsequent runs will follow the schedule you defined.

Reference

Field Description
Source Integration The configured source system providing the data.
Target Integration The configured target system receiving the data.
Source or Target Settings Options specific to the selected integrations and pipeline. Availability, defaults, and editability vary by integration.
Polling Rate The interval for pipeline runs. See Polling Rates for vendor-specific recommendations.
Backtrack How far back to fetch data on the first run.
Nightly Resync Runs an additional 24-hour backfill at midnight UTC for supported pipelines. See Nightly Resync.
Pipeline Name A unique, human-friendly identifier for the pipeline.
Flag Errored Marks the pipeline Errored after 1, 2, 3, 5, or 10 failed executions. New pipelines default to 3 consecutive failed executions.
Flag Stale Marks the pipeline Stale after 1, 3, 6, or 12 hours; 1, 2, or 3 days; or 1 week without a successful execution. New pipelines default to 1 day.
Pause Stale Timer on Weekends Excludes weekend time from the stale timer using the organization timezone.