Once a pipeline is created, you can either edit it to make changes or duplicate it to create a new pipeline based on an existing one. These are two separate features, and each serves a different purpose.

Editing Pipelines

Editing a pipeline is straightforward. You might update details such as the pipeline’s name, adjust its polling rate, or enable additional execution behaviors.

How to Edit a Pipeline

  1. Go to the Pipelines page.
  2. Click the pipeline you want to edit.
  3. Select Edit Settings from the menu (three vertical dots).
  4. Make your changes and save.

Common use cases for editing:

  • Changing the polling rate to run more or less frequently.
  • Renaming the pipeline for clarity.
  • Fixing or updating credentials (e.g., rotating an API key).
  • Enabling recovery or backfill behavior such as Nightly Resync.

Nightly Resync

Nightly Resync is an optional pipeline feature that performs an automatic 24-hour data resynchronization at the end of each day.

When enabled, IMSURGE will re-fetch the previous day’s data to account for:

  • Temporary connectivity issues
  • Sensor or network outages
  • Delayed or late-arriving data from the source system

This helps ensure data completeness without requiring manual re-runs.

How Nightly Resync Works

  • Runs once per day, automatically
  • Re-syncs the previous 24 hours of data
  • Requires a polling frequency to be set
  • Does not run for pipelines configured as Manual Only
  • Uses the same source and target configuration as the pipeline
  • Does not affect the pipeline’s normal polling schedule

When to Enable Nightly Resync

  • Pipelines connected to cellular or intermittently connected instruments
  • Sources known to deliver delayed data
  • Critical monitoring pipelines where data gaps are unacceptable

Enabling Nightly Resync You can enable Nightly Resync:

  • When creating a new pipeline
  • When editing an existing pipeline via Edit Settings

Duplicating Pipelines

Duplicating a pipeline creates a new one that copies the configuration of the original, including its source and target integrations and any metadata established during prior executions. This ensures that the new pipeline points to the same sensors in the target system and avoids creating duplicates there.

The duplicated pipeline is identical to the original but starts with no execution history. You can then adjust its settings, such as giving it a new name or assigning a different polling rate.

Common use cases for duplicating:

  • Creating a second pipeline with the same source and target but a different polling interval (e.g., hourly sync plus daily monitoring).
  • Setting up a backup or alternative schedule for pipelines connected to instruments with intermittent connectivity.
  • Preserving existing sensor mappings when creating a new pipeline to the same target system.

Step-by-Step

  1. Navigate to the Pipeline: Go to the Pipelines page and click on the pipeline you wish to manage.
  2. Choose an Action:
    • Select Edit Settings to modify the pipeline in place.
    • Select Duplicate to clone it into a new pipeline.
  3. Configure Settings:
    • Adjust polling frequency, Nightly Resync, or other options as needed.
  4. Save Your Changes:
    • For Edit, changes apply immediately to the existing pipeline.
    • For Duplicate, provide a unique name and save. The new pipeline begins fresh, with up to seven days of backtrack.