Release Version: BetaRelease VersionBeta Vendor Support: OfficialVendor SupportOfficial

The Specto DataLake integration allows IMSURGE pipelines to deliver processed event data directly into Specto’s managed S3-based data lake. From there, data can be analyzed using Specto’s Tableau dashboards or connected directly to your own BI tools for deeper insights.

Setup Instructions

1. Creating the Credentials

When adding your Specto DataLake integration, you will be prompted to create or select a set of Credentials. To create them, you need:

  • SDL License Key – The key provided by Specto upon account setup.

2. Creating the Integration

After selecting or creating your Credentials, configure the integration with the following parameters:

  • Data Rollover Period – The number of days of data to retain in the root project directory parquet file. Older data is automatically moved to the archive directory.
  • Project – The project folder name where event files will be written.

Reference

FieldDescriptionRequired
SDL License KeyLicense key provided by Specto upon account setup✓
FieldDescriptionRequired
Data Rollover PeriodNumber of days of data present in the root directory parquet file before being archived✓
ProjectProject folder to write event files to✓

Additional Notes

  • Event files are written in Parquet format to a Specto-managed AWS S3 bucket.
  • Specto will provide AWS credentials for accessing the data, or grant access through a Tableau instance for direct visualization.
  • Event files are organized by pipeline name within the S3 bucket.
  • Each project folder includes an archive directory, containing data older than the specified rollover period.

External References