Documentation
APICreate a Ticket
  • 📔Documentation Guide
  • 📨Data & Engagement Platform
    • Data Model
      • Customer Data
      • Product Catalogue
      • Events
        • Onsite Events
        • Outbound Events
        • Transactions
    • App User Management
      • User Roles
      • App Users
    • Data Management
      • Data Management with API
      • Data Management with Data Feeds
        • Users
          • Import Users
          • Update or Delete a User Trait via File Upload
          • List of Standard User Traits/Attributes
        • Products
        • Events
          • Import Orders
          • Import Events
          • List of Events and Properties
      • Data Export
    • Web Tracking
      • Web Tracking v1 (Legacy)
        • Setup
        • Set up business Units
        • Tracking User Behavior
          • Identify Method
          • Page Method
          • Track Method
          • Opt-Out from Tracking
        • Testing & Debugging
      • Web Tracking v2
        • Setup using Google Tag Manager
        • Setup without a Tag Manager
        • Configuration & Config Commands
        • Tracking User Behavior
          • Identify Method
          • Update Method
          • Page Method
          • Track Method
          • Opt-Out from Tracking
        • Testing & Debugging
      • Migrate Tracking SDK v1 -> v2
      • Mobile Web Tracking
    • User Segmentation
      • Creating a Segment
      • Conditions
      • Combining Segments
    • Messages
      • Frequency Capping
      • Templates
        • Template Builder
      • Integrations
        • Channels
          • SMS
            • Twilio
              • Set up a Twilio Account
              • Set up Twilio Integration
              • Create a Campaign Message
            • Link Mobility
              • Set Up Link Mobility Account
              • Set up Link Mobility Integration
              • Create a Campaign Message
          • SFTP
            • Set up SFTP Integration
            • Create a Campaign Message
          • Direct Mail
            • Optilyz
              • Set up Optilyz Integration
              • Create a Campaign Message
          • Webhooks
            • Single Webhook
              • Set up Webhook Integration
              • Create a Campaign Message
              • Response Data and Custom Events
            • Batch Webhook
              • Set up Batch Webhook Integration
              • Create a Campaign Message
            • Zenloop via CrossEngage Webhook
              • Set up Zenloop Integration (via CrossEngage Webhook)
              • Set up Zenloop Survey
              • Create a Campaign Message
              • Set up Zenloop Survey via ESP
              • Obtain Response Data
            • Google Analytics via Webhook
          • Segment Transfer
            • Facebook
              • Set up Facebook Developer Account
            • Optimizely
              • Set up Optimizely Account
            • Google Analytics
              • Set up Google Analytics Integration
              • Create a Campaign with Google Analytics
              • Using the Google Analytics Integration
            • Airship
              • Set up an Airship Account
              • Set up an Airship Integration
              • Create a Campaign Message with Airship
          • Onsite Display
            • Trbo
              • Set up Trbo Integration
              • Create a Campaign Message in CrossEngage
              • Configure Campaign Message in Trbo
              • Obtain Response Data
          • Email
            • Mailjet
              • Set up Mailjet Integration
              • Obtain Response Data via Webhook
              • Create a Campaign Message
              • Personalize Preview Texts in Mailjet
            • Mandrill (by MailChimp)
              • Set up Mandrill Integration
              • Obtain Response Data via Webhook
              • Create a Campaign Messege
            • Inxmail
              • Set up Inxmail Integration
              • Create a Campaign Message
            • Sendgrid (by Twilio)
              • Set up SendGrid Integration
              • Obtain Response Data via Webhook
              • Create a Campaign Message
            • Mailgun
              • Set up Mailgun Integration
              • Obtain Response Data via Webhooks
              • Create a Campaign Message with Mailgun
            • Episerver (Optimizely)
              • Set up Episerver Integration
              • Create a Campaign Message with Episerver
          • Push Notifications
            • Airship
              • Set up an Airship Account
              • Set up an Airship Integration
              • Create a Campaign Message with Airship
        • Attachments
        • Delete an Integration
      • Personalization
        • Import Data
          • User Profile Data
          • Campaign Data
          • Cart Data
        • Formatting Functions
          • Date Formatting
          • Number Formatting
          • String Formatting
          • Hash Functions
        • General Helper Functions
          • Conditional Functions
          • Filtering Arrays
        • Product Helper Functions
          • Fetch from Product Feed
          • Fetch from Tracking Event
          • Fetch from User Journey
        • Misc. Helper Functions
          • Voucher Helper Function
            • Vouchers: Use Case
          • Event Helper Functions
          • Opt Out Helper Functions
        • Operators
      • Vouchers
        • Creating Vouchers
        • Using Vouchers
    • Campaign Management
      • Campaigns
        • Create an Audience Campaign
        • Create a Real-Time Campaign
        • Control Group
      • Stories
        • Building a Story
        • Use Case: Welcome Story
      • Segment Transfer
        • Create a Segment Transfer Campaign
    • Consent Management
      • Subscription/Consent Management
      • System Opt-Out/Opt-In
      • System Blacklist/Whitelist
    • Prediction Models
      • Create a new Model
      • Feature Engineering
      • SQL Filter
    • System Monitoring
      • Dashboard
        • Segment Tracker
      • Events Overview
      • Activity Log
      • Slack Notifications
        • Setting up Slack Notifications
    • Help & Support
      • System Status
      • Reach out to Customer Support
      • Suggest an Improvement
      • Privacy Policy
    • Glossary - Data & Engagement
  • 📈Predictions Platform
    • Data Model
      • Customer Data
      • Transactions
      • Activities
    • Overview
      • Data Tab
        • Data Tables in the Predictions Platform
      • Insights Tab
      • Model Builder Tab
        • Feature Engineering
        • Custom SQL Filter
        • Model Report
      • Prediction Tab
      • Selections Tab
    • Tutorials
      • Prepare and Validate Data
      • Analyze RFM Customers
      • Create a new Model
      • Predict Campaign Profit
    • Glossary - Predictions
Powered by GitBook
On this page
  • Data Format
  • File Structure and Naming
  • Customer Data
  • Transaction Data
  • Activity Data
  1. Predictions Platform

Data Model

In the CrossEngage Predictions Platform, data is divided into three types; Customer, Transaction and Activity data. Customer and Transaction data are required for creating Prediction Models, while Activity data is optional.

Data Format

All data is uploaded to the Predictions Platform in the form of CSV files, encoded in UTF-8. The structure of the CSV files should comply with the RFC4180 Standard, with the Extension that a header with the field names is included in the file.

  • The fields of the CSV are separated by a comma (,) by default. Note that you can also use other characters, by changing the default settings of how the file is read.

  • Each line has the same number of fields.

  • Each line has a carriage return at the end (CRLF). The last line may or may not have a carriage return after it.

  • If there are quotation marks, the delimiter (,), or a new line in the fields, the fields must be separated with quotation marks and quotation marks within the fields must be indicated by writing them in pairs.

Examples

The following is an example of valid CSV, where values are not enclosed in quotation marks.

customer_id, comment, activity_type, activity_timestamp
209147, template: Campaign1, email_sent, 13:00:00
209147, , email_click, 2020-06-17 13:09:14

Now if one or more values contain Carriage Return or the Delimiter (,), we can re-write this file with quotation marks:

"customer_id", "comment", "activity_type", "activity_timestamp"
"209147", "template: Campaign1, upsell", "email_sent", "13:00:00"
"209147", "", "email_click", "2020-06-17 13:09:14"

In this example, the first value of "comment" now contains a comma (template: Campaign1, upsell).

Similarly, if one of more values contain Quotation marks, we can re-write the file where every value is enclosed in quotation marks, and quotation marks inside fields are doubled.

"customer_id", "comment", "activity_type", "activity_timestamp"
"209147", "template: ""Campaign1"", upsell", "email_sent", "13:00:00"
"209147", "", "email_click", "2020-06-17 13:09:14"

In this example, the first value of "comment now contains quotation marks, that are written twice (template: ""Campaign1"", upsell).

File Structure and Naming

The CrossEngage Predictions Platform uses two types of files to maintain up-to-date data, Base files and Delta files.

Base files contain data of a certain period and are part of each data package, e.g. transaction files for individual years. These files contain complete data for a specific time period, and are not expected to be replaced later.

Delta files contain data of a time period that has not yet fully elapsed, for example, transactions of the current year. Subsequent uploads of current year transactional data will replace the previous data, as the new file is a superset of the old one.

Customer Data

Since your collection of Customers is never considered "complete", customer files are always Delta files. This means that with every Upload, you upload the complete and most-up-to-date Customer data to replace the previous version. Customer Data files use the same name every time: customers_delta.csv.

Transaction Data

Transaction files containing all transaction records of 1 year or multiple years follow the following naming scheme:

  • Transactions 2015 -> transactions_base_2015.csv

  • Transactions 2016 -> transactions_base_2016.csv

  • Transactions 2017-2020 -> transactions_base_2017_2020.csv

Transaction files containing partial records of the current year use the following naming scheme:

  • Transactions 2023 Q1 -> transactions_delta_2023.csv

  • Transactions 2023 Q1+Q2+Q3 -> transactions_delta_2023.csv

Once the year is complete, and a single file with complete records of the year is created, it should be named as a base file again. Uploading a base file for 2023 will cause the platform to disregard the previous delta files for that year.

Activity Data

Activity files follow a similar naming scheme to the transaction files:

  • Outbound Activities 2015 -> activities_outbound_base_2015.csv

  • Outbound Activities 2016 -> activities_outbound_base_2016.csv

  • Outbound Activities 2017-2020 -> activities_outbound_base_2017_2020.csv

Activity files containing partial records of the current year use the following naming scheme:

  • Outbound Activities 2023 Q1 -> activities_outbound_delta_2023.csv

  • Outbound Activities 2023 Q1+Q2+Q3 -> activities_outbound_delta_2023.csv

Once the year is complete, and a single file with complete records of the year is created, it should be named as a base file again. Uploading a base file for 2023 will cause the platform to disregard the previous delta files for that year.

PreviousGlossary - Data & EngagementNextCustomer Data

Last updated 1 year ago

For more information on the Data Tab, please .

For more information on Preparing and Validating your data, please .

📈
click here
click here