Documentation

Lytics File Service

Suggest Edits

Overview

Lytics File Service allows you to import or export data from sources that Lytics does not have a built-in integration with. Data can imported in either a JSON or CSV file and exported as a CSV file for use in ads or campaign targeting.

Authorization

Lytics uses SSH File Transfer Protocol (SFTP) to import files into Lytics and to export user data into files on an SFTP server. You can use your own SFTP service or Lytics can provide a managed SFTP server account. Adding an authorization for use in an import or export job differs if you use our own or a Lytics SFTP account.

If you are new to creating authorizations in Lytics, see the Authorizations Dashboard documentation for more information.

  1. Select Lytics File Service from the list of providers.
  2. Select the method for authorization. Note that different methods may support different job types. Custom supports the following authorization methods:
  3. Enter a Label to identify your authorization.
  4. (Optional) Enter a Description for further context on your authorization.
  5. Complete the configuration steps needed for your authorization. These steps will vary by method.
  6. Click Save Authorization.

Client SFTP Server Authorization

  1. In the Host text field, enter the host name or IP address of the SFTP server you want to connect to.
  2. In the Port text field, enter the port number for the SFTP server.
  3. In the Username text field, enter the username for the SFTP server.
  4. In the Password text field, enter the password for the SFTP server. If you wish to use a Private Key, leave this field empty.
  5. In the Private Key text field, enter the ssh private key for the user for the SFTP server. This key is usually generated by you as a part of a SSH private/public key combination. Lytics is compatible with keys in the OpenSSH format. If you used a tool like PuTTY to generate your key make sure to export it in the OpenSSH format before adding it to Lytics.

    NOTE: If you choose to provide both, then Lytics will try to connect to the SFTP server with the private key first and if that fails then we will try with the password.
  6. In the Passphrase text field, enter the passphrase for the private key you entered if any.
  7. In the Folder text field, enter the relative path to the folder to import or export file. If the path is not given, then we will export the file to the user's home directory.
    client-sftp-server-auth

Client SFTP Server Authorization with PGP Private Key

This authorization lets you import JSON or CSV files to Lytics that have been encrypted using your PGP private key. In order to add this authorization, please make sure you have your PGP keys set up first. See PGP encryption for more information.

  1. In the Host text field, enter the host name or IP address of the SFTP server you want to connect to.
  2. In the Port text field, enter the port number for the SFTP server.
  3. In the Username text field, enter the username for the SFTP server.
  4. In the Password text field, enter the password for the SFTP server. If you wish to use a Private Key, leave this field empty.
  5. In the Private Key text field, enter the ssh private key for the user for the SFTP server. This key is usually generated by you as a part of a SSH private/public key combination. Lytics is compatible with keys in the OpenSSH format. If you used a tool like PuTTY to generate your key make sure to export it in the OpenSSH format before adding it to Lytics.

    NOTE: If you choose to provide both, then Lytics will try to connect to the SFTP server with the private key first and if that fails then we will try with the password.
  6. In the Passphrase text field, enter the passphrase for the private key you entered if any.
  7. In the PGP Private Key text field, enter your PGP private key. See PGP private keys for more information.
  8. In the PGP Private Key Passphrase text field, enter the passphrase for your PGP private key.
  9. In the Folder text field, enter the relative path to the folder to import the file from. If the path is not given, then we will connect to the user's directory.
    client-sftp-pgp-private

Client SFTP Server Authorization with PGP Public Key

This authorization lets you export Lytics audiences to a CSV file while also encrypting the content using the provided PGP public key. In order to add this authorization, please make sure you have your PGP keys set up first. See PGP encryption for more information.

  1. In the Host text field, enter the host name or IP address of the SFTP server you want to connect to.
  2. In the Port text field, enter the port number for the SFTP server.
  3. In the Username text field, enter the username for the SFTP server.
  4. In the Password text field, enter the password for the SFTP server. If you wish to use a Private Key, leave this field empty.
  5. In the Private Key text field, enter the ssh private key for the user for the SFTP server. This key is usually generated by you as a part of a SSH private/public key combination. Lytics is compatible with keys in the OpenSSH format. If you used a tool like PuTTY to generate your key make sure to export it in the OpenSSH format before adding it to Lytics.

    NOTE: If you choose to provide both, then Lytics will try to connect to the SFTP server with the private key first and if that fails then we will try with the password.
  6. In the Passphrase text field, enter the passphrase for the private key you entered if any.
  7. In the PGP Public Key text field, enter PGP public key. See PGP public keys for more information.
  8. In the Folder text field, enter the relative path to the folder to export file to. If the path is not given, then we will connect to the user's directory.
    client-sftp-pgp-public

Lytics Managed SFTP Server

Lytics provides a managed SFTP server where you can provision an account to move data into and out of Lytics.

The Lytics managed SFTP server URL and port are:

SFTP Host Name: sftp.lytics.io 
SFTP Port Number: 22

For credentials to connect to Lytics Managed SFTP, you will create it according to your requirements using one of the following authorizations:

When you add an authorization, Lytics creates a new user using the information provided in the Lytics managed SFTP server. You can use those user credentials to upload or download a file.

📘

The file in Lytics Managed SFTP server will remain for 90 days. Please manage it accordingly.

📘

Updating the file modified time is not supported by the Lytics Managed SFTP server. The modified time is set when the file is uploaded to the server and cannot be changed.

📘

Deleting the a Lytics Managed SFTP server authorization in the Lytics platform, will remove the username from accessing the SFTP server.

Lytics Managed SFTP Server Authorization

  1. In the Username text box, enter a username. This username must be between 3 to 32 characters. Username can have characters a-z, A-Z, 0-9, hyphen(-), and underscore(_). It cannot start with hyphen(-).
  2. In the Password password box, enter a password. The password must be between 8 to 32 characters and include at least one uppercase letter, lowercase letter, number and special character.
  3. In the Public Key text box, enter your SSH public key. This key is usually generated by you as a part of a SSH private/public key combination. Lytics is compatible with keys in the OpenSSH format. If you used a tool like PuTTY to generate your key make sure to export it in the OpenSSH format before adding it to Lytics.

    NOTE: Either password or public key must be provided.
    lytics-sftp-auth

Lytics Managed SFTP Server Authorization with PGP Private Key

This authorization lets you import JSON or CSV files that have been encrypted with your PGP private key. In order to add this authorization, please make sure you have your PGP keys set up. See PGP encryption for more information.

  1. Create a user for Lytics managed SFTP server if you have not already.
  2. Use the credentials of that user and Lytics managed SFTP server host details (Host name: sftp.lytics.io Port: 22) to create SFTP Server authorization with PGP Private Key using Client SFTP Server Authorization with PGP Private Key.

Lytics Managed SFTP Server Authorization with PGP Public Key

This authorization lets you export Lytics audiences to a CSV file while encrypting the content using the provided PGP public key. In order to add this authorization, please make sure you have your PGP keys set up. See PGP encryption for more information.

  1. Create a user for Lytics managed SFTP server if you have not already.
  2. Use the credentials of that user and Lytics managed SFTP server host details (Host name: sftp.lytics.io Port: 22) to create SFTP Server authorization with PGP Public Key using Client SFTP Server Authorization with PGP Public Key.

JSON Import

Lytics allows you to import data via a JSON file for data sources we do not natively integrate with.

Integration Details

  • Implementation Type: Server-side Integration.
  • Implementation Technique: File Based Transfer Integration.
  • Frequency: Batch Integration. once a day at a scheduled time.
  • Resulting data: User Profiles once the imported data is mapped via LQL.

This integration parses JSON files and imports fields and associated data. The JSON file must be a JSON New Line Delimited file where each line contains a single record. Once the integration is started it will:

  1. Connect to the SFTP server.
  2. Navigate to the configured folder.
  3. Read the selected JSON file.
  4. Import all the fields in the JSON file.

Fields

Once you choose the file to import during configuration, Lytics will read the file and list all the fields that can be imported. You may select fields that you wish to import.

Configuration

Follow these steps to set up and configure an import job for Lytics File Service in the Lytics platform. If you are new to creating jobs in Lytics, see the Data Sourcesdocumentation for more information.

  1. Select Lytics File Service from the list of providers.
  2. Select the Import JSON job type from the list.
  3. Select the Authorization you would like to use or create a new one.
  4. Enter a Label to identify this job you are creating in Lytics.
  5. (Optional) Enter a Description for further context on your job.
  6. Complete the configuration steps for your job.
  7. From the Stream input, select the data stream where the data will be posted or enter a name to create a new stream.
  8. (Optional) From the Folder input, select the folder where the JSON file is located.
  9. (Optional) From the File input, select the file to import. Listing files may take up to a couple minutes after the bucket is chosen.
  10. (Optional) From the Timestamp Field input, select the field name that contains the timestamp of an event. If no field is specified, the event will be time stamped with the time of the import.
  11. (Optional) Select the Keep Updated checkbox, to continuously run this import.
    custom-json-import-pt1
  12. (Optional) Toggle Show Advanced Options tab to expand the advanced configuration.
  13. (Optional) In the Prefix text box, enter the file name prefix. You may use regular expressions for pattern matching. The prefix must match the file name up to the timestamp. A precalculated prefix derived from the selected file will be available as a dropdown.
  14. (Optional) From the Time of Day input, select time of day to start importing each day.
  15. (Optional) From the Timezone input, select the timezone for the time of day you selected above.
  16. Click Start Import.
    custom-json-import-pt2

CSV Import

Lytics allows you to import data via a CSV file from data sources we do not natively integrate with.

Integration Details

  • Implementation Type: Server-side Integration.
  • Implementation Technique: User Field.
  • Frequency: Batch Integration, frequency can be configured.
  • Resulting data: User Profiles once the imported data is mapped via LQL.

This integration parses CSV files and imports fields and associated data. Once the integration is started it will:

  1. Connect to the SFTP server.
  2. Navigate to the configured folder.
  3. Read the selected CSV file.
  4. Import the fields that are chosen during configuration.

Fields

Once you choose the file to import during configuration, Lytics will read the file and list all the fields that can be imported. You may select fields that you wish to import.

Configuration

Follow these steps to set up and configure an import job for Lytics File Service integrations in the Lytics platform. If you are new to creating jobs in Lytics, see the Data Sourcesdocumentation for more information.

  1. Select Lytics File Service from the list of providers.
  2. Select the Import CSV job type from the list.
  3. Select the Authorization you would like to use or create a new one.
  4. Enter a Label to identify this job you are creating in Lytics.
  5. (Optional) Enter a Description for further context on your job.
  6. Complete the configuration steps for your job.
  7. In the Stream text box, select the data stream where the data will be posted or enter a name to create a new stream.
  8. (Optional) From the Folder input, select the folder where the CSV file is located.
  9. From the File input, select the file to import, or if this is a continuous import of dated files e.g., file_20160218.csv, choose the earliest dated file to import, and all files will be imported from that date forward.
  10. (Optional) In the Custom Delimiter text box, enter the delimiter of the file. Default delimiter is a comma, ,. Other common delimiters may be "|" (pipe) or ";". For tab delimited files enter "t".
  11. (Optional) From the Timestamps input, select one or more columns in the CSV that contain the timestamp of an event. Sort in order of importance (e.g., "Updated" should be above "Created"). If no fields are specified, the event will be time stamped with the time of the import. Timestamp selection is unavailable for zip files larger than 100MB.
  12. (Optional) From the Fields input, select fields to import. If left empty all fields will be imported. If no field names appear, check to ensure the CSV file has an appropriate header row or the delimiter may need to be changed. Field selection is unavailable for zip files larger than 100MB.
  13. (Optional) In the Prefix text box, enter the file name prefix. You may use regular expressions for pattern matching. The prefix must match the file name up to the timestamp. A precalculated prefix derived from the selected file will be available as a dropdown.
  14. (Optional) Select the Keep Updated checkbox to run this import continuously checking for new files periodically.
  15. (Optional) From the File Upload Frequency input, select how often to check for a new file.
  16. (Optional) From the Time of Day input, select the time of day to start import.
  17. (Optional) From the Timezone input, select the timezone for the time of day you selected above.
  18. (Optional) Select the Diff checkbox to compare file contents to the previous file contents during continuous import and import only rows that have changed. This is useful when large amounts of data remain unchanged in each file.
  19. Click Start Import.
    custom-csv-import

CSV Export

Lytics allows you to export your audiences to a CSV file for importing into other tools that Lytics does not natively integration with.

Integration Details

  • Implementation Type: Server-side Integration.
  • Implementation Technique: File Based Transfer Integration.
  • Frequency: Batch Integration, frequency can be configured.
  • Resulting data: User Fields in a file.

Fields

You can select and export any Lytics user fields to a CSV file.

Export CSV for Download

Export Lytics users to a CSV file that can be downloaded. You will receive an email with a link to download the file.

Follow these steps to set up and configure an export job for Lytics File Service in the Lytics platform.

  1. Select Lytics File Service from the list of providers.
  2. Select the Email CSV job type from the list.
  3. Select the Authorization you would like to use or create a new one.
  4. Enter a Label to identify this job you are creating in Lytics.
  5. (Optional) Enter a Description for further context on your job.
  6. Select the audience to export.
  7. Complete the configuration steps for your job.
  8. In the Email text box, enter the email address to recieve the link to download.
  9. Click on the Show Advanced Options tab to expand the advanced configuration.
  10. (Optional) From the Compress input, select the compression type.
  11. (Optional) Select the Include Headers checkbox to include the field names in the first row of the exported file.
  12. (Optional) Select the Audience Name checkbox to add a column with the name of the exported Lytics audience.
  13. (Optional) Select the Additional Audiences checkbox to add a column containing all the Lytics audiences a user is a member of.
  14. (Optional) From the Fields to Export input, select a list of user fields to export.
  15. (Optional) In the User Limit numeric field, enter the maximum number of users to export. Leave blank to export all users.
  16. (Optional) In the Custom Delimiter text box, enter a custom delimiter for your file. Default delimiter is ",". A recommended custom delimiter is "|" (pipe).
  17. (Optional) From the Custom Join input, select a custom join character for fields that have multiple values. Default is to join multi-value fields with "&", e.g., emails=[email protected]&[email protected].
  18. (Optional) Select the Continuous Export checkbox, to email CSV file every day.
  19. (Optional) From the Time of Day input, select the time of day to start export each day.
  20. (Optional) From the Timezone input, select the timezone for time of day.
  21. Click Start Export.
    csv-email-export

Once the CSV file has been created, you will receive an email with a link to download it. The download link will expire after 24 hours.

Export the CSV to an SFTP Server

You can opt to export your CSV to an SFTP server. This server can be your own or a Lytics managed SFTP server. The audience CSV file will be named lytics_{segment_slug}_{date_time}.csv, and can optionally be compressed into a zip or gz archive.

Follow these steps to set up and configure an export job for Lytics File Service in the Lytics platform.

  1. Select Lytics File Service from the list of providers.
  2. Select the Export CSV via SFTP job type from the list.
  3. Select the Authorization you would like to use or create a new one.
  4. Enter a Label to identify this job you are creating in Lytics.
  5. (Optional) Enter a Description for further context on your job.
  6. From the Audience Name drop-down, select the Lytics audience that contains the users to export.
  7. (Optional) From the Compress input, select the compression type for the file to be exported.
  8. (Optional) Select the Include Headers checkbox, to include the field names as the first row of the exported file.
  9. (Optional) Select the Audience Name checkbox, to select to add a column with the name of the exported Lytics audience.
  10. (optional) Select the Audience Slug checkbox, to add a column with the slug of the exported Lytics audience.
  11. (Optional) Select the Additional Audiences checkbox, to add a column containing all the Lytics audiences a user is a member of.
  12. (Optional) From the Fields to Export input, select the list of user fields to export. If none are selected then every user field will be included.
  13. (Optional) In the User Limit numeric field, enter the maximum number of users to export. Leave blank to export all users.
  14. (Optional) In the Custom Delimiter text box, enter a custom delimiter for your file. Default delimiter is ",". A recommended custom delimiter is "|" (pipe).
  15. (Optional) From the Custom Join input, select a custom join character for fields that have multiple values. Default is to join multi-value fields with "&", e.g., emails=[email protected]&[email protected].
  16. (Optional) Select the Continuous Export checkbox, to export at the specified frequency selected in File Export Frequency option.
  17. (Optional) From the File Export Frequency input, select how often to export the new file.
  18. (Optional) From the Time of Day input, select time of day to start export each day.
  19. (Optional) From the Timezone input, select timezone for time of day.
  20. (Optional) From the Timestamp Format input, select the format for the timestamp that will be added at the end of the filename. MM = month, mm = minute. If None (omit timestamp) is selected, the file name will not contain a timestamp. For continuous exports, this will result in the file being overwritten on each run.
  21. Click Start Export.
    csv-export-server.png

Updated 11 days ago