Overview

Amazon Kinesis Data Streams is a real-time data streaming service provided by Amazon.

Integrating Lytics with AWS Kinesis allows you to bring in data from Kinesis Data Streams and leverage Lytics to enrich user profiles and build behavioral audiences in Lytics. You can then send Lytics audience events as triggers back to your Kinesis Data Streams.

Authorization

If you have not already done so, you will need to set up an AWS Kinesis account before you begin the process described below.

You can authorize the integration in one of two ways:

Providing your AWS keys
Delegating Access via AWS IAM

Providing your AWS Keys

Follow the steps below to authorize AWS with Lytics using your AWS keys. For more information on obtaining your keys, see Amazon's documentation on secret and access keys.

Select Amazon Web Services from the list of providers.
Select the AWS Keys method for authorization.
Enter a Label to identify your authorization.
(Optional) Enter a Description for further context on your authorization.
Enter your Access Key and Secret Key.
Click Save Authorization.

Delegating Access via AWS IAM

Instead of providing AWS credentials directly, you can grant Lytics access to your Kinesis streams by creating an IAM role in your AWS account. Below is a set of instructions for how to set up delegated authorization.

1. Create an IAM Role

Create a role with the following trust policy, which allows Lytics to assume it:

{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Principal": {
      "AWS": "arn:aws:iam::358991168639:user/gce1"
    },
    "Action": "sts:AssumeRole"
  }
}

2. Attach a Permission Policy

Attach a policy to the role granting the Kinesis actions Lytics needs. Replace YOUR_ACCOUNT_ID, region, and stream name with your values.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "kinesis:PutRecord",
        "kinesis:PutRecords",
        "kinesis:GetRecords",
        "kinesis:GetShardIterator",
        "kinesis:ListShards"
      ],
      "Resource": "arn:aws:kinesis:us-east-1:YOUR_ACCOUNT_ID:stream/YOUR_STREAM"
    },
    {
      "Effect": "Allow",
      "Action": "kinesis:ListStreams",
      "Resource": "*"
    }
  ]
}

The kinesis:ListStreams action requires "Resource": "*" because it does not support resource-level permissions. This permission is used by the Lytics UI to display available streams when configuring a job.

If you prefer not to include kinesis:ListStreams in your IAM policy, you can manually enter the stream name in the job configuration.

You may also separate permissions into distinct IAM policies for import-only and export-only use cases, as appropriate.

3. Configure in Lytics

When configuring your Kinesis import or export job, enter the role ARN (e.g. arn:aws:iam::YOUR_ACCOUNT_ID:role/Lytics/LyticsKinesisAccess) in the Role ARN field.

To learn more, review the following Amazon reference materials:

Import Activity Data

Import data from your AWS Kinesis Data Streams to use Lytics data science scoring and Insights to build rich, behavioral audiences.

Integration Details

Implementation Type: Server-side Integration
Type: REST API Integration
Frequency: Real-time Integration
Resulting Data: Import of Event Data published in AWS Kinesis Data Stream.

Fields

Any fields that are present in the event data in your AWS Kinesis Stream are posted in your Lytics Stream that is configured during the job setup. You can map them to Lytics User Profiles using Custom LQL. Please contact Lytics Support to find out more about this option.

Configuration

Follow these steps to set up and configure an AWS Kinesis Import job in the Lytics platform. If you are new to creating authorizations in Lytics, see the Authorizations documentation for more information.

Select Amazon Web Services from the list of providers.
Select the Import Activity Data (Kinesis) job type from the list.
Select the Authorization you would like to use or create a new one.
Enter a Label to identify this job you are creating in Lytics.
(Optional) Enter a Description for further context on your job.
Enter the name of the Lytics Stream where the data will be imported into.
From the Region drop-down, select the AWS region that has the Kinesis Stream you want to import data from.
Using the AWS Role ARN textbox , enter the valid AWS Role ARN to AssumeRole into that has access to Kinesis read operations. Only necessary if you are using delegated auth.
From the Kinesis Stream drop-down, select the AWS Kinesis Stream from which you are bringing in the data.
Click Start Import.

Export Audience Triggers

Send Lytics audience event triggers to your AWS Kinesis Data Streams to trigger a message when a user enters or exits a Lytics audience.

Integration Details

Implementation Type: Server-side Integration
Type: REST API Integration
Frequency: Real-time Integration, with an optional one-time Backfill of the audience after setup.
Resulting Data: Event Data published to AWS Kinesis Stream.

Fields

The fields included depend on the raw event in Lytics. All user fields will be included in the data published to your AWS Kinesis Data Stream unless specified in the Export Fields selection.

Configuration

Follow these steps to set up and configure the AWS Kinesis Trigger job in Lytics Platform.

Select Amazon Web Services from the list of providers.
Select the Export Audience Triggers job type from the list.
Select the Authorization you would like to use or create a new one.
Enter a Label to identify this job you are creating in Lytics.
(Optional) Enter a Description for further context on your job.
From the Audiences list, select the Lytics audiences with events you want to send to the Kinesis Stream.
From the Stream Region drop-down, select the AWS region for the Kinesis Stream.
(Optional) Using AWS Role ARN input, enter the Role ARN that will be used if you selected the delegated authorization method.
From the Kinesis Stream drop-down, select the AWS Kinesis Stream that you want to publish Lytics events.
(Optional) From the User Identifier Field drop-down, select the field that will be used as Kinesis Partition Key.
(Optional) From Export Fields select all the user fields to include. If none are selected then all user fields will be sent.
(Optional) Select the Existing Users checkbox to enable a backfill of current members of the audience(s) as enter event.
Click Complete to start job.