Overview

Amazon Kinesis Data Streams is a real-time data streaming service provided by Amazon.

Integrating Lytics with AWS Kinesis allows you to bring in data from Kinesis Data Streams and leverage Lytics to enrich user profiles and build behavioral audiences in Lytics. You can then send Lytics audience events as triggers back to your Kinesis Data Streams.

Authorization

If you have not already done so, you will need to set up an AWS Kinesis account before you begin the process described below.

You can authorize the integration in one of two ways:

Providing your AWS keys
Lytics Delegated authorization For Export

Providing your AWS Keys

Follow the steps below to authorize AWS with Lytics using your AWS keys. For more information on obtaining your keys, see Amazon's documentation on secret and access keys.

Select Amazon Web Services from the list of providers.
Select the AWS Keys method for authorization.
Enter a Label to identify your authorization.
(Optional) Enter a Description for further context on your authorization.
Enter your Access Key and Secret Key.
Click Save Authorization.

Lytics Delegated Authorization for Export

Below is a set of instructions for how to set up delegated authorization. This method of doing authorization is more complicated than the AWS Keys method but some people prefer it:

Set up Policy and Roles in your AWS Kinesis account.
Contact Lytics support with your role Amazon Resource Name (ARN) that we grant permission to.

Here are related Amazon reference documents:

Stream Name: The name of the Kinesis stream that you will be writing to. It can be any alphanumeric string plus underscores, periods, or dashes.

This example shows lytics_triggers_stream.

# 1 Setup AWS CLI Permissions, change name of "my_aws_account"
# Assumes you have setup the AWS cli https://aws.amazon.com/cli/
aws configure --profile=my_aws_account
> ....

# 2 Ensure we have a kinesis stream, Create stream if need be
aws kinesis list-streams --profile=my_aws_account
aws kinesis describe-stream --stream-name lytics_triggers_stream --profile=my_aws_account
aws kinesis create-stream --stream-name lytics_triggers_stream --shard-count=1 --profile=my_aws_account
# cleanup if needed
aws kinesis delete-stream --stream-name lytics_triggers_stream --profile=my_aws_account

# 3 replace your AWS account-id below wherever you see 111111111

# 3a Create a Role in NonLytics account that allows a lytics user to
# assume identity/permission inside this account in order to write to kinesis.
aws iam list-roles --profile=my_aws_account --path-prefix="/Lytics"
aws iam create-role --profile=my_aws_account --role-name="LyticsWriteToKinesis" \
  --path="/Lytics/WriteToKinesis/" \
  --assume-role-policy-document '{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Principal": { 
      "AWS": "arn:aws:iam::358991168639:user/gce1"
    },
    "Action": "sts:AssumeRole"
  }
}'

# 3b NonLytics Account: Create A policy allowing access to a specific Kinesis stream
aws iam list-policies --profile=my_aws_account  --scope=Local
aws iam get-policy --profile=my_aws_account --policy-arn="arn:aws:iam::111111111:policy/KinesisAssumeWrites"
aws iam create-policy --profile=my_aws_account \
  --policy-name KinesisAssumeWrites --policy-document '{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "kinesis:PutRecord",
        "kinesis:PutRecords"
      ],
      "Resource": [
        "arn:aws:kinesis:us-west-2:111111111:stream/lytics_triggers_stream"
      ]
    }
  ]
}'

# 3c attach that new policy to previously created role
aws iam attach-role-policy --profile=my_aws_account \
  --role-name LyticsWriteToKinesis \
  --policy-arn "arn:aws:iam::111111111:policy/KinesisAssumeWrites"

# 3d view that role policy
aws iam get-role-policy --profile=my_aws_account --role-name LyticsWriteToKinesis --policy-name KinesisAssumeWrites

Cleanup

# kinesis streams cost $ when idle, so cleanup if test
aws kinesis delete-stream --stream-name lytics_triggers_stream --profile=my_aws_account

# detach role-policy
aws iam detach-role-policy --profile=my_aws_account \
  --role-name LyticsWriteToKinesis \
  --policy-arn "arn:aws:iam::111111111:policy/KinesisAssumeWrites"

# if you need to delete
aws iam list-role-policies --profile=my_aws_account --role-name="LyticsWriteToKinesis"
aws iam delete-role-policy --profile=my_aws_account --role-name="LyticsWriteToKinesis" --policy-name="simulate-inbound-lytics" 
aws iam delete-role --profile=my_aws_account --role-name "LyticsWriteToKinesis"

aws iam list-policies --profile=my_aws_account  --scope=Local
aws iam get-policy --profile=my_aws_account --policy-arn="arn:aws:iam::111111111:policy/KinesisAssumeWrites"
aws iam delete-policy --profile=my_aws_account --policy-arn="arn:aws:iam::111111111:policy/KinesisAssumeWrites"

aws iam delete-role --profile=my_aws_account --role-name "LyticsWriteToKinesis"

Contact Lytics to grant permission

Lytics will need the role Amazon Resource Name (ARN) from your AWS account to grant permission to. Contact Lytics Support with the ARN. It will look like this example if you followed the instructions above.

arn:aws:iam::111111111:role/Lytics/WriteToKinesis/LyticsWriteToKinesis

Import Activity Data

Import data from your AWS Kinesis Data Streams to use Lytics data science scoring and Insights to build rich, behavioral audiences.

Integration Details

Implementation Type: Server-side Integration
Type: REST API Integration
Frequency: Real-time Integration
Resulting Data: Import of Event Data published in AWS Kinesis Data Stream.

Fields

Any fields that are present in the event data in your AWS Kinesis Stream are posted in your Lytics Stream that is configured during the job setup. You can map them to Lytics User Profiles using Custom LQL. Please contact Lytics Support to find out more about this option.

Configuration

Follow these steps to set up and configure an AWS Kinesis Import job in the Lytics platform. If you are new to creating authorizations in Lytics, see the Authorizations documentation for more information.

Select Amazon Web Services from the list of providers.
Select the Import Activity Data (Kinesis) job type from the list.
Select the Authorization you would like to use or create a new one.
Enter a Label to identify this job you are creating in Lytics.
(Optional) Enter a Description for further context on your job.
Enter the name of the Lytics Stream where the data will be imported into.
From the Region drop-down, select the AWS region that has the Kinesis Stream you want to import data from.
From the Kinesis Stream drop-down, select the AWS Kinesis Stream from which you are bringing in the data.
Click Start Import.

Export Audience Triggers

Send Lytics audience event triggers to your AWS Kinesis Data Streams to trigger a message when a user enters or exits a Lytics audience.

Integration Details

Implementation Type: Server-side Integration
Type: REST API Integration
Frequency: Real-time Integration, with an optional one-time Backfill of the audience after setup.
Resulting Data: Event Data published to AWS Kinesis Stream.

Fields

The fields included depend on the raw event in Lytics. All user fields will be included in the data published to your AWS Kinesis Data Stream unless specified in the Export Fields selection.

Configuration

Follow these steps to set up and configure the AWS Kinesis Trigger job in Lytics Platform.

Select Amazon Web Services from the list of providers.
Select the Export Audience Triggers job type from the list.
Select the Authorization you would like to use or create a new one.
Enter a Label to identify this job you are creating in Lytics.
(Optional) Enter a Description for further context on your job.
From the Audiences list, select the Lytics audiences with events you want to send to the Kinesis Stream.
From the Stream Region drop-down, select the AWS region for the Kinesis Stream.
From the Kinesis Stream drop-down, select the AWS Kinesis Stream that you want to publish Lytics events.
(Optional) Using AWS Role ARN input, enter the Role ARN that will be used if you selected the delegated authorization method.
(Optional) From the User Identifier Field drop-down, select the field that will be used as Kinesis Partition Key.
(Optional) From Export Fields select all the user fields to include. If none are selected then all user fields will be sent.
(Optional) Select the Existing Users checkbox to enable a backfill of current members of the audience(s) as enter event.
Click Complete to start job.