Lytics Probabilistic Identity Resolution with Lytics Warehouse and Cloud Connect


Lytics' approach to identity resolution encompasses two common approaches: deterministic and probabilistic.

Deterministic stitching is a data linkage method used by default in the Lytics Identity Graph that relies on precise relationships between keys and identifiers from diverse data sources. This method ensures high accuracy and consistency when connecting and managing user profiles, as it follows a strict and predictable set of rules for matching and merging data. Deterministic matching is often used to create a unified and reliable view of user profiles. This allows organizations to maintain data integrity and make informed decisions based on consolidated and accurate information.

Probabilistic stitching is a data linking technique that optionally enhances the Lytics Identity Graph's capabilities. Unlike deterministic stitching, statistical analysis establishes likely connections between user profiles when clear, definitive relationships are missing. This method provides flexibility, making it valuable in scenarios where strict deterministic criteria are not required, such as upper funnel engagements, thus improving data insights and adaptability for organizations. It retains the graph's core deterministic structure while incorporating probabilistic links.

With probabilistic use cases, Lytics provides an interface for creating a "confidence" score based upon configured "match fields" which can then be associated with your deterministically matched profiles based on business rules and desired use cases.

Regardless of the desired usage of Deterministic and Probabilistic strategies, Lytics offers three essential tools to ensure your organization remains flexible into the future: Conductor, Cloud Connect, and Lytics Warehouse.

Lytics Warehouse + Cloud Connect

For this use case, the assumption is that the core data lives in a data warehouse and has not/will not be streamed directly to Lytics. The customer would like to append data onto Lytics profiles based on defined probabilistic identity rules. The architecture could look like the following.

In this case, the customer can use a Lytics Warehouse or their cloud warehouse as the destination for the data needed to facilitate probabilistic matching. In this example, we use BigQuery; however, customers can also choose to use Cloud Connect with Snowflake, Azure, or Amazon. Customers can then use SQL to generate a Lytics Data Model that will append these attributes to Lytics using the fuzzy matching Logic. These additional attributes can be used for segmentation and channel sync through Lytics Decision Engine. If the fuzzy matching rules need to be changed or modified, it is as easy as modifying the Cloud Connect data model(s) (SQL Query) or creating new ones. This profile enrichment can all be done without damaging the integrity of Lytics’ existing deterministic ID Resolution.

Lytics Entity Matching Workflow to BigQuery + Lytics Warehouse + Cloud Connect

For this use case, the assumption is that all of the needed data is already in Lytics, and the goal is to figure out if profiles in Lytics can be merged using different ID logic, such as business or household-level resolution, rather than individual. For this use case, the customer will export an audience of users out of Lytics using the probabilistic workflow. Customers are able to decide which profile attributes should be used when generating the "confidence score", such as first name, last name, and/or address. This will output a table like the following:

Entity Match Table Schema

Entity Match Table Example

The customer can then combine the resulting entity table with additional profile data in the cloud warehouse to create probabilistic profiles based on different confidence scores. This data can be appended to profiles in Lytics using Cloud Connect. An example of the architecture for this is below:

Regardless of the method, the result will be a robust ID strategy flexible enough to engage with all of your consumers as they move throughout your marketing funnel.