Read Blog Post

Read Blog Post

Contributed on July 4, 2023

1

33

Data Monetization: How Snowflake Data Share and CDC can help monetise your data?

Blog: Indium Software - Big Data

Data monetization

It is the practice of generating revenue or extracting value from data assets by utilizing owned or accessed data to gain insights, make informed decisions, and establish fresh revenue streams. It has become increasingly crucial in the digital era, where businesses, organizations, and individuals accumulate and generate vast quantities of data.

How can you monetise and why is it prominent in the data world?

In the realm of data, one prominent method of monetization is through targeted advertising. In this process, companies gather extensive data on user behavior, preferences, and demographics, enabling them to gain insights into individual interests and requirements. Subsequently, this valuable data is harnessed to deliver highly personalized advertisements to specific individuals or segmented groups within the population.

Targeted advertising occupies a prominent position in the data world due to multiple compelling reasons:

1. Enhanced effectiveness: By leveraging data insights, advertisers can customize their messaging to specific audiences who are more likely to show interest in their products or services. This results in improved conversion rates and a more optimized utilization of advertising budgets.

2. Elevated user experience: Relevant and personalized advertisements deliver value to users by showcasing offerings that align with their interests. This elevates the overall user experience and minimizes the perception of intrusive or irrelevant advertising.

3. Increased revenue potential: Targeted advertising has the potential to generate higher revenues for both advertisers and publishers. Advertisers are willing to invest premium amounts to reach their ideal audience, while publishers can command higher rates for ad space when they can demonstrate the effectiveness of their targeted advertising capabilities.

4. Data-driven decision making: Monetizing data through targeted advertising necessitates sophisticated data analytics and insights. This drives the advancement of cutting-edge data analytics tools, machine learning algorithms, and data science techniques. Consequently, the data world continues to progress and innovate, enabling improved decision making and business strategies rooted in data-driven insights.

Snowflake

Data Warehousing is the process of bringing data from various sources into one place to gather different business insights. This is largely helping to understand the business users and make various decisions on demand. Snowflake is playing a crucial role as a unified and fully managed warehouse cloud platform to store and compute huge amounts of data. The decoupled model of Snowflake, storage, and compute greatly facilitates organizations attainment of a cost-effective warehouse system based on demand. One of the cool features called Snowflake Share enables data sharing among the organizational accounts, which enables the segregation of data production and consumption with their computation adapted. Let us see how this Snow Share works and enables us to do the change tracking from the consumer account.

How can data be monetized in snowflakes?

• Snowflake provides a platform for data monetization, enabling businesses to leverage data effectively.
• It facilitates data collection from diverse source systems.
• The platform enables the transformation of large datasets into valuable business insights through analytics.
• Snowflake ensures the secure sharing of raw or processed data with third parties.

Snowflake sharing

Snowflake Sharing is one of the features that allows users to share data securely and efficiently with customers, partners, and suppliers. It enables users to share data without compromising security or control. Users should define access policies and rules, including user rules and permissions, through which authorized users have access to see the data. In general, a snowflake database object can be shared with the direct target, a list, listed and a group of read-only accounts (consumers) within and across regions (through replication) from a producer account. Let us see how this sharing can be done as a producer account and consumed as a consumer account.

Different forms of sharing

➔   Snowflake Secure Data Sharing between the Same Regions

Snowflake provides a secure and efficient way to share data between Snowflake accounts within the same region.

To share data between Snowflake accounts in the same region, you need to set up the required roles and privileges on the data. ACCOUNTADMIN: The ACCOUNTADMIN role is required to set up secure data sharing. This role can create and manage the required database objects and grant privileges to other roles. In addition to the roles, you will need to grant the appropriate privileges to each role. The specific privileges required will depend on the requirements of your data sharing use case.

The following commands need to be executed by the producer.

CREATE OR REPLACE SHARE SHARE1;
GRANT USAGE ON DATABASE PRIMARY_DB TO SHARE SHARE1;
GRANT USAGE ON SCHEMA PRIMARY_DB.SCHEMA1 TO SHARE SHARE1;
GRANT SELECT ON TABLE PRIMARY_DB.SCHEMA1.EMPLOYEE TO SHARE SHARE1;

Include the account in the SHARE1 share

ALTER SHARE SHARE1 ADD ACCOUNTS=org_name.consumer_name;

Consumers are required to execute the following commands.                                     

CREATE DATABASE SECONDARY_DB FROM SHARE org_name.producer_name.SHARE1;

➔   Snowflake Secure Data Sharing between regions

Snowflake provides a solution for securely sharing data between regions. By leveraging Snowflake’s cloud-based architecture and advanced security features, users can share sensitive data with other region accounts without compromising on security or performance.

To execute the following commands,run them on ACCOUNT PRODUCER 1.

USE ROLE ORGADMIN;
SELECT SYSTEM$GLOBAL_ACCOUNT_SET_PARAMETER(‘org_name.AP_SOUTH_EAST_ACCOUNT’,’ENABLE_ACCOUNT_DATABASE_REPLICATION’, ‘true’);

Creating primary database

CREATE DATABASE PRODUCE_DB_1;
USE PRODUCE_DB_1;
CREATE SCHEMA PROCDUCER_SCHEMA;
CREATE TABLE PRODUCER_TABLE (ID INT,NAME VARCHAR(255),BRANCH_CODE INT,LOCATION VARCHAR(255));
ALTER TABLE PRODUCER_TABLE SET CHANGE_TRACKING = TRUE;

Creating AWS_AP_SOUTH_1 account

USE ROLE ORGADMIN;
CREATE ACCOUNT AP_SOUTH_PRODUCER_ACCOUNT
admin_name=ADMIN_NAME
admin_password=’PASSWORD’
first_name=AKHIL
last_name=TUMMAPUDI
email=’****@gmail.com’
edition=ENTERPRISE
region=AWS_AP_SOUTH_1;
Select system$global_account_set_parameter(org_name.AP_SOUTH_PRODUCER_ACCOUNT’,’ENABLE_ACCOUNT_DATABASE_REPLICATION’,’TRUE’);

You can replicate to the AWS_AP_SOUTH_1 account and promote an existing database in your local account as the primary one.

use role accountadmin;
alter database PRODUCER_DB_1 enable replication to accounts org_name.AP_SOUTH_PRODUCER_ACCOUNT;

Following commands need to be run on ACCOUNT PRODUCER 2

CREATE WAREHOUSE MY_WH;

Replicate the existing database to a secondary database in the other region

create database PRODUCER_DB_12 as replica of org_name.AP_SOUTH_EAST_ACCOUNT.PRODUCER_DB_1;

Create a database for stored procedures

create database PRODUCER_DB_SP_12;
use database PRODUCER_DB_SP_12;

Schedule refresh of the secondary database

create or replace task refresh_PRODUCER_DB_12_task
warehouse = MY_WH
schedule = ‘1 MINUTE’
as
alter database PRODUCER_DB_12 refresh;

alter task refresh_PRODUCER_DB_12_task resume;

Refresh the secondary database now

alter database PRODUCER_DB_12 refresh;

Create a share.

create OR REPLACE share share1;

Add objects to the share.

grant usage on database PRODUCER_DB_12 to share share1;
grant usage on schema PRODUCER_DB_12.PROCDUCER_SCHEMA to share share1;
grant select on TABLE PRODUCER_DB_12.PROCDUCER_SCHEMA.PRODUCER_TABLE to share share1;

Add consumer accounts to the share

alter share share1 add accounts=org_name.AP_SOUTH_ACCOUNT;

Following commands need to be run on ACCOUNT CONSUMER.

use role ORGADMIN;
select system$global_account_set_parameter(‘org_name.AP_SOUTH_ACCOUNT’,’ENABLE_ACCOUNT_DATABASE_REPLICATION’,’TRUE’);
use role accountadmin;
CREATE DATABASE CONSUMER_DB_12 FROM SHARE org_name.AP_SOUTH_PRODUCER_ACCOUNT.SHARE1;

Types of data sharing in Snowflake:

Direct Data Share in Snowflake

Why?

Direct data sharing in Snowflake enables the secure sharing of real-time data sets among different Snowflake accounts, eliminating the need for data duplication or movement. This feature facilitates seamless real-time collaboration and analysis across various entities, including partners, subsidiaries, and customers.

Pros:

1. Seamless collaboration: By enabling immediate data sharing, it fosters seamless collaboration and swift decision-making among multiple entities in real time.

2. Cost-effective: It eliminates the necessity for data replication or ETL processes, thereby minimizing storage and processing expenses related to data movement.

3. Robust security and governance: Snowflake incorporates robust security features that guarantee data privacy and control, empowering organizations to share data with the utmost confidence.

4. Streamlined data sharing: Data providers can effortlessly share targeted data sets with chosen recipients, granting precise control over data access in a simplified manner.

Cons:

1. Reliance on data providers: The accessibility and accuracy of data for data recipients depend on the data providers. Any challenges or delays faced by the providers can have an impact on the recipient’s ability to access the shared data.

2. Restricted data transformation capabilities: Direct data sharing primarily revolves around the sharing of raw or minimally transformed data, which imposes limitations on the recipient’s capacity to execute intricate data transformations within Snowflake.

Change Data Capture (CDC) Data Share in Snowflake

Why?

CDC data sharing in Snowflake enables organisations to share real-time data changes extracted from source databases with other Snowflake accounts. It facilitates nearly instantaneous data replication and synchronisation between systems.

Pros:

1. Instantaneous data synchronisation: CDC data sharing ensures swift replication of changes made in the source databases, making the data promptly available to the receiving Snowflake accounts. This enables real-time analytics and reporting.
2. Minimised latency: CDC captures and delivers only the modified data, significantly reducing data replication time and minimising latency compared to traditional batch-based data sharing methods.
3. Optimised resource utilisation: With CDC data sharing, only the changed data is captured and replicated, leading to efficient resource utilisation. This helps reduce network bandwidth usage and storage requirements.
4. Uninterrupted data availability: The near-real-time nature of CDC data sharing guarantees that the receiving Snowflake accounts have access to the most up-to-date data continuously.

Cons:

1. Reliance on source database compatibility: CDC data sharing relies on the support of change data capture capabilities in the source databases. Incompatibility with certain databases may restrict its usability and functionality.

2. Heightened complexity: The implementation and management of CDC data sharing entail configuring and monitoring data capture processes, introducing additional complexity compared to traditional data sharing methods.

How at Indium have we helped customers monetise their customer data?

• One of the customers use cases is to replicate the data from Snowflake tables and views into other target systems in real-time.
• Where the customer has the primary Snowflake account, data is collected from various sources, and they want to replicate the changes immediately to other targets through Striim.
• Striim is a platform providing a real-time change data capture solution from various data sources like databases, file systems, Snowflake, and others.
• Here, we used Snowflake’s share feature to share the data from the primary account to the secondary accounts.
• As explained above, Striim picked up the changes from the shared data from the secondary accounts in real-time.

Conclusion

In the digital era, the significance of data monetization has grown, enabling organisations to derive value from their data assets. A prominent approach to monetizing data is through targeted advertising, leveraging comprehensive data insights. While data sharing in Snowflake brings advantages like real-time collaboration and reduced latency, it also entails challenges such as dependency on source database compatibility and increased complexity in implementation and management. Overall, Snowflake empowers organisations to effectively monetize their data while offering robust data warehousing capabilities. Striim, as a real-time replication platform, plays a major role in consuming changes from Snowflake tables and views from the secondary accounts.

The post Data Monetization: How Snowflake Data Share and CDC can help monetise your data? appeared first on IndiumSoftware.