data beats

Share this post

Warehouse-native Apps Explained

databeats.community
Audience Data Pro

Warehouse-native Apps Explained

Part 1 of the series on warehouse-native apps — SaaS tools built on the data warehouse, also known as Connected Apps

Arpit Choudhury
Jun 24, 2022
4
10
Share this post

Warehouse-native Apps Explained

databeats.community

→ Link to the series.

A warehouse-native app is any SaaS tool that can run on top of a customer’s data warehouse, without the need to ingest and store any data.

In other words, a warehouse-native app vendor enables its customers to bring their own data warehouse to the vendor’s application which then performs tasks using the customer’s data. 

The data doesn’t need to be synced from the customer’s data warehouse to the vendor’s application, which is typically the case with traditional SaaS solutions.

Snowflake refers to Warehouse-native apps as Connected apps and their traditional counterparts as Managed apps, which, besides offering a set of functionality, manage their customer’s data too.

But there’s more.

Besides storing a copy of every customer’s customer data, managed apps also store the data generated as a result of their own product being used by their customers. On the other hand, warehouse-native apps automatically load this data back into the customer’s data warehouse. 

Let’s break this down using an email marketing tool as an example. 

Managed Email Marketing App

Before creating a segment or activating a campaign using a traditional email tool, you must either upload CSV files with data about your customers or sync data from your first-party app to the email tool using their API.

The diagram above depicts the flow of data to a managed email marketing tool such as Mailchimp or Customer.io.

The fact that such companies have to store a copy of every customer’s data explains why their pricing model is based on subscribers/profiles — basically, customers have to pay for their own data to be stored at the vendor’s end.

Additionally, all usage data like campaign metrics, subscription status, etc is also stored by the vendors in their own databases, which customers can later download as CSVs or extract and load into their data warehouse using an ELT tool. 

Warehouse-native Email Marketing App

With a warehouse-native or connected email app, you don’t need to upload or sync any data to start using it — you just need to connect your data warehouse to start building segments and campaigns. 

As you can see above, a warehouse-native app or connected app is like a component that sits on top of a data warehouse and offers certain functionality, which in this case is email marketing.

Connected apps are able to read data from a warehouse, enabling its users to select appropriate tables from the warehouse and then build segments and campaigns using data from those tables.

In terms of sending emails, connected apps either have their own infrastructure or allow customers to connect to yet another email service provider like Sendgrid. Either way, the connected app vendor loads usage data (campaign metrics, etc) back into the customer’s warehouse, doing away with the need for an ELT pipeline. 

Reverse ETL vs Connected Apps

It’s worth mentioning that there’s some overlap between reverse ETL (rETL) tools and some connected apps. In fact, rETL tools behave a lot like connected apps — both sit on top of a data warehouse but don’t store any data at their end, while allowing customers to perform certain actions.

However, even though rETL tools offer features like sending Slack alerts based on changes to your data, moving data from the warehouse to downstream managed apps remains the primary use case for rETL.

Instead of uploading CSVs or using the managed app’s API to sync data, one can move data from the warehouse to a managed app using rETL as middleware. 

Similarly, as depicted below, one can move usage data from the managed app back to the warehouse using an ELT tool (or the managed app’s API).

Summary and closing thoughts

I’d like to conclude this post with a quick summary followed by some thoughts.

  • A warehouse-native app is any SaaS tool that can run on top of a customer’s data warehouse, without the need to ingest and store any data

  • Warehouse-native apps are also referred to as Connected apps (since they’re connected to the customer’s own data platform)

  • Warehouse-native apps or connected apps automatically load usage data back into the customer’s data warehouse.

  • There’s some overlap in the functionality of connected apps and reverse ETL tools — both sit on top of a data warehouse but don’t store any data at their end, while allowing customers to perform certain actions.

While unlikely to happen anytime soon, connected apps can become more like rETL tools by building destination integrations, allowing customers to also move data downstream to other third-party tools.

Reverse ETL tools, on the other hand, have the potential to enable traditional SaaS companies to embrace the warehouse-native architecture.


Exploring data tools?

Check out the tools directory

Looking to explain the MDS to someone? We made a song that does just that!

DATA BEATS 🥁
The Modern Data Track 🎶
Read more
6 months ago · Arpit Choudhury
10
Share this post

Warehouse-native Apps Explained

databeats.community
Previous
Next
10 Comments
Sarah Krasnik
Writes Sarah's Newsletter
Jun 24, 2022Liked by Arpit Choudhury

I think there's an additional important difference and that's optimization and query cost. Warehouse-native apps owned by other teams run queries directly on the warehouse, owned the by the data team. With additional queries comes a price, absorbed by the data team. If this doesn't have oversight, it's easy to rack up a pretty hefty Snowflake bill. The optimization of query efficiency and cost is up to the vendor, but most warehouse-native tools are very early and aren't talking about it yet. It remains to be seen if there is an additional cost, and if so, if there's a savings that will make an organization come out net-positive.

Expand full comment
Reply
4 replies by Arpit Choudhury and others
Gaurav Kotak
Jan 1Liked by Arpit Choudhury

Thanks for this post.

When we first moved from on-premise to SaaS, the analogy was it's like plugging your appliance into a socket vs running your own electricity generator at home. SaaS was compelling

Today, indeed, it does feel like each appliance is powered it's own fully integrated electric companies. And those electric companies are sharing data (ELT - RETL) to get 360 view of your house. Now that Warehouses are SaaS, it feels like warehouse native apps are compelling with a singular backend (also in the cloud)

Which are the top Warehouse native apps you're tracking?

Expand full comment
Reply
1 reply by Arpit Choudhury
8 more comments…
TopNewCommunity

No posts

Ready for more?

© 2023 astorik Pte. Ltd.
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing