Announcing Qwak Feature Store support for Snowflake data sources

Qwak Feature Store provides a unified feature store for training and real-time operations without the need to write additional code or create manual processes to keep features consistent.  There are many ways to create features in the Qwak Feature Store, including Batch-based features, streaming or non-materialized features.

Starting today, Qwak supports Snowflake as a data source for Batch features

The Data Source connectors provide you with a consistent data source interface for any database and create a standard way to combine stream and batch data sources for Feature Transformations.

Integrating Qwak Feature Store and Snowflake

Defining a Feature Set enables you to create features from your analytical data: when calculating feature values, Qwak will simply read from the underlying data source.

For example, in a fraud detection model use case, we might have two values to pull from a Snowflake data source:

• Average transaction per customer - avg_amount
• Standard deviation of a transaction per customer - sttdev_amount

And two from Streaming events from Kafka:

• Last transaction amount
• Last transaction time
  

Architecture

How to configure Snowflake as Data Source

Snowflake data source connector definition:

from qwak.feature_store.sources.data_sources import SnowflakeSource

some_snowflake_source = SnowflakeSource(name='users_demo',
                                        description='a snowflake source description',
                                        date_created_column='insert_date_column',
                                        host='Snowflake DNS address',
                                        username_secret_name='snowflake-username',
                                        password_secret_name='snowflake-password',
                                        database='db_name',
                                        schema='schema_name',
                                        warehouse='data_warehouse_name',
                                        sql='SQL query for feature extraction')

Register batch feature using the Snowflake connector:

BatchFeatureSet(
  name=”batch_transaction_features”,
  data_sources=[“snowflake_transactions_history”],
  scheduling_policy=”daily”,
  validations=[expect_column_values_to_be_between(
    column=”amount”, min_value=0, max_value=None)]
  function=SqlFunction(
    “””
    SELECT User_ID,
      avg(Amount) AS avg_amount,
      sttdev(Amount) AS stddev_amount
    FROM snowflake_trasactions_history
    GROUP BY User_ID
    “””)
)

Qwak Feature Store helps ensure models make accurate predictions by making the same features available for both training and inference.