dbt-ydb

dbt-ydb is a plugin for dbt that provides support for working with YDB. dbt-ydb adapter is in preview stage and does not currently support all dbt features. The sections below list the supported features and known limitations.

Installation

To install plugin, execute the following command:

pip install dbt-ydb

Supported features

• Table materialization
• View materialization
• Seeds
• Docs generate
• Tests
• Incremental materializations (merge strategy only)
• Snapshots

Limitations

• YDB does not support CTE
• YDB requires a primary key to be specified for its tables. See the configuration section for instructions on how to set it.
• source() macro requires you to specify a schema. Use / if your source is in root folder.

Usage

Profile Configuration

To configure YDB connection, fill profile.yml file as below:

profile_name:
  target: dev
  outputs:
    dev:
      type: ydb
      host: [localhost] # YDB host
      port: [2136] # YDB port
      database: [/local] # YDB database
      schema: [<empty string>] # Optional subfolder for DBT models
      secure: [False] # If enabled, grpcs protocol will be used
      root_certificates_path: [<empty string>] # Optional path to root certificates file

      # Static Credentials
      username: [<empty string>]
      password: [<empty string>]

      # Access Token Credentials
      token: [<empty string>]

      # Service Account Credentials
      service_account_credentials_file: [<empty string>]

Model Configuration

View

Option	Description	Required	Default

Table

Option	Description	Required	Default
primary_key	Primary key expression to use during table creation	yes
store_type	Type of table. Available options are row and column	no	row
auto_partitioning_by_size	Enable automatic partitioning by size. Available options are ENABLED and DISABLED	no
auto_partitioning_partition_size_mb	Partition size in megabytes for automatic partitioning	no
ttl	Time-to-live (TTL) expression for automatic data expiration	no

Incremental

Option	Description	Required	Default
incremental_strategy	Strategy of incremental materialization. Current adapter supports only merge strategy, which will use YDB's UPSERT operation.	no	default
primary_key	Primary key expression to use during table creation	yes
store_type	Type of table. Available options are row and column	no	row
auto_partitioning_by_size	Enable automatic partitioning by size. Available options are ENABLED and DISABLED	no
auto_partitioning_partition_size_mb	Partition size in megabytes for automatic partitioning	no
ttl	Time-to-live (TTL) expression for automatic data expiration	no

Example table configuration

{{ config(
    primary_key='id, created_at',
    store_type='row',
    auto_partitioning_by_size='ENABLED',
    auto_partitioning_partition_size_mb=256,
    ttl='Interval("P30D") on created_at'
) }}

select
    id,
    name,
    created_at
from {{ ref('source_table') }}

Seed

Option	Description	Required	Default
primary_key	Primary key expression to use during table creation	no	The first column of CSV will be used as default.