Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions providers/microsoft/azure/docs/connections/analysis_services.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

.. http://www.apache.org/licenses/LICENSE-2.0

.. Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.

.. _howto/connection:azure_analysis_services:

Microsoft Azure Analysis Services Connection
============================================

The Microsoft Azure Analysis Services connection type enables model refresh operations through the
`asynchronous refresh REST API <https://learn.microsoft.com/en-us/analysis-services/azure-analysis-services/analysis-services-async-refresh>`__.

The :class:`~airflow.providers.microsoft.azure.hooks.analysis_services.AzureAnalysisServicesHook`,
:class:`~airflow.providers.microsoft.azure.operators.analysis_services.AzureAnalysisServicesRefreshOperator`,
and :class:`~airflow.providers.microsoft.azure.sensors.analysis_services.AzureAnalysisServicesSensor`
use this connection.

Default Connection ID
---------------------

The default connection ID is ``azure_analysis_services_default``.

Configuring the Connection
--------------------------

Region Endpoint
Specify the rollout endpoint in ``host``, for example ``westus.asazure.windows.net``. Do not include
``https://``, a port, the Analysis Services server name, or a path. You can find this endpoint in the
server's full name, such as ``asazure://westus.asazure.windows.net/example-server``.

Client ID
Specify the Microsoft Entra service principal application (client) ID in ``login``.

Client Secret
Specify the service principal client secret in ``password``.

Tenant ID
Specify the Microsoft Entra tenant ID in the ``tenantId`` extra field.

Azure Analysis Services currently requires the service principal to be a server administrator for
asynchronous refresh REST API calls. Add it to the server administrator role using the format
``app:<client-id>@<tenant-id>``. See `Add a service principal to the server administrator role
<https://learn.microsoft.com/en-us/analysis-services/azure-analysis-services/analysis-services-addservprinc-admins>`__.

This connection supports service principal client-secret authentication. Managed identity authentication is
not supported because Azure Analysis Services does not support managed identities for these operations.

.. spelling:word-list::

rollout
85 changes: 85 additions & 0 deletions providers/microsoft/azure/docs/operators/analysis_services.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

.. http://www.apache.org/licenses/LICENSE-2.0

.. Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.

Microsoft Azure Analysis Services Operators
===========================================

Use these components to start and monitor asynchronous model refresh operations in Microsoft Azure Analysis
Services. Configure an :ref:`Azure Analysis Services connection <howto/connection:azure_analysis_services>`
before using them.

Prerequisite Tasks
------------------

.. include:: /operators/_partials/prerequisite_tasks.rst

.. _howto/operator:AzureAnalysisServicesRefreshOperator:

Start a model refresh
---------------------

Use :class:`~airflow.providers.microsoft.azure.operators.analysis_services.AzureAnalysisServicesRefreshOperator`
with ``wait_for_termination=False`` to start a refresh and immediately return its refresh ID.

.. exampleinclude:: /../tests/system/microsoft/azure/example_azure_analysis_services.py
:language: python
:dedent: 4
:start-after: [START howto_operator_azure_analysis_services_refresh]
:end-before: [END howto_operator_azure_analysis_services_refresh]

Wait for the refresh to finish
------------------------------

With the default ``wait_for_termination=True`` the operator also waits for the refresh to reach a
terminal status. The ``timeout`` parameter limits that wait and starts once the refresh has been
submitted, while ``request_timeout`` limits each individual REST request.

.. exampleinclude:: /../tests/system/microsoft/azure/example_azure_analysis_services.py
:language: python
:dedent: 4
:start-after: [START howto_operator_azure_analysis_services_refresh_and_wait]
:end-before: [END howto_operator_azure_analysis_services_refresh_and_wait]

.. note::

The operator and the sensor always run deferred. Both the request that starts the refresh and
the status polling are performed by the triggerer, so no worker slot is held while the model is
refreshing. A triggerer must be running in your deployment.

.. _howto/sensor:AzureAnalysisServicesSensor:

Wait with a sensor
------------------

The operator return value can be passed directly to
:class:`~airflow.providers.microsoft.azure.sensors.analysis_services.AzureAnalysisServicesSensor`.
This separates starting the refresh from waiting for it.

.. exampleinclude:: /../tests/system/microsoft/azure/example_azure_analysis_services.py
:language: python
:dedent: 4
:start-after: [START howto_sensor_azure_analysis_services_refresh]
:end-before: [END howto_sensor_azure_analysis_services_refresh]

Azure Analysis Services accepts only one active refresh for a model. A second request returns HTTP 409, so
serialize refresh tasks that target the same model, for example with task dependencies or an Airflow pool.

Reference
---------

For more information, see `Asynchronous refresh with the REST API
<https://learn.microsoft.com/en-us/analysis-services/azure-analysis-services/analysis-services-async-refresh>`__.
34 changes: 34 additions & 0 deletions providers/microsoft/azure/provider.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,12 @@ versions:
- 1.0.0

integrations:
- integration-name: Microsoft Azure Analysis Services
external-doc-url: https://learn.microsoft.com/en-us/analysis-services/azure-analysis-services/
how-to-guide:
- /docs/apache-airflow-providers-microsoft-azure/operators/analysis_services.rst
logo: /docs/integration-logos/Microsoft-Azure.png
tags: [azure]
- integration-name: Microsoft Azure Batch
external-doc-url: https://azure.microsoft.com/en-us/services/batch/
how-to-guide:
Expand Down Expand Up @@ -213,6 +219,9 @@ integrations:
tags: [azure]

operators:
- integration-name: Microsoft Azure Analysis Services
python-modules:
- airflow.providers.microsoft.azure.operators.analysis_services
- integration-name: Microsoft Azure Compute
python-modules:
- airflow.providers.microsoft.azure.operators.compute
Expand Down Expand Up @@ -254,6 +263,9 @@ operators:
- airflow.providers.microsoft.azure.operators.powerbi

sensors:
- integration-name: Microsoft Azure Analysis Services
python-modules:
- airflow.providers.microsoft.azure.sensors.analysis_services
- integration-name: Microsoft Azure Compute
python-modules:
- airflow.providers.microsoft.azure.sensors.compute
Expand All @@ -275,6 +287,9 @@ filesystems:
- airflow.providers.microsoft.azure.fs.msgraph

hooks:
- integration-name: Microsoft Azure Analysis Services
python-modules:
- airflow.providers.microsoft.azure.hooks.analysis_services
- integration-name: Microsoft Azure Compute
python-modules:
- airflow.providers.microsoft.azure.hooks.compute
Expand Down Expand Up @@ -327,6 +342,9 @@ hooks:
- airflow.providers.microsoft.azure.hooks.powerbi

triggers:
- integration-name: Microsoft Azure Analysis Services
python-modules:
- airflow.providers.microsoft.azure.triggers.analysis_services
- integration-name: Microsoft Azure Batch
python-modules:
- airflow.providers.microsoft.azure.triggers.batch
Expand Down Expand Up @@ -387,6 +405,22 @@ transfers:
python-module: airflow.providers.microsoft.azure.transfers.gcs_to_wasb

connection-types:
- hook-class-name: airflow.providers.microsoft.azure.hooks.analysis_services.AzureAnalysisServicesHook
hook-name: "Azure Analysis Services"
connection-type: azure_analysis_services
ui-field-behaviour:
hidden-fields: ["schema", "port", "extra"]
relabeling:
host: Region Endpoint
login: Client ID
password: Client Secret
placeholders:
host: westus.asazure.windows.net
conn-fields:
tenantId:
label: Tenant ID
schema:
type: ["string", "null"]
- hook-class-name: airflow.providers.microsoft.azure.hooks.base_azure.AzureBaseHook
hook-name: "Azure"
connection-type: azure
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,15 @@ def get_provider_info():
"name": "Microsoft Azure",
"description": "`Microsoft Azure <https://azure.microsoft.com/>`__\n",
"integrations": [
{
"integration-name": "Microsoft Azure Analysis Services",
"external-doc-url": "https://learn.microsoft.com/en-us/analysis-services/azure-analysis-services/",
"how-to-guide": [
"/docs/apache-airflow-providers-microsoft-azure/operators/analysis_services.rst"
],
"logo": "/docs/integration-logos/Microsoft-Azure.png",
"tags": ["azure"],
},
{
"integration-name": "Microsoft Azure Batch",
"external-doc-url": "https://azure.microsoft.com/en-us/services/batch/",
Expand Down Expand Up @@ -149,6 +158,10 @@ def get_provider_info():
},
],
"operators": [
{
"integration-name": "Microsoft Azure Analysis Services",
"python-modules": ["airflow.providers.microsoft.azure.operators.analysis_services"],
},
{
"integration-name": "Microsoft Azure Compute",
"python-modules": ["airflow.providers.microsoft.azure.operators.compute"],
Expand Down Expand Up @@ -203,6 +216,10 @@ def get_provider_info():
},
],
"sensors": [
{
"integration-name": "Microsoft Azure Analysis Services",
"python-modules": ["airflow.providers.microsoft.azure.sensors.analysis_services"],
},
{
"integration-name": "Microsoft Azure Compute",
"python-modules": ["airflow.providers.microsoft.azure.sensors.compute"],
Expand All @@ -229,6 +246,10 @@ def get_provider_info():
"airflow.providers.microsoft.azure.fs.msgraph",
],
"hooks": [
{
"integration-name": "Microsoft Azure Analysis Services",
"python-modules": ["airflow.providers.microsoft.azure.hooks.analysis_services"],
},
{
"integration-name": "Microsoft Azure Compute",
"python-modules": ["airflow.providers.microsoft.azure.hooks.compute"],
Expand Down Expand Up @@ -299,6 +320,10 @@ def get_provider_info():
},
],
"triggers": [
{
"integration-name": "Microsoft Azure Analysis Services",
"python-modules": ["airflow.providers.microsoft.azure.triggers.analysis_services"],
},
{
"integration-name": "Microsoft Azure Batch",
"python-modules": ["airflow.providers.microsoft.azure.triggers.batch"],
Expand Down Expand Up @@ -379,6 +404,21 @@ def get_provider_info():
},
],
"connection-types": [
{
"hook-class-name": "airflow.providers.microsoft.azure.hooks.analysis_services.AzureAnalysisServicesHook",
"hook-name": "Azure Analysis Services",
"connection-type": "azure_analysis_services",
"ui-field-behaviour": {
"hidden-fields": ["schema", "port", "extra"],
"relabeling": {
"host": "Region Endpoint",
"login": "Client ID",
"password": "Client Secret",
},
"placeholders": {"host": "westus.asazure.windows.net"},
},
"conn-fields": {"tenantId": {"label": "Tenant ID", "schema": {"type": ["string", "null"]}}},
},
{
"hook-class-name": "airflow.providers.microsoft.azure.hooks.base_azure.AzureBaseHook",
"hook-name": "Azure",
Expand Down
Loading