Data Factory

Filters

  • Standard Value Filter (see Generic Filters)
  • ARM Resource Filters (see Generic Filters)
    • Metric Filter - Filter on metrics from Azure Monitor - (see Data Factory Supported Metrics)
    • Tag Filter - Filter on tag presence and/or values
    • Marked-For-Op Filter - Filter on tag that indicates a scheduled operation for a resource

Actions

Example Policies

This set of policies will mark all Data Factories for deletion in 7 days that have ‘test’ in name (ignore case), and then perform the delete operation on those ready for deletion.

policies:
  - name: mark-test-datafactories-for-deletion
    resource: azure.datafactory
    filters:
      - type: value
        key: name
        op: in
        value_type: normalize
        value: test
     actions:
      - type: mark-for-op
        op: delete
        days: 7
  - name: delete-test-datafactories
    resource: azure.datafactory
    filters:
      - type: marked-for-op
        op: delete
    actions:
      - type: delete

This policy will find all Data Factories with 10 or more failures in pipeline runs over the last 72 hours and notify user@domain.com

policies:
  - name: notify-datafactory-dropping-messages
    resource: azure.datafactory
    filters:
      - type: metric
        metric: PipelineFailedRuns
        op: ge
        aggregation: total
        threshold: 10
        timeframe: 72
     actions:
      - type: notify
        template: default
        priority_header: 2
        subject: Datafactory Pipeline Failing
        to:
          - user@domain.com
        transport:
          - type: asq
            queue: https://accountname.queue.core.windows.net/queuename