Azure Common Filters

Filters

active-key-name

Filter Key Vault Managed Storage Account Resource on Active Key Name property.

This is Value based filter, you can provide string value property.

value_type is always normalize.

example

List all Key Vault managed Storage Accounts with Active Key Name key1

policies:
  - name: keyvault-storage-active-key-name
    resource: azure.keyvault-storage
    filters:
      - type: active-key-name
        value: key1

properties:
  default:
    type: object
  key: null
  op: null
  type:
    enum:
    - active-key-name
  value:
    type: string
  value_from:
    additionalProperties: 'False'
    properties:
      expr:
        oneOf:
        - type: integer
        - type: string
      format:
        enum:
        - csv
        - json
        - txt
        - csv2dict
      url:
        type: string
    required:
    - url
    type: object
  value_regex:
    type: string
  value_type: null
required:
- value
- type

auto-regenerate-key

Filter Key Vault Managed Storage Account Resource on Auto Regenerate property.

This is Value based filter, you can provide boolean value property.

example

List all Key Vault managed Storage Accounts with disabled automatic keys regeneration

policies:
  - name: keyvault-storage-auto-regenerate
    resource: azure.keyvault-storage
    filters:
      - type: auto-regenerate-key
        value: False

properties:
  default:
    type: object
  key: null
  op: null
  type:
    enum:
    - auto-regenerate-key
  value:
    type: boolean
  value_from:
    additionalProperties: 'False'
    properties:
      expr:
        oneOf:
        - type: integer
        - type: string
      format:
        enum:
        - csv
        - json
        - txt
        - csv2dict
      url:
        type: string
    required:
    - url
    type: object
  value_regex:
    type: string
  value_type: null
required:
- type

configuration

properties:
  default:
    type: object
  key:
    type: string
  op:
    enum:
    - eq
    - equal
    - ne
    - not-equal
    - gt
    - greater-than
    - ge
    - gte
    - le
    - lte
    - lt
    - less-than
    - glob
    - regex
    - regex-case
    - in
    - ni
    - not-in
    - contains
    - difference
    - intersect
  type:
    enum:
    - configuration
  value:
    oneOf:
    - type: array
    - type: string
    - type: boolean
    - type: number
    - type: 'null'
  value_from:
    additionalProperties: 'False'
    properties:
      expr:
        oneOf:
        - type: integer
        - type: string
      format:
        enum:
        - csv
        - json
        - txt
        - csv2dict
      url:
        type: string
    required:
    - url
    type: object
  value_regex:
    type: string
  value_type:
    enum:
    - age
    - integer
    - expiration
    - normalize
    - size
    - cidr
    - cidr_size
    - swap
    - resource_count
    - expr
    - unique_size
    - date
    - version
required:
- type

cost

Filter resources by the cost consumed over a timeframe.

Total cost for the resource includes costs for all of it child resources if billed separately (e.g. SQL Server and SQL Server Databases). Warning message is logged if we detect different currencies.

Timeframe options:

  • Number of days before today

  • All days in current calendar period until today:

    • WeekToDate

    • MonthToDate

    • YearToDate

  • All days in the previous calendar period:

    • TheLastWeek

    • TheLastMonth

    • TheLastYear

examples

SQL servers that were cost more than 2000 in the last month.

policies:
    - name: expensive-sql-servers-last-month
      resource: azure.sqlserver
      filters:
      - type: cost
        timeframe: TheLastMonth
        op: gt
        value: 2000

SQL servers that were cost more than 2000 in the last 30 days not including today.

policies:
    - name: expensive-sql-servers
      resource: azure.sqlserver
      filters:
      - type: cost
        timeframe: 30
        op: gt
        value: 2000

properties:
  default:
    type: object
  key: null
  op:
    enum:
    - eq
    - equal
    - ne
    - not-equal
    - gt
    - greater-than
    - ge
    - gte
    - le
    - lte
    - lt
    - less-than
    - glob
    - regex
    - regex-case
    - in
    - ni
    - not-in
    - contains
    - difference
    - intersect
  timeframe:
    oneOf:
    - enum:
      - WeekToDate
      - MonthToDate
      - YearToDate
      - TheLastWeek
      - TheLastMonth
      - TheLastYear
    - minimum: 1
      type: number
  type:
    enum:
    - cost
  value:
    oneOf:
    - type: array
    - type: string
    - type: boolean
    - type: number
    - type: 'null'
  value_from:
    additionalProperties: 'False'
    properties:
      expr:
        oneOf:
        - type: integer
        - type: string
      format:
        enum:
        - csv
        - json
        - txt
        - csv2dict
      url:
        type: string
    required:
    - url
    type: object
  value_regex:
    type: string
  value_type:
    enum:
    - age
    - integer
    - expiration
    - normalize
    - size
    - cidr
    - cidr_size
    - swap
    - resource_count
    - expr
    - unique_size
    - date
    - version
required:
- timeframe
- type

diagnostic-settings

The diagnostic settings filter is implicitly just the ValueFilter on the diagnostic settings for an azure resource.

example

Find Load Balancers that have logs for both LoadBalancerProbeHealthStatus category and LoadBalancerAlertEvent category enabled. The use of value_type: swap is important for these examples because it swaps the value and the evaluated key so that it evaluates the value provided is in the logs.

policies:
  - name: find-load-balancers-with-logs-enabled
    resource: azure.loadbalancer
    filters:
      - type: diagnostic-settings
        key: logs[?category == 'LoadBalancerProbeHealthStatus'][].enabled
        value: True
        op: in
        value_type: swap
      - type: diagnostic-settings
        key: logs[?category == 'LoadBalancerAlertEvent'][].enabled
        value: True
        op: in
        value_type: swap
example

Find KeyVaults that have logs enabled for the AuditEvent category.

policies:
  - name: find-keyvaults-with-logs-enabled
    resource: azure.keyvault
    filters:
      - type: diagnostic-settings
        key: logs[?category == 'AuditEvent'][].enabled
        value: True
        op: in
        value_type: swap

properties:
  default:
    type: object
  key:
    type: string
  op:
    enum:
    - eq
    - equal
    - ne
    - not-equal
    - gt
    - greater-than
    - ge
    - gte
    - le
    - lte
    - lt
    - less-than
    - glob
    - regex
    - regex-case
    - in
    - ni
    - not-in
    - contains
    - difference
    - intersect
  type:
    enum:
    - diagnostic-settings
  value:
    oneOf:
    - type: array
    - type: string
    - type: boolean
    - type: number
    - type: 'null'
  value_from:
    additionalProperties: 'False'
    properties:
      expr:
        oneOf:
        - type: integer
        - type: string
      format:
        enum:
        - csv
        - json
        - txt
        - csv2dict
      url:
        type: string
    required:
    - url
    type: object
  value_regex:
    type: string
  value_type:
    enum:
    - age
    - integer
    - expiration
    - normalize
    - size
    - cidr
    - cidr_size
    - swap
    - resource_count
    - expr
    - unique_size
    - date
    - version
required:
- type

event

Filter a resource based on an event.

properties:
  default:
    type: object
  key:
    type: string
  op:
    enum:
    - eq
    - equal
    - ne
    - not-equal
    - gt
    - greater-than
    - ge
    - gte
    - le
    - lte
    - lt
    - less-than
    - glob
    - regex
    - regex-case
    - in
    - ni
    - not-in
    - contains
    - difference
    - intersect
  type:
    enum:
    - event
  value:
    oneOf:
    - type: array
    - type: string
    - type: boolean
    - type: number
    - type: 'null'
  value_from:
    additionalProperties: 'False'
    properties:
      expr:
        oneOf:
        - type: integer
        - type: string
      format:
        enum:
        - csv
        - json
        - txt
        - csv2dict
      url:
        type: string
    required:
    - url
    type: object
  value_regex:
    type: string
  value_type:
    enum:
    - age
    - integer
    - expiration
    - normalize
    - size
    - cidr
    - cidr_size
    - swap
    - resource_count
    - expr
    - unique_size
    - date
    - version
required:
- type

firewall-rules

Filters resources by the firewall rules

Rules can be specified as x.x.x.x-y.y.y.y or x.x.x.x or x.x.x.x/y.

With the exception of equal all modes reference total IP space and ignore specific notation.

include: True if all IP space listed is included in firewall.

any: True if any overlap in IP space exists.

only: True if firewall IP space only includes IPs from provided space (firewall is subset of provided space).

equal: the list of IP ranges or CIDR that firewall rules must match exactly.

IMPORTANT: this filter ignores all bypass rules. If you want to ensure your resource is not available for other Azure Cloud services or from the Portal, please use firewall-bypass filter.

example

policies:
    - name: servers-with-firewall
      resource: azure.sqlserver
      filters:
          - type: firewall-rules
            include:
                - '131.107.160.2-131.107.160.3'
                - 10.20.20.0/24

oneOf:
- required:
  - type
  - include
- required:
  - type
  - any
- required:
  - type
  - only
- required:
  - type
  - equal
properties:
  any:
    items:
      type: string
    type: array
  equal:
    items:
      type: string
    type: array
  include:
    items:
      type: string
    type: array
  only:
    items:
      type: string
    type: array
  type:
    enum:
    - firewall-rules

instance-view

properties:
  default:
    type: object
  key:
    type: string
  op:
    enum:
    - eq
    - equal
    - ne
    - not-equal
    - gt
    - greater-than
    - ge
    - gte
    - le
    - lte
    - lt
    - less-than
    - glob
    - regex
    - regex-case
    - in
    - ni
    - not-in
    - contains
    - difference
    - intersect
  type:
    enum:
    - instance-view
  value:
    oneOf:
    - type: array
    - type: string
    - type: boolean
    - type: number
    - type: 'null'
  value_from:
    additionalProperties: 'False'
    properties:
      expr:
        oneOf:
        - type: integer
        - type: string
      format:
        enum:
        - csv
        - json
        - txt
        - csv2dict
      url:
        type: string
    required:
    - url
    type: object
  value_regex:
    type: string
  value_type:
    enum:
    - age
    - integer
    - expiration
    - normalize
    - size
    - cidr
    - cidr_size
    - swap
    - resource_count
    - expr
    - unique_size
    - date
    - version
required:
- type

marked-for-op

Filter resources for tag specified future action

Filters resources by a ‘custodian_status’ tag which specifies a future date for an action.

The filter parses the tag values looking for an ‘op@date’ string. The date is parsed and compared to do today’s date, the filter succeeds if today’s date is gte to the target date.

The optional ‘skew’ parameter provides for incrementing today’s date a number of days into the future. An example use case might be sending a final notice email a few days before terminating an instance, or snapshotting a volume prior to deletion.

The optional ‘skew_hours’ parameter provides for incrementing the current time a number of hours into the future.

Optionally, the ‘tz’ parameter can get used to specify the timezone in which to interpret the clock (default value is ‘utc’)

example

policies:
 - name: vm-stop-marked
   resource: azure.vm
   filters:
     - type: marked-for-op
       # The default tag used is custodian_status
       # but that is configurable
       tag: custodian_status
       op: stop
       # Another optional tag is skew
       tz: utc

properties:
  op:
    type: string
  skew:
    minimum: 0
    type: number
  skew_hours:
    minimum: 0
    type: number
  tag:
    type: string
  type:
    enum:
    - marked-for-op
  tz:
    type: string
required:
- type

metric

Filters Azure resources based on live metrics from the Azure monitor

Click here for a full list of metrics supported by Azure resources.

example

Find all VMs with an average Percentage CPU greater than 75% over last 2 hours

policies:
  - name: vm-percentage-cpu
    resource: azure.vm
    filters:
      - type: metric
        metric: Percentage CPU
        aggregation: average
        op: gt
        threshold: 75
        timeframe: 2
example

Find KeyVaults with more than 1000 API hits in the last hour

policies:
  - name: keyvault-hits
    resource: azure.keyvault
    filters:
      - type: metric
        metric: ServiceApiHit
        aggregation: total
        op: gt
        threshold: 1000
        timeframe: 1
example

Find SQL servers with less than 10% average DTU consumption across all databases over last 24 hours

policies:
  - name: dtu-consumption
    resource: azure.sqlserver
    filters:
      - type: metric
        metric: dtu_consumption_percent
        aggregation: average
        op: lt
        threshold: 10
        timeframe: 24
        filter:  "DatabaseResourceId eq '*'"

properties:
  aggregation:
    enum:
    - total
    - average
    - count
    - minimum
    - maximum
  filter:
    type: string
  interval:
    enum:
    - PT1M
    - PT5M
    - PT15M
    - PT30M
    - PT1H
    - PT6H
    - PT12H
    - P1D
  metric:
    type: string
  no_data_action:
    enum:
    - include
    - exclude
  op:
    enum:
    - eq
    - equal
    - ne
    - not-equal
    - gt
    - greater-than
    - ge
    - gte
    - le
    - lte
    - lt
    - less-than
  threshold:
    type: number
  timeframe:
    type: number
  type:
    enum:
    - metric
required:
- type
- metric
- op
- threshold

offer

CosmosDB Offer Filter

Allows access to the offer on a collection or database.

example

This policy will find all collections with a V2 offer which indicates throughput is provisioned at the collection scope.

policies:
  - name: cosmosdb-collection-high-throughput
    resource: azure.cosmosdb-collection
    filters:
      - type: offer
        key: offerVersion
        op: eq
        value: 'V2'

properties:
  default:
    type: object
  key:
    type: string
  op:
    enum:
    - eq
    - equal
    - ne
    - not-equal
    - gt
    - greater-than
    - ge
    - gte
    - le
    - lte
    - lt
    - less-than
    - glob
    - regex
    - regex-case
    - in
    - ni
    - not-in
    - contains
    - difference
    - intersect
  type:
    enum:
    - offer
  value:
    oneOf:
    - type: array
    - type: string
    - type: boolean
    - type: number
    - type: 'null'
  value_from:
    additionalProperties: 'False'
    properties:
      expr:
        oneOf:
        - type: integer
        - type: string
      format:
        enum:
        - csv
        - json
        - txt
        - csv2dict
      url:
        type: string
    required:
    - url
    type: object
  value_regex:
    type: string
  value_type:
    enum:
    - age
    - integer
    - expiration
    - normalize
    - size
    - cidr
    - cidr_size
    - swap
    - resource_count
    - expr
    - unique_size
    - date
    - version
required:
- type

offhour

Schedule offhours for resources see offhours for features and configuration.

properties:
  default_tz:
    type: string
  offhour:
    maximum: 23
    minimum: 0
    type: integer
  opt-out:
    type: boolean
  skip-days:
    items:
      pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}
      type: string
    type: array
  skip-days-from:
    additionalProperties: 'False'
    properties:
      expr:
        oneOf:
        - type: integer
        - type: string
      format:
        enum:
        - csv
        - json
        - txt
        - csv2dict
      url:
        type: string
    required:
    - url
    type: object
  tag:
    type: string
  type:
    enum:
    - offhour
  weekends:
    type: boolean
  weekends-only:
    type: boolean
required:
- offhour
- default_tz
- type

onhour

Schedule offhours for resources see offhours for features and configuration.

properties:
  default_tz:
    type: string
  onhour:
    maximum: 23
    minimum: 0
    type: integer
  opt-out:
    type: boolean
  skip-days:
    items:
      pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}
      type: string
    type: array
  skip-days-from:
    additionalProperties: 'False'
    properties:
      expr:
        oneOf:
        - type: integer
        - type: string
      format:
        enum:
        - csv
        - json
        - txt
        - csv2dict
      url:
        type: string
    required:
    - url
    type: object
  tag:
    type: string
  type:
    enum:
    - onhour
  weekends:
    type: boolean
  weekends-only:
    type: boolean
required:
- onhour
- default_tz
- type

parent

Meta filter that allows you to filter child resources by applying filters to their parent resources.

You can use any filter supported by corresponding parent resource type.

examples

Find Azure KeyVault Keys from Key Vaults with owner:ProjectA tag.

policies:
  - name: kv-keys-from-tagged-keyvaults
    resource: azure.keyvault-key
    filters:
      - type: parent
        filter:
          type: value
          key: tags.owner
          value: ProjectA

properties:
  filter:
    type: object
  type:
    enum:
    - parent
required:
- type
- type

policy-compliant

Filter resources based on Azure Policy compliance status

Filter resources by their current Azure Policy compliance status.

You can specify if you want to filter compliant or non-compliant resources.

You can provide a list of Azure Policy definitions display names or names to limit amount of non-compliant resources. By default it returns a list of all non-compliant resources.

policies:
 - name: non-compliant-vms
   resource: azure.vm
   filters:
     - type: policy-compliant
       compliant: false
       definitions:
         - "Definition display name 1"
         - "Definition display name 2"

properties:
  compliant:
    type: boolean
  definitions:
    type: array
  type:
    enum:
    - policy-compliant
required:
- type
- compliant
- type

reduce

Generic reduce filter to group, sort, and limit your resources.

This example will select the longest running instance from each ASG, then randomly choose 10% of those, maxing at 15 total instances.

example

- name: oldest-instance-by-asg
  resource: ec2
  filters:
    - "tag:aws:autoscaling:groupName": present
    - type: reduce
      group-by: "tag:aws:autoscaling:groupName"
      sort-by: "LaunchTime"
      order: asc
      limit: 1

Or you might want to randomly select a 10 percent of your resources, but no more than 15.

example

- name: random-selection
  resource: ec2
  filters:
    - type: reduce
      order: randomize
      limit: 15
      limit-percent: 10

properties:
  discard:
    minimum: 0
    type: number
  discard-percent:
    maximum: 100
    minimum: 0
    type: number
  group-by:
    oneOf:
    - type: string
    - key:
        type: string
      type: object
      value_regex: string
      value_type:
        enum:
        - string
        - number
        - date
  limit:
    minimum: 0
    type: number
  limit-percent:
    maximum: 100
    minimum: 0
    type: number
  null-order:
    enum:
    - first
    - last
  order:
    enum:
    - asc
    - desc
    - reverse
    - randomize
  sort-by:
    oneOf:
    - type: string
    - key:
        type: string
      type: object
      value_regex: string
      value_type:
        enum:
        - string
        - number
        - date
  type:
    enum:
    - reduce
required:
- type

regeneration-period

Filter Key Vault Managed Storage Account Resource on Regeneration Period property.

This is Value based filter, you can provide any value and op properties.

example

List all Key Vault managed Storage Accounts with regeneration period not equal to P90D

policies:
  - name: keyvault-storage-regeneration-period
    resource: azure.keyvault-storage
    filters:
      - type: regeneration-period
        op: ne
        value: P90D

properties:
  default:
    type: object
  key: null
  op:
    enum:
    - eq
    - equal
    - ne
    - not-equal
    - gt
    - greater-than
    - ge
    - gte
    - le
    - lte
    - lt
    - less-than
    - glob
    - regex
    - regex-case
    - in
    - ni
    - not-in
    - contains
    - difference
    - intersect
  type:
    enum:
    - regeneration-period
  value:
    oneOf:
    - type: array
    - type: string
    - type: boolean
    - type: number
    - type: 'null'
  value_from:
    additionalProperties: 'False'
    properties:
      expr:
        oneOf:
        - type: integer
        - type: string
      format:
        enum:
        - csv
        - json
        - txt
        - csv2dict
      url:
        type: string
    required:
    - url
    type: object
  value_regex:
    type: string
  value_type: null
required:
- type

resource-lock

Filter locked resources. Lock can be of 2 types: ReadOnly and CanNotDelete. To filter any lock, use “Any” type. Lock type is optional, by default any lock will be applied to the filter. To get unlocked resources, use “Absent” type.

example

Get all keyvaults with ReadOnly lock:

policies:
 - name: locked-keyvaults
   resource: azure.keyvault
   filters:
     - type: resource-lock
       lock-type: ReadOnly
example

Get all locked sqldatabases (any type of lock):

policies:
 - name: locked-sqldatabases
   resource: azure.sqldatabase
   filters:
     - type: resource-lock
example

Get all unlocked resource groups:

policies:
 - name: unlock-rgs
   resource: azure.resourcegroup
   filters:
     - type: resource-lock
       lock-type: Absent

properties:
  lock-type:
    enum:
    - ReadOnly
    - CanNotDelete
    - Any
    - Absent
  type:
    enum:
    - resource-lock
required:
- type
- type

storage-diagnostic-settings

Filters storage accounts based on its diagnostic settings. The filter requires specifying the storage type (blob, queue, table, file) and will filter based on the settings for that specific type.

example

Find all storage accounts that have a ‘delete’ logging setting disabled.

policies:
    - name: find-accounts-with-delete-logging-disabled
      resource: azure.storage
      filters:
        - or:
            - type: storage-diagnostic-settings
              storage-type: blob
              key: logging.delete
              op: eq
              value: False
            - type: storage-diagnostic-settings
              storage-type: queue
              key: logging.delete
              op: eq
              value: False
            - type: storage-diagnostic-settings
              storage-type: table
              key: logging.delete
              op: eq
              value: False
example

Find Load Balancers that have logs for both LoadBalancerProbeHealthStatus category and LoadBalancerAlertEvent category enabled. The use of value_type: swap is important for these examples because it swaps the value and the evaluated key so that it evaluates the value provided is in the logs.

policies:
  - name: find-load-balancers-with-logs-enabled
    resource: azure.loadbalancer
    filters:
      - type: diagnostic-settings
        key: logs[?category == 'LoadBalancerProbeHealthStatus'][].enabled
        value: True
        op: in
        value_type: swap
      - type: diagnostic-settings
        key: logs[?category == 'LoadBalancerAlertEvent'][].enabled
        value: True
        op: in
        value_type: swap
example

Find KeyVaults that have logs enabled for the AuditEvent category.

policies:
  - name: find-keyvaults-with-logs-enabled
    resource: azure.keyvault
    filters:
      - type: diagnostic-settings
        key: logs[?category == 'AuditEvent'][].enabled
        value: True
        op: in
        value_type: swap

properties:
  default:
    type: object
  key:
    type: string
  op:
    enum:
    - eq
    - equal
    - ne
    - not-equal
    - gt
    - greater-than
    - ge
    - gte
    - le
    - lte
    - lt
    - less-than
    - glob
    - regex
    - regex-case
    - in
    - ni
    - not-in
    - contains
    - difference
    - intersect
  storage-type:
    enum:
    - blob
    - queue
    - table
    - file
    type: string
  type:
    enum:
    - storage-diagnostic-settings
  value:
    oneOf:
    - type: array
    - type: string
    - type: boolean
    - type: number
    - type: 'null'
  value_from:
    additionalProperties: 'False'
    properties:
      expr:
        oneOf:
        - type: integer
        - type: string
      format:
        enum:
        - csv
        - json
        - txt
        - csv2dict
      url:
        type: string
    required:
    - url
    type: object
  value_regex:
    type: string
  value_type:
    enum:
    - age
    - integer
    - expiration
    - normalize
    - size
    - cidr
    - cidr_size
    - swap
    - resource_count
    - expr
    - unique_size
    - date
    - version
required:
- storage-type
- type

value

Generic value filter using jmespath

properties:
  default:
    type: object
  key:
    type: string
  op:
    enum:
    - eq
    - equal
    - ne
    - not-equal
    - gt
    - greater-than
    - ge
    - gte
    - le
    - lte
    - lt
    - less-than
    - glob
    - regex
    - regex-case
    - in
    - ni
    - not-in
    - contains
    - difference
    - intersect
  type:
    enum:
    - value
  value:
    oneOf:
    - type: array
    - type: string
    - type: boolean
    - type: number
    - type: 'null'
  value_from:
    additionalProperties: 'False'
    properties:
      expr:
        oneOf:
        - type: integer
        - type: string
      format:
        enum:
        - csv
        - json
        - txt
        - csv2dict
      url:
        type: string
    required:
    - url
    type: object
  value_regex:
    type: string
  value_type:
    enum:
    - age
    - integer
    - expiration
    - normalize
    - size
    - cidr
    - cidr_size
    - swap
    - resource_count
    - expr
    - unique_size
    - date
    - version
required:
- type