gcp.bq-dataset

GCP resource: https://cloud.google.com/bigquery/docs/reference/rest/v2/datasets

Filters

event

list-item

marked-for-op

metrics

reduce

scc-findings

value

marked-for-op

Filter resources for label specified future action

Filters resources by a ‘custodian_status’ label which specifies a future date for an action.

The filter parses the label values looking for an ‘op@date’ string. The date is parsed and compared to do today’s date, the filter succeeds if today’s date is gte to the target date.

The optional ‘skew’ parameter provides for incrementing today’s date a number of days into the future. An example use case might be sending a final notice email a few days before terminating an instance, or snapshotting a volume prior to deletion.

The optional ‘skew_hours’ parameter provides for incrementing the current time a number of hours into the future.

Optionally, the ‘tz’ parameter can get used to specify the timezone in which to interpret the clock (default value is ‘utc’)

example:

policies:
 - name: vm-stop-marked
   resource: gcp.instance
   filters:
     - type: marked-for-op
       # The default label used is custodian_status
       # but that is configurable
       label: custodian_status
       op: stop
       # Another optional label is skew
       tz: utc

properties:
  label:
    type: string
  op:
    type: string
  skew:
    minimum: 0
    type: number
  skew_hours:
    minimum: 0
    type: number
  type:
    enum:
    - marked-for-op
  tz:
    type: string
required:
- type

metrics

Supports metrics filters on resources.

All resources that have cloud watch metrics are supported.

Docs on cloud watch metrics

Google Supported Metrics https://cloud.google.com/monitoring/api/metrics_gcp
Custom Metrics https://cloud.google.com/monitoring/api/v3/metric-model#intro-custom-metrics

- name: firewall-hit-count
  resource: gcp.firewall
  filters:
    - type: metrics
      name: firewallinsights.googleapis.com/subnet/firewall_hit_count
      aligner: ALIGN_COUNT
      days: 14
      value: 1
      op: greater-than

The period-start key allows you to align the metric window in two ways. By default, using auto, the window is computed relative to the current time. Alternatively, setting it to start-of-day aligns the window to full UTC calendar days, beginning at 00:00:00 UTC and ending at current day 00:00:00 UTC.

- name: instance-low-cpu-last-full-day
  resource: gcp.instance
  filters:
    - type: metrics
      name: compute.googleapis.com/instance/cpu/utilization
      aligner: ALIGN_MEAN
      days: 1
      value: 0.05
      op: less-than
      period-start: start-of-day

properties:
  aligner:
    enum:
    - ALIGN_NONE
    - ALIGN_DELTA
    - ALIGN_RATE
    - ALIGN_INTERPOLATE
    - ALIGN_MIN
    - ALIGN_MAX
    - ALIGN_MEAN
    - ALIGN_COUNT
    - ALIGN_SUM
    - REDUCE_COUNT_FALSE
    - ALIGN_STDDEV
    - ALIGN_COUNT_TRUE
    - ALIGN_COUNT_FALSE
    - ALIGN_FRACTION_TRUE
    - ALIGN_PERCENTILE_99
    - ALIGN_PERCENTILE_95
    - ALIGN_PERCENTILE_50
    - ALIGN_PERCENTILE_05
    - ALIGN_PERCENT_CHANG
    type: string
  days:
    type: number
  filter:
    type: string
  group-by-fields:
    items:
      type: string
    type: array
  metric-key:
    type: string
  missing-value:
    type: number
  name:
    type: string
  op:
    enum:
    - eq
    - equal
    - ne
    - not-equal
    - gt
    - greater-than
    - ge
    - gte
    - le
    - lte
    - lt
    - less-than
    - glob
    - regex
    - regex-case
    - in
    - ni
    - not-in
    - contains
    - difference
    - intersect
    - mod
    type: string
  period-start:
    enum:
    - auto
    - start-of-day
    type: string
  reducer:
    enum:
    - REDUCE_NONE
    - REDUCE_MEAN
    - REDUCE_MIN
    - REDUCE_MAX
    - REDUCE_MEAN
    - REDUCE_SUM
    - REDUCE_STDDEV
    - REDUCE_COUNT
    - REDUCE_COUNT_TRUE
    - REDUCE_COUNT_FALSE
    - REDUCE_FRACTION_TRUE
    - REDUCE_PERCENTILE_99
    - REDUCE_PERCENTILE_95
    - REDUCE_PERCENTILE_50
    - REDUCE_PERCENTILE_05
    type: string
  type:
    enum:
    - metrics
  value:
    type: number
required:
- value
- name
- op

Permissions - monitoring.timeSeries.list

Actions

delete

mark-for-op

notify

set-access

set-labels

update

webhook

delete

Invoke an api call on each resource.

Quite a number of procedural actions are simply invoking an api call on a filtered set of resources. The exact handling is mostly boilerplate at that point following an 80/20 rule. This class is an encapsulation of the 80%.

properties:
  type:
    enum:
    - delete
required:
- type

Permissions - bigquery.datasets.get, bigquery.datasets.delete

mark-for-op

Label resources for future action.

The optional ‘tz’ parameter can be used to adjust the clock to align with a given timezone. The default value is ‘utc’.

If neither ‘days’ nor ‘hours’ is specified, Cloud Custodian will default to marking the resource for action 4 days in the future.

example:

policies:
 - name: vm-mark-for-stop
   resource: gcp.instance
   filters:
     - type: value
       key: name
       value: instance-to-stop-in-four-days
   actions:
     - type: mark-for-op
       op: stop
       days: 2

properties:
  days:
    exclusiveMinimum: false
    minimum: 0
    type: number
  hours:
    exclusiveMinimum: false
    minimum: 0
    type: number
  label:
    type: string
  msg:
    type: string
  op:
    type: string
  type:
    enum:
    - mark-for-op
  tz:
    type: string
required:
- type

Permissions - bigquery.datasets.update

set-access

Add or remove entries in a BigQuery dataset’s access list.

The BigQuery datasets.patch API replaces the entire access array, so this action reads each dataset’s current access entries, applies the add and remove lists, and patches the dataset with the result. Existing entries not affected by either list are kept.

add entries are appended unless an equal entry already exists. remove entries are match criteria: an existing entry is removed when every key in a criterion equals the entry’s value for that key. An add entry that also matches a remove criterion fails validation.

Roles may be given in either the legacy form (READER) or the IAM form (roles/bigquery.dataViewer); BigQuery returns only the legacy form.

Datasets whose access list would not change are not patched.

example:

Remove public access from datasets (CIS GCP Foundations 7.1)

policies:
  - name: bq-dataset-remove-public-access
    resource: gcp.bq-dataset
    filters:
      - or:
        - type: value
          key: access[].iamMember
          op: contains
          value: allUsers
        - type: value
          key: access[].specialGroup
          op: contains
          value: allAuthenticatedUsers
    actions:
      - type: set-access
        remove:
          - iamMember: allUsers
          - specialGroup: allAuthenticatedUsers

example:

Grant a group read access to datasets

policies:
  - name: bq-dataset-grant-analysts
    resource: gcp.bq-dataset
    actions:
      - type: set-access
        add:
          - role: READER
            groupByEmail: analysts@example.com

properties:
  add:
    items:
      additionalProperties: false
      oneOf:
      - required:
        - specialGroup
      - required:
        - iamMember
      - required:
        - userByEmail
      - required:
        - groupByEmail
      - required:
        - domain
      properties:
        domain:
          type: string
        groupByEmail:
          type: string
        iamMember:
          type: string
        role:
          type: string
        specialGroup:
          type: string
        userByEmail:
          type: string
      required:
      - role
      type: object
    minItems: 1
    type: array
  remove:
    items:
      additionalProperties: false
      minProperties: 1
      properties:
        domain:
          type: string
        groupByEmail:
          type: string
        iamMember:
          type: string
        role:
          type: string
        specialGroup:
          type: string
        userByEmail:
          type: string
      type: object
    minItems: 1
    type: array
  type:
    enum:
    - set-access
required:
- type

Permissions - bigquery.datasets.get, bigquery.datasets.update

set-labels

Set labels to GCP resources

example:

This policy will label all existing resource groups with a value such as environment

policies:
  - name: gcp-add-multiple-labels
    resource: gcp.instance
    description: |
      Label all existing instances with multiple labels
    actions:
     - type: set-labels
       labels:
         environment: test
         env_type: customer

  - name: gcp-add-label-from-resource-attr
    resource: gcp.instance
    description: |
      Label all existing instances with label taken from resource attribute
    actions:
     - type: set-labels
       labels:
         environment:
          type: resource
          key: name
          default-value: name_not_found

  - name: gcp-add-default-label
    resource: gcp.instance
    description: |
      Omitting key gives a conditional default - the label is set only
      when it is absent, leaving an existing value untouched
    actions:
     - type: set-labels
       labels:
         owner:
          type: resource
          default-value: platform

  - name: gcp-remove-label
    resource: gcp.instance
    description: |
      Remove label from all instances
    actions:
     - type: set-labels
       remove: [env]

properties:
  labels:
    additionalProperties:
      oneOf:
      - additionalProperties: false
        anyOf:
        - required:
          - key
        - required:
          - default-value
        properties:
          default-value:
            type: string
          key:
            type: string
          type:
            enum:
            - resource
            type: string
        required:
        - type
        type: object
      - type: string
    type: object
  remove:
    items:
      type: string
    type: array
  type:
    enum:
    - set-labels
required:
- type

Permissions - bigquery.datasets.update

update

Update BigQuery dataset attributes.

This action supports partial updates via the BigQuery datasets.patch API.

example:

policies:
  - name: bq-dataset-update-access
    resource: gcp.bq-dataset
    filters:
      - type: value
        key: id
        op: contains
        value: c7n_bq_dataset
    actions:
      - type: update
        access:
          - role: READER
            userByEmail: analyst@example.com

properties:
  access:
    items:
      type: object
    type: array
  defaultPartitionExpirationMs:
    type: string
  defaultTableExpirationMs:
    type: string
  description:
    type: string
  friendlyName:
    type: string
  maxTimeTravelHours:
    type: string
  storageBillingModel:
    type: string
  type:
    enum:
    - update
required:
- type

Permissions - bigquery.datasets.get, bigquery.datasets.update