Enable GKE logging

Job driver logs

See Managed Service for Apache Spark job logs in Logging for information on configuring and viewing Managed Service for Apache Spark on GKE Spark job driver logs.

Cloud Logging

Enable Logging to view, search, filter, and archive Managed Service for Apache Spark on GKE logs in Cloud Logging.

You can search for Spark driver and executor logs in Cloud Logging using the following labels:

  • dataproc.googleapis.com/cluster-uuid = Managed Service for Apache Spark cluster UUID
  • dataproc.googleapis.com/job-uuid = Managed Service for Apache Spark job UUID
  • dataproc.googleapis.com/role

Sample queries:

labels."k8s-pod/dataproc_googleapis_com/cluster-uuid"="Managed Service for Apache Spark cluster UUID"
labels."k8s-pod/dataproc_googleapis_com/job-uuid"="Managed Service for Apache Spark job UUID"
labels."k8s-pod/dataproc_googleapis_com/role"="driver"

What's next