Google Cloud Fundamentals
Getting started with BigQuery

Getting started with BigQuery

We will stream our App Engine logs into BigQuery and query the data.

We will use the App Engine project created in #14 to store queries.

Setup Cloud Logging

  1. Confirm that you have a running instance of your app at App Engine (opens in a new tab)
  2. Visit Cloud Logging (opens in a new tab) and select your currently running application
  3. Click 'Create Export' to create a dataset to write to
    1. Add a meaningful name
    2. Select 'BigQuery' for sink service
    3. Create a sink destination (or select one if already created)
    4. Create sink
  4. Visit Exports (opens in a new tab) from the sidebar

Making queries

  1. Visit BigQuery (opens in a new tab) where the dataset should be visible (you may need to select the correct project)
  2. Create a few requests in your application
  3. Back in BigQuery, click on the table generated by Google below the export name in the sidebar
  4. In the table details click a few fields (only leaf nodes can be selected for the query) and make the query

Make a custom query

  1. Get the top 10 requests ordered by latency

    SELECT protoPayload.resource, protoPayload.latency
    FROM <your-table-id>
    ORDER BY protoPayload.latency DESC
    LIMIT 10
  2. Get all status codes and the number of times those status codes were returned in a request:

      COUNT(protoPayload.status) AS COUNT
      protoPayload.latency DESC


  1. Under Log Exports (opens in a new tab) delete the export
  2. In BigQuery (opens in a new tab) delete your dataset

Make sure to also delete the compute instance and storage created in the earlier demos.