Google Cloud BigQuery against AWS S3

The AWS ecosystem includes other cloud providers and software providers.

There have been requests to look beyond AWS-owned products. Google’s BigQuery Omni is an excellent opportunity to look at services run on AWS infrastructure but managed by competitors and partners. We looked at Athena as a potential alternative to Splunk a few months ago. Athena enables you to run SQL Queries against data stored in S3. However, it’s slow compared to Splunk.

BigQuery can run SQL queries against S3 as well. The cost of egress between AWS and GCP is a concern. However, BigQuery Omni is hosted within a select number of AWS regions. As a managed service, you can run high-performance SQL queries against the same S3 data as Athna without the egress cost.

Join the Conversation and register for the CTO Advisor Hybrid Cloud Virtual Event March 9th at 9:00am CT to 5:00pm CT.

Generally speaking, BigQuery isn’t cheaper or even the same effective cost as Athena. AWS charges per compressed TB queried while GCP charges per uncompressed TB queried. The tradeoff for the much faster results is cost.

Use Cases

Log Data Analysis

You can now get closer to the types of use cases you’d use Splunk for vs. Athena.

MultiCloud

If your primary cloud is Google, but you have customers with log or IoT data in AWS, you can use BigQuery Omni to analyze the data in place and transfer the results to your GCP console.

Hybrid Analysis

No rule says you have to pick between the two solutions. You may provide both solutions depending on service levels as a platform group. A data scientist may be OK with Athena’s performance, while a business analyst may need faster answers from BigQuery Omni.