Configure Streaming Resources for Data Ingestion

What is the best way to configure resources for a streaming ingestion? We currently have a Google pub/sub topic we are ingesting with very low data volume. Base on the calculations from the docs, it appears we are using ~8 streaming cores which feels a bit unnecessary and is increasing costs significantly. Note, this is for the raw ingest, not any downstream pipeline builder datasets.

How should I go about reducing the compute utilization for this ingest?

1 Like