After doing some calculations in a streaming transform, I want to export the results back to a kafka cluster. I have a streaming export set up and I want to set up health checks for the export.
I can monitor the stream that feeds the export, how I can I monitor the export itself? I’d like to ensure that the export is always active and there are an expected number of records / minute flowing through the export.
Hi! This feature is being prioritized. We will reach out once it is generally available.
Out of curiosity – how would you configure this alert? For example, “alert if less than 100 records are flowing through in the last 1 minute”.
Hi @tpark for streaming exports I think the best measurement would be lag. i.e. “Alert if there are more than X records in the source stream that have not been exported” or “Alert if there are records in the source stream that are X minutes old that have not been exported”