There are cases where data larger than the defined column size is sent.
In this case, we would like to detect that such data has been sent and notify the operator.
I have tried to do this with Data Health, but there does not seem to be an option.
How can we achieve this kind of check?
You can monitor the length of the values in a string column with Data Expectations, in either Python Transforms or Pipeline Builder. For example, in Pipeline Builder, the expectation definition would look like this (assuming a maximum allowed length of ten characters):
A Data Health check will be automatically generated from the expectation. You can “Watch” this check to receive an e-mail notification when the check fails (or you can add the check to a Monitoring View to configure more advanced notification functionality).
Thank you for your answer.
It looks like you can do what you want by using Data Expectation.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.