When creating data ingestion syncs for an AWS S3 Source Type, I have the option to implement a filter of type “customFileProcessor” (see screenshot below). However, there is no guidance on what types of filters I can actually implement. Is there an example of a yaml configuration that I could use to filter based on filenames, file contents, timestamp of modification etc. or any other supported conditions?
+1 I would like to know more about this as well.
It seems to be a relict of the past where things are only managed by Palanteerians?