I have a pipeline, including datasources, datasets, restricted-views and then that backs Objects Types.
How are the markings propagating in this situation ? Are builds required for the markings to propagate ? Do they propagate immediately when applied ?
For datasets, it’s instantly. I did not have other artefacts which you mention in my pipeline when I learned this sad lesson. I once applied a marking on a source or a directory, cannot remember. It went downstream instantly, so users were not able to see their own datasets. Then I removed the marking using code and then everything downstream had to be rebuild.
However, I have tested a case where I apply a marking, but downstream I already had a code which removes that marking. In this case, the marking did not go downstream beyond the point where it was removed.
If you apply a marking to a resource (either by applying it directly on the dataset or by applying it on a parent project/folder) then the resources that are downstream of that dataset in the pipeline will immediately have this marking propagated. This is because you’ve marked the data as sensitive and therefore all derived data is marked as well.
Similarly, if you are removing the marking from a resource where the marking is directly applied (or via a parent project/folder), then it will immediately be removed from downstream resources in the pipeline. (That is of course, unless these resources have the marking separately applied or propagated.)
You can simulate these effect as described in https://www.palantir.com/docs/foundry/data-lineage/see-impact-marking-changes/.
Now, if you are removing an inherited marking in a pipeline, then those effects are not immediate and require re-building downstream resources. This is because, unlike the case above, you’ve declared that the transformation is redacting the sensitive data in some way that makes the output not sensitive (e.g. removing a sensitive column). Thus, it requires running this transformation for the output to lose its marking.
Note that this can be complicated with incremental pipelines. Read more in https://www.palantir.com/docs/foundry/building-pipelines/remove-inherited-markings/.