I am collaborating on a pipeline with 2 other users; at one point we all had the same pipeline/transform step open on our respective computers. However, when we each clicked “calculate row count”, each returned a different table length (eg one user’s computer said 29 rows, one said 28, another 31, etc), which is also different from the length of the final output table which was 90 rows. Why does this discrepancy occur? Is it ok to ignore or does it indicate something wrong with the pipeline? Thanks!
Hey @ctsig
A couple questions:
- Is there any non determinism in your pipeline ?
- Any LLMs?
- What’s the size of your input tables?
- Are any of you using input sampling?
In general I wouldn’t compare preview to your final output as preview won’t always use all the rows of your backing datasets
1 Like