When i use “collect array” within an aggregate, is the order of rows preserved? I have a sort block before my agggregate block but wanted to know if this is preserved (from a few examples I saw, it is preserved)
If not, what’s the best way to collect an array of values where the order is defined by a field called “sequence”
Hey @prithvim, order in a ‘collect array’ expression used in an aggregation is not deterministic so this isn’t guaranteed behaviour, because the order could be altered by a shuffle prior to aggregation.
You could try using an ordered (ie, Rows between, Sorted) window with ‘collect array’ sorting on ‘sequence’ instead, and then filter down to get the desired result.
Hi @tstrudwick
Thanks for getting back. This seemed to work
1 Like