What type of window is used for "Outer caching join" in pipeline builder?

Ask the Community
,
1

In the “Outer caching join” board in streaming pipelines in Pipeline builder, what type of cache is used? Is the cache timer reset every time a new row with the same ID appears or is it based on the first ID?

Example:

I have a join between two streams A and B with joining based on id and cache set to 4 hours.

Then these events come in stream A:

  • T+0:00 id: 1, val: a
  • T+0:30 id: 1, val: b
  • T+1:30 id: 1, val: c

Does the record for val: c expire at T+4:00 or T+5:30?

3

Hey there, in this situation after all three records are processed, the 3rd record with val: c will be stored in the cache and will be expired at T+5:30. We update the timer every time a new record with the given key is processed