Real-time update on Workshop

Ask the Community
,
1

Hi guys, does any one have an idea how to enable real-time update to be seen on a workshop application ?

So far i am seeing 5 min delay from the time the pipeline build is completed and data appearing in workshop (Refresh is set to 10 sec for the active objects i am using on the workshop)

How do we handle the latency that is observed from getting data as soon as the pipeline build is complete

2

Depending on your setup, after a pipeline build is complete the ontology will still need to run a few things such as indexing which may explain the 5 min delay. You should monitor the object to see if that’s about how long it takes:
https://www.palantir.com/docs/foundry/object-indexing/funnel-batch-pipelines/

1 Like
3

If your data is fundamentally coming in batches (i.e. you have a scheduled sync that brings in data, say, every 15 minutes) then you’ll probably be best to consider the final sync from the backing dataset to the ontology object type as the final “job” of the pipeline. Fundamentally there is still “work” done - and you can go in and observe the sync and index jobs from the Object Type Datasource configuration page - that is going to take some amount of time, depending on the scale of the updated data.

If you actually have streaming data and want the app to update continuously as new data arrives, then you can configure lower-latency throughput all the way to the ontology with streams.

2 Likes
4

Adding to this thread:

It would be extremely beneficial to have ‘Build Status Widget’, where you can select pipelines, schedules, or Ontology Objects and it provides their current status and last build time.

Even when the schedules & pipeline builds complete, it still takes a few more minutes for the Ontology Object to re-index also.

Our users uploading data/files into their workshops, need a visual means to know when to refresh the workshop to see the fully complete / ingested results.

Forrest

2 Likes