What’s the best way of getting around the 100k object set load limit? I have a function that is called from a workshop with many thousands of objects (often more than 100k). Ive attached the part of the function that is failing due to the limit (line 99). Since the logic ultimately edits each and every one of the objects in the input object set, i dont believe aggregations would be a relevant solution here. Would appreciate any guidance or thoughts!
Actions currently allow editing up to 10,000 objects at a time.
Workflows where more objects have to be edited at a time are best supported by setting up a hybrid mechanism using both actions (to record an intent) and then running transforms to execute the intent at scale. Product teams are working on new features to make interactions between transforms and ontology edits smoother (for example being able to run a transforms job which is guaranteed to run only after the prior ontology edit has reached a materialization dataset)
I also faced similar issue where I had to apply edit on more than 10K (40K objects)
I suggest creating a flag column which helps to define which all objects will receive edits, use slice to limit your object filter to 9999 objects or less, then apply your edits and also revert the flag column to false.
Now run this logic with an automation in sequence.
