Hey developer community,
I would like to share some of the questions i find myself asking recently with the answers from our product team. This isn’t for posing new questions, but to provide quick, ready-made solutions for common queries. Hope this will be useful for the community.
What does task and sync stand for?
Here is the up-to-date description of sync and task in the public documentation: https://www.palantir.com/docs/foundry/data-connection/core-concepts/#syncs
How can we configure the number of jobs running in an agent?
In the agent configuration page, max concurrent syncs represents the maximum number of jobs that an agent can run at once. This includes batch syncs, exports, tasks and other sync types.
What are the advantages of using multiple agents on the source? How is the load distributed to them?
HA agents are meant to cover their counterpart agents being offline for maintenance. The HA agent setup also offers a very naive load balancing. It chooses the agent with highest bandwidth (bandwidth = maxConcurrentSyncs - currentlyRunningSyncs) however it is recommended to set up your agent/source configuration such that a single agent in an HA setup can handle the full load of what the sources assigned to it are trying to do.
What is the best way to investigate if an agent needs more heap memory?
Looking at the ‘Percentage of heap used’ metric clarifies if this needs remediation (i.e. the heap is being at or close to 100%). ‘OS Physical Memory’ metric can use more memory than is allocated specifically for JVM heap since -Xms is being used to allocate the heap with the size specified in the agent configuration. Moreover, it can also show memory usage from non-agent processes as well.
How does each concurrent sync allocate CPU/Memory?
There is no isolation between syncs on the agent. They are all operating in the same process.