What is the difference between System Prompt and Task Prompt in an LLM Block?

As the title says, I want to know when and how best to use the system prompt vs the task prompt. Currently, I have all my logic in the system prompt, and only provide input variables into the task prompt so that the system prompt can read the variables provided in the task prompt. Is there a better way of using each respective prompt?

This seems roughly right to me - under the hood (which you can see in the Preview panel clicking the Show Raw option at the very top) AIP Logic concatenates together your System prompt, then a generated System prompt that describes how to use any tools you’ve configured as well as how to respond correctly, and then your task prompt.

image

There isn’t too much here that is Foundry-specific (other than the generated prompt in the middle that describes how to use the tools and format responses) so you can learn from any of the myriad prompt-crafting guides or develop your own intuition for what works best.

In my usage, I’ve generally put something like “Your job is…” as the start of the System prompt and written a paragraph or two that defines the agent. I normally haven’t templated this too much, though you could if you wanted to switch the “persona” based on different data-defined task.

But assuming you’re configuring an AIP Logic for a single task, it’s then in the Task prompt that I write what I actually want the LLM to do, any broad rules or heuristics, specific guidance on when to use certain provided tools, what sequence of logic to follow, and of course the relevant templated data.

I’m no means an expert, so this is just want I’ve landed on for a couple of text-centric production workflows.