I’m looking for a way to access the internal probability distribution (e.g., logits or token probabilities) that an LLM computes during generation. My goal is to analyze or manipulate these probabilities directly for use in detecting hallucination
Is there a way to expose or extract this probability state when working with LLMs inside the Palantir Foundry / AIP environment?
Any pointers or examples would be appreciated.