Analyze Media References Images with AI

Ask the Community
1

Hi, I have a use case where I need to upload images and analyze them with AI to get a description.

I’m currently working with attachments and passing them to the Code Repository and deploying a model in it, but I’d like to use Media References.

The only problem I have using media references is that I can’t find a method to perform that analysis with a media reference LLM.

Any advice or steps I can take to solve this problem?

Thanks a lot!

2

Hello!

You could leverage Pipeline Builder or AIP Logic for this. You can upload media to a media reference property via an Action, and then pass this to AIP Logic and use the LLM board there. Alternatively, you can use the media set that the media was uploaded to in pipeline builder and use the pipeline to feed the description property on the object.

Hopefully these docs will help:

https://www.palantir.com/docs/foundry/media-sets-advanced-formats/upload-media/

https://www.palantir.com/docs/foundry/pipeline-builder/transforms-transform-media

https://www.palantir.com/docs/foundry/pipeline-builder/pipeline-builder-llm/

https://www.palantir.com/docs/foundry/logic/blocks/#use-llm

Let me know if you have any questions about how to achieve this.

2 Likes
3

Thank you for your reply!!

Could you guide me to use the AIP Logic to do this, I get an error telling me I can’t use media references in AIP Logic.

Thanks,
Mateo

4

What does the error specifically say and when does the error pop up?

5

Sorry, It’s not exactly an error, i have this output from the AIP Logic

I am unable to view images directly. Please provide a description or specify what you would like analyzed about the image.

6

Are you using a vision model (i.e. GPT 41) and can you validate that you are properly sending a media reference to the useLLM board. You can send a media reference by either defining an input of media reference + passing that into useLLM board or by reading an MediaReference object property + passing that into the useLLM board.

It doesn’t seem like you are actually sending an image to the LLM if that is the proper message you get back from it.

7

Hello,

Now it’s working, I was sending in wrong way the image to the LLM.

Thanks for your help!!

8

This topic was automatically closed 91 days after the last reply. New replies are no longer allowed.