I have a code repository that creates a dataset of dataset metadata. I need to be able to access the end-point that AIP Assist suggests as the solution to my problem, which is ‘{base_url}/foundry-metadata/api/datasets/{dataset_rid}/branches/{branch}/metadata’.
Using curl I hit that endpoint with the requisite headers, but it returns {“errorCode”:“NOT_FOUND”,“errorName”:“Default:NotFound”,“errorInstanceId”:“bla-bla-bla-bla”,“parameters”:{}}.
Can you give example values of what you mean with dataset metadata? Asking because that can mean many things in Foundry.
Metrics about last runtime, whether a stream is up or not, and I’d be interested in learning what else the metadata object could tell me.
For something like this I’d suggest opening a page with the browser’s DevTools open. Then in the Network tab you can search for part of the URL, e.g. “metadata” and you’ll probably find some examples of the API usage that might help you.
When I do that, looking at a dataset’s preview page I see one example with a Request URL in the form:
https://<base_url>/foundry-metadata/api/schemas/datasets/<rid>/branches/<branch_name>?endTransactionRid=ri.foundry.main.transaction.<snip>
The response will be transaction based (as things like schemas etc. can change between builds). So I imagine first you’ll have to do a call to a branch to get the transaction rid, and then the metadata call that mirrors the example above.
Within Slate under the Queries tab there are also various platform queries possible, like to the Build service, which will give you build histories and status etc. So if you’re visualising it there, these may be easier to use (as you don’t have to do the same authentication process if you’re calling it from the Slate app on the same platform).
These tend to be slightly different in endpoint format than those listed in the documentation, or what you see in the network traffic (like the above example doesn’t contain the /v1/ in the path that is in the documentation, so it’s probably a different endpoint).
1 Like
I have tried that, however I am only seeing that ‘schema’ endpoint you referenced when I filter by ‘metadata’. That’s not enough. I understand you are saying that’s one example, but I cannot find more there. I appreciate the Slate feedback, but I’m not trying to learn a new app right now. I did pull it up and look at it at least, but it’s too complicated.
Does anyone have further ideas on this? I reached out to Palantir for assistance and their reply was that AIP Assist “must have hallucinated that endpoint”. You can’t make this stuff up.
I think your requirements are not clear.
Can you show some screenshots of the metadata that the UI shows that you want to programmatically retrieve?
Here we have a streaming dataset and it is constantly going down. But when it goes down it doesn’t alert even with several Data Health checks.
There are also times when it goes down, reporting an error code that is swallowed.
Our on call team has to watch it like a hawk b/c monitoring does not alert us.
I don’t want this topic to expire. Any other ideas here?