Cursor, the AI-enabled IDE, has a @Docs feature that lets Cursor crawl and index a set of Developer Docs to give context to the model when prompting for code.
I wanted to have Cursor crawl the Foundry Docs but it seems like the way Palantir is rendering the docs is not conducive to crawling:
Unfortunately the Palantir docs seems to use client-side rendering and have anti-bot measures that prevent our crawler from properly indexing multiple pages.
We’re aware of this limitation and I’m throwing this to the team to see if we can add an alternative method for handling these types of docs. For now you’ll need to index individual pages manually, or you can try to get a local copy and @ it within your codebase!
Is Palantir able to provide a hosted and/or downloadable version of their docs that can be fed into Cursor?
Hey @Bougs, thanks for flagging this. At the moment we don’t but we are working on something very similar around Foundry context for LLMs that might be interesting to you. I’d love to know exactly how you’re trying to use Cursor for Foundry dev? Is it for building with the OSDK or something else?
Yes I am building Typescript Functions with OSDK. I create a Code Repo in Foundry and then work on it locally to harness the power of Cursor. A great use case for me would be:
Hey Cursor,
Given the performance and optimization techniques listed here, find ways to adjust my code to increase parallelization without changing the return value.
Unfortunately, AIP Assist has been woefully inadequate to help (often times just plain hallucinating functionality that doesn’t exist) and AIP Assist has very minimal integration with Code Repo Web IDE.
I just tested out cursor crawling using the url: https://www.palantir.com/docs/foundry and it seems to work. It indexed 3488 pages and it seems to generate code based off the docs from my basic testing.
I think it was the entry url, I tried some others that didn’t work at all. Also, yes it only took like a second. I had the same suspicion so I asked it to write me a function and it seemed to pick the relevant doc pages in the composer view. One for the cursor team to see the exact text that was pulled and indexed for each page.
