Programatically upgrade a library version

Ask the Community
1

I know that the environment is fixed and there is a lock file that keeps the consistency of the code, but I have a use case where I want to generate a dataset based on all the configs that exist in that library. Currently, I have a process that reads all the configs in the lib and generates a dataset, but every time there is a new version, I have to manually point to the newest version of the library…making it unmaintainable…

I would like that:

  • Every time the pipeline runs, get the latest version available of the library
  • The process doesn’t require manual intervention to increase the library tag

How can I achieve this?

2

Library version can only be updated trough at least running checks once and regenerating the jobspec of your dataset.

You could automate the commit to bump the library version. This code could run on a schedule to periodically check if there is a new library version and if yes, perform the commit.

You could use a TPA with client credentials and editor permissions on your project to make the API call.

You might want to protect the branch of your repo that uses the library, so you would need to do API calls to unlock the branch and re-lock it after the library version bump commit.

1 Like
3

So you think the only way would be to externally do this with API calls? Or how would you propose to do it? (with more detail, please)

I’ve checked this, dont know if there is any possibility to do something with it that can trigger the library bump:

https://www.palantir.com/docs/foundry/code-repositories/create-custom-checks

We have a master library and we have a process that gets all the info on this library, but we cannot see how we can automate the info extraction when a new version is published…

4

Custom checks won’t work since a) there is no way to securely store any kind of credentials and b) no network connections possible.

You don’t have to run my proposal outside of foundry. You can use an external transform, compute modules or even a function that you trigger on demand.

5

Can you please be more specific? I currently dont see how this could be feasible.

How would you do something like the following:
1- programmatically verify if there is a new version
2- unlock branch
3- automate the commit to bump the lib version to the latest
4- lock branch
5- run transform that extracts all the info from the lib

In a function (Typescript or Python)