What are the Changes on a Repository when having Project Scope Exemption?

[green]

Hello,

We have a repository that is project-scope exempted, so we can make API calls on the platform from Python code within the repository.

Recently we need to manage changes in security markings in the same repository, e.g. adding the stop_propagating=Markings(["marking_id"], on_branches=["master"]) on another transform in the same repo.

The “Allow changes to security markings in this repository” setting under “Security Approvals” is greyed out, and we think it’s because of the project scope exemption.The remove markings documentation doesn’t mention this facet.

I’ve checked the documentation, but can’t see anything listing the differences; but is there an overview somewhere please? The ability to make API calls will then disappear, but are there other consequences?

I don’t want to change this property of the repo without knowing what all the changes are going to be.

[redboyben]

@green

I’ve checked the documentation, but can’t see anything listing the differences; but is there an overview somewhere please? The ability to make API calls will then disappear, but are there other consequences?

I don’t want to change this property of the repo without knowing what all the changes are going to be.

Hi @green ,
I do not think PSJ exemptions and markings propagations are tied. Do you have a protected branch on your repository? You need one to enable the PR approval process required for markings.

[green]

Hello, @redboyben

Yes, this repo, repo1, has a protected branch. Just to back up a little on how I got here:

I tried to add stop_propagating, for the first time, to repo1. It has a protected branch, but on the development branch where the stop propagating was added, the checks would fail with this message:

Detailed errors: {This CI run attempted to introduce security marking changes,
but this repository is not running project-scoped CI.
Project-scoped CI can be enabled by repository owners in the "Settings" tab.,
This CI run attempted to introduce security marking changes,
but this repository is not running project-scoped CI
Project-scoped CI can be enabled by repository owners in the "Settings" tab.}

I cannot see a ‘Project-scoped CI’ entry anywhere in the repo settings. The closest I see is Security Approvals, but I cannot enable that (even with a protected branch) when Project Scope requirements are in force apparently:

The test, I created a new, second repo, repo2, in the same project folder, created a protected branch, enabled allow changes in security markings (which is possible in repo2), added stop_propagating on the branch, and the checks passed. With the PR approved by the marking owner we could merge it into the Main branch and everything worked as expected. So it seems to a repo, and not project, based issue.

Unsure what the problem is, I contacted our platform team, who got the following feedback from Palantir:

This is likely to do with PSJ Exemptions.
So if they’ve got Project Scoped Jobs Exemptions, they will not be able to change that setting
They can either setup a new repository, or remove the exemption
But removing the exemption may affect other workflows

We’re aware of the issue with not being able to make API calls when PSJ Exemptions are removed - but what other impact might there be? No-one seems to be able to point me to a list of the changes, or effects. Repo1 builds a number of very widely used datasets within our org, and so this is not a change I’m happy to just roll the dice on - especially if it’s not clear if there’s a trivial roll back or not.

If you think it isn’t to do with PSJ Exemptions, and there’s another setting, that would be great - as removing this to enable stop_propagating would be a pain for this and other repos we would then be forced to split because we can’t have API calls and changes in security markings in the same repo.

Thanks

[redboyben]

Hi @green - my bad, I did further testing and it does affect security approvals indeed.
Few things:

• I’m not sure why you cannot see the Project Scope tab - are you owner of the project?
• You’ll need that access to turn PSJ exemption off to enable security markings
• In terms of impact, the only thing that changes is which token is being used by the builds. So either a token restricted to the input/output datasets (ie more secure), or if you are PSJ exempt, a user token. So enforcing PSJ will have an impact on builds that are relying on the token to do some things - the vast majority of use cases here are indeed API calls.
• To turn it off, you only need the Project Scope tab access (if you are already owner, please ask your Palantir rep to investigate what you need to get access to it). To rollback in case of unexpected issue, you’ll need manual intervention from your Palantir reps as this is an insecure operation.

I would:

• crawl the repository for API calls - if you’re not making any, unlikely you’re using user tokens and PSJ is safe to remove
• if you don’t find any, I would coordinate with your Palantir rep to get access to Project Scope + orchestrate removing PSJ exemption (since you’ll need them for a potential rollback, I would coordinate with them especially since people might be off around end of year )
• if you do find some, then the best solution would probably be to isolate in a different repo the API calls in the PSJ exempt repo, and move everything else outside of the repo. You can “steal” an existing dataset from one repo by:

1. deleting the transform creating the dataset from repo 1
2. deleting the job spec on said dataset (keep a copy of it somewhere to be extra safe)
3. add the transform creating the dataset to repo 2 & run the CI - you should be good from there.

Eventually, isolating the API calls (and using external transform if that’s an option for you) would be the best course of action here for a long term setup following best practices

[green]

Hello @redboyben ,

Ok, thanks for that answer; that means other impact should be limited.

I am the owner on both the project and repo; but it may be that my org reserves some of those settings rights for the platform admin team - I have to ask them to add/remove the PSJ exemption.

In this repo we’ve identified the one case we make an API call, and can carve that out, the impact isn’t too big in this particular case.

For other repos we have, this is more annoying. Apart from having additional repos to maintain, it means if you’re making code changes that affect both transforms with API calls, and other transforms involving markings changes, you can’t sync the changes by having them on the same branch - you have to edit two in parallel, and test by coding in the other branch on dataset inputs etc. and when deploying have to sequence the merges correctly.