Error: Code references a non-dataset resource (ri.blobster.main.configuration)

Ask the Community
1

I’m running into an issue in a Python Code Repository while trying to ingest a raw JSON file

The Error:

Code references a non-dataset resource: ri.blobster.main.configuration.bf6ec23e-bee6-46ef-a13d-fbbcd9dba4cd

Context

I have imported the raw data into my project.The file is a JSON Lines file.I am using the following decorator:

@transform(
my_input=Input(“/Path/to/my/raw_data_folder”), # Points to the folder/file
output=Output(“/Path/to/output_dataset”)
) I’ve verified the path matches the Foundry filesystem, but it seems Foundry is resolving the path to a configuration or folder type rather than a dataset type

How can I reference a raw uploaded JSON file as an Input if it hasn’t been “serialized” as a dataset yet? Do I need to use FileSystem access differently, or is there a way to force Foundry to treat this RI as a valid input?

2

The issue you’re facing happens because Foundry requires datasets as inputs for transforms, not raw file references. When you upload a raw JSON file, it needs to be imported as a dataset first.

To resolve this, import your JSON file as a dataset without any schema. This will create a wrapper around your raw file, allowing you to access it properly in your transform code.

Use filesystem().ls() filesystem methods to access the raw file content without schema.

3

That worked. Thank you!

1 Like