Ingest files from many subfolders from sFTP / Azure Blob Storage / etc

Ask the Community
1

I have a structure of files that is highly nested in folders in my source system:

- Root
  - Folder1
    - Subfolder1
      - SubSubfolder1
        - File1.txt
        - File2.md
      - SubSubfolder2
        - File3.txt
    - Subfolder2
      - SubSubfolder3
        - SubSubSubfolder1
          - File4.md
  - Folder2
    - Subfolder3
      - SubSubfolder4
        - File5.txt
      - SubSubfolder5
        - SubSubSubfolder2
          - File6.md
          - File7.txt

How can I configure my Data Connection Sync so that I can ingest all files from all subfolders ?

Note: I see the path matches option, but I want to know if I can specify something like */file* as a pattern, or something like it, to match all files regardless of their path.

image

2

The “Path matches” option takes a regular expression, not a UNIX-style glob. So if you wanted to match all text in a folder you would want folder/.*.txt