Consuming External API Data into Pipeline Builder

Ask the Community
,
1

We’re working to pull in external data from various systems via API; specifically Workday.

We’ve tried the Workday connector but that doesn’t seem to be working correctly AND there’s no way to connect to Reports.

We’re able to setup the connection + webhook, and we can see the response, but 2 challenges:

  1. When we try and set Parameters to store the request response, it throws an error. Not a blocker because we can store the whole response, but still a big (note: the response is quite large, so the auto feature doesn’t work for this report.
  2. The bigger problem is we can’t figure out how to have this API (or Webhook) consume those API responses in Pipeline Builder.

I know we can build a Notebook to do all this, but I really want to avoid that if possible.

Anyone had success?

2

Hi @mjacoby, thanks for all the context. You can try following the steps below which is the standard workflow we’ve seen and recommend.

  1. Create and configure a REST API source with all the necessary configurations as-per your workday connection
  2. Create a External Python Function repository.
  3. Implement the necessary functionality to query your API, consume the response as needed, and return the data needed in your pipeline. For more information on using Source’s in Python environments see our external-systems library documentation
  4. Publish the python function and import into your Pipeline via User Defined Functions (UDFS)

Hope this helps!

1 Like
3

@uzaheer, appreciate the response. I don’t want to use anything custom.

This seems like a pretty simple thing to consume; API to Pipeline builder.

4

Mind elaborating on the bug you are encountering? Would be useful to see your webhook configuration and the error message