I have a workflow where users are sending each other emails with Excels, to do simulations and ultimately make decisions.
How can I migrate this to Foundry ? How can my users have a one-source of truth to operate from instead of a scattered workflow ?
The main approach is to use AI FDE to convert and migrate your Excel workbook into a hardcoded pipeline, specific to this particular workflow.
This is essentially an AIP-Agent-assisted migration.
Note: There are ways to make this fully automatic, or self-evolving, but let’s cover that as an extension, at the end of this post.
Given AI FDE can iterate on your source resources, and create Foundry resources (Code Repository Pipelines, Pipeline builder, Datasets, Workshop application, Ontology Objects, exports, etc.) this ensures that the business “goal” of the Excel (and all its complexity) can be encoded in a workflow that will support your users day to day.
In an initial iteration, AI FDE can create pipelines (code repository-backed, for maximum flexibility) from raw Excels, and wire it to the different data sources/materialization of Ontology Object types, etc. that are already in your Foundry enrollment.
Create a Code Repository to serve as the working environment for the migration.
Upload the workbook and supporting materials (notes, auxiliary files, documentation) directly into a folder within the repository. AI FDE may need some libraries (e.g. openpyxl) for AI FDE to be able to interpret your Excel files.
Personal or ad-hoc Excel files used alongside the “official” workbook (or event transcripts of meetings of interview of end users) can be uploaded and incorporated.
Collect supporting materials such as business definitions, excel calculation notes, source-system documentation, mapping tables, example exports, expected results, known exception, and screenshots of important dashboards or worksheets
More source material means better pipeline construction and richer validation.
Start an AI FDE session and point AI FDE at the repository with a prompt similar to “the Excel is located at this path in the code repository” or by adding the file using the “Files” button. The migration proceeds through three focused agent sessions:
Session A - Extract logic and context: Start a first session of AI FDE to this code repository.
The agent iteratively inspects the workbook and produces a migration report for each metric, KPI, and calculated value. Reports capture formulas, static data, sample inputs/outputs, and edge cases. Given the agent queries the workbook directly, it can extract and validate logic in multiple passes, catching edge cases that a one-shot static extractor would miss.
Session B - Build pipeline: Start a second session of AI FDE on the same code repository that reads the migration reports and the raw materials, then generates Foundry transforms that reproduce the workbook’s logic. Input datasets you provide are wired in as pipeline sources.
The idea is for AI FDE to “build the new logic” from all the context that was accumulated in the repository.
Session C - Validate: Start a third session of AI FDE on the same code repository that compares the pipeline’s outputs to the original workbook’s values using the extracted example data, flags discrepancies, and iterates on fixes.
The idea is for AI FDE to validate the quality of what was produced so far, and correct it if needs be.
If end users needs visibility or even should be able to tweak the logic, it is possible to extract the logic of each cell in a separate row, synced as an object type. This object type can be surfaced in Workshop, for users to validate the logic.
Once reviewed, the decision/mapping/logic/formulas modified by end users via Workshop, in the Ontology, need to be flowed-back to the code repository.
This will require the code to be changed.
A developer can launch a new AI FDE session to perform the edits and open a PR on the code repository (which can then be reviewed by someone that can validate the changes).
Alternatively, when a user edits a field definition through the ontology, an automated agent can analyze the change and open a PR against the Code Repository. The PR is reviewed and approved by a developer, preserving auditability.
Pro-code agents are capable of doing this.
If the output is a decision (e.g. someone looking at numbers and deciding on a course of action) the output of the pipeline/workflow can very well be a Workshop application (so that the decision is captured as well as part of the ontology/app).
If the output has to be an excel (e.g. for regulatory or reporting workflows that must produce an Excel deliverable), it is possible to produce this excel as a last step of the transform.
Regardless if an Excel is the final artifact, it is possible to surface the pipeline-computed values in Workshop first, for commentary, manual overrides, and approvals (and then generate the final workbook via a static pipeline if useful).
The main downside of the above approach is that the onboarding is manual: one must validate the pipeline and run the AI FDE sessions manually. Each new workbook/workflow requires a fresh run through the three sessions. Although the agent does the heavy lifting, a skilled developer is needed to onboard and steer it.
The idea here is to use headless agents as a way to create new pipelines. To some extend, the idea is to automate the AI-FDE workflow described above.
At a high level:
The three build-time agents can also be implemented as a single agent with subagents. The whole workflow can be orchestrated from a Workshop app or a custom front end. The whole workflow could be orchestrated from Workshop or a custom app frontend.
This allows for a fully self-service onboarding of new workbooks and to scales to large portfolios of similar workbooks without a developer bottleneck.
However, the prompts and orchestration must be mature enough to run reliably without a human in the loop for most cases. So, this requires investment in evaluation. You need automated confidence that the generated pipelines are correct before letting users self-serve.
Session 1
I have an Excel workbook in this Code Repository:
The workbook and supporting documents are located at: /dropzone/.xlsx
Your working directory:Goal: eventually convert this workbook into a data pipeline that takes datasets
as inputs and produces the workbook as output. When source data updates, a new
version of the workbook should be produced.This Code Repository is your working environment. Keep all context in the repo.
- Store commands you use in a “Workbook interaction notes” file (markdown) for reuse.
- Store metric/cell/KPI/output definitions in dedicated file(s).
- Store progress as a checklist in a “Progress checklist” file so any session can resume.
Iteratively discover the workbook and build a “migration report” for each metric or
value, logging each completed task alongside next steps. Use terminal commands to run
Python scripts that inspect the workbook. Decompose how the workbook is built, extract
the underlying logic, capture edge cases, and produce guidance a downstream agent can
use to build the equivalent pipeline.
Session 2
I have an Excel workbook in this Code Repository:
Workbook path: /dropzone/.xlsx
Your working directory:Goal: convert the workbook into a data pipeline that takes datasets as inputs and
produces the workbook as output. When source data updates, a new version is produced.The repository contains migration reports at docs/migration_reports/ — the output
of prior analysis of the workbook and supporting context.Reference files:
“Workbook interaction notes” — previously used commands for inspecting the workbook.
Metric/cell/KPI/output definitions — in their own files.
“Progress checklist” — what previous agents completed.
Input data is available at:Your task: translate the migration reports into working transforms.
Pipeline outputs should include:
[Define the primary output — e.g., a transform that regenerates the Excel file
populated with pipeline-computed values. Before the file is produced, values should
also be written to a tabular dataset (with cell position and metadata) so they can
be ontologized and reviewed by users pre-generation.]
A dataset of field/cell definitions — flowing into an object type that captures
the business logic and rationale for each cell.
A dataset of computed field instances — flowing into an object type users review
before the final report is generated.
[Optional] Agent [Orchestration around the pipeline]
Create orchestration around the main pipeline.
Object types needed:
“Field” — for reviewing the extracted logic (from the markdown migration reports,
converted into a reviewable tabular dataset).
“Document” — a new instance created by a user in a Workshop app, marked “validated”
once all fields are approved.
“Field instance” — for reviewing generation on a specific document’s data.
When a field is edited, regenerate the final Excel and link it to its Document instance.
If input data is dated, parameterize the date on the Document object and filter accordingly.
Session 3
I have an Excel workbook in this Code Repository:
Workbook path: /dropzone/.xlsx
Your working directory:Goal: confirm we have a correct data pipeline equivalent to the original workbook.
When source data updates, a new version of the workbook is produced, and the results
must be correct.The repository contains migration reports at docs/migration_reports/ and the transforms
written as the pipeline equivalent of the workbook.Reference files:
- “Workbook interaction notes” — commands for inspecting the workbook.
- Metric/cell/KPI/output definitions.
- “Progress checklist” — prior agent progress.
Your task: validate that the pipeline output is correct against the original workbook.
Document your findings so another agent can fix any discrepancies you identify.
