I’m working on a personal project using Foundry/AIP, where a user can upload an image of their meal and then get a summary of the nutrients in that meal. In the workshop, I’m trying to figure out how I can make it so that the summary is generated for the most recent meal that was added to the object set. I use AIP Logic to generate this summary, and I pass in a “meal” object along with additional information. However, I don’t know how to specify which meal object is passed into this function. Is there a way to accomplish what I want to do?
I have included screenshots of my Workshop application for additional clarity.
Hi @jaysen_quan is there a specific reason why you are passing a multi-selected object set into your function - and not a single selection?
How would the user flow be in your imaginary app? I could imagine this in a way that users select a meal from a object list or table. If yes, then you could enfore the sorting of the table/list to be desc. by creation/updated date and enable auto-selection. This way the single selected object would always be the latest (until the user selects a different). Your meals object set would be the input to the object table/list.
Hi @Phil-M, when I added the “Meal” object type into Workshop and created a variable for that object type, I couldn’t find a good way to filter out a specific meal from the entire object set.
The user flow would be something like this:
The user uploads an image of a meal to a media uploader. That image gets processed in a data pipeline, where the food items are extracted from the image, and then a new Meal object gets created. Once this object is created, I am hoping to pass it into an AIP Logic function, where I generate a “meal summary” that presents the user with estimated nutrition facts about their meal. (This is just a Markdown block under the “Generate Description” button that displays the output string from the AIP logic function)
Here is a screenshot for clarity of what the UI in Workshop looks like right now. There’s probably an easier / cleaner way to do this, but I’m relatively new to Foundry so not quite familiar with the ins-and-outs of the platform yet.
If you were using the standard Button group widget, there’s a configuration option at the very bottom that allows you to bind the newly created object against an Object set variable that you could then use as an input to your AIP Logic function.
Given that you’re using the Media Uploader widget instead, that is not available. Does your object type have a Timestamp property to the effect of “created on” or “uploaded on”? If so, you can write a TypeScript function that always pulls the latest object when you sort in descending order, convert the object set to an object list, and access the first element (0-index). You’d publish this function, and use it to back an Object set variable in your workshop module (and let it recompute automatically).
Hey @joshOntologize, my object type does have a Timestamp property, and I think by using it, I was able to come up with a workable way to handle my desired functionality.
The only downside of this approach is that a user would need to wait a few minutes before uploading their next image (right now I have it set at the past hour, but I might change it to a few minutes). But, I think this is a good enough workaround for my purposes. Let me know what you think.
That’s a solid workaround! If this approach is acceptable for your use case, you should be good to go. I’ve pasted below a code snippet that you could use with a TypeScript function instead (where you create a new Object Set variable in your Workshop module, and back it by the published function with the below code).
import { Function } from "@foundry/functions-api";
import { Objects, YourObjectType } from "@foundry/ontology-api";
export class MyFunctions {
@Function()
public returnLatestObject(): YourObjectType {
// This returns an Object list/array
const latestObjectList = Objects.search().yourObjectType()
.orderBy(obj => obj.nameOfTimestampProperty.desc())
.take(1);
// This array contains exactly one element/object, return that exact object
return latestObjectList[0];
}
}
Regarding the following:
The latency you’re observing here is a byproduct of the workflow design and the way Foundry works in general. Based on something you mentioned in a previous post in this thread (see below), I’m assuming you’re storing the user-uploaded image in a Media Set you’ve created, and you have a schedule configured to run every time it detects an update/new file appended to that media set. This would then build the dataset that ultimately backs your Meal object type (please correct me if I’m wrong!).
The latency is the sum of:
time taken for the schedule to detect the update (new image uploaded),
time taken for the dataset to actually build (you can optimize this by making this process incrementally),
time taken for the Ontology to detect the new version of a backing dataset, and finally
time taken for the changelog, indexing, and hydrating steps into your Object Type in OSv2
Yes, that’s exactly how I have my workflow set up! Thanks for spelling out the causes of the latency, that makes perfect sense.
I’ll try implementing the code you sent. In terms of my current workaround, what I’ve done is that I pass in an object set containing objects that have been created within the past day into the AIP logic function (this is done by filtering the object set based on Timestamp as I mentioned before). I then use an LLM block in the AIP Logic function to figure out which of these objects in the object set has been created most recently. I originally passed in all objects, but I figured that eventually the whole object set would get increasingly large as more images are updated, so it’s more realistic to use the set of objects that have been created in the past day. I’m not sure how much computation time this actually saves in terms of the LLM figuring out the most recently created object, though.
Something you can do to let everything happen in the backend is string together your workflow as follows:
User uploads image to Workshop module
Create an Automation (condition: Object added to set → effect: run AIP Logic function)
Create your AIP Logic function*
Have a function-backed string variable in the Workshop module that calls the AIP Logic function
Your AIP Logic function doesn’t need to contain any inputs. The very first block at the top can be the implemented TypeScript function that returns the most recently created Meal object. The Use LLM block below it can then implement your logic and return a string that you store in the string variable in workshop (automatic recalculation), which you can display using the Markdown widget. Because the logic function only gets triggered when a new object is detected (via the automation), it is guaranteed to show the nutrients for the most recent Meal object.
Of course it’ll take a few minutes to show up in the frontend due to the above described latency, but this should work for what you’re trying to achieve.
Hey @joshOntologize, sorry for the late response. I think I was able to figure out a fairly consistent workflow using that TypeScript function. I basically just created an object variable that’s backed by the TypeScript function and I pass that object into an AIP Logic function that backs a string variable which is used in a Markdown widget, and then I have a button which triggers the widget to recalculate and that runs the AIP Logic function for me.
I appreciate the suggestion though, that seems like it would be a more straightforward way to accomplish this workflow because I wouldn’t need this extra button that the user has to press to see the description of their meal.
