I have an AIP Logic built into a workshop page where it is summarizing the data of a table. The table can be filtered and then causes the AIP Logic to recompute using the new filtered data.
I was wondering if there was a way to save/preload the output of the AIP to reduce the need for recomputation and less “thinking” time. I’ve experimented with state-saving but that doesn’t work for displaying to end-users.
There’s no way of knowing up front how a given user will filter a table so there’s no way of knowing what to cache. Assuming you somehow had this foresight - perhaps there’s some state saving thing you can do where several “saved filters” are saved and can be chosen from instead of open ended filtering, you may be able to do something like display a separate AIP logic widget on the screen but in a 0 width section, one for each request you’d like to precompute.
There is currently no better first class way to “preload” a widget or variables, but I will add a +1 to our internal request tracking this.
hi @matthewqin,
Just to be clear:
Right?
If so, there are a few workarounds, but none are “pretty”. Below I am assuming that you are working with Objects, but this can be tailored to fit other data inputs.
Start with a helper Object Type:
Where you might pass the query directly to the LLM, you now need a helper function to do this instead. Depending on how much you rely on the widgets for UI work, there will be a few caveats listed below:
The helper function should:
Now, if you are relying on the Workshop widgets for your UI, your helper function might need to output a struct with a few more variables than the summary itself (e.g. ‘summaryExists: boolean’), and you will need to bind this all together with Workshop’s variable transforms (summaryExists === true, then return the object and don’t send the summary to the LLM widget,etc.)
Let me know how this solution works in your use case, and we can reiterate if needed.
@jakehop ‘s solution certainly works if your problem is that the same inputs should not recompute the AIP Logic output.
But each time you change the filter, its going to change the inputs. And, depending on your solution, that could trigger multiple requests while the user works his way through the filter inputs.
So caching isnt going to help for this situation.
If your problem is that each filter change (assuming there can be many) kicks off a new request, is it not better simply to have the user click a button to call the AIP Logic function once they are done filtering? Then the LLM delay is only happening when the user requests the AIP Logic output. Not a complex solution but it all depends on your aims.
@mad_scientist raises a fair point.
A last way to get these summaries calculated (if that is needed) is to either use a faster model or combine a faster model with a precomputed output.
I have a pipeline where we batch compute descriptions of large amounts of data, where a (cheap) LLM is creating summaries and a list with specific keywords. This is done once at ingestion, so we are able to use a very fast model to do summaries on the fly, at low cost.