We are planning to use OTel log streaming capability to stream Palantir operational logs to log aggregator. Planning to use these logs to generate alerts rather than relying on monitoring views to generate alerts. Question - Would there be any gaps in logs which would prevent us from configuring alerts (using OTel logs) similar to available Palantir platform?
Hey, sorry for the delay in getting to this,
Internally all the monitor rules we have rely on Metrics and Events. For example the stream liveness monitor measures the time between consecutive checkpoint Events, and the streaming lag monitor relies on a lag Metric.
Currently, neither of these log types are present in the Log Export Stream as they are considered quite internal to the platform.
Depending on what you want to monitor, some relevant logs may be present (for example functions log when there is a success/failure), but some may not (for example actions don’t currently have this log). Additionally, we don’t currently publish any schema for the logs and so there are no guarantees about how the logs may change over time. As such, you may find such monitoring difficult.
Are there particular reasons you don’t want to use the in-platform monitoring? If there are blockers I’m happy to chat through them and see if we can find ways to unblock using it.