I have a Java Map UDF running in a streaming pipeline. In my Java code, I have a SafeLogger that I use to output some debugging data. However, when I download the logs from the build screen, I do not see any logs from my UDF in the downloaded file.
Is there anything else I need to configure to get logs from my UDF?
It sounds like you have set up everything that is necessary to log output from your UDF in a builder pipeline. Based on my testing, it appears that logs are properly being displayed when run as part of a batch pipeline, but not as part of a streaming pipeline.
This is not expected and is a bug that we are currently investigating. In the meantime, if you would like to confirm that logging is correctly set up within your UDF, you can run it as part of a batch pipeline and see if the logs appear there. If so, then once this is fixed, you can expect them to appear in your streaming pipeline logs as well.
Thanks for surfacing this issue, and we will let you know once it has been resolved!
Hi, any update on this? I also have a Java Map UDF, and I have a branch with this setup
try {
throw new Exception("something went wrong!");
} catch (Exception e) {
System.out.println(e.getMessage());
logger.error("what are you doing!" + e.getMessage());
}
Even when I use the UDF in a batch pipeline, I don’t see any logs in the browser or when I download them.
Hi, could you share more detail about how you’re constructing this logger? I was able to see logs from a UDF running as part of a batch pipeline with the following setup:
Adding: implementation 'com.palantir.safe-logging:logger' under the dependencies section of the subproject’s build.gradle file
Constructing the logger as: private static final SafeLogger log = SafeLoggerFactory.get(MyMapUdfImpl.class);
Logging as: log.warn("Hello world"); from inside my UDF logic
If you try the above and it still doesn’t work, any other details you can share about how I may be able to reproduce this behavior would be much appreciated!