In code repositories, we’ve found that iterating with Python often ends up losing us time in the long run. Python is fantastic to work with, but working with it in code repositories can be somewhat of a hassle.
Partly, AIP autocomplete, and regular system autocomplete are very slow, compared to authoring functions in TypeScript. This is (probably) just Python being Python, and autocomplete is a pain on interpreted languages in general. However a larger issue comes to iterating and testing while developing new transforms. Due to Python being interpreted, very often there are errors not caught by the development environment (using a function with incorrect types, indexing things that can’t be indexed, etc.). This isn’t so much an issue in a local environment, but when testing consists of “Preview” for builds, it means that very simple errors end up killing tests during development rather quickly, which can be frustrating for devs.
TypeScript is supported across much of the platform, but it seems rather impracticable to implement for transforms, given their underlying spark runtime.
Rather, we would prefer to write more of our production code in Java, since it is actively supported for transforms, but there is some additional friction for doing so, due to the lack of examples on foundry documentation. It would be nice to have examples be toggle-able between languages, where appropriate.
Sourced from geeksforgeeks