Foundry Python Library Repos for Dependency Packaging in Spark
Summary A data engineer attempted to modularize their code by moving shared transformation logic from a Codified Transform Pipeline into a standalone Function Repository. Despite successfully publishing the repository and seeing it appear in the UI, the engineer was unable to import the functions into their main transformation pipeline. The failure occurred because the engineer … Read more