Last year we released Declarative Resource Definitions and Declarative Pipeline Workflows, a declarative, YAML based approach for defining your dataflows in Ascend that can easily be checked into code repositories, and applied to your Ascend environment via the Ascend CLI. We’ve found a number of teams adopting this as a part of their DataOps best practices, and leveraging it in their dev-stage-prod lifecycle. For users who enjoy declarative models, YAML, and Jinja, this can be an incredibly power tool for managing your Ascend dataflows.
With teams using Ascend.io to automate an increasingly large number of their data pipelines, however, we’ve seen another common pattern emerge: programmatic creation of Ascend dataflows. For users who have highly templatized dataflows, we found that many of them were writing Python scripts to programmatically generate Dataflow Resource Definition YAML files. While exciting to see our users continue to build on and around Ascend’s existing capabilities, this also compelled us to create something new. This Python to YAML to CLI to API adds unnecessary steps and requires users to understand our YAML definitions even though they were just an intermediate stage. Additionally, many of our users expressed the desire to imperatively create, manage, and observe their dataflows.
As a result, we are excited to announce the release of our new Python SDK! This SDK sits on top of Ascend’s public API, and is dynamically generated based upon the Protocol Buffer and gRPC (which we use extensively) definitions of all components within the Ascend platform.
Want to give the new SDK a spin? Head on over to our Python SDK section in our developer portal.
New to Ascend and interested in giving our SDK and other new features a test drive? Sign up for a free trial and start building! Want to learn more about this feature and the other parts of the Ascend Unified Data Engineering platform? Schedule a live demo with one of our data engineers. Have a favorite function you’d like to see us add support for? Join us in Slack (#feature-requests) and let us know — we’d love to hear from you!
