As I’ve written in articles before, data is “heavy” in that it costs money to move around, process, and extract value out of. Some emerging features in Ascend address the source of these costs, with some incredible side benefits. Let’s get into it.
Pinpoint Storage Location for BigQuery
For many customers, data location requirements are often driven by regulation and data mesh strategies.
As you know, data for pipelines is sourced from any location through Read Connectors, and finished data product is delivered to any location with Write Connectors. But as we described in our February update, the location for interim storage in intelligent pipelines is determined by the data cloud accounts to which you connect an Ascend data service.
Up until now, for BigQuery this location was by default in the GCP Americas zones. In this release, users can designate specific GCP regions where their interim data should be stored.
Start Your Pipeline with Pre-Loaded Data
Sometimes, your data pipeline starts with data that is already located in a table in your data cloud. Maybe you’ve used another tool to load data, or the data is the result of an application running natively in that data cloud. In any case, there should be no need to run it through another ETL ingestion process to get it into your data pipeline!
On Ascend, you can hook these existing tables into your intelligent data pipelines with zero reprocessing, eliminating redundant storage and unnecessary data movement. Ascend monitors these tables for new data just like any other data source, and automatically propagates all changes through the data pipelines. To reduce reprocessing even further, Incremental processing from these tables will leverage the native capabilities of each data cloud.
Look for these upcoming “My Tables” functions on the platform.
Leverage Cost-Saving Storage Tiers
As you know, AWS S3 (“Simple Storage Service”, remember?) is the OG of massive cloud storage used by many systems, including Ascend, when deployed on AWS. Less known is how to best utilize S3 storage tiers to save costs.
S3 now provides intelligent tiering, which automatically moves data into cheaper (and slower) tiers based not on age, but access patterns to the data. This pattern is a great fit for intelligent pipelines, which access older and newer data based on its relevance to delivering data products.
These savings are unique to Ascend, and will appear only as lower storage costs on your AWS bill.
Faster Cost Transparency
In January we highlighted the new data plane usage reports in the Observe panel of the platform. Users are increasingly drawn to this panel to see resource usage of their intelligent pipelines, pinpoint areas to refactor, and quantify the costs of delivering individual data products. These reports rely on queries on Ascend-managed cost metadata.
In this release, the reports’ performance is dramatically improved, running over 10X faster across all the data clouds. Regardless of the time span or depth of the resources you are looking at, these reports simply pop!
Also, check out the new Sunburst visualization!
Saving Network Costs from Ascend Updates
The relentless pace of innovation from Ascend means frequent updates to the many instances of the platform running in customers’ private cloud accounts. While the benefits of new capabilities are profound, these updates affect the networking costs of these customers.
In this release, we are revising how updates are deployed, resulting in smaller update payloads on customer networks. Unlike other tools that offer “dedicated installs,” all Ascend instances continue to operate as true SaaS, with zero-touch required by customers’ IT teams. These savings are unique to Ascend, and will appear only as lower networking costs on your AWS bill.
