Data Virtuality Platform 2.4.25: mostly about the Server

This is a rather big release and it brings quite a lot of improvements and bug fixes, mainly for the Server. Most importantly, we have fixed a bug affecting work with Google Ads API: previously, running several queries simultaneously might cause out-of-memory errors. Now you can safely run multiple Google Ads jobs without encountering errors. Another important bug fix improves REST API queries performance: we have resolved a bug causing the JsonStreamer.write method to not use a buffer and send the data byte by byte to the RESTEasy output stream. 

A whole bunch of fixes concerns Column Level Data Lineage: the path is now highlighted for a view if the query contains aggregation or DISTINCT or arithmetic operation or concatenation; the SYSADMIN.getDataLineageWithRelationsOnly procedure will not display temporary tables in the plan and will not fail with an assertion error anymore and will return the correct information for materialized views with enabled optimization; and WITH queries (also known as CTEs) will return correct results on generation of data lineage.

As for the Studio, we have fixed three bugs, thus improving performance and security. The job tab will no longer freeze if there is a job with a comma in the description; if a Dependency graph or an Impact analysis tab is open for a non-existent object, no error message will be shown upon reconnecting (the tab will now display a relevant message, 'The %procedure|table|view|schema' does not exist;); and last but not least, virtual schemas will not be shown in the Data Sources section if permissions are set to objects in those schemas but not the schemas themselves.

And finally, an improvement and a bug fix for the connectors: for Amazon Selling Partner, the Orders procedures will now save data into subordinate tables, and for Google Ads API, reports with the bidding_strategy and accessible_bidding_strategy parameters will yield correct results.

Here are all issues in this release:

Server

• DVCORE-7755 (Improvement): Google Ads API: fix mapping of date fields and add non-pushing down for specific tables
• DVCORE-7694 (Improvement): MongoDB: update the driver to version 4.5.0
• DVCORE-7657 (Improvement): Make it possible to directly assign values to array items
• DVCORE-7646 (Improvement): MySQL: update the JDBC driver to version 8.0.27
• DVCORE-7754 (Bug Fix): REST API: JsonStreamer.write method does not use a buffer and sends the data byte by byte to the RESTEasy output stream
• DVCORE-7744 (Bug Fix): Column Level Data Lineage: a path is not highlighted for a view if a query contains aggregation or DISTINCT or arithmetic operation or concatenation
• DVCORE-7741 (Bug Fix): Dependency graph generation fails when there is an uppercased virtual schema with a procedure respectively eponymous to an uppercased schema and table in a data source configured with importer.useFullSchemaName set to true
• DVCORE-7728 (Bug Fix): Column Level Data Lineage: SYSADMIN.getDataLineageWithRelationsOnly procedure returns temporary tables in the plan and in some cases fails with an assertion error
• DVCORE-7727 (Bug Fix): Redshift: no tables are shown for a data source with importer.schemaPattern value consisting of only wildcards
• DVCORE-7723 (Bug Fix): Google Ads API: running several queries in parallel may result in out of memory errors
• DVCORE-7574 (Bug Fix): SYSADMIN.getClientId adds a linebreak to client IDs defined with Windows line endings
• DVCORE-7395 (Bug Fix): Column Level Data Lineage: an empty targetColumnSchema value is returned when plan contains a materialized table in Snowflake
• DVCORE-6827 (Bug Fix): Column Level Data Lineage: WITH queries cause incorrect result on generation of data lineage

Studio

• DVCORE-7693 (Bug Fix): job tab freezes if there is a job with a comma in the description
• DVCORE-7674 (Bug Fix): an error message is shown upon reconnecting if a Dependency graph or an Impact analysis tab is open for a non-existent object
• DVCORE-7009 (Bug Fix): virtual schemas are shown in Data Sources section if permissions are set to objects in those schemas but not the schemas themselves

Connectors

• SQL-497 (Improvement): Amazon Selling Partner: Orders procedures should save data into subordinate tables
• SQL-537 (Bug Fix): Google Ads API: fix bidding_strategy and accessible_bidding_strategy parameters in reports