Data Virtuality Platform 3.5: Many important improvements and some bug fixes

This release contains quite a lot of improvements. Most importantly, we've updated the Google Ads API to v12, which means significant changes for the Server and the Google Ads API connector because, in v12, the "campaign_experiment" table has been removed, as well as some columns (e.g. "ad_group_ad"."ad_gmail_ad_marketing_image", "video"."ad_group_ad_ad_gmail_ad_header_image") from tables. For the Google Ads API connector, there is one additional change: all columns with the "ad_gmail_ad" prefix (e.g. "ad_gmail_ad_header_image", "ad_group_ad_ad_gmail_ad_header_image", etc.) were removed from the following procedures: "Performance_AdGroupAd", "Performance_AdGroupAdAsset", "Performance_AdGroupAdAssetCombination", "Performance_FeedItem", "Performance_FeedPlaceholder", "Performance_SearchTerm" and "Performance_Video". If you need the complete lists of removed and added objects, please get in touch with our support.

This is not the only update to a later version in this release: we've also updated the Cassandra driver to v3.1.3.

As more minor, but functional improvements, we've added handling for incomplete UTF-8 multibyte sequences, and more information to the "Max block number exceeded" error message.

Of the bug fixes, two deal with issues affecting BigQuery: one prevented the RECORD data type in mode repeated from getting flattened, and another caused parallel queries getting data from TIMESTAMP columns to fail on tables larger than 30K rows. Now all works fine.

Another couple of bug fixes improve Excel performance: in case of an incorrect value in a formula cell, a clearer error message is now returned, and for cells with functions, the data type mapping is now correct. Here's what changed: previously, all formula cells were mapped to the STRING data type, and now they are mapped to data types corresponding to the return value of the function used. For example, if a cell contains a date function, it is mapped to the DATE data type; if it is an "equal to cell A1" function, it is mapped to the A1 cell data type.

Speaking of functions, we've fixed the behaviour of the JsonToXml function: previously, it escaped the special characters instead of replacing them with blank spaces. Now it processes them correctly.

Last but not least for the Server, we've worked on a bug affecting the Web service connector and preventing authCode from being saved in the connection properties. This works fine now, too.

Last but not least overall, another bug fix for the connectors, namely, for the Google Analytics connector: now, we've added ORDER BY for custom pipes to ensure columns are sorted in the expected order.

Here are all issues in this release:

Server

• DVCORE-7918 (Improvement): Google Ads API: update to v12
• DVCORE-7909 (Improvement): Add handling for incomplete UTF-8 multibyte sequences
• DVCORE-7869 (Improvement): Cassandra: update the driver to v3.1.3
• DVCORE-7862 (Improvement): Add more information to the "Max block number exceeded" error message
• DVCORE-7906 (Bug Fix): BigQuery: RECORD data type in mode repeated does not get flattened
• DVCORE-7788 (Bug Fix): Excel: an unclear error message is returned in case of an incorrect value in a formula cell
• DVCORE-7757 (Bug Fix): Excel: incorrect data type mapping for cells with functions
• DVCORE-7612 (Bug Fix): BigQuery: parallel queries getting data from TIMESTAMP columns fail on tables larger than 30K rows
• DVCORE-7084 (Bug Fix): Web service connector: authCode is not saved in the connection properties
• DVCORE-4722 (Bug Fix): JsonToXml function escapes the special characters instead of replacing them with blank spaces

Connectors

• SQL-646 (Bug Fix): Google Ads API: data source creation fails due to an updated list of columns returned by some templates
• SQL-577 (Bug Fix): Google Analytics: apply column ordering for custom pipes