Data Virtuality Platform 3.13: another big and diverse release

This is another big release with many new additions, improvements, and bug fixes, so in this announcement, we'll highlight the most important of them.

On the Server side, we've implemented two new features and several improvements. The Server now supports the JSONTABLE, JSONPATHVALUE, JSONQUERY, and JSONARRAY functions, and we've added the DATE_TRUNC function to the list of already supported date and time functions.

Two of the improvements concern the Google Analytics Data connector: we've added token exchange for the OAuth workflow and support of custom dimensions and metrics. Column names on table creation now support more special characters, and we've updated the JSch library and exposed SSH configuration in the connection properties for SFTP data sources.

As for bug fixes, we've resolved an issue with wildcards in the importer.catalog property which couldn't be escaped (and which caused unnecessary requirement for using full schema names). Please note that underscores in the importer.catalog property are now treated as wildcards, and you can use the '\' symbol to escape underscores in order to explicitly use one catalog with an underscore if the importer.useFullSchemaName is set to false.

The Google Data Analytics connector also received a bug fix: the mandatory propertyId in runReport is no longer nullable. Now everything works properly.

For the Studio, we've updated Eclipse RCP to version 4.25.0.

In this release, we've added four new connectors: Greenhouse, Otto Market, Woocommerce, and Chargebee. As for the existing connectors, we've updated the Facebook connector to API v17.0. For the Amazon Vendor Partner connector, we've added support of Coupon Performance report and procurableProductOutOfStockRate and vendorConfirmationRate columns to the GET_VENDOR_INVENTORY_REPORT procedure - please note that there are new columns in the Vendor_Report_Inventory report procedure: procurableProductOutOfStockRate, receiveFillRate, uft, vendorConfirmationRate. For the Amazon Selling Partner connector, we've fixed two bugs: one causing the Report_FlatFileV2Settlement procedure to throw conversion errors due to an incorrect data type and another resulting in Inventory Ledger reports failing with "Non-whitespace character found between the qualifier". Another bug we've fixed affected the Google Data Analytics connector where Google Analytics Data: using reserved words as dimensions would throw exception. We've also resolved a bug with the Walmart connector which caused the Report_Items and Report_ItemPerformance pipe procedures to throw conversion errors and another bug with the Walmart Marketplace connector where pipes__ReturnOrders failed with 'Unexpected multi-valued result was returned for XMLTable column'. Now all is well. 

Here are all issues in this release:

Server

• DVCORE-8083 (New Feature): Add support for JSONTABLE, JSONPATHVALUE, JSONQUERY, and JSONARRAY functions
• DVCORE-7932 (New Feature): Implement a DATE_TRUNC function
• DVCORE-8105 (Improvement): Allow more special characters in column names on table creation
• DVCORE-8104 (Improvement): Google Analytics Data: add token exchange for the OAuth workflow
• DVCORE-8081 (Improvement): Google Analytics Data: support custom dimensions and metrics
• DVCORE-7940 (Improvement): SFTP: update JSch library and expose SSH configuration in the connection properties
• DVCORE-8103 (Bug Fix): Google Analytics Data: mandatory propertyId in runReport procedure is nullable
• DVCORE-8095 (Bug Fix): Wildcards in importer.catalog property cannot be escaped causing unnecessary requirement for using full schema names
  Underscores in importer.catalog property are now treated as wildcards.
  A '\' symbol can be used to escape underscores in order to explicitly use one catalog with an underscore if the importer.useFullSchemaName is set to false.
• DVCORE-7896 (Bug Fix): Trailing spaces are trimmed in table names
• DVCORE-7850 (Bug Fix): Parsing time fails if the part to be omitted is numeric
• DVCORE-7802 (Bug Fix): EXPLAIN statement used on a procedure call creates table metadata when the procedure has a SELECT INTO query in its definition
• DVCORE-7393 (Bug Fix): BigQuery: select from a table fails when the columnNameType translator property is set to UPPERCASE
• DVCORE-7114 (Bug Fix): "History Update" and "Upsert" jobs fail when the name of the identity column starts with a digit
• DVCORE-7063 (Bug Fix): Variable assignment behaves inconsistently when used with a result of a procedure
• DVCORE-6999 (Bug Fix): ARRAY_AGG function fails for some PostgreSQL data types

Studio

• DVCORE-7858 (Improvement): Update Eclipse RCP to version 4.25.0

Exporter

• DVCORE-7923 (Bug Fix): Separator is commented if there is a line comment in the end of an export element

Connectors

• SQL-695 (New Feature): Greenhouse: create connector
• SQL-469 (New Feature): Otto Market: create connector
• SQL-406 (New Feature): Woocommerce: create connector
• SQL-96 (New Feature): Chargebee: create connector
• SQL-738 (Improvement): Amazon Vendor: add procurableProductOutOfStockRate And vendorConfirmationRate columns to GET_VENDOR_INVENTORY_REPORT procedure
  New columns are added to Vendor_Report_Inventory report procedure: procurableProductOutOfStockRate, receiveFillRate, uft, vendorConfirmationRate
• SQL-711 (Improvement): Keepa: add Rating and Review columns for the Products procedure
• SQL-706 (Improvement): OTTO Market: update Products and Receipts to use the latest V3 endpoint instead of the obsolete v2 endpoint
• SQL-694 (Improvement): Amazon Vendor: add support of Coupon Performance report
• SQL-647 (Improvement): Facebook: update connector to API v17.0
• SQL-436 (Improvement): SEMrush connector: refactor connector
• SQL-421 (Improvement): Lead Alliance: add support of the new version (v2) of API
• SQL-752 (Bug Fix): Walmart: Report_Items and Report_ItemPerformance pipe procedures throw conversion errors
• SQL-744 (Bug Fix): Amazon Selling Partner: Report_FlatFileV2Settlement procedure throws conversion errors due to an incorrect data type
• SQL-734 (Bug Fix): Google Analytics Data: using reserved words as dimensions throws exception
• SQL-732 (Bug Fix): Walmart Marketplace: pipes__ReturnOrders fails with 'Unexpected multi-valued result was returned for XMLTable column'
• SQL-730 (Bug Fix): Amazon Selling Partner: Inventory Ledger reports fail with "Non-whitespace character found between the qualifier"
• SQL-371 (Bug Fix): Lead Alliance: executing LeadSalesStatistic pipe throws 'Invalid Integer format in String' error