Data Virtuality Platform 4.0.7: a big and diverse release

This is a pretty big release which includes several new features, quite a lot of improvements, and many bug fixes, so in this announcement, we'll focus on the most important things.

The Server now supports Amazon Athena as a data source and includes an OpenAPI (Swagger) connector. The improvements include OAuth support for Google Data Analytics and "jdbc-properties" connection property support for SAP HANA. We've also adapted the Oracle ADWC connector to use the new JDBC driver version and updated the JDBC driver to ojdbc11 v23.2.0.0. Instead of the old SYSADMIN.resetAtomicSchedule, there's a new SYSADMIN.resetSchedule procedure which resets both atomic and non-atomic CHAINED schedules. Last but not least, our REST API now returns the information regarding the parameters associated with a procedure.

As for the bug fixes, most importantly, we've fixed a bug affecting S3 and Azure Blob Storage and causing the connector procedures to handle relative and absolute paths incorrectly when data source prefix is set. As a result of another important bug fix, the timestamp delivered by our REST API now conforms to the ISO 8601 standard ('yyyy-MM-dd HH:mm:ss.SSS'). We've also fixed a bug resulting in the Server sending too many queries to configuration database when storing query and data lineage plans. For SQL Server, we've resolved a problem when selecting from table valued functions in schemas or data bases with dashes in their names resulted in error, and for Common Table Expressions, another problem causing SELECT INTO from CTE to fail. For PostgreSQL, we've fixed a bug causing improper parsing of capitalized column names for JSON columns and ambiguity error when joining tables with JSON columns. Now everything works well.

For the Studio, we've fixed a bug affecting the SQL Editor for Linux users: if you use the dark theme for your OS, the SQL Editor in the Studio will use the dark theme, too.

The Connector family welcomes two new members: Piwik and Partnerize connectors. The Linkedin connector received an important update: as the legacy Content API and Share API had been deprecated, we've replaced the Shares and UgcPosts procedures with the Posts procedure, and we've also fixed a bug preventing posts-endpoint from returning all posts of the API. For the Amazon Selling Partner connector, we've resolved the bug causing FinancialEvents not to generate the TaxWithheld_Taxes child table, and for the Amazon Vendor connector, we've fixed the problem with GET_VENDOR_NET_PURE_PRODUCT_MARGIN_REPORT returning FAILED status instead of data. Last but not least, we've fixed two bugs affecting the Adobe Analytics connector: one resulting in failure when receiving the http 400 'report_not_ready' response message, and another which caused installing examples to fail. All is well now.

Here are all issues in this release:

Server

• DVCORE-6140 (New Feature): Add support for Amazon Athena as a data source
• DVCORE-5270 (New Feature): OpenAPI (Swagger): implement a connector
• DVCORE-8072 (Improvement): Google Analytics Data: add support for OAuth
• DVCORE-8051 (Improvement): Oracle ADWC: adapt the connector to use the new JDBC driver version
• DVCORE-7765 (Improvement): SAP HANA: add support for "jdbc-properties" connection property
• DVCORE-7561 (Improvement): Oracle: update JDBC driver to ojdbc11 v23.2.0.0
• DVCORE-6583 (Improvement): Create the SYSADMIN.resetSchedule procedure for resetting both atomic and non-atomic CHAINED schedules
• DVCORE-6193 (Improvement): Data Virtuality REST API: send procedure parameters in response when the source endpoint is used
• DVCORE-8073 (Bug Fix): Rest API: timestamp format does not conform to the ISO 8601 standard
• DVCORE-8023 (Bug Fix): JsonToXML parsing escapes allowed symbols
• DVCORE-7983 (Bug Fix): PostgreSQL: improper parsing of capitalized column names for JSON columns and ambiguity error when joining tables with JSON columns
• DVCORE-7961 (Bug Fix): SQL Server: selecting from table valued functions in schemas or data bases with dashes in their names results in error
• DVCORE-7960 (Bug Fix): FTP/SFTP/SCP: UTILS.unzip procedure fails if archive contains a folder
• DVCORE-7959 (Bug Fix): FTP/SFTP/SCP: UTILS.unzip procedure fails if output folder does not exist
• DVCORE-7787 (Bug Fix): Web service connector: content-length header cannot be set for empty requests
• DVCORE-7776 (Bug Fix): Common Table Expressions: SELECT INTO from CTE fails
• DVCORE-7753 (Bug Fix): S3 and Azure Blob Storage: connector procedures handle relative and absolute paths incorrectly when data source prefix is set
• DVCORE-7603 (Bug Fix): BigQuery: replications from BigQuery into BigQuery fail for TIMESTAMP data type
• DVCORE-6646 (Bug Fix): BigQuery: source views are shown as tables in SYS.Tables
• DVCORE-6614 (Bug Fix): Server sends too many queries to configuration database when storing query and data lineage plans

Studio

• DVCORE-7881 (Bug Fix): SQL Editor color scheme stays light when dark theme in Linux is set

Connectors

• SQL-700 (New Feature): Piwik: create connector and pipes
• SQL-429 (New Feature): Partnerize: create connector
• SQL-645 (Improvement): LinkedIn: replace procedures due to deprecation of the legacy Content API and Share API
• SQL-710 (Bug Fix): LinkedIn: posts-endpoint does not return all posts of the API
• SQL-705 (Bug Fix): Amazon Selling Partner:FinancialEvents does not generate TaxWithheld_Taxes child table
• SQL-639 (Bug Fix): Adobe Analytics: failure when receiving the http 400 'report_not_ready' response message
• SQL-600 (Bug Fix): Amazon Vendor: GET_VENDOR_NET_PURE_PRODUCT_MARGIN_REPORT returns FAILED status instead of data
• SQL-331 (Bug Fix): Adobe Analytics: installing examples fails