Data Virtuality Platform 4.0.1: brand-new version with many improvements, and not only

We're very happy to present this very special release of version 4.0.1 bringing many new features, improvements, and bug fixes!

Starting with this version, the Data Virtuality Server supports JPA as a connector. We've added a new, even more secure, type of authorization: OAuth2/OpenID (JWT) authorization via a third-party identity provider. We've also introduced clustering mode for the Server and versioning support for remarks of the data source objects. Git Integration is no longer a preview feature - it's got a complete overhaul and is now complete with a new set of system procedures (check out the fully updated documentation). Furthermore, you can create or replace data sources with a single procedure call using the new CREATE OR REPLACE command.

As for improvements, for S3 data sources, we've added CEPH support, and for the JDBC driver, support of OAuth2 authorization type. We've also enabled "OPTIONS Annotation" on views and procedures which means that from now on, remarks are stored in two columns in SYS.Table, SYS.Columns, SYS.Procedures and SYS.ProcedureParams system tables: the description column contains remarks added on the Data Virtuality side, and the OriginalDescription column contains remarks from the data source added outside of Data Virtuality.

Another significant improvement in this release makes it possible to configure WildFly metrics host and port and use Prometheus JMX Exporter as an alternative to standard WildFly metrics.

The bug fixes are also present in this release: we've fixed a bug which made it impossible to create a copyOverSQL job if the source object is missing despite the OPTION $NOFAIL and a bug causing an escape sequence parse error on selecting from a TEXTTABLE if the escape character is followed by a non-escapable character. And for S3 data sources, the UTILS.unzip procedure will now work as it should.

The Connectors have a new addition: the brand-new Linnworks connector. As for our existing connectors, we've added, among many other things, five new reports for Amazon Selling Partner: GET_SALES_AND_TRAFFIC_REPORT, Inventory Ledger, GET_BRAND_ANALYTICS_SEARCH_TERMS_REPORT, GET_BRAND_ANALYTICS_REPEAT_PURCHASE_REPORT, and GET_FLAT_FILE_SALES_TAX_DATA. And our Amazon Vendor connector now supports GET_VENDOR_FORECASTING_REPORT. We've also updated the TikTok API version to v1.3 and fixed the bug causing the BasicReport procedure to return an "HTTP 400 Bad Request" error.  

Here are all issues in this release:

Server:

• DVCORE-7936 (New Feature): Implement JPA translator
• DVCORE-7864 (New Feature): Implement CREATE OR REPLACE and $NOFAIL support for views and procedures
• DVCORE-7675 (New Feature): Enable creation or replacement of data sources using a single procedure call
• DVCORE-7596 (New Feature): Add support of OAuth2/OpenID (JWT) authorization via a third-party identity provider
• DVCORE-7419 (New Feature): Introduce clustering mode for the server
• DVCORE-6936 (New Feature): Git integration (Part 2)
• DVCORE-4813 (New Feature): Add versioning support for remarks of the data source objects
• DVCORE-7954 (Improvement): Git integration: allow creating tags
• DVCORE-7935 (Improvement): S3: add support for CEPH by providing endpoint parameter when creating an S3 data source
• DVCORE-7910 (Improvement): Enable configuring WildFly and Prometheus JMX Exporter metrics host and port
• DVCORE-7786 (Improvement): JDBC driver: add support of OAuth2 authorization type
• DVCORE-6203 (Improvement): Enable "OPTIONS Annotation" on views and procedures
• DVCORE-7951 (Bug Fix): It is impossible to create a copyOverSQL job if the source object is missing despite the OPTION $NOFAIL
• DVCORE-7934 (Bug Fix): Escape sequence parse error on selecting from a TEXTTABLE if the escape character is followed by a non-escapable character
• DVCORE-7886 (Bug Fix): S3: UTILS.unzip procedure does not work for S3 data source

Connectors:

• SQL-514 (New Feature): Linnworks: create a connector
• SQL-669 (Improvement): Linnworks: create Pipes procedures to add new Linnworks tables
• SQL-666 (Improvement): Tiktok: update API version to v1.3
• SQL-649 (Improvement): Meltwater: address the breaking changes in the exports API
• SQL-632 (Improvement): Amazon Vendor: add support of GET_VENDOR_FORECASTING_REPORT reports
• SQL-629 (Improvement): Amazon Selling Partner: make Orders procedure to load the newest data during long runs
• SQL-619 (Improvement): Amazon Selling Partner: add GET_FLAT_FILE_SALES_TAX_DATA report
• SQL-618 (Improvement): Amazon Selling Partner: add rateLimit header to debug log
• SQL-611 (Improvement): Microsoft Dynamics: introduce saving data into subtables
• SQL-574 (Improvement): Amazon Vendor: add shipWindow columns from the API response
• SQL-566 (Improvement): Amazon Selling Partner: add GET_BRAND_ANALYTICS_SEARCH_TERMS_REPORT and GET_BRAND_ANALYTICS_REPEAT_PURCHASE_REPORT reports
• SQL-562 (Improvement): Amazon Selling Partner: add incremental options to procedures
• SQL-556 (Improvement): Amazon Vendor: add support of reportPeriod parameter
• SQL-520 (Improvement): Amazon Seller Partner: implement GET_SALES_AND_TRAFFIC_REPORT and Inventory Ledger reports
• SQL-651 (Bug Fix): Microsoft Dynamics: connector has no quick start mode, and incremental download ignores the filter parameter

• SQL-587 (Bug Fix): Amazon Selling Partner: Inventory Summaries report does not check the maximum start_date

• SQL-578 (Bug Fix): Amazon Selling Partner: Sales and Traffic and FBA fulfilled shipments reports operate incorrectly with multiple accessible marketplaces
• SQL-565 (Bug Fix): Amazon Selling Partner: Report_InventoryLedgerSummaryView skips the last day in a batch
• SQL-522 (Bug Fix): Tiktok: the BasicReport procedure returns an "HTTP 400 Bad Request" error
• SQL-511 (Bug Fix): Microsoft Dynamics: connector is incompatible with version 2.4 of the server
• SQL-18 (Bug Fix): Microsoft Dynamics: internal_Authenticate procedure fails to run and there is no documentation to get cloud version credentials