LDW 2.3.20: ODBC improvements, TLSv1.2, incremental replication

This release addresses a number of issues around the usage of ODBC for connecting to Data Virtuality Server (such as Power BI). We reworked the cache that holds metadata for ODBC connections to better handle concurrent access and different permissions depending on the user. It improves performance and fixes scenarios like two users with different permissions requesting metadata via ODBC at the same time. There are still pending improvements when using ODBC with LDAP authentication and SSL which will be addressed in one of the next minor releases.

On the connector side, we extended the BigQuery connector to support UPDATE statements. Together with previous improvements, the connector is now capable of handling DELETE, INSERT and UPDATE statements. Please note that we still do not formally support BigQuery as an Analytical Storage yet, but it can be used as a data source with the capability to replicate/modify data.

For all connectors that are using JDBC drivers (PostgreSQL, MS SQL, etc) we added TLSv1.2 as an available encryption protocol.

In the Studio, a new type of replication jobs is available: Incremental replication. It works similarly to incremental materializations and can use e.g. a timestamp column to determine the last replicated state and query the data source only for rows with a greater value (further explanations are available as tooltips in the dialogue).

Studio

• DVCORE-6350 (Improvement): Add a wizard for creating incremental replication jobs
• DVCORE-7206: Refreshing a filtered job list fails with IndexOutOfBoundsException
• DVCORE-6870: Commas get removed from password when editing a data source

Connectors

• SQL-294 (Improvement): Amazon MWS: Add Flat File Order Report procedure
• SQL-279 (Improvement): Amazon Advertising: Support dimensions for reports
• SQL-332 SharePoint: Loading data returns only 100 items and does not support pagination
• DVCORE-6925 PostgreSQL: SUM and AVG functions fail when used on NULL literal cast to any data type
• DVCORE-7204 BigQuery: The STRING_AGG function is not pushed down
• DVCORE-6620 (Improvement) BigQuery: Add pushdown for UPDATE statements
• DVCORE-7196 Snowflake: INSERT INTO SELECT of BLOB column from Microsoft SQL Server fails on empty binary values
• DVCORE-7207 Snowflake: FIRST_VALUE, LAST_VALUE, LEAD and LAG window functions are not pushed down
• DVCORE-6573 SQL Server: TIMESTAMPDIFF function returns NULL if the result is too big for INTEGER data type
• DVCORE-7199 (Improvement) JDBC data sources: Add support for TLSv1.2

Backend

• DVCORE-7136: Virtual Databases: Recommended optimization does not get a proper state after refreshing Analytical Storage if it is performed after creating a secondary VDB and restarting the server
• DVCORE-7134: Virtual Databases: wrong error message when creating a virtual database with a duplicate name
• DVCORE-7221: Schedule creation might sporadically fail with a Hibernate exception
• DVCORE-7165: LEFT JOIN produces incorrect result if not pushed down and ON clause contains a filtering condition
• DVCORE-7110: HTML line breaks are not presented correctly by UTILS.sendmail if attachments are present
• DVCORE-7135: ErrorHandler: incorrect error message is returned on casting of too big value to INTEGER
• DVCORE-7073: Metadata is not loaded correctly via ODBC