LDW 2.1.26 released: S3/Blob Storage prefix, configurable column casing

AWS S3/Azure Blob Storage Prefix

The last release in 2019 has a number of connector improvements, most notably, we extended both the AWS S3 and Azure Blob Storage connectors to support exposing only a subdirectory of the bucket/storage. The new prefix parameter can be set to the name of a subdirectory which will then become the root directory of the data source, preventing parent directories from being accessible via the data source.

Configurable column casing

When creating new tables in the analytical storage or other data sources, column names are automatically converted into lower case letters for compatibility reasons. This behavior can now be changed by setting the columnNameType translator property on the analytical storage/target data source. It can be set to always use lower case letters, upper case letters or preserving the original casing.

System schemas in Studio

The Studio also received a small usability improvement by reorganizing system schemas. Instead of showing them as part of the data sources, system schemas will now be grouped separately to improve clarity.

Studio

• DVCORE-6728 (Improvement): Group system schemas in a separate folder
• DVCORE-6727 (Improvement): Reorganize data sources in 'Add data source' wizard
• DVCORE-6721 (Improvement): Sort the list of data sources in the create Copy Over job wizard alphabetically
• DVCORE-6718 (Improvement): Update the license terms
• DVCORE-6717 (Improvement): Update the license expiration warning
• DVCORE-6729 (Improvement): Add a hint to the error message when trying to connect to a server that is still starting up
• DVCORE-6719: Tooltips in Linux version are unreadable
• DVCORE-5170: For lower resolution settings, windows get out of the scope

Connectors

• DVCORE-6705 (Improvement) BigQuery: Introduce a property to replace a CHAR(0) character when inserting
• DVCORE-6713 BigQuery: Upload job timeout causes errors when replicating large tables
• DVCORE-6703 (Improvement) NetSuite: Remove the mandatory 'Ciphersuites' property from the CLI template
• DVCORE-6735 (Improvement) Salesforce: Add a property disabling the usage of fully qualified column names in projections of rewritten queries
• DVCORE-6618 (Improvement) Azure Blob Storage: Allow specifying a prefix to limit result sets
• DVCORE-6632 Azure Blob Storage: Poor performance when listing a limited number of files from a heavily populated storage
• DVCORE-6609 (Improvement) S3 connector: Allow specifying a prefix to limit result sets
• DVCORE-3006 (Improvement) Vertica: Allow usage of CLOBs and BLOBs for Vertica using its LONG VARCHAR and LONG VARBINARY data types
• DVCORE-6730 Vertica: WITH queries are not pushed down and return incorrect results
• DVCORE-6697 Vertica: Incorrect data types mapping and function pushdown
• DVCORE-3473 Vertica: Querying data from BigDecimal columns of PostgreSQL or Oracle tables fails after materialization to Vertica and refreshing analytical storage
• DVCORE-3462 Vertica: Materialization of PostgreSQL tables with columns of types REAL and NAME fails and causes errors on refreshing analytical storage
• DVCORE-6639 kdb+: Subqueries fail if used with IN operator

Backend

• DVCORE-6694 (Improvement): Allow reading dynamic connector properties in the same session, when they were written
• DVCORE-6601 (Improvement): Create a translator property to control column names case
• DVCORE-6739: Old materialized table is picked instead of the most recent
• DVCORE-6704: UPDATE fails silently when the value of the field being updated is NULL