LDWH 2.0.54 released: Redshift S3 Load via IAM Role

In version 2.0.54 we added another improvement for our Redshift connector. For ingesting data into Redshift, it is recommended to use the so-called S3 Load mechanism, which writes the to-be-ingested data to CSV files on an S3 bucket. The authentication for writing to/reading from the S3 bucket so far required an IAM User with an Access Keys ID/Secret Access Key. In case your company has a policy to regularly rotate credentials, this requires you to re-generate those credentials and update them in Data Virtuality.

From this version on, you can use an IAM Role instead. The IAM Role automatically generates short-lived credentials that no longer need to be rotated. You can read more about the setup of different S3 Load strategies in our dedicated post.

Studio

• DVCORE-5450: 'Kill session' in Query Logs window does nothing

Connectors

• DVCORE-5494 (Improvement): Redshift: Implement push down of VAR_POP, VAR_SAMP, STDDEV_POP, STDDEV_SAMP window functions
• DVCORE-5358 (Improvement): Redshift: Enable S3 Load to use IAM Role authorization
• DVCORE-5905: PostgreSQL CHARACTER VARYING column causes an error on querying after materialization to Redshift and server restart
• DVCORE-5449: Data type 'string' becomes 'Clob' after PostgreSQL data source refresh
• DVCORE-5631: Creation of a searched CASE expression index for a PostgreSQL table fails

Backend

• DVCORE-5198 (Improvement): CopyOverSourceTable job alters the target table even if the structure has not changed
• DVCORE-4015 (Improvement): Add retryCounter, retryDelay and runTimeout options to copyOver jobs
• DVCORE-5088: Data Virtuality REST API: Very small double/float values produce invalid JSON
• DVCORE-5454: Data Virtuality REST API: Calls to procedure without return parameters are executed but yield an error
• DVCORE-5629: Data Virtuality REST API: The server always assumes pagination is used and so connections might be blocked unnecessarily
• DVCORE-5932: Some unnecessary queries are sent to configuration database
• DVCORE-5919: Newly created chained jobs or schedules should not consider outdated executions of base jobs
• DVCORE-5910: JOIN of a table and sub-query containing DISTINCT and a table function of a procedural call with arguments fails
• DVCORE-5571: Dynamic SQL: Executing correct dynamic SQL statements throws an error
• DVCORE-5554: Query Planner throws an NPE on complex queries with procedure calls in the SELECT clause
• DVCORE-5515: Recursive CTE: Rewriting error for multiple sources
• DVCORE-5900: Unable to refresh data source metadata if the data source was unavailable on server start up