Data Virtuality Platform 4.3: many improvements and bug fixes

This release includes many improvements and bug fixes and introduces the GraphQL connector, as a preview feature - your feedback is most appreciated!

We've improved the Server's performance with caching of license (please note that if you manually copy the license file to the "dvserver/standalone/data/datavirtuality" folder, the license will be updated in 5 minutes rather than immediately). We've also significantly improved the Cloud Agent which now has a master management WebSocket connection, a procedure to check if an agent is connected, an improved logging. Also, we've split the Cloud Agent's configuration into several files and introduced a configuration directory for them.

We've also updated the Google Ads API to v15 which means the removal of the "product_bidding_category_constant" table and the addition of a number of tables: "android_privacy_shared_key_google_ad_group", "android_privacy_shared_key_google_campaign", "android_privacy_shared_key_google_network_type", "asset_group_top_combination_view", "campaign_lifecycle_goal", "customer_lifecycle_goal", "local_services_lead", "local_services_lead_conversation", "local_services_verification_artifact", "offline_conversion_upload_client_summary", "product_category_constant", "product_link_invitation" and "recommendation_subscription". Also, some columns (e.g. "customer_offline_conversion_client_summaries", "campaign_shopping_setting_sales_country") were removed from tables, and some new columns were added - for a full list of removed and added objects, please get in touch with our support team.

As for bug fixes, we've worked particularly extensively on Git Integration where we've fixed a bug preventing the "allowUserLogin" parameter for users from being tracked in Git, another bug causing the system role dv-developer-role to be unnecessarily tracked, and a bug with the procedure gitExclude which did not accept the "-" symbol. Now all is well. 

We've also worked on our Apache Parquet connector and fixed several bugs: one resulting in inability to create a data source and read tables with certain data types, another preventing the connector from working properly with connectors other than local files, yet another causing the "writeSingleFile" property to only work for the first table being created, and last but not least, a bug leading to excessive logging when selecting from a table after several inserts.

Among other news, the Azure Synapse connector now supports authentication via Azure Active Directory, and the hasRole function now returns correct result regarding membership in 'odata-role'.

In the Studio, we've improved the visibility of the "Apply" button in CSV, XML, and JSON wizards, added preference option for SQL editor window size, and fixed a bug causing the "AccessTokenEndpoint" parameter of a web service data source set via script to get overridden by the wizard. Now everything is working properly.

As for the connectors, we've worked on the Exact Online connector to make refresh token exchange expiration-aware and protect against racing issues and to fix two bugs: one causing an 'Encountered "'Description of Opportunity Type' error when creating a data source and another where an expired Refresh toket did not allow to use the token stored in properites. We've also fixed a nasty bug causing the Freshdesk connector to fail on installation and various minor bugs affecting the Walmart, Exactag, Matomo, and Linnworks connectors.

Here are all issues in this release:

Server

• DVCORE-8092 (New Feature): GraphQL: implement a connector (preview)

• DVCORE-8319 (Improvement): Update Data Virtuality Server configuration to allow 25 concurrent queries

• DVCORE-8289 (Improvement): Add caching for the Data Virtuality Server license

• DVCORE-8286 (Improvement): Azure Synapse: add support for authentication via Azure Active Directory

• DVCORE-8279 (Improvement): Cloud Agent: add a master management WebSocket connection
• DVCORE-8277 (Improvement): Cloud Agent: provide a procedure to check if an agent is connected

• DVCORE-8276 (Improvement): Cloud Agent: split configuration into several files and introduce a configuration directory for them

• DVCORE-8275 (Improvement): Cloud Agent: improve logging

• DVCORE-8268 (Improvement): Google Ads API: update to v15

• DVCORE-8304 (Bug Fix): MAINTENANCE mode: recommended optimizations get corrupted after the server restart when MAINTENANCE mode is enabled

• DVCORE-8290 (Bug Fix): Inner BEGIN END block is removed when IF statement with a boolean literal as a condition is used

• DVCORE-8285 (Bug Fix): Retrieving data from system tables related to jobs and schedules is slow

• DVCORE-8256 (Bug Fix): Running "createOrReplaceDatasource" procedure followed by a server restart causes the data source to fail

• DVCORE-8245 (Bug Fix): Git Integration: "allowUserLogin" parameter for users is not tracked in Git

• DVCORE-8242 (Bug Fix): Git integration: system role dv-developer-role is tracked by git

• DVCORE-8206 (Bug Fix): Git integration: the procedure gitExclude does not accept the "-" symbol

• DVCORE-8054 (Bug Fix): Apache Parquet: unable to create a data source and read tables with certain data types

• DVCORE-8052 (Bug Fix): Apache Parquet: connectors other than ufile are not functional

• DVCORE-8050 (Bug Fix): Apache Parquet: the "writeSingleFile" property only works for the first table being created

• DVCORE-8049 (Bug Fix): Apache Parquet: excessive logging when selecting from a table after several inserts

• DVCORE-7962 (Bug Fix): The result of HASROLE function for "odata-role" is always FALSE
• DVCORE-7543 (Bug Fix): Excel: columns with empty cells in header row are missed

• DVCORE-7398 (Bug Fix): Incorrect behavior for certain combinations of "chainedToScheduleId", "chainCondition" and "chainString" values in SYSADMIN.updateSchedule procedure

• DVCORE-7357 (Bug Fix): Dropping and recreating a table in the same procedure results in an error

• DVCORE-6216 (Bug Fix): MAT_JOIN recommended optimization is not used when JOIN predicate is ON TRUE

Studio

• DVCORE-8295 (Improvement): Improve the visibility of the "Apply" button in CSV, XML, and JSON wizards

• DVCORE-8282 (Improvement): Add preference option for SQL editor window size

• DVCORE-8108 (Bug Fix): "AccessTokenEndpoint" parameter of a web service data source set via script gets overridden by the wizard

Connectors

• SQL-821 (Improvement): Google Ads API: ensure compatibility with v15

• SQL-507 (Improvement): Exact Online: make refresh token exchange expiration-aware and protect against racing issues

• SQL-834 (Bug Fix): Walmart: "Report_Promotions" procedure throws a conversion error
• SQL-831 (Bug Fix): Exactag: Conversions enddate value is always ignored

• SQL-829 (Bug Fix): Exact Online: an 'Encountered "'Description of Opportunity Type' error when creating a data source

• SQL-828 (Bug Fix): Matomo: "SitesManager_AllSites" procedure fails with an "Element not defined" error

• SQL-827 (Bug Fix): Linnworks: Orders_GetOpenOrders prodecure does not specify fulfilment center

• SQL-444 (Bug Fix): Freshdesk: connector fails on installation

• SQL-20 (Bug Fix): Exact Online: an expired Refresh toket does not allow to use the token stored in properites