Incomplete values in csv import
AnsweredHi,
I'm trying to import a csv file via DV.
Additionally I'm trying to join a filepath column containing the respective filepath in each cell of the column.
As soon as I join the filepath column only the first character of imported csv columns are being displayed.
I've already tried to use other encodings but 'WINDOWS-1252' seems to be the right fit.
Expected behaviour: Four columns. No loss of data.
Actual behaviour: Four columns as expected. Loss of data in each imported csv column.
SELECT f.filePath,data.*
FROM
(CALL "some_server"."getFiles"("pathAndPattern" => '*.csv')) f,
TEXTTABLE(to_chars ("f.file", 'WINDOWS-1252')
COLUMNS
"Timestamp" string
,
"SomeID" string
,
"Another_ID" string
DELIMITER ' '
QUOTE ''
SKIP 1
) "data"
WHERE f.filePath LIKE '%adform/lightspeed%'
AND data."Timestamp" IS NOT NULL;;
Help will be much appreciated.
Best,
Emmanuel
-
Comment actionsOfficial comment
Hi Emmanuel,
would you be able to share one of those CSV files from the data source (you can shorten/anonymize the content)?
Also, it would be interesting to know if this is an issue with how the columns are displayed, or if the data is actually truncated. To verify this, could you please export the result of your query from above via the Studio by clicking on the "Export to CSV file" button? In the CSV file that gets generated, are the rows also truncated?
Thanks
Niklas
-
Hi Niklas,
after some investigation we've discovered
that the respective file is encoded in UCS2 (SMS-Encoding).
We have recoded the file to UTF-8.
So my problem is solved for now.
There's a similar request from my colleague. He shared a csv-file in UCS-Encoding (Post "UCS2-Kodierung von CSV-Dateien").
Thanks a lot!
Emmanuel
Please sign in to leave a comment.
Comments
2 comments