could someone tell me which function is behind ONE DATA’s “Timestamp Extraction” Processor? I’ve run into the problem that the SQL function “date_format()” returns calendar-week 1 for the week that is half in 2020 and half in 2021 and at the same time our “Timestamp Extraction” Processor returns calendar-week 53.
It’s not a spark SQL function but a custom implementation based on
ZonedDateTime from java.
And just for reference: Spark’s weekofyear(), month(), and year() seem to give the same output as our “Timestamp Extraction” processor and both seem to follow ISO 8601
For Postgres extract(week from timestamp_column), extract(month from timestamp_column), and extract(year from timestamp_column) work the same.