Timestamp Extraction Processor

ONE DATA Core
1

Dear all :slight_smile:

could someone tell me which function is behind ONE DATA’s “Timestamp Extraction” Processor? I’ve run into the problem that the SQL function “date_format()” returns calendar-week 1 for the week that is half in 2020 and half in 2021 and at the same time our “Timestamp Extraction” Processor returns calendar-week 53.

2

It’s not a spark SQL function but a custom implementation based on ZonedDateTime from java.

1 Like
3

Thanks Peter!
And just for reference: Spark’s weekofyear(), month(), and year() seem to give the same output as our “Timestamp Extraction” processor and both seem to follow ISO 8601 :slight_smile:
For Postgres extract(week from timestamp_column), extract(month from timestamp_column), and extract(year from timestamp_column) work the same.