What i want:
A spline visualization.
What i have:
Three columns of random numbers. One of them with UTC time stamps in ms, uniformly distributed. I use a query processor to apply a left join and a ORDER BY time.
What i get:
- An error message from Highcharts: The x-Axis (time) is not ordered:
Highcharts Error
- A graph with an unordered a-axis (probably)
Remarks:
Highcharts time series example
Highcharts clearly recognizes the x-axis and interprets it as a time series. (see x axis labels)
Question: Why are the data points unordered?
Hello Jonas,
you order but you do it before saving. But you need to order on read. Because on read a lot of additions are executed by ONE DATA, which do not consider the e.g. the ordering of the stored data. In addition saving in hdfs will split files at some points in comparison to saving in postgres, due to its pure nature. You then have effects like ordering of buckets but the buckets not ordererd ( Example: 8, 9 ,10, 11, 1, 2, 3, 4, 5, 6, 7).
So my suggesion to try here is to order on read at the very last config happening before the plot gets created.
Thank You very much! It works.
Just a minor remark to what @johannesheininger wrote: For small amounts of rows (read: few thousands) it is possible to preserve the ordering even on Parquet and ORC files - you have to use the “Manipulate partitions” processor and set the number of partitions to 1. This only works if the data fits into a single file partition and is quite expensive to perform.
But as Johannes wrote, the better solution is to store to Postgres and apply the required sorting in Apps.
1 Like
