feat: index daily_stations by day

[bajtos]

Speed up spark-stats queries, they always filter by a date range.

Before - 7.2 seconds:

 ->  Parallel Seq Scan on daily_stations  (cost=0.00..899791.13 rows=2746355 width=144) (actual time=273.312..7516.967 rows=2091389 loops=3)
       Filter: (day = date((now() - '1 day'::interval)))
       Rows Removed by Filter: 8091296

After - 0.8 seconds:

 ->  Parallel Index Scan using daily_stations_day on daily_stations  (cost=0.45..283087.78 rows=2746355 width=144) (actual time=143.526..988.537 rows=2091389 loops=3)
       Index Cond: (day = date((now() - '1 day'::interval)))

I tested various combinations of indexed and included columns. The simplest option - index on the date value only - is performing best.

Links:

• feat: add endpoint for participants with highest measurement count spark-stats#164

[juliangruber]

👏 massive speed up!

[bajtos]

👏 massive speed up!

Thank you 😊

I think it's good to put this into context - the entire query takes more than 30 seconds to complete; this index improves the time from 36s down to 30s.

[bajtos]

Oops, I should have added CONCURRENTLY to let Postgres build the index in the background and don't block the startup of the spark-evaluate service 🙈

[bajtos]

The index is live, the migration finished in ~29 second 😅

2024-07-03T12:22:23Z app[e2867541be3e68] cdg [info]Migrating DB schema from version 10 to version 11
2024-07-03T12:22:52Z app[e2867541be3e68] cdg [info]Migrated DB schema to version 11