Skip to content

Improve clickhouse events table schema #5684

Description

@macobo

Is your feature request related to a problem?

  1. We can't update beyond 21.6 currently as our events table currently does SAMPLE BY uuid.
    This does not work on clickhouse >= 21.7 due to Hash functions should be checked at table creation, not at sampling time ClickHouse/ClickHouse#822 and Check hash function at table creation ClickHouse/ClickHouse#26256.
    We can't e.g. SAMPLE BY cityHash64(uuid) because the expression needs to be in the sort key and a number. Related: ClickHouse M1 Docker #5215
  2. We should include event column in the sort key since it's used in most of our queries

Both of these are tragically pretty heavy changes which would require re-reading the whole schema again, so it might make sense to bundle these up.

Proposed solution:

ORDER BY (team_id, event, toStartOfHour(timestamp), distinct_id, cityHash64(uuid))
SAMPLE BY cityHash64(uuid) 

Open questions:

  1. What's the difference in ORDER BY (..., timestamp, ...) and ORDER BY (..., toDate(timestamp), ...) etc? When to prefer one to the other?
  2. How to handle this migration on cloud?
  3. How to handle this migration on self-hosted?

cc @fuziontech @tiina303 @yakkomajuri would you mind handling this as #team-platform?

Metadata

Metadata

Assignees

Labels

P0Critical, breaking issue (page crash, missing functionality)enhancementNew feature or requestperformanceHas to do with performance. For PRs, runs the clickhouse query performance suite

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions