Skip to content

Add a scanner error check to ClusterCounter.Count

David Fifield requested to merge dcf/snowflake:clustercounter-scan-err into main

It was silently exiting at the "recordingStart":"2022-09-23T17:06:59.680537075Z" line, the first line whose length (66873) exceeds bufio.MaxScanTokenSize. Now distinctcounter exits with an error status instead of reporting partial results.

$ ./distinctcounter -from 2023-01-01T00:00:00Z -to 2023-01-10T00:00:00Z -in metrics-ip-salted.jsonl
2023/04/20 13:54:11 unable to count:bufio.Scanner: token too long

Merge request reports