gharchive-traps
过去的一个半月花了很多时间在gharchive（GitHub Archive）这个数据到clickhouse的同步上。最后就在记录一些gharchive数据问题。
数据文件的下载方式就在这里不具体说明了，源网站都有。另外建议分开做下载和解析导入，因为下面坑真的很多一步没做好程序就寄了。
先简单说明，gharchive：
- 活动档案自 2011 年 2 月 12 日起提供。
- 2011 年 2 月 12 日至 2014 年 12 月 31 日之间的活动档案是通过（现已弃用）时间线 API 记录的。
- 从 2015 年 1 月 1 日开始的活动档案是通过事件 API 记录的。
因此这里的坑也分成Timeline和Event两部分（数据库也是
Timeline
2012-2014
因为表结构为数据定制，因此皆无异常
Events
2015
无异常
2016
无文件
2017
无异常
2018
无文件
- http://data.gharchive.org/2018-10-21-23.json.gz
- http://data.gharchive.org/2018-10-22-0.json.gz
- http://data.gharchive.org/2018-10-22-1.json.gz
2019
无文件
- https://data.gharchive.org/2019-05-08-12.json.gz
- https://data.gharchive.org/2019-05-08-13.json.gz
- https://data.gharchive.org/2019-05-08-12.json.gz
- https://data.gharchive.org/2019-05-08-13.json.gz
- https://data.gharchive.org/2019-09-12-8.json.gz
- https://data.gharchive.org/2019-09-12-9.json.gz
- https://data.gharchive.org/2019-09-12-10.json.gz
- https://data.gharchive.org/2019-09-12-11.json.gz
- https://data.gharchive.org/2019-09-12-12.json.gz
- https://data.gharchive.org/2019-09-12-13.json.gz
- https://data.gharchive.org/2019-09-12-14.json.gz
- https://data.gharchive.org/2019-09-12-15.json.gz
- https://data.gharchive.org/2019-09-12-16.json.gz
- https://data.gharchive.org/2019-09-12-17.json.gz
- https://data.gharchive.org/2019-09-12-18.json.gz
- https://data.gharchive.org/2019-09-12-19.json.gz
- https://data.gharchive.org/2019-09-12-20.json.gz
- https://data.gharchive.org/2019-09-12-21.json.gz
- https://data.gharchive.org/2019-09-12-22.json.gz
- https://data.gharchive.org/2019-09-12-23.json.gz
- https://data.gharchive.org/2019-09-13-0.json.gz
- https://data.gharchive.org/2019-09-13-1.json.gz
- https://data.gharchive.org/2019-09-13-2.json.gz
- https://data.gharchive.org/2019-09-13-3.json.gz
- https://data.gharchive.org/2019-09-13-4.json.gz
- https://data.gharchive.org/2019-09-13-5.json.gz
2020
无文件
- https://data.gharchive.org/2020-03-05-22.json.gz
- https://data.gharchive.org/2020-06-10-12.json.gz
- https://data.gharchive.org/2020-06-10-13.json.gz
- https://data.gharchive.org/2020-06-10-14.json.gz
- https://data.gharchive.org/2020-06-10-15.json.gz
- https://data.gharchive.org/2020-06-10-16.json.gz
- https://data.gharchive.org/2020-06-10-17.json.gz
- https://data.gharchive.org/2020-06-10-18.json.gz
- https://data.gharchive.org/2020-06-10-19.json.gz
- https://data.gharchive.org/2020-06-10-20.json.gz
- https://data.gharchive.org/2020-06-10-21.json.gz
- https://data.gharchive.org/2020-08-21-9.json.gz
- https://data.gharchive.org/2020-08-21-10.json.gz
- https://data.gharchive.org/2020-08-21-11.json.gz
- https://data.gharchive.org/2020-08-21-12.json.gz
- https://data.gharchive.org/2020-08-21-13.json.gz
- https://data.gharchive.org/2020-08-21-14.json.gz
- https://data.gharchive.org/2020-08-21-15.json.gz
- https://data.gharchive.org/2020-08-21-16.json.gz
- https://data.gharchive.org/2020-08-21-17.json.gz
- https://data.gharchive.org/2020-08-21-18.json.gz
- https://data.gharchive.org/2020-08-21-19.json.gz
- https://data.gharchive.org/2020-08-21-20.json.gz
- https://data.gharchive.org/2020-08-21-21.json.gz
- https://data.gharchive.org/2020-08-21-22.json.gz
- https://data.gharchive.org/2020-08-21-23.json.gz
- https://data.gharchive.org/2020-08-22-0.json.gz
- https://data.gharchive.org/2020-08-22-1.json.gz
- https://data.gharchive.org/2020-08-22-2.json.gz
- https://data.gharchive.org/2020-08-22-3.json.gz
- https://data.gharchive.org/2020-08-22-4.json.gz
- https://data.gharchive.org/2020-08-22-5.json.gz
- https://data.gharchive.org/2020-08-22-6.json.gz
- https://data.gharchive.org/2020-08-22-7.json.gz
- https://data.gharchive.org/2020-08-22-8.json.gz
- https://data.gharchive.org/2020-08-22-9.json.gz
- https://data.gharchive.org/2020-08-22-10.json.gz
- https://data.gharchive.org/2020-08-22-11.json.gz
- https://data.gharchive.org/2020-08-22-12.json.gz
- https://data.gharchive.org/2020-08-22-13.json.gz
- https://data.gharchive.org/2020-08-22-14.json.gz
- https://data.gharchive.org/2020-08-22-15.json.gz
- https://data.gharchive.org/2020-08-22-16.json.gz
- https://data.gharchive.org/2020-08-22-17.json.gz
- https://data.gharchive.org/2020-08-22-18.json.gz
- https://data.gharchive.org/2020-08-22-19.json.gz
- https://data.gharchive.org/2020-08-22-20.json.gz
- https://data.gharchive.org/2020-08-22-21.json.gz
- https://data.gharchive.org/2020-08-22-22.json.gz
- https://data.gharchive.org/2020-08-22-23.json.gz
- https://data.gharchive.org/2020-08-23-0.json.gz
- https://data.gharchive.org/2020-08-23-1.json.gz
- https://data.gharchive.org/2020-08-23-2.json.gz
- https://data.gharchive.org/2020-08-23-3.json.gz
- https://data.gharchive.org/2020-08-23-4.json.gz
- https://data.gharchive.org/2020-08-23-5.json.gz
- https://data.gharchive.org/2020-08-23-6.json.gz
- https://data.gharchive.org/2020-08-23-7.json.gz
- https://data.gharchive.org/2020-08-23-8.json.gz
- https://data.gharchive.org/2020-08-23-9.json.gz
- https://data.gharchive.org/2020-08-23-10.json.gz
- https://data.gharchive.org/2020-08-23-11.json.gz
- https://data.gharchive.org/2020-08-23-12.json.gz
- https://data.gharchive.org/2020-08-23-13.json.gz
- https://data.gharchive.org/2020-08-23-14.json.gz
- https://data.gharchive.org/2020-08-23-15.json.gz
内容错乱
- https://data.gharchive.org/2020-10-30-17.json.gz md5sum 71d21fc7d9feed6c479d817360c2327d
文件最后缺失"site_admin":false},"repo":nul这个null都没打完，修改为,"repo":null}}},"public":true,"created_at":"2020-10-30T18:00:20Z"}完全闭合并补充时间信息（根据comment的updated时间），另外查的这个人当时没org信息因此不补充
2021
无文件非常多：
- https://data.gharchive.org/2021-08-25-17.json.gz
- https://data.gharchive.org/2021-08-25-18.json.gz
- https://data.gharchive.org/2021-08-25-19.json.gz
- https://data.gharchive.org/2021-08-25-20.json.gz
- https://data.gharchive.org/2021-08-25-21.json.gz
- https://data.gharchive.org/2021-08-25-22.json.gz
- https://data.gharchive.org/2021-08-25-23.json.gz
- https://data.gharchive.org/2021-08-26-0.json.gz
- https://data.gharchive.org/2021-08-26-1.json.gz
- https://data.gharchive.org/2021-08-26-2.json.gz
- https://data.gharchive.org/2021-08-26-3.json.gz
- https://data.gharchive.org/2021-08-26-4.json.gz
- https://data.gharchive.org/2021-08-26-5.json.gz
- https://data.gharchive.org/2021-08-26-6.json.gz
- https://data.gharchive.org/2021-08-26-7.json.gz
- https://data.gharchive.org/2021-08-26-8.json.gz
- https://data.gharchive.org/2021-08-26-9.json.gz
- https://data.gharchive.org/2021-08-26-10.json.gz
- https://data.gharchive.org/2021-08-26-11.json.gz
- https://data.gharchive.org/2021-08-26-12.json.gz
- https://data.gharchive.org/2021-08-26-13.json.gz
- https://data.gharchive.org/2021-08-26-14.json.gz
- https://data.gharchive.org/2021-08-26-15.json.gz
- https://data.gharchive.org/2021-08-26-16.json.gz
- https://data.gharchive.org/2021-08-26-17.json.gz
- https://data.gharchive.org/2021-08-26-18.json.gz
- https://data.gharchive.org/2021-08-26-19.json.gz
- https://data.gharchive.org/2021-08-26-20.json.gz
- https://data.gharchive.org/2021-08-26-21.json.gz
- https://data.gharchive.org/2021-08-26-22.json.gz
- https://data.gharchive.org/2021-08-26-23.json.gz
- https://data.gharchive.org/2021-08-27-0.json.gz
- https://data.gharchive.org/2021-08-27-1.json.gz
- https://data.gharchive.org/2021-08-27-2.json.gz
- https://data.gharchive.org/2021-08-27-3.json.gz
- https://data.gharchive.org/2021-08-27-4.json.gz
- https://data.gharchive.org/2021-08-27-5.json.gz
- https://data.gharchive.org/2021-08-27-6.json.gz
- https://data.gharchive.org/2021-08-27-7.json.gz
- https://data.gharchive.org/2021-08-27-8.json.gz
- https://data.gharchive.org/2021-08-27-9.json.gz
- https://data.gharchive.org/2021-08-27-10.json.gz
- https://data.gharchive.org/2021-08-27-11.json.gz
- https://data.gharchive.org/2021-08-27-12.json.gz
- https://data.gharchive.org/2021-08-27-13.json.gz
- https://data.gharchive.org/2021-08-27-14.json.gz
- https://data.gharchive.org/2021-08-27-15.json.gz
- https://data.gharchive.org/2021-08-27-16.json.gz
- https://data.gharchive.org/2021-08-27-17.json.gz
- https://data.gharchive.org/2021-08-27-18.json.gz
- https://data.gharchive.org/2021-08-27-19.json.gz
- https://data.gharchive.org/2021-08-27-20.json.gz
- https://data.gharchive.org/2021-08-27-21.json.gz
- https://data.gharchive.org/2021-08-27-22.json.gz
- https://data.gharchive.org/2021-10-22-5.json.gz
- https://data.gharchive.org/2021-10-22-6.json.gz
- https://data.gharchive.org/2021-10-22-7.json.gz
- https://data.gharchive.org/2021-10-22-8.json.gz
- https://data.gharchive.org/2021-10-22-9.json.gz
- https://data.gharchive.org/2021-10-22-10.json.gz
- https://data.gharchive.org/2021-10-22-11.json.gz
- https://data.gharchive.org/2021-10-22-12.json.gz
- https://data.gharchive.org/2021-10-22-13.json.gz
- https://data.gharchive.org/2021-10-22-14.json.gz
- https://data.gharchive.org/2021-10-22-15.json.gz
- https://data.gharchive.org/2021-10-22-16.json.gz
- https://data.gharchive.org/2021-10-22-17.json.gz
- https://data.gharchive.org/2021-10-22-18.json.gz
- https://data.gharchive.org/2021-10-22-19.json.gz
- https://data.gharchive.org/2021-10-22-20.json.gz
- https://data.gharchive.org/2021-10-22-21.json.gz
- https://data.gharchive.org/2021-10-22-22.json.gz
- https://data.gharchive.org/2021-10-23-2.json.gz
- https://data.gharchive.org/2021-10-23-3.json.gz
- https://data.gharchive.org/2021-10-23-4.json.gz
- https://data.gharchive.org/2021-10-23-5.json.gz
- https://data.gharchive.org/2021-10-23-6.json.gz
- https://data.gharchive.org/2021-10-23-7.json.gz
- https://data.gharchive.org/2021-10-23-8.json.gz
- https://data.gharchive.org/2021-10-23-9.json.gz
- https://data.gharchive.org/2021-10-23-10.json.gz
- https://data.gharchive.org/2021-10-23-11.json.gz
- https://data.gharchive.org/2021-10-23-12.json.gz
- https://data.gharchive.org/2021-10-23-13.json.gz
- https://data.gharchive.org/2021-10-23-14.json.gz
- https://data.gharchive.org/2021-10-23-15.json.gz
- https://data.gharchive.org/2021-10-23-16.json.gz
- https://data.gharchive.org/2021-10-23-17.json.gz
- https://data.gharchive.org/2021-10-23-18.json.gz
- https://data.gharchive.org/2021-10-23-19.json.gz
- https://data.gharchive.org/2021-10-23-20.json.gz
- https://data.gharchive.org/2021-10-23-21.json.gz
- https://data.gharchive.org/2021-10-23-22.json.gz
- https://data.gharchive.org/2021-10-24-3.json.gz
- https://data.gharchive.org/2021-10-24-4.json.gz
- https://data.gharchive.org/2021-10-24-5.json.gz
- https://data.gharchive.org/2021-10-24-6.json.gz
- https://data.gharchive.org/2021-10-24-7.json.gz
- https://data.gharchive.org/2021-10-24-8.json.gz
- https://data.gharchive.org/2021-10-24-9.json.gz
- https://data.gharchive.org/2021-10-24-10.json.gz
- https://data.gharchive.org/2021-10-24-11.json.gz
- https://data.gharchive.org/2021-10-24-12.json.gz
- https://data.gharchive.org/2021-10-24-13.json.gz
- https://data.gharchive.org/2021-10-24-14.json.gz
- https://data.gharchive.org/2021-10-24-15.json.gz
- https://data.gharchive.org/2021-10-24-16.json.gz
- https://data.gharchive.org/2021-10-24-17.json.gz
- https://data.gharchive.org/2021-10-24-18.json.gz
- https://data.gharchive.org/2021-10-24-19.json.gz
- https://data.gharchive.org/2021-10-24-20.json.gz
- https://data.gharchive.org/2021-10-24-21.json.gz
- https://data.gharchive.org/2021-10-24-22.json.gz
- https://data.gharchive.org/2021-10-25-1.json.gz
- https://data.gharchive.org/2021-10-25-2.json.gz
- https://data.gharchive.org/2021-10-25-3.json.gz
- https://data.gharchive.org/2021-10-25-4.json.gz
- https://data.gharchive.org/2021-10-25-5.json.gz
- https://data.gharchive.org/2021-10-25-6.json.gz
- https://data.gharchive.org/2021-10-25-7.json.gz
- https://data.gharchive.org/2021-10-25-8.json.gz
- https://data.gharchive.org/2021-10-25-9.json.gz
- https://data.gharchive.org/2021-10-25-10.json.gz
- https://data.gharchive.org/2021-10-25-11.json.gz
- https://data.gharchive.org/2021-10-25-12.json.gz
- https://data.gharchive.org/2021-10-25-13.json.gz
- https://data.gharchive.org/2021-10-25-14.json.gz
- https://data.gharchive.org/2021-10-25-15.json.gz
- https://data.gharchive.org/2021-10-25-16.json.gz
- https://data.gharchive.org/2021-10-25-17.json.gz
- https://data.gharchive.org/2021-10-25-18.json.gz
- https://data.gharchive.org/2021-10-25-19.json.gz
- https://data.gharchive.org/2021-10-25-20.json.gz
- https://data.gharchive.org/2021-10-25-21.json.gz
- https://data.gharchive.org/2021-10-25-22.json.gz
- https://data.gharchive.org/2021-10-26-0.json.gz
- https://data.gharchive.org/2021-10-26-1.json.gz
- https://data.gharchive.org/2021-10-26-2.json.gz
- https://data.gharchive.org/2021-10-26-3.json.gz
- https://data.gharchive.org/2021-10-26-4.json.gz
- https://data.gharchive.org/2021-10-26-5.json.gz
- https://data.gharchive.org/2021-10-26-6.json.gz
- https://data.gharchive.org/2021-10-26-7.json.gz
- https://data.gharchive.org/2021-10-26-8.json.gz
- https://data.gharchive.org/2021-10-26-9.json.gz
- https://data.gharchive.org/2021-10-26-10.json.gz
- https://data.gharchive.org/2021-10-26-11.json.gz
- https://data.gharchive.org/2021-10-26-12.json.gz
- https://data.gharchive.org/2021-10-26-13.json.gz
- https://data.gharchive.org/2021-10-26-14.json.gz
- https://data.gharchive.org/2021-10-26-15.json.gz
- https://data.gharchive.org/2021-10-26-16.json.gz
- https://data.gharchive.org/2021-10-26-17.json.gz
- https://data.gharchive.org/2021-10-26-18.json.gz
- https://data.gharchive.org/2021-10-26-19.json.gz
- https://data.gharchive.org/2021-10-26-20.json.gz
- https://data.gharchive.org/2021-10-26-21.json.gz
- https://data.gharchive.org/2021-10-26-22.json.gz
- https://data.gharchive.org/2021-10-26-23.json.gz
- https://data.gharchive.org/2021-10-27-0.json.gz
- https://data.gharchive.org/2021-10-27-1.json.gz
- https://data.gharchive.org/2021-10-27-2.json.gz
- https://data.gharchive.org/2021-10-27-3.json.gz
- https://data.gharchive.org/2021-10-27-4.json.gz
- https://data.gharchive.org/2021-10-27-5.json.gz
- https://data.gharchive.org/2021-10-27-6.json.gz
- https://data.gharchive.org/2021-10-27-7.json.gz
- https://data.gharchive.org/2021-10-27-8.json.gz
- https://data.gharchive.org/2021-10-27-9.json.gz
- https://data.gharchive.org/2021-10-27-10.json.gz
- https://data.gharchive.org/2021-10-27-11.json.gz
- https://data.gharchive.org/2021-10-27-12.json.gz
- https://data.gharchive.org/2021-10-27-13.json.gz
- https://data.gharchive.org/2021-10-27-14.json.gz
- https://data.gharchive.org/2021-10-27-15.json.gz
- https://data.gharchive.org/2021-10-27-16.json.gz
- https://data.gharchive.org/2021-10-27-17.json.gz
- https://data.gharchive.org/2021-10-27-18.json.gz
- https://data.gharchive.org/2021-10-27-19.json.gz
- https://data.gharchive.org/2021-10-27-20.json.gz
- https://data.gharchive.org/2021-10-27-21.json.gz
- https://data.gharchive.org/2021-10-27-22.json.gz
- https://data.gharchive.org/2021-10-27-23.json.gz
- https://data.gharchive.org/2021-10-28-0.json.gz
- https://data.gharchive.org/2021-10-28-1.json.gz
- https://data.gharchive.org/2021-10-28-2.json.gz
- https://data.gharchive.org/2021-10-28-3.json.gz
- https://data.gharchive.org/2021-10-28-4.json.gz
- https://data.gharchive.org/2021-10-28-5.json.gz
- https://data.gharchive.org/2021-10-28-6.json.gz
- https://data.gharchive.org/2021-10-28-7.json.gz
- https://data.gharchive.org/2021-10-28-8.json.gz
- https://data.gharchive.org/2021-10-28-9.json.gz
- https://data.gharchive.org/2021-10-28-10.json.gz
- https://data.gharchive.org/2021-10-28-11.json.gz
- https://data.gharchive.org/2021-10-28-12.json.gz
- https://data.gharchive.org/2021-10-28-13.json.gz
- https://data.gharchive.org/2021-10-28-14.json.gz
- https://data.gharchive.org/2021-10-28-15.json.gz
- https://data.gharchive.org/2021-10-28-16.json.gz
- https://data.gharchive.org/2021-10-28-17.json.gz
- https://data.gharchive.org/2021-10-28-18.json.gz
- https://data.gharchive.org/2021-10-28-19.json.gz
- https://data.gharchive.org/2021-10-28-20.json.gz
- https://data.gharchive.org/2021-10-28-21.json.gz
- https://data.gharchive.org/2021-10-28-22.json.gz
- https://data.gharchive.org/2021-10-28-23.json.gz
- https://data.gharchive.org/2021-10-29-0.json.gz
- https://data.gharchive.org/2021-10-29-1.json.gz
- https://data.gharchive.org/2021-10-29-2.json.gz
- https://data.gharchive.org/2021-10-29-3.json.gz
- https://data.gharchive.org/2021-10-29-4.json.gz
- https://data.gharchive.org/2021-10-29-5.json.gz
- https://data.gharchive.org/2021-10-29-6.json.gz
- https://data.gharchive.org/2021-10-29-7.json.gz
- https://data.gharchive.org/2021-10-29-8.json.gz
- https://data.gharchive.org/2021-10-29-9.json.gz
- https://data.gharchive.org/2021-10-29-10.json.gz
- https://data.gharchive.org/2021-10-29-11.json.gz
- https://data.gharchive.org/2021-10-29-12.json.gz
- https://data.gharchive.org/2021-10-29-13.json.gz
- https://data.gharchive.org/2021-10-29-14.json.gz
- https://data.gharchive.org/2021-10-29-15.json.gz
- https://data.gharchive.org/2021-10-29-16.json.gz
- https://data.gharchive.org/2021-10-29-17.json.gz
内容错乱
- https://data.gharchive.org/2021-09-11-9.json.gz 第42321行有缺失内容，导致后一个PushEvent和前一个重叠。被覆盖部分根据 https://api.github.com/repos/baogaopro/images13/commits/f5cf22c7 信息手动补充
2022
无问题
2023
内容错乱
77743 行，一堆空字符，空字符后面又是一个json，把这些空字符换成一个换行符