CRL sharding notes

In order to prevent quite large CRLs if we wish to implement them we will need to use IDP (Issuing Distribution Point) based CRL sharding. In a IDP based sharding scheme the number of CRL shards is defined by how many unique CRL Distribution Point URLs and present in issued certificates. For instance if there are three different CRL DP URLs that indicates there are three CRL shards. The scope of each of these shards is defined by URL in the IDP extension in the CRL, every certificate with a CRL DP URL matching the CRL IDP URL is covered by that particular CRL shard.

What to shard on

There are a ton of things we can decide to shard on, but there are some considerations to take into account.

An obvious variable would be the notAfter of certificates, but this can be somewhat dangerous as it could allow a user to cause ballooning of a single shard. For instance if a user issues a large number of certificates in a short period, and then chooses to revoke all of those certificates it would cause all of those certificates to be added to a single shard.

Likely what we'll want to shard on is something that will provide uniform distribution across the shards, for instance by using some part of the certificate serial, such that revocations of certificates in clumps are properly shared across shards.

Boulder design considerations

With our current database design we'd need to scan the certificateStatus table each time we wanted to generate CRLs for certificates that had been revoked since the last time we generated CRLs. This would be a rather expensive and, likely, slow process. Instead we should introduce a new table which only records revocations. this can then be used to quickly determine what new entries need to be added to the updated CRLs.