Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace BigInt based elliptic curve library #4027

Open
12 of 20 tasks
randombit opened this issue Apr 20, 2024 · 9 comments
Open
12 of 20 tasks

Replace BigInt based elliptic curve library #4027

randombit opened this issue Apr 20, 2024 · 9 comments
Assignees
Labels
enhancement Enhancement or new feature
Milestone

Comments

@randombit
Copy link
Owner

randombit commented Apr 20, 2024

Botan 3.5.0

In this release pcurves is really just used for hash to curve

  • Initial pcurves (point arithmetic, fixed curve params) - that's Add library for compile time instantiation of elliptic curves #3979
  • Deprecate all the functionality that existed just to support elliptic curves using BigInt, eg mod_sub, ct_reduce_below, many more.
  • Support for providing parameterized curves, where we eg compute Montgomery params at runtime. This is required not just for user provided/application specific curves but also I don't think it's worthwhile to provide the fully parameterized/hardcoded support for obscure curves like secp160r1. In the short term, application curves, secp160r1, etc fall back to the currently used BigInt code

Botan 3.6.0

In this release, we tie together EC_Scalar/EC_AffinePoint to pcurves so that everything goes fast 🚀

Bumped to 3.7.0

Work originally planned for 3.6.0 but bumped to 3.7.0

  • Add new constructors to EC keys, deprecate old constructors
  • Technical debt patrol - remove duplicated logic between the various types
  • Start removing stuff replaced by pcurves, eg CurveGFP_NIST (we can just use Montgomery for everything)

Botan 3.6.0 or later. Nice optimizations / cleanups but not critical

@randombit randombit added this to the Botan 3.6.0 milestone Apr 20, 2024
@randombit randombit self-assigned this May 5, 2024
@reneme
Copy link
Collaborator

reneme commented Jun 5, 2024

We should consider compile time configuration of which curves get instantiated (as pointed out here). Much like the compile-time configuration of having "kyber" and/or "kyber_90s", for instance.

Adding one build module per curve should do the trick, and probably provides the best affordance for users. On the other hand, it produces quite a bit of boilerplate due to subdirectories, info.txt files and code machinery.

Alternatively, I could see this as a dedicated compile-time switch: Like:

  • ./configure.py --enable-elliptic-curves=all (the default)
  • ./configure.py --enable-elliptic-curves=p256,brainpool256 (a restricted set)

and then simply #ifdef the instantiations in pcurves.cpp accordingly.

@randombit
Copy link
Owner Author

The --enable-elliptic-curves approach is potentially confusing since even if you disable the pcurves module entirely, the elliptic curves still exist. All that ends up happening is you fall back to the slower BigInt code. [*]

[*] I mean right now ECDSA etc still use BigInt, even after both #3979 and #4042 land. There are several more steps here until the whole thing hangs together. But in the end state, we'll have fast ECC for specific curves, plus fallback code for weird curves, application curves, etc. If you disable the fast curve, it doesn't disable the curve, just disables it going fast.

That said there may be environments where the code size becomes an issue. I tried out a module per curve and it was not so bad. There are likely to not be more than ~5 new curves added here over time so I don't think it's an issue in an ongoing way.

In the end we could consider also having a way of disabling the slowpath BigInt stuff, which would benefit environments that are using just P-256 or something.

@reneme
Copy link
Collaborator

reneme commented Jun 9, 2024

Nice! I saw that you split the curve into compile-time modules. Indeed the boilerplate isn't so bad. Also provides a nice place to keep curve-specific special cases. 👍

I added a few minor suggestions, in which the type -> "Internal" might need some extra attention. Perhaps have a new module type "Feature" (which might also be better suited for "Kyber" vs. "Kyber90s", and similar situations...)?

randombit added a commit that referenced this issue Jul 12, 2024
This is analagous to the DL scheme key types added in #3210, but here
we have to retain the existing classes as we are constrained by SemVer.

The new types contain both our old types (BigInt, EC_Point) and new types
(EC_Scalar, EC_AffinePoint). Eventually the legacy types will be removed,
but we can't do that until the next major version. GH #4027
randombit added a commit that referenced this issue Jul 12, 2024
This is analagous to the DL scheme key types added in #3210, but here
we have to retain the existing classes as we are constrained by SemVer.

The new types contain both our old types (BigInt, EC_Point) and new types
(EC_Scalar, EC_AffinePoint). Eventually the legacy types will be removed,
but we can't do that until the next major version. GH #4027
randombit added a commit that referenced this issue Jul 12, 2024
This is analagous to the DL scheme key types added in #3210, but here
we have to retain the existing classes as we are constrained by SemVer.

The new types contain both our old types (BigInt, EC_Point) and new types
(EC_Scalar, EC_AffinePoint). Eventually the legacy types will be removed,
but we can't do that until the next major version. GH #4027
randombit added a commit that referenced this issue Jul 13, 2024
For P-256 this speeds up projective->affine conversion by about 5%,
saving ~2K cycles.

For P-384 it speeds up conversion by 25%, saving 8K cycles.

For P-521 it speeds up conversion by 20%, saving 33K cycles.

GH #4027
The savings are most pronounced with ECDSA signing (since it is the
fastest operation), improving by 2%, 4%, and 2% resp.
randombit added a commit that referenced this issue Jul 13, 2024
For P-256 this speeds up projective->affine conversion by about 5%,
saving ~2K cycles.

For P-384 it speeds up conversion by 25%, saving 8K cycles.

For P-521 it speeds up conversion by 20%, saving 33K cycles.

The savings are most pronounced with ECDSA signing (since it is the
fastest operation), improving by 2%, 4%, and 2% resp.

GH #4027
randombit added a commit that referenced this issue Jul 13, 2024
For P-256 this speeds up projective->affine conversion by about 5%,
saving ~2K cycles.

For P-384 it speeds up conversion by 25%, saving 8K cycles.

For P-521 it speeds up conversion by 20%, saving 33K cycles.

The savings are most pronounced with ECDSA signing (since it is the
fastest operation), improving by 2%, 4%, and 2% resp.

GH #4027
randombit added a commit that referenced this issue Jul 14, 2024
This is analagous to the DL scheme key types added in #3210, but here
we have to retain the existing classes as we are constrained by SemVer.

The new types contain both our old types (BigInt, EC_Point) and new types
(EC_Scalar, EC_AffinePoint). Eventually the legacy types will be removed,
but we can't do that until the next major version. GH #4027
randombit added a commit that referenced this issue Jul 14, 2024
This is analagous to the DL scheme key types added in #3210, but here
we have to retain the existing classes as we are constrained by SemVer.

The new types contain both our old types (BigInt, EC_Point) and new types
(EC_Scalar, EC_AffinePoint). Eventually the legacy types will be removed,
but we can't do that until the next major version. GH #4027
randombit added a commit that referenced this issue Jul 14, 2024
This is analagous to the DL scheme key types added in #3210, but here
we have to retain the existing classes as we are constrained by SemVer.

The new types contain both our old types (BigInt, EC_Point) and new types
(EC_Scalar, EC_AffinePoint). Eventually the legacy types will be removed,
but we can't do that until the next major version. GH #4027
randombit added a commit that referenced this issue Jul 15, 2024
For P-256 this speeds up projective->affine conversion by about 5%,
saving ~2K cycles.

For P-384 it speeds up conversion by 25%, saving 8K cycles.

For P-521 it speeds up conversion by 20%, saving 33K cycles.

The savings are most pronounced with ECDSA signing (since it is the
fastest operation), improving by 2%, 4%, and 2% resp.

GH #4027
randombit added a commit that referenced this issue Jul 15, 2024
For P-256 this speeds up projective->affine conversion by about 5%,
saving ~2K cycles.

For P-384 it speeds up conversion by 25%, saving 8K cycles.

For P-521 it speeds up conversion by 20%, saving 33K cycles.

The savings are most pronounced with ECDSA signing (since it is the
fastest operation), improving by 2%, 4%, and 2% resp.

GH #4027
randombit added a commit that referenced this issue Jul 15, 2024
Saves approximately 11K cycles for P-384 and 36K cycles for P-521,
improving ECDSA signing for both curves by ~4%.

GH #4027 #1479
randombit added a commit that referenced this issue Jul 15, 2024
Saves approximately 11K cycles for P-384 and 36K cycles for P-521,
improving ECDSA signing for both curves by ~4%.

GH #4027 #1479
randombit added a commit that referenced this issue Jul 15, 2024
Saves approximately 11K cycles for P-384 and 36K cycles for P-521,
improving ECDSA signing for both curves by ~4%.

GH #4027 #1479
randombit added a commit that referenced this issue Jul 16, 2024
This one is a little more involved since this is the first curve where
p != 3 mod 4, and in fact since P-224 is == 1 mod 16 we must use
Shanks-Tonelli

GH #4027
randombit added a commit that referenced this issue Jul 16, 2024
This one is a little more involved since this is the first curve where
p != 3 mod 4, and in fact since P-224 is == 1 mod 16 we must use
Shanks-Tonelli

GH #4027
randombit added a commit that referenced this issue Jul 16, 2024
This one is a little more involved since this is the first curve where
p != 3 mod 4, and in fact since P-224 is == 1 mod 16 we must use
Shanks-Tonelli

GH #4027
randombit added a commit that referenced this issue Jul 16, 2024
This one is a little more involved since this is the first curve where
p != 3 mod 4, and in fact since P-224 is == 1 mod 16 we must use
Shanks-Tonelli

GH #4027
randombit added a commit that referenced this issue Jul 17, 2024
This one is a little more involved since this is the first curve where
p != 3 mod 4, and in fact since P-224 is == 1 mod 16 we must use
Shanks-Tonelli

GH #4027
randombit added a commit that referenced this issue Jul 18, 2024
This one is a little more involved since this is the first curve where
p != 3 mod 4, and in fact since P-224 is == 1 mod 16 we must use
Shanks-Tonelli

GH #4027
randombit added a commit that referenced this issue Jul 22, 2024
This is analagous to the DL scheme key types added in #3210, but here
we have to retain the existing classes as we are constrained by SemVer.

The new types contain both our old types (BigInt, EC_Point) and new types
(EC_Scalar, EC_AffinePoint). Eventually the legacy types will be removed,
but we can't do that until the next major version. GH #4027
randombit added a commit that referenced this issue Jul 22, 2024
This is analagous to the DL scheme key types added in #3210, but here
we have to retain the existing classes as we are constrained by SemVer.

The new types contain both our old types (BigInt, EC_Point) and new types
(EC_Scalar, EC_AffinePoint). Eventually the legacy types will be removed,
but we can't do that until the next major version. GH #4027
randombit added a commit that referenced this issue Jul 23, 2024
This is analagous to the DL scheme key types added in #3210, but here
we have to retain the existing classes as we are constrained by SemVer.

The new types contain both our old types (BigInt, EC_Point) and new types
(EC_Scalar, EC_AffinePoint). Eventually the legacy types will be removed,
but we can't do that until the next major version. GH #4027
randombit added a commit that referenced this issue Jul 23, 2024
This is analagous to the DL scheme key types added in #3210, but here
we have to retain the existing classes as we are constrained by SemVer.

The new types contain both our old types (BigInt, EC_Point) and new types
(EC_Scalar, EC_AffinePoint). Eventually the legacy types will be removed,
but we can't do that until the next major version. GH #4027
randombit added a commit that referenced this issue Jul 23, 2024
This is analagous to the DL scheme key types added in #3210, but here
we have to retain the existing classes as we are constrained by SemVer.

The new types contain both our old types (BigInt, EC_Point) and new types
(EC_Scalar, EC_AffinePoint). Eventually the legacy types will be removed,
but we can't do that until the next major version. GH #4027

Co-Authored-By: René Meusel <[email protected]>
randombit added a commit that referenced this issue Jul 23, 2024
This is analagous to the DL scheme key types added in #3210, but here
we have to retain the existing classes as we are constrained by SemVer.

The new types contain both our old types (BigInt, EC_Point) and new types
(EC_Scalar, EC_AffinePoint). Eventually the legacy types will be removed,
but we can't do that until the next major version. GH #4027

Co-Authored-By: René Meusel <[email protected]>
randombit added a commit that referenced this issue Jul 24, 2024
This is analagous to the DL scheme key types added in #3210, but here
we have to retain the existing classes as we are constrained by SemVer.

The new types contain both our old types (BigInt, EC_Point) and new types
(EC_Scalar, EC_AffinePoint). Eventually the legacy types will be removed,
but we can't do that until the next major version. GH #4027

Co-Authored-By: René Meusel <[email protected]>
randombit added a commit that referenced this issue Jul 24, 2024
This is analagous to the DL scheme key types added in #3210, but here
we have to retain the existing classes as we are constrained by SemVer.

The new types contain both our old types (BigInt, EC_Point) and new types
(EC_Scalar, EC_AffinePoint). Eventually the legacy types will be removed,
but we can't do that until the next major version. GH #4027

Co-Authored-By: René Meusel <[email protected]>
randombit added a commit that referenced this issue Jul 31, 2024
This one is a little more involved since this is the first curve where
p != 3 mod 4, and in fact since P-224 is == 1 mod 16 we must use
Shanks-Tonelli

GH #4027
thillux pushed a commit to thillux/botan that referenced this issue Aug 16, 2024
This one is a little more involved since this is the first curve where
p != 3 mod 4, and in fact since P-224 is == 1 mod 16 we must use
Shanks-Tonelli

GH randombit#4027
@randombit randombit added the enhancement Enhancement or new feature label Aug 21, 2024
@randombit randombit modified the milestones: Botan 3.6.0, Botan 3.7.0 Oct 14, 2024
@Honzaik
Copy link

Honzaik commented Nov 5, 2024

Hello,

I am new to this library so I might be missing something, but from what I understand, you are deprecating EC_Point and replacing it with EC_AffinePoint. My concern is, that currently I dont see currently how to add two distinct EC_AffinePoints. Of course, for Diffie-Hellman all one needs is interface that takes a scalar n and calculates n*P for some point P. However, in other applications you may want to add distinct points.

So my question is, is there a way to add two EC_AffinePoints?

Thank you in advance.

@randombit
Copy link
Owner Author

@Honzaik this is not currently supported simply because the interface of EC_AffinePoint was quite intentionally done as the absolute minimum required to implement the relevant algorithms. This is to a large extent a resposne to EC_Point which has a huge number of very specialized interfaces, many of which make it difficult to implement EC_Point using any other approach than exactly how it is currently implemented.

In the short term the answer is if you want point addition use EC_Point - it’s deprecated but thats just an advisory that it will be removed down the line (next major release in ~~~ 2027)

It’s very likely EC_AffinePoint will gain point addition. I debated adding it during the initial implemenetation and then decided to wait until it’s actually in use. That will come anway when SPAKE2 is implemented, since that requires point additions.

I would be curious as to what kind of protocol you are implementing.

@Honzaik
Copy link

Honzaik commented Nov 11, 2024

@randombit thank you for the explanation. My use case is similar to SPAKE2, i.e., some PAKE-adjacent constuctions. Specifically, PAPKE (https://eprint.iacr.org/2019/199.pdf) which can be used to build PAKE requires addition of different points. Similarly, the M2F construction from https://eprint.iacr.org/2023/295.pdf also requires addition.

These constructions also need hashing to curve, but that is already implemented (for prime curves), however, I'd also note that the "uniform hashing" construction (https://www.ietf.org/archive/id/draft-irtf-cfrg-hash-to-curve-16.html#section-3-4.2.2) also requires adding two points.

Don't get me wrong, I am doing just a proof of concept implementation and all these use cases basically boil down to PAKEs so I understand I am a tiny minority of users that need it. At the end, I understand why a lot of crypto libraries try to minimize the opportunity for their users to shoot themselves in the foot and power users can in the end fork it and expose the hidden functions (with a little bit of work). On the other hand, I feel like leaving the addition implemented doesn't pose that much of a risk (maybe as some sort of a “hazmat” layer, like in python cryptography).

@randombit
Copy link
Owner Author

It's not specifically about preventing users from shooting themselves in the foot. TBH working at this level -- using raw elliptic curve points to define a new protocol -- you had better know what you are doing anyway! It was more that the interface of EC_AffinePoint was defined to be just what was required to implement the various elliptic curve algorithms already included, and it so happened that we did not need at this time to expose point addition to the rest of the library, thus it was not added.

Point addition will be added when SPAKE2 is added, or perhaps I'll just add it for 3.7.0 right away since addition plus point negation probably unblocks most of the basic protocols one might want to implement.

randombit added a commit that referenced this issue Nov 25, 2024
This adds point negation and point addition.

Point addition is not particularly efficient since it converts the
result immediately to an affine coordinate, as we do not currently
expose projective points at all. That said it is more than sufficient
for simple protocols that just need to add a few points together.

This also corrects the return result of EC_AffinePoint::mul_px_qy,
which would previously crash if the resulting point was the identity
element. Instead it should return nullopt.

Discussion in #4027
randombit added a commit that referenced this issue Nov 27, 2024
This adds point negation and point addition.

Point addition is not particularly efficient since it converts the
result immediately to an affine coordinate, as we do not currently
expose projective points at all. That said it is more than sufficient
for simple protocols that just need to add a few points together.

This also corrects the return result of EC_AffinePoint::mul_px_qy,
which would previously crash if the resulting point was the identity
element. Instead it should return nullopt.

Discussion in #4027
@randombit
Copy link
Owner Author

randombit commented Nov 27, 2024

@Honzaik fyi with #4446 master now has point addition and negation available in EC_AffintPoint. There is also a new constant time 2-ary mul (p*x + q*y) that may be useful to you.

@Honzaik
Copy link

Honzaik commented Nov 28, 2024

Great! Thank you very much.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or new feature
Projects
None yet
Development

No branches or pull requests

3 participants