Efficient Linear Multiparty PSI and Extension to Circuit/Quorum PSI

psi/psi21_experiment/README.md

@@ -0,0 +1,15 @@
+# Efficient Linear Multiparty PSI and Extension to Circuit/Quorum PSI
+
+(https://www.xueshufan.com/publication/3150904314)
+
+&emsp;&emsp;Our protocols for all three problem settings, namely, mPSI, circuit PSI and qPSI, broadly have two phases. At a high level, in the first phase, a fixed designated party, say $P_1$, interacts with all other parties $P_2,...,P_n$ using 2-party protocols. In the second phase, all parties engage in 𝑛-party protocols to compute a circuit to get the requisite output.<br>
+**multiparty_psi**<br>
+&emsp;&emsp;For mPSI, in the first phase, we invoke a two-party functionality, which we call weak private set membership (wPSM) functionality,
+between a leader, $P_1$ and each $P_i$ (for $i ∈ {2, · · · , 𝑛}$). Informally, the wPSM functionality, when invoked between $P_1$ and $P_i$ on their individual private sets does the following: for each element in $P_1$’s set, it outputs the same random value to both $P_1$ and $P_i$ , if that element is in $P_i$’s set, and outputs independent random values, otherwise. By invoking only n instances ofthe wPSM functionality overall, we ensure that the total communication complexity of this phase is linear in n. <br>
+&emsp;&emsp;In the second phase, all the parties together run a secure multiparty computation to obtain shares of 0 for each element in $P_1$’s set that is in the intersection and shares of a random element for other elements.<br>
+**circuit_psi**<br>
+&emsp;&emsp;For circuit psi, the first phase additionally includes conversion ofthe outputs from the wPSM functionality to arithmetic shares of 1 if $P_1$ and $P_i$ received the same random value, and shares of 0, otherwise.<br>
+&emsp;&emsp;In the second phase, for every element of $P_1$, all parties must get shares of 1 if that element belongs to the intersection, and shares of 0, otherwise.<br>
+**quorum_psi**<br>
+&emsp;&emsp;For qpsi, the first phase additionally includes conversion ofthe outputs from the wPSM functionality to arithmetic shares of 1 if $P_1$ and $P_i$ received the same random value, and shares of 0, otherwise.<br>
+&emsp;&emsp;In the second phase, we appropriately choose another polynomial such that for each element in $P_1$’s set, the polynomial evaluates to 0 if and only if that element belongs to the quorum intersection, and random otherwise.<br>

psi/psi21_experiment/el_c_psi/BUILD.bazel

@@ -0,0 +1,61 @@
+# Copyright 2024 zhangwfjh
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+load("//bazel:psi.bzl",  "psi_cc_binary", "psi_cc_library", "psi_cc_test")
+
+package(default_visibility = ["//visibility:public"])
+
+psi_cc_binary(
+    name = "el_c_psi_benchmark",
+    srcs = ["el_c_psi_benchmark.cc"],
+    deps = [
+        ":el_c_psi",
+        "@com_github_google_benchmark//:benchmark_main",
+    ],
+)
+
+psi_cc_library(
+    name = "el_c_psi",
+    srcs = [
+        "el_hashing.cc",
+        "el_c_psi.cc",
+        "el_opprf.cc",
+    ],
+    hdrs = [
+        "el_hashing.h",
+        "el_c_psi.h",
+        "el_opprf.h",
+    ],
+    deps = [
+        "//psi/utils:communication",
+        "//psi/utils:sync",
+        "//psi/utils:test_utils",
+        "@com_google_absl//absl/types:span",
+        "@yacl//yacl/base:exception",
+        "@yacl//yacl/base:int128",
+        "@yacl//yacl/crypto/hash:hash_utils",
+        "@yacl//yacl/crypto/rand",
+        "@yacl//yacl/kernel/algorithms:base_ot",
+        "@yacl//yacl/kernel/algorithms:iknp_ote",
+        "@yacl//yacl/kernel/algorithms:kkrt_ote",
+        "@yacl//yacl/link",
+    ],
+)
+
+psi_cc_test(
+    name = "el_c_psi_test",
+    srcs = ["el_c_psi_test.cc"],
+    tags = ["manual"],
+    deps = [":el_c_psi"],
+)

psi/psi21_experiment/el_c_psi/el_c_psi.cc

@@ -0,0 +1,165 @@
+// Copyright 2024 zhangwfjh
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "psi/psi/psi21_experiment/el_c_psi/el_c_psi.h"
+
+#include <future>
+
+#include "psi/psi/psi21_experiment/el_c_psi/el_opprf.h"
+#include "psi/psi/utils/communication.h"
+#include "psi/psi/utils/sync.h"
+#include "yacl/crypto/hash/hash_utils.h"
+#include "yacl/crypto/rand/rand.h"
+#include "yacl/utils/serialize.h"
+
+namespace psi::psi {
+
+namespace {
+
+constexpr uint32_t kLinkRecvTimeout = 60 * 60 * 1000;
+
+}  // namespace
+
+NcParty::NcParty(const Options& options) : options_{options} {
+  auto [ctx, wsize, me, leader] = CollectContext();
+  ctx->SetRecvTimeout(kLinkRecvTimeout);
+  p2p_.resize(wsize);
+  for (size_t dst{}; dst != wsize; ++dst) {
+    if (me != dst) {
+      p2p_[dst] = CreateP2PLinkCtx("el_c_psi", ctx, dst);
+    }
+  }
+}
+
+std::vector<std::string> NcParty::Run(
+    const std::vector<std::string>& inputs) {
+  auto [ctx, wsize, me, leader] = CollectContext();
+  auto counts = AllGatherItemsSize(ctx, inputs.size());
+  size_t count{};
+  for (auto cnt : counts) {
+    if (cnt == 0) {
+      return {};
+    }
+    count = std::max(cnt, count);
+  }
+
+  auto items = EncodeInputs(inputs, count);
+  auto shares = ZeroSharing(count);
+  auto recv_share = SwapShares(items, shares);
+  auto recons = Reconstruct(items, recv_share);
+  std::vector<std::string> intersection;
+  for (size_t k{}; k != count; ++k) {
+    if (recons[k] == 0) {
+      intersection.push_back("1");
+    } else {
+      intersection.push_back("0");
+    }
+  }
+  return intersection;
+}
+
+std::vector<uint128_t> NcParty::EncodeInputs(
+    const std::vector<std::string>& inputs, size_t count) const {
+  std::vector<uint128_t> items;
+  items.reserve(count);
+  std::transform(
+      inputs.begin(), inputs.end(), std::back_inserter(items),
+      [](std::string_view input) { return yacl::crypto::Blake3_128(input); });
+  //  Add random dummy elements
+  std::generate_n(std::back_inserter(items), count - inputs.size(),
+                  yacl::crypto::FastRandU128);
+  return items;
+}
+
+auto NcParty::ZeroSharing(size_t count) const -> std::vector<Share> {
+  auto [ctx, wsize, me, leader] = CollectContext();
+  std::vector<Share> shares(wsize, Share(count));
+  for (size_t k{}; k != count; ++k) {
+    uint64_t sum{};
+    for (size_t dst{1}; dst != wsize; ++dst) {
+      sum ^= shares[dst][k] = yacl::crypto::FastRandU64();
+    }
+    shares[0][k] = sum;
+  }
+  return shares;
+}
+
+auto NcParty::SwapShares(const std::vector<uint128_t>& items,
+                            const std::vector<Share>& shares) const -> Share {
+  auto [ctx, wsize, me, leader] = CollectContext();
+  auto count = shares.front().size();
+  std::vector<Share> recv_shares(count);
+  std::vector<std::future<Share>> futures(wsize);
+  // NOTE: First Send Then Receive for peers of smaller ranks
+  for (size_t id{}; id != me; ++id) {
+    futures[id] = std::async(
+        [&](size_t id) {
+          ElOpprfSend(p2p_[id], items, shares[id]);
+          return ElOpprfRecv(p2p_[id], items);
+        },
+        id);
+  }
+  // NOTE: First Receive Then Send for peers of larger ranks
+  for (size_t id{me + 1}; id != wsize; ++id) {
+    futures[id] = std::async(
+        [&](size_t id) {
+          auto ret = ElOpprfRecv(p2p_[id], items);
+          ElOpprfSend(p2p_[id], items, shares[id]);
+          return ret;
+        },
+        id);
+  }
+  for (size_t id{}; id != wsize; ++id) {
+    recv_shares[id] = (me == id ? shares[id] : futures[id].get());
+  }
+
+  Share share(count);  // S(x_k)
+  for (size_t k{}; k != count; ++k) {
+    for (size_t src{}; src != wsize; ++src) {
+      share[k] ^= recv_shares[src][k];
+    }
+  }
+  return share;
+}
+
+auto NcParty::Reconstruct(const std::vector<uint128_t>& items,
+                             const Share& share) const -> Share {
+  auto [ctx, wsize, me, leader] = CollectContext();
+  auto count = items.size();
+  if (me == leader) {
+    std::vector<Share> recv_shares(count);
+    std::vector<std::future<Share>> futures(wsize);
+    for (size_t src{}; src != wsize; ++src) {
+      if (me != src) {
+        futures[src] = std::async(
+            [&](size_t src) { return ElOpprfRecv(p2p_[src], items); }, src);
+      }
+    }
+    for (size_t src{}; src != wsize; ++src) {
+      recv_shares[src] = (me == src ? share : futures[src].get());
+    }
+    Share recons(count);  // sum of S_i(x_k) over i
+    for (size_t k{}; k != count; ++k) {
+      for (size_t src{}; src != wsize; ++src) {
+        recons[k] ^= recv_shares[src][k];
+      }
+    }
+    return recons;
+  } else {
+    ElOpprfSend(p2p_[leader], items, share);
+    return share;
+  }
+}
+
+}  // namespace psi::psi

psi/psi21_experiment/el_c_psi/el_c_psi.h

@@ -0,0 +1,59 @@
+// Copyright 2024 zhangwfjh
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+#pragma once
+
+#include <memory>
+#include <string>
+#include <vector>
+
+#include "yacl/base/int128.h"
+#include "yacl/link/link.h"
+
+namespace psi::psi {
+
+// Practical Multi-party Private Set Intersection from Symmetric-Key Techniques
+// https://eprint.iacr.org/2017/799.pdf
+
+class NcParty {
+ public:
+  struct Options {
+    std::shared_ptr<yacl::link::Context> link_ctx;
+    size_t leader_rank;
+  };
+
+  NcParty(const Options& options);
+  virtual std::vector<std::string> Run(const std::vector<std::string>& inputs);
+
+ private:
+  using Share = std::vector<uint64_t>;
+
+  std::vector<uint128_t> EncodeInputs(const std::vector<std::string>& inputs,
+                                      size_t count) const;
+  std::vector<Share> ZeroSharing(size_t count) const;
+  Share SwapShares(const std::vector<uint128_t>& items,
+                   const std::vector<Share>& shares) const;
+  Share Reconstruct(const std::vector<uint128_t>& items,
+                    const Share& share) const;
+
+  // (ctx, world_size, my_rank, leader_rank)
+  auto CollectContext() const {
+    return std::make_tuple(options_.link_ctx, options_.link_ctx->WorldSize(),
+                           options_.link_ctx->Rank(), options_.leader_rank);
+  }
+
+  Options options_;
+  std::vector<std::shared_ptr<yacl::link::Context>> p2p_;
+};
+
+}  // namespace psi::psi

psi/psi21_experiment/el_c_psi/el_c_psi_benchmark.cc

@@ -0,0 +1,87 @@
+// Copyright 2022 Ant Group Co., Ltd.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include <future>
+#include <iostream>
+
+#include "benchmark/benchmark.h"
+#include "psi/psi/psi21_experiment/el_c_psi/el_c_psi.h"
+#include "psi/psi/psi21_experiment/el_c_psi/el_opprf.h"
+#include "yacl/base/exception.h"
+#include "yacl/crypto/hash/hash_utils.h"
+#include "yacl/link/test_util.h"
+
+namespace {
+std::vector<uint128_t> CreateRangeItems(size_t begin, size_t size) {
+  std::vector<uint128_t> ret(size);
+  for (size_t i = 0; i < size; i++) {
+    auto hash = yacl::crypto::Blake3(std::to_string(begin + i));
+    memcpy(&ret[i], hash.data(), sizeof(uint128_t));
+  }
+  return ret;
+}
+
+void ElCPsiSend(const std::shared_ptr<yacl::link::Context>& link_ctx,
+                const std::vector<uint128_t>& items_hash) {
+  // auto ot_recv = psi::kkrt::GetKkrtOtSenderOptions(link_ctx, 512);
+  // return psi::kkrt::KkrtPsiSend(link_ctx, ot_recv, items_hash);
+  std::vector<uint64_t> shares;
+  for (size_t i = 0; i < items_hash.size(); i++) {
+    uint64_t item = 0;
+    shares.push_back(item);
+  }
+
+  return psi::psi::ElOpprfSend(link_ctx, items_hash, shares);
+}
+
+std::vector<uint64_t> ElCPsiRecv(
+    const std::shared_ptr<yacl::link::Context>& link_ctx,
+    const std::vector<uint128_t>& items_hash) {
+  // auto ot_send = psi::kkrt::GetKkrtOtReceiverOptions(link_ctx, 512);
+  // return psi::kkrt::KkrtPsiRecv(link_ctx, ot_send, items_hash);
+  return psi::psi::ElOpprfRecv(link_ctx, items_hash);
+}
+
+}  // namespace
+
+static void BM_El_C_Psi(benchmark::State& state) {
+  for (auto _ : state) {
+    state.PauseTiming();
+    size_t n = state.range(0);
+    auto alice_items = CreateRangeItems(1, n);
+    auto bob_items = CreateRangeItems(2, n);
+
+    auto contexts = yacl::link::test::SetupWorld(2);
+
+    state.ResumeTiming();
+
+    std::future<void> kkrt_psi_sender =
+        std::async([&] { return ElCPsiSend(contexts[0], alice_items); });
+    std::future<std::vector<uint64_t>> kkrt_psi_receiver =
+        std::async([&] { return ElCPsiRecv(contexts[1], bob_items); });
+
+    kkrt_psi_sender.get();
+    auto results_b = kkrt_psi_receiver.get();
+  }
+}
+
+// [256k, 512k, 1m, 2m, 4m, 8m]
+BENCHMARK(BM_El_C_Psi)
+    ->Unit(benchmark::kMillisecond)
+    ->Arg(256 << 10)
+    ->Arg(512 << 10)
+    ->Arg(1 << 20)
+    ->Arg(2 << 20)
+    ->Arg(4 << 20)
+    ->Arg(8 << 20);