Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

change: Move from maybe_rayon to rayon-1.8 #122

Merged
merged 7 commits into from
Jan 5, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ env:
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true


jobs:
test:
Expand All @@ -42,8 +41,10 @@ jobs:
strategy:
matrix:
include:
- feature:
- feature: default
target:
- wasm32-unknown-unknown
- wasm32-wasi
CPerezz marked this conversation as resolved.
Show resolved Hide resolved
steps:
- uses: actions/checkout@v2
- uses: actions-rs/toolchain@v1
Expand All @@ -54,6 +55,10 @@ jobs:
# This build will be reused by nextest,
# and also checks (--all-targets) that benches don't bit-rot
run: cargo build --release --all-targets --no-default-features --features "${{ matrix.feature }}"
# We run WASM build (for tests) which compiles the lib allowig us to have
# `getrandom` as a dev-dependency.
- name: Build (WASM compat)
run: cargo build --tests --release --no-default-features --features "${{ matrix.feature }}"
- name: Test
run: |
cargo nextest run --profile ci --release --workspace --no-default-features --features "${{ matrix.feature }}"
Expand Down
10 changes: 7 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@ serde_json = "1.0.105"
hex = "0.4"
rand_chacha = "0.3.1"

# Added to make sure we are able to build the lib in the CI.
# Notice this will never be loaded for someone using this lib as dep.
[target.'cfg(all(target_arch = "wasm32", target_os = "unknown"))'.dev-dependencies]
getrandom = { version = "0.2", features = ["js"] }

[dependencies]
subtle = "2.4"
ff = { version = "0.13.0", default-features = false, features = ["std"] }
Expand All @@ -35,11 +40,10 @@ serde = { version = "1.0", default-features = false, optional = true }
serde_arrays = { version = "0.1.0", optional = true }
hex = { version = "0.4", optional = true, default-features = false, features = ["alloc", "serde"] }
blake2b_simd = "1"
maybe-rayon = { version = "0.1.0", default-features = false }
rayon = "1.8"

[features]
default = ["bits", "multicore"]
multicore = ["maybe-rayon/threads"]
Copy link
Collaborator

@kilic kilic Jan 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[[bench]]
name = "msm"
harness = false
required-features = ["multicore"]

Seems like we had a leftover feature for msm benchs

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohh dammit!!
Will update. Hopefully this will not affect the released crate.

default = ["bits"]
asm = []
bits = ["ff/bits"]
bn256-table = []
Expand Down
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,19 @@ The implementations were originally ported from [matterlabs/pairing](https://git
* Various features related to serialization and deserialization of curve points and field elements.
* Curve-specific optimizations and benchmarking capabilities.

## Controlling parallelism

`halo2curves` currently uses [rayon](https://github.com/rayon-rs/rayon) for parallel
computation.

The `RAYON_NUM_THREADS` environment variable can be used to set the number of
threads.

When compiling to WASM-targets, notice that since version `1.7`, `rayon` will fallback automatically (with no need to handle features) to require `getrandom` in order to be able to work.
For more info related to WASM-compilation.

See: [Rayon: Usage with WebAssembly](https://github.com/rayon-rs/rayon#usage-with-webassembly) for more info.

## Benchmarks

Benchmarking is supported through the use of Rust's built-in test framework. Benchmarks can be run without assembly optimizations:
Expand Down
5 changes: 2 additions & 3 deletions src/fft.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
use crate::multicore;
pub use crate::{CurveAffine, CurveExt};
use ff::Field;
use group::{GroupOpsOwned, ScalarMulOwned};
Expand Down Expand Up @@ -38,7 +37,7 @@ pub fn best_fft<Scalar: Field, G: FftGroup<Scalar>>(a: &mut [G], omega: Scalar,
r
}

let threads = multicore::current_num_threads();
let threads = rayon::current_num_threads();
let log_threads = threads.ilog2();
let n = a.len();
assert_eq!(n, 1 << log_n);
Expand Down Expand Up @@ -107,7 +106,7 @@ pub fn recursive_butterfly_arithmetic<Scalar: Field, G: FftGroup<Scalar>>(
a[1] -= &t;
} else {
let (left, right) = a.split_at_mut(n / 2);
multicore::join(
rayon::join(
|| recursive_butterfly_arithmetic(left, n / 2, twiddle_chunk * 2, twiddles),
|| recursive_butterfly_arithmetic(right, n / 2, twiddle_chunk * 2, twiddles),
);
Expand Down
1 change: 0 additions & 1 deletion src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ pub mod ff_ext;
pub mod fft;
pub mod hash_to_curve;
pub mod msm;
pub mod multicore;
pub mod serde;

pub mod bn256;
Expand Down
15 changes: 5 additions & 10 deletions src/msm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ use ff::PrimeField;
use group::Group;
use pasta_curves::arithmetic::CurveAffine;

use crate::multicore;

fn get_booth_index(window_index: usize, window_size: usize, el: &[u8]) -> i32 {
// Booth encoding:
// * step by `window` size
Expand Down Expand Up @@ -155,12 +153,12 @@ pub fn small_multiexp<C: CurveAffine>(coeffs: &[C::Scalar], bases: &[C]) -> C::C
pub fn best_multiexp<C: CurveAffine>(coeffs: &[C::Scalar], bases: &[C]) -> C::Curve {
assert_eq!(coeffs.len(), bases.len());

let num_threads = multicore::current_num_threads();
let num_threads = rayon::current_num_threads();
if coeffs.len() > num_threads {
let chunk = coeffs.len() / num_threads;
let num_chunks = coeffs.chunks(chunk).len();
let mut results = vec![C::Curve::identity(); num_chunks];
multicore::scope(|scope| {
rayon::scope(|scope| {
let chunk = coeffs.len() / num_threads;

for ((coeffs, bases), acc) in coeffs
Expand All @@ -186,10 +184,7 @@ mod test {

use std::ops::Neg;

use crate::{
bn256::{Fr, G1Affine, G1},
multicore,
};
use crate::bn256::{Fr, G1Affine, G1};
use ark_std::{end_timer, start_timer};
use ff::{Field, PrimeField};
use group::{Curve, Group};
Expand All @@ -200,12 +195,12 @@ mod test {
fn best_multiexp<C: CurveAffine>(coeffs: &[C::Scalar], bases: &[C]) -> C::Curve {
assert_eq!(coeffs.len(), bases.len());

let num_threads = multicore::current_num_threads();
let num_threads = rayon::current_num_threads();
if coeffs.len() > num_threads {
let chunk = coeffs.len() / num_threads;
let num_chunks = coeffs.chunks(chunk).len();
let mut results = vec![C::Curve::identity(); num_chunks];
multicore::scope(|scope| {
rayon::scope(|scope| {
let chunk = coeffs.len() / num_threads;

for ((coeffs, bases), acc) in coeffs
Expand Down
16 changes: 0 additions & 16 deletions src/multicore.rs

This file was deleted.

Loading