1B rides NYC taxi native benchmark

This repository reflects progress of performance and design study for 1B ride NYC taxi benchamarks.

Compilation

Install and activate miniconda
Create development environment: conda env create -n arrow-def -f conda/default-env.yml
Activate this environment: conda activate arrow-def
Configure: cmake -GNinja -DCMAKE_BUILD_TYPE=release -S . -B ../build
Build: cmake --build ../build
Allocate huge memory pages: sudo bash -c 'echo 30000 > /proc/sys/vm/nr_hugepages'
Download trips.xaa.csv.gz, gzip -d it
Run ../build/bench_all for a single run of all execution nodes.
Adjust and run bash run-aff-bsz.sh for collecting times of CSV reader in different modes.

Name		Name	Last commit message	Last commit date
Latest commit History 48 Commits
conda		conda
.gitignore		.gitignore
.travis.yml		.travis.yml
CMakeLists.txt		CMakeLists.txt
README.md		README.md
aggregate.h		aggregate.h
benchmark.cpp		benchmark.cpp
group_by.h		group_by.h
load.h		load.h
pandas_variant.py		pandas_variant.py
print.h		print.h
run-aff-bsz.sh		run-aff-bsz.sh
sort.h		sort.h
timegm.h		timegm.h
transform.h		transform.h
trips_test.csv		trips_test.csv
util.h		util.h