Skip to content

Function to iterate and save to delta lake format using Calcbench API

Notifications You must be signed in to change notification settings

duotangx/calcbench-iterate-and-save-delta

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 

Repository files navigation

Calcbench Iterate and Save to Delta Lake

Function to iterate and save to Delta Lake format using Calcbench API.

Tested on Databricks + AWS but should work for other platforms supporting Delta Lake. You will need to manually set spark.config, as well as S3 bucket name and Delta Lake table name.

Logic is adapted from and largely similar to Calcbench's other iterate_and_save functions.

About

Function to iterate and save to delta lake format using Calcbench API

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages