Skip to content

Commit

Permalink
feat: declare the schema of the Parquet file (#8)
Browse files Browse the repository at this point in the history 
* deps: remove ipykernel dep

* fix: remove timer decorator

* feat: declare the schema of the Parquet file

We used to use duckdb, but there were issues to to incorrect type
guessing by duckdb. By validating and modifying the data before
Parquet generation, we can control completely the schema.
  • Loading branch information
@raphael0202
raphael0202 authored Nov 14, 2024
1 parent 48e21b3 commit 309b4f7
Show file tree
Hide file tree
Showing 3 changed files with 576 additions and 659 deletions.
Loading

0 comments on commit 309b4f7

Please sign in to comment.