Skip to content

Latest commit

 

History

History
127 lines (100 loc) · 10 KB

README.md

File metadata and controls

127 lines (100 loc) · 10 KB
Raw

Awesome Dataframes

An awesome list of dataframe (and dataframe-like) libraries. This list focuses on libraries and tools intended for local (on your personal computer) manipulation of tabular data.

Libraries

Python

  • pandas - Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more.
  • Polars - Fast multi-threaded DataFrame library in Rust and Python.
  • Modin - Speed up your Pandas workflows by changing a single line of code.
  • Ibis - A pandas-like deferred expression system, with first-class SQL support.
  • Narwhals - Lightweight and extensible compatibility layer between dataframe libraries!
  • agate - agate is a Python data analysis library that is optimized for humans instead of machines. It is an alternative to numpy and pandas that solves real-world problems with readable code.
  • Lemuras - A small pure Python library to deal with big tables.
  • datatable - A Python package for manipulating 2-dimensional tabular data structures.
  • Prosto - A Python data processing toolkit to programmatically author and execute complex data processing workflows. Conceptually, it is an alternative to purely set-oriented approaches to data processing like map-reduce, relational algebra, SQL or data-frame-based tools like pandas.
  • siuba - Python library for using dplyr like syntax with pandas and SQL.
  • Vaex - A high performance Python library for lazy Out-of-Core DataFrames (similar to Pandas), to visualize and explore big tabular datasets.
  • dfply - dplyr-style piping operations for pandas dataframes.
  • kadro - A friendly pandas wrapper with a more composable grammar support.
  • dexplo - Data exploration library with a pandas-like API.
  • pands_cub - A detailed project that teaches you how to build your own Python data analysis library, pandas_cub, from scratch.
  • fletcher - Pandas ExtensionDType/Array backed by Apache Arrow.
  • tidypandas - A grammar of data manipulation for pandas inspired by tidyverse.
  • redframes - [re]ctangular[d]ata[frames]
  • static-frame - Immutable and grow-only Pandas-like DataFrames with a more explicit and consistent interface

R

  • dplyr - A grammar of data manipulation, providing a consistent set of verbs that help you solve the most common data manipulation challenges.
  • data.table - Provides a high-performance version of base R's data.frame with syntax and feature enhancements for ease of use, convenience and programming speed.
  • dance - Dancing 💃 with the stats, aka tibble() dancing 🕺. dance is a sort of reinvention of dplyr classic verbs, with a more modern stack underneath, i.e. it leverages a lot from vctrs and rlang.

JavaScript

  • Arquero - A JavaScript library for query processing and transformation of array-backed data tables. Following the relational algebra and inspired by the design of dplyr, Arquero provides a fluent API for manipulating column-oriented data frames.
  • dataflow-api - JavaScript API for dataflow processing using the vega-dataflow reactive engine. Perform common database operations (sorting, filtering, aggregation, window calculations) over JavaScript objects.
  • datalib - A JavaScript data utility library. It provides facilities for data loading, type inference, common statistics, and string templates.
  • Tidy.js - Tidy up your data with JavaScript, inspired by dplyr and the tidyverse.
  • Data-Forge - The JavaScript data transformation and analysis toolkit inspired by Pandas and LINQ.
  • zebras - A data manipulation and analysis library written in JavaScript offering the convenience of pandas or R.
  • dataframe-js - A javascript library providing a new data structure for datascientists and developers.
  • Simple Data Analysis (SDA) - Easy-to-use JavaScript library for most common data analysis tasks.

Julia

  • DataFrames.jl - Tools for working with tabular data in Julia.
  • DataKnots.jl - A Julia library for querying data with an extensible, practical and coherent algebra of query combinators.
  • Volcanito.jl - Backend agnostic for tabular data operations in Julia.
  • Query.jl - A package for querying julia data sources. It can filter, project, join and group data from any iterable data source, including all the sources supported in IterableTables.jl.
  • TidierData.jl - 100% Julia implementation of the dplyr and tidyr R packages.

Clojure

  • tech.ml.dataset - A Clojure high performance data processing system.
  • tablecloth - Dataset manipulation library build on the top of tech.ml.dataset.

Common Lisp

C++

  • DataFrame - A C++ statistical library that provides an interface similar to Pandas package in Python.

Elixir

  • explorer - Explorer brings series (one-dimensional) and dataframes (two-dimensional) for fast data exploration to Elixir.

Elm

  • tidy - Leaning heavily on the principles of the tidyverse, and especially tidy data, this package makes it easy to reshape and tidy tabular data for easier data analysis and visualization.

Go

  • column - High-performance, columnar, in-memory store with bitmap indexing in Go.
  • gambas - Data analysis tool for Go. Similar to the famous Python library pandas.

Haskell

  • DataFrame - An intuitive, dynamically-typed DataFrame library.

Java

  • Tablesaw - Java dataframe and visualization library.

Kotlin

Lua

  • Assistant - A data science library providing flexible dataframes for Lua 5.1+

Q

  • qSQL - Query language embedded into Q.

Raku

  • Data::Reshapers - Raku package with data reshaping functions for different data structures.

Ruby

  • rover - Simple, powerful data frames for Ruby.
  • daru - daru (Data Analysis in RUby) is a library for storage, analysis, manipulation and visualization of data in Ruby.
  • polars-ruby - Blazingly fast DataFrames for Ruby.

Rust

  • polars - A blazingly fast DataFrames library implemented in Rust.
  • datafusion - DataFrame API in Apache Arrow DataFusion

Wolfram

  • Dataset - Represents a structured dataset based on a hierarchy of lists and associations.

Database

  • SQLite - A C-language library that implements a small, fast, self-contained, high-reliability, full-featured, SQL database engine
  • DuckDB - An embeddable SQL OLAP Database Management System.

CLI

  • VisiData - A terminal spreadsheet multitool for discovering and arranging data.

GUI

  • Power Query - A core capability of Power Query is to filter and combine, that is, to mash-up data from one or more of a rich collection of supported data sources.

Other

  • prql - A modern language for transforming data — a simple, powerful, pipelined SQL replacement.
  • Malloy - An experimental language for describing data relationships and transformations.
  • Arrow - A cross-language development platform for in-memory data.
  • Substrait - A cross platform way to express data transformation, relational algebra, standardized record expression and plans.
  • Consortium for Python Data APIs

Papers

Other Lists