Skip to content

vineyard v0.2.7
Compare
Choose a tag to compare
@github-actions github-actions released this 19 Aug 07:49
· 1101 commits to main since this release
v0.2.7
491fbbc

Vineyard v0.2.7 brings many new features into experimental stage, including

  • Airflow Integration with Vineyard: vineyard serves as a XCom backend for airflow to support the tasks in user-defined workflow shares large volume of intermediate data using vineyard, and enables more possibilities for orchestrating bigdata analytical workloads with Airflow. To try Airflow with vineyard, please refer to https://v6d.io/notes/airflow.html

  • Vineyard brings Dask integration in this release, data in vineyard can be consumed and produced by dask procedures, and can be further shared by other compute engines that works on Vineyard. For a end-to-end demo, please refer to https://v6d.io/notes/dask.html

  • Vineyard now can serve as the data source for ML frameworks, e.g., tensorflow, pytorch and xgboost. (thanks @chaitravi-ce )

  • Vineyard Go SDK (thanks @linlih) and Java SDK (thanks @zhanglei1949) comes into a very preliminary stage.

Vineyard v0.2.7 will the last release of v0.2.x series and v0.3.0 will come next week!

Commits

  • [491fbbc]: Upgrade vineyard to v0.2.7, and add a version number config in headers. (Tao He)
  • [cc1d2af]: Fixes a bug in read response and native lib output directory, and adapt to Vineyard Java Fragment. (#453) (Tao He)
  • [7050e63]: Fix a tiny bug in tensorflow resolver (#451) (Diwen Zhu)
  • [00c5bfe]: Fixes the RPATH problem on Mac. (#450) (Tao He)
  • [bbe1c69]: Add doc for a dask tensorflow example (#440) (Diwen Zhu)
  • [be6f4ad]: Bump junit from 4.11 to 4.13.1 in /java/modules (#447) (dependabot[bot])
  • [1b36571]: Add a vineyard::Blob -> ArrowBuf resolver. (#446) (Tao He)
  • [8c6357a]: The objectmeta interface, and packaging script for FFI resolver on metadata (#441) (Tao He)
  • [c2ccc65]: Implements a static constructor to get metadata from unsafe sources. (#445) (Tao He)
  • [7d99778]: Ensure mmap in the same order with send_fd in server. (#444) (Tao He)
  • [84f2164]: Drop unused rust code from the /src directory. (#443) (Tao He)
  • [a558ed1]: Catch JSON error as well when reading the RPC request from client. (#442) (Tao He)
  • [65482c5]: Memory allocator tests and benchmarks. (#438) (Pei Li)
  • [1ec9f19]: Add ipc, proposal and rpc for Rust (#437) (Sijie)
  • [2724ca7]: Add doc and quick fix to xgboost (#436) (Diwen Zhu)
  • [46d820d]: Fix invalid step (#435) (Siyuan Zhang)
  • [5830e1a]: Add CI for Rust (#434) (Siyuan Zhang)
  • [d4db525]: Use unique_ptr to allow better flexibility for FFI interfaces. (#433) (Tao He)
  • [c964ae3]: Add Go SDK put/get/drop name api implementation (#432) (linlih)
  • [aa9bd17]: Add a troubleshooting section about the "permission error". (#431) (Tao He)
  • [00278cf]: Added Global Resolvers and Tests for PyTorch and Documentation (#416) (Chaitravi Chalke)
  • [3add0e7]: Add docs and example dags for airflow integration. (#429) (Tao He)
  • [d34a4ca]: Bug Fix: property_graph_util::VertexDataColumn subtraction between shared_ptr and ptrdiff_t (#428) (Zhang Lei)
  • [364cbfa]: Refine the resolver context implementation, and refactor duplicated code (#425) (Tao He)
  • [e7ed7c3]: fix template type in vertexDataColum (#424) (Zhang Lei)
  • [38d6957]: Lint: reduce code duplication, remove unused imports, isort, etc. (#421) (Tao He)
  • [794ce15]: Add a command line arguments to control whether printing the metrics. (#420) (Tao He)
  • [793b5df]: Add missing init.py to dask module otherwise it is an empty package. (#417) (Tao He)
  • [05820a5]: Adds migrate command to vineyard-ctl (#419) (Rohan Gupta)
  • [a5d3c5c]: [MRG]🎉 init prometheus setting (#418) (Jiang Shanshan)
  • [06fe9df]: Add Dask integration: vineyard as data sink (#415) (Diwen Zhu)
  • [d800c4f]: Airflow: support running among distributed celery workers. (#414) (Tao He)
  • [5b16173]: Add Dask integration: vineyard as data source (#409) (Diwen Zhu)
  • [262c484]: Added Tests for Tensorflow (#410) (Chaitravi Chalke)
  • [22055c2]: Associate the client object to resolved value as well when possible. (#413) (Tao He)
  • [dca1b69]: Vineyard XCom backend for apache-airflow. (#411) (Tao He)
  • [f729e93]: Upgrade etcd-cpp-apiv3 to make it compatible with cmake less than 3.10. (Tao He)
  • [b8efbb6]: Added Global Resolvers for Tensorflow and Local Builders and Resolvers for PyTorch (#395) (Chaitravi Chalke)
  • [fb448e3]: Allow the IPC socket connectable by other users. (#407) (Tao He)
  • [44bf8f4]: Adds documentation for vineyard-ctl (#405) (Rohan Gupta)
  • [b1268cf]: Adds new commands, arguments and examples to vineyard-ctl (#401) (Rohan Gupta)
  • [e03cf51]: Run xgboost tests use a scoped fixture, rather than populating the global ctx (#403) (Tao He)
  • [b448a9f]: Add docs for the resolver/builder/driver context. (Tao He)
  • [5505cc7]: Doc chapter for Vineyard ctl. (#402) (Tao He)
  • [dd987b1]: Tidy up contrib/ml/xgboost with new resolver context (#400) (Diwen Zhu)
  • [ad03562]: Contextual resolver/builder/driver for flexible nested dynamic registry. (#399) (Tao He)
  • [4b4f9f8]: Enable contrib-ml test, and install xgboost deps. (#396) (Tao He)
  • [26b40d4]: Add a doc chapter for ML integration. (#397) (Tao He)
  • [7e0d44a]: Add xgboost integration and test (#394) (Diwen Zhu)
  • [71e326b]: Add go sdk rpc/ipc client connect and close api implementation (#392) (linlih)
  • [7f6ec86]: Fixes write bytes and formatting of objectmapper. (#390) (Tao He)
  • [e5978cc]: Bump junit from 4.11 to 4.13.1 in /java/core (#389) (dependabot[bot])
  • [316db4f]: The framework of java-client for vineyard. (#388) (Tao He)
  • [e5065d2]: Adds new commands and arguments to vineyard-ctl (#387) (Rohan Gupta)
  • [15f9553]: Add a version string to vineyardd by gflags::SetVersionString. (#386) (Tao He)
  • [91b7ace]: Adds stat command to vineyard-ctl (#384) (Rohan Gupta)
  • [b58f91b]: Add a helper API for debugging. (#383) (Tao He)
  • [292500f]: Revisit the default visibility, and only "default" when necessary. (#382) (Tao He)
  • [0a64994]: Upgrade the nlohmann::json submodule, fixes #358. (Tao He)
  • [784c14b]: The generated setup.cfg worth to be checked-in. (#381) (Tao He)
  • [02b3832]: Added tensorflow builders and resolvers for datatypes tensor and dataframe (#379) (Chaitravi Chalke)
  • [d919f76]: Start the go sdk project. (#380) (Tao He)
  • [55c69e8]: RPC & IPC service depends on the meta & bulk only. (#377) (Tao He)
  • [13878e6]: Init rust sdk framework for client (#376) (Siyuan Zhang)
  • [f1b2eac]: Fixes the installation order in CI. (Tao He)

Contributors

zhanglei1949, linlih, and chaitravi-ce
Assets 16
Loading