We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A clear and concise description of what the bug is.
To help us to reproduce this bug, please provide information below:
# download dataset wget https://atp-modelzoo.oss-cn-hangzhou.aliyuncs.com/release/datasets/WuDaoCorpus2.0_base_sample.tgz tar zxvf WuDaoCorpus2.0_base_sample.tgz
import xorbits.pandas as pd import xorbits.datasets as xdatasets from datasets import load_dataset import xorbits xorbits.init() data = load_dataset("./WuDaoCorpus2.0_base_sample") print(data) df = pd.DataFrame(pd.DataFrame(data['train']), chunk_size=1000) print(df.shape) print(df.dtypes) print(df.head()) from xorbits.experimental import dedup res = dedup(df, col="content") print(10*"---") print(res)
A clear and concise description of what you expected to happen.
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered:
add a line is ok: xorbits.shutdown()
Sorry, something went wrong.
So, it was raised when process exit?
No branches or pull requests
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
To help us to reproduce this bug, please provide information below:
# download dataset wget https://atp-modelzoo.oss-cn-hangzhou.aliyuncs.com/release/datasets/WuDaoCorpus2.0_base_sample.tgz tar zxvf WuDaoCorpus2.0_base_sample.tgz
Expected behavior
A clear and concise description of what you expected to happen.
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: