Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding new documents to mydig #255

Open
saggu opened this issue Aug 17, 2018 · 0 comments
Open

Adding new documents to mydig #255

saggu opened this issue Aug 17, 2018 · 0 comments

Comments

@saggu
Copy link
Member

saggu commented Aug 17, 2018

  1. When a user uploads a new file, json lines or csv or any format. Catalog the documents in the hbase table <project_name>_catalog as described in Creating a new project #253 .
    Initially the date_processed is empty and status is 0.

  2. If the uploaded file is a json lines file, catalog each individual json object, if its a csv or tsv or excel etc, create a wrapping json object and catalog the wrapping json object.

  3. Update the table dataset_view as described in Starting up myDIG for the very first time #254 . This table will only updated when documents are either uploaded or deleted from myDIG.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant