Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Headless/DuckDB Data Explorer tab does not update if underlying file is changed #5551

Closed
juliasilge opened this issue Nov 27, 2024 · 2 comments
Closed

Headless/DuckDB Data Explorer tab does not update if underlying file is changed #5551

juliasilge opened this issue Nov 27, 2024 · 2 comments
Assignees
@wesm
Labels
area: data explorer Issues related to Data Explorer category. enhancement New feature or request support
Milestone
2025.01.0 Pre-Rel...

Comments

@juliasilge
Copy link
Contributor

Starting the discussion here to ascertain if this is a bug or a skill issue.

Using latest positron:

Positron Version: 2024.12.0 (Universal) build 80
Code - OSS Version: 1.93.0
Commit: c3c489641477362638b2a80d94355a86a63eeb21
Date: 2024-11-26T02:46:47.938Z
Electron: 30.4.0
Chromium: 124.0.6367.243
Node.js: 20.15.1
V8: 12.4.254.20-electron.0
OS: Darwin arm64 23.6.0

Using R 4.4.1

Issue

  • Open positron
  • Create a parquet file library(arrow); write_parquet(iris, "iris.parquet")
  • Open iris.parquet in viewer ✅
  • Overwrite a parquet file write_parquet(iris[,2:4], "iris.parquet")
  • Open iris.parquet in viewer ❌ <- does not update
  • Confirm correct parquet actually got written names(read_parquet("iris.parquet")) ✅

So it seems like somewhere the parquet viewer in not updating. Also this is not an R thing. Also replicated with pandas.

Update - replicated with a csv file as well

write.csv(iris, "iris.csv")
write.csv(iris[,2:4], "iris.csv")
names(read.csv("iris.csv"))

Originally posted by @boshek in #5548
@juliasilge juliasilge added area: data explorer Issues related to Data Explorer category. enhancement New feature or request support labels Nov 27, 2024
@wesm wesm self-assigned this Dec 2, 2024
@wesm
Copy link
Contributor

wesm commented Dec 2, 2024

I agree that this should work -- I will look at how to watch the underlying file for changes and fire the data/schema change events to get live updates.

@wesm wesm mentioned this issue Dec 6, 2024
Merged
@wesm wesm added this to the 2025.01.0 Pre-Release milestone Dec 6, 2024
wesm added a commit that referenced this issue Dec 9, 2024
@wesm
Data Explorer: Watch for file changes and trigger live updates in hea…
4d18d86 
…dless/DuckDB backend (#5651)

Addresses #5551.

This uses `fs.watchFile` with a 1-second polling interval (which seemed
more robust in testing than `fs.watch`) to respond to changes to files
on disk. I ran into some weird file caching issue in duckdb-wasm, so I
had to work around it by registering small files in duckdb-wasm's
tempfs, and then loading these into a database table for querying.

Live updates in action:

https://github.com/user-attachments/assets/c22b6fd7-cdce-4e06-a8d7-7e888124ef35
@testlabauto
Copy link
Contributor

Verified Fixed

Positron Version(s) : 2025.01.0-104
OS Version          : OSX

Test scenario(s)

Verified with TSV file

Link(s) to TestRail test cases run or created:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wesm wesm

Labels
area: data explorer Issues related to Data Explorer category. enhancement New feature or request support
Projects
None yet
Milestone
2025.01.0 Pre-Release
Development

No branches or pull requests
3 participants
@wesm @juliasilge @testlabauto