[FEATURE REQUEST] Improve the `removed from original folder` option in Auto Uploads #4437

Aitorbp · 2024-07-16T07:14:10Z

Related Issues

App:

Add changelog files for the fixed issues in folder changelog/unreleased. More info here
Add feature to Release Notes in ReleaseNotesViewModel.kt creating a new ReleaseNote() with String resources (if required)

QA

Checks:

#4437 (comment)

Reports:

(1) Server down and retry does not remove the local copies with "remove" behaviour [FEATURE REQUEST] Improve the removed from original folder option in Auto Uploads #4437 (comment) [FIXED]

JuancaG05

Some changes requested here @Aitorbp

...main/java/com/owncloud/android/usecases/transfers/uploads/UploadFileFromContentUriUseCase.kt

owncloudApp/src/main/java/com/owncloud/android/workers/RemoveLocalFileWorker.kt

JuancaG05

One comment here @Aitorbp

...main/java/com/owncloud/android/usecases/transfers/uploads/UploadFileFromContentUriUseCase.kt

JuancaG05

LGTM

Aitorbp · 2024-07-30T08:14:07Z

The solution that has been attempted in this issue is the following:
We need the file deletion to be done after that file has been uploaded to the server.
For this we have a worker UploadFileFromContentUriWorker, which performs the upload task. So we must create another worker in the background that runs when UploadFileFromContentUriWorker has been successful. If it has not been successful, the file will not be deleted, avoiding data loss.
Technically, to do this, the workManager's then() function has been used. The content of this function is executed ONLY when the first worker has been successful.

jesmrec · 2024-07-30T08:22:32Z

jesmrec · 2024-07-30T08:56:49Z

(1) [FIXED]

i did the following steps:

Enable Auto uploads for pictures with removed from original folder behaviour
Take some pictures
Switch server off (it will make the uploads fail)
Wait till pictures are enqueued/failed
Switch server on
Retry failed uploads

Current: uploads are finally uploaded, but they are still in local
Expected: uploads are finally uploaded, but they are removed from local because the selected behaviour in Settings

NOTE: with connection loss, it works fine.

Pixel 2, Android11
41c2016a0

Aitorbp · 2024-08-06T07:50:54Z

(1)

i did the following steps:

Enable Auto uploads for pictures with removed from original folder behaviour

Take some pictures

Switch server off (it will make the uploads fail)

Wait till pictures are enqueued/failed

Switch server on

Retry failed uploads

Current: uploads are finally uploaded, but they are still in local Expected: uploads are finally uploaded, but they are removed from local because the selected behaviour in Settings

NOTE: with connection loss, it works fine.

Pixel 2, Android11 41c2016a0

To put this problem into context, first of all we need to know the different behaviors between how the worker works when there is no connection and when the server is off.

No connection: The worker is not executed, therefore the upload remains queued until the connection is recovered.

Server off: The worker is executed, but the upload ends up failing because the server is not operational and the upload gives a Connection error. We will need to click Retry to try the upload again.

The problem is that within the UploadFileFromContentUriWorker worker the following copyFileToLocalStorage() function is being executed. This method updates the localPath to tmp. Therefore when we do the Retry it will delete the file that we have in the tmp folder and not the file that is on the device.

As a solution, we have created a new column (sourcePath) in the Transfers table, where we save the path of the file on the device, since we have no way to recover it once the upload is ready to do a Retry. In this way, when we click on Retry, the file we have on the device will be deleted when the upload is finished.

mplazaspalacio

Some comments

owncloudApp/src/main/java/com/owncloud/android/workers/RemoveSourceFileWorker.kt

owncloudApp/src/main/java/com/owncloud/android/workers/UploadFileFromFileSystemWorker.kt

mplazaspalacio

LGTM

…tUriWorker

…some refactor

…ransfer table

…itory

…ryUploadFromSystemUseCase

jesmrec · 2024-08-19T09:09:47Z

Let's move this one. Now, with move behaviour, original files are removed just when the file is uploaded and submitted to the server, preventing data loss caused by non-recoverable failures.

Good one, improving our reliability 💯

Aitorbp self-assigned this Jul 16, 2024

Aitorbp linked an issue Jul 16, 2024 that may be closed by this pull request

[FEATURE REQUEST] Improve the removed from original folder option in Auto Uploads #4357

Closed

11 tasks

Aitorbp requested a review from JuancaG05 July 16, 2024 09:40

JuancaG05 changed the title ~~[FEATURE REQUEST] Improve the removed from original folder option in Auto Uploads~~ [FEATURE REQUEST] Improve the removed from original folder option in Auto Uploads Jul 19, 2024

JuancaG05 requested changes Jul 19, 2024

View reviewed changes

Aitorbp force-pushed the feature/improve_removed_from_original_folder_option_auto_upload branch 3 times, most recently from dc40855 to 84ec62b Compare July 22, 2024 10:25

JuancaG05 requested changes Jul 23, 2024

View reviewed changes

...main/java/com/owncloud/android/usecases/transfers/uploads/UploadFileFromContentUriUseCase.kt Outdated Show resolved Hide resolved

Aitorbp requested a review from JuancaG05 July 24, 2024 07:05

JuancaG05 requested changes Jul 29, 2024

View reviewed changes

...main/java/com/owncloud/android/usecases/transfers/uploads/UploadFileFromContentUriUseCase.kt Outdated Show resolved Hide resolved

Aitorbp force-pushed the feature/improve_removed_from_original_folder_option_auto_upload branch from 3b981b6 to 41c2016 Compare July 29, 2024 15:20

Aitorbp requested a review from JuancaG05 July 29, 2024 15:23

JuancaG05 approved these changes Jul 30, 2024

View reviewed changes

Aitorbp force-pushed the feature/improve_removed_from_original_folder_option_auto_upload branch 2 times, most recently from 8a346d8 to b18aeb7 Compare August 13, 2024 08:35

mplazaspalacio suggested changes Aug 14, 2024

View reviewed changes

Aitorbp requested a review from mplazaspalacio August 14, 2024 10:49

mplazaspalacio approved these changes Aug 14, 2024

View reviewed changes

Aitorbp and others added 7 commits August 14, 2024 14:39

feat: create new worker to remove local file after upload file

6bc6d9d

feat: added removeLocalFileWorker to UploadFileFromContentUriUseCase

de1b4ef

feat: remove code relate to removeLocalFile() in UploadFileFromConten…

8b1c648

…tUriWorker

feat: changed comment place to UploadFileFromContentUriUseCase

197a1be

chore: added calens file

7fba59c

docs: calens changelog updated

ea0becd

fix: moved behavior condition to UploadFileFromContentUriUseCase and …

8c60e07

…some refactor

Aitorbp added 11 commits August 14, 2024 14:40

fix: changed logic of worContinuation and added an else condition

2f74054

fix: removed beginWith when it is not necessary

22858e7

fix: auto-migration room database with the new column sourcePath in T…

b723794

…ransfer table

fix: added updateTransferSourcePath func to dao, datasource and repos…

02077f6

…itory

fix: added sourcePath parameter to uploadFileFromSystemUseCase in Ret…

e37c04f

…ryUploadFromSystemUseCase

fix: updated sourcePath before uploading the file

5b1ac77

fix: Removed source file if the retry upload is successful

48cd37c

fix: removed condition, the tmp file will be always removed

01db1a6

fix: added new func test in data source

b6bda56

fix: renamed to RemoveSourceFileWorker

39a8cbe

refactor: added comma

57a3476

Aitorbp force-pushed the feature/improve_removed_from_original_folder_option_auto_upload branch from 0f2c8b0 to 57a3476 Compare August 14, 2024 13:40

docs: calens changelog updated

048fd74

jesmrec added the Approved by qa label Aug 19, 2024

Aitorbp merged commit 254388d into master Aug 19, 2024
5 checks passed

Aitorbp deleted the feature/improve_removed_from_original_folder_option_auto_upload branch August 19, 2024 10:04

jesmrec mentioned this pull request Oct 24, 2024

[RELEASE] 4.4.1 #4501

Closed

22 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEATURE REQUEST] Improve the `removed from original folder` option in Auto Uploads #4437

[FEATURE REQUEST] Improve the `removed from original folder` option in Auto Uploads #4437

Aitorbp commented Jul 16, 2024 •

edited by jesmrec

Loading

JuancaG05 left a comment

JuancaG05 left a comment

JuancaG05 left a comment

Aitorbp commented Jul 30, 2024 •

edited by JuancaG05

Loading

jesmrec commented Jul 30, 2024 •

edited

Loading

jesmrec commented Jul 30, 2024 •

edited

Loading

Aitorbp commented Aug 6, 2024

(1)

mplazaspalacio left a comment

mplazaspalacio left a comment

jesmrec commented Aug 19, 2024

[FEATURE REQUEST] Improve the removed from original folder option in Auto Uploads #4437

[FEATURE REQUEST] Improve the removed from original folder option in Auto Uploads #4437

Conversation

Aitorbp commented Jul 16, 2024 • edited by jesmrec Loading

Related Issues

QA

JuancaG05 left a comment

Choose a reason for hiding this comment

JuancaG05 left a comment

Choose a reason for hiding this comment

JuancaG05 left a comment

Choose a reason for hiding this comment

Aitorbp commented Jul 30, 2024 • edited by JuancaG05 Loading

jesmrec commented Jul 30, 2024 • edited Loading

jesmrec commented Jul 30, 2024 • edited Loading

(1) [FIXED]

Aitorbp commented Aug 6, 2024

(1)

mplazaspalacio left a comment

Choose a reason for hiding this comment

mplazaspalacio left a comment

Choose a reason for hiding this comment

jesmrec commented Aug 19, 2024

[FEATURE REQUEST] Improve the `removed from original folder` option in Auto Uploads #4437

[FEATURE REQUEST] Improve the `removed from original folder` option in Auto Uploads #4437

Aitorbp commented Jul 16, 2024 •

edited by jesmrec

Loading

Aitorbp commented Jul 30, 2024 •

edited by JuancaG05

Loading

jesmrec commented Jul 30, 2024 •

edited

Loading

jesmrec commented Jul 30, 2024 •

edited

Loading