Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding intent causes the app in release builds to throw realmexception #1355

Closed
milindgoel15 opened this issue Jul 15, 2023 · 51 comments
Closed

Comments

@milindgoel15
Copy link

milindgoel15 commented Jul 15, 2023

What happened?

I am using a local configuration-based realm and i want to create a scheduled task and add it to my calendar app. For this, i am using android_intent_plus to create an intent. But whenever i add this plugin to a project and build a release apk. The app throws a realm exception.

If I remove the plugin from the project, do flutter clean and build a release apk. The app works fine. is it a plugin issue?

I tried adding the plugin package name to Proguard and tested without any proguard or obfuscation.

Repro steps

  1. Added android intent plus
  2. Built release apk and ran apk on device
  3. the app threw realmexception and whole blank white screen.

Version

3.10.5

What Atlas Services are you using?

Local Database only

What type of application is this?

Flutter Application

Client OS and version

Android 13

Code snippets

No response

Stacktrace of the exception/crash you're getting

realmexception: error opening realm at path ./default.realm. error code: 1018 . message: failed to open file at path './default.realm.lock': read-only file system flutter

Relevant log output

No response

@desistefanova
Copy link
Contributor

Hi @milindgoel15!
How about changing the path where the Realm is stored? Have you already tried that?
You can set a different location using path parameter of Configuration.local constructor. There is also another approach described here: https://www.mongodb.com/docs/realm/sdk/flutter/realm-database/configure-and-open/#customize-default-configuration
You can also try to set fifoFilesFallbackPath
https://www.mongodb.com/docs/realm/sdk/flutter/realm-database/configure-and-open/#set-custom-fifo-special-files

@milindgoel15
Copy link
Author

No, I have not tried it yet. Will do

Do you happen to know or any idea why the realm is getting locked after adding android intent plugin to the project?

@milindgoel15
Copy link
Author

milindgoel15 commented Jul 15, 2023

Setting a different path is much more difficult with Android 13. Because google doesn't let apps to access external storage. Is there any other workaround to this issue?

Edit 1: Plus all the existing data of users won't be there in the new directory location.

Edit 2: Is there any option to export the realm db file and import it later for restoring data?

@desistefanova
Copy link
Contributor

@milindgoel15, unfortunately i couldn't reproduce this error. It works for me. Maybe what I do is different from your code. It will be good if you can share some parts of your code. What do you launch with android_intent_plus?

If I understand correctly, you have already the realm file on your device and it is locked by some reason.
In order to keep your user data you can copy the file to another location.

  final String realmPath = "${Configuration.defaultStoragePath}\realm\\";
  if (!Directory(realmPath).existsSync()) {
    Directory(realmPath).createSync();
    File("${Configuration.defaultRealmPath}\\${Configuration.defaultRealmName}").copySync(realmPath);
    Configuration.defaultRealmPath = "$realmPath\\${Configuration.defaultRealmName}";
  }
  final realm = Realm(Configuration.local([Task.schema]));

You can use this as an workaround, but it will be good to know why the files are locked.
Please let me know, if this helped.

@milindgoel15
Copy link
Author

milindgoel15 commented Jul 16, 2023

@milindgoel15, unfortunately i couldn't reproduce this error. It works for me. Maybe what I do is different from your code. It will be good if you can share some parts of your code. What do you launch with android_intent_plus?

Yes, sure.
Regarding the intent package, whenever i add the package to pubspec.yaml and do a release build, Then only the realm gets locked and throws that realm exception( Even if i don't write any code related to intent). I also tried the add_2_calendar package and its the same exception. So idk whats locking the realm here.

This is my realm service and i am using it with riverpod provider to watch/read changes.

class RealmServices {
  late Realm realm;

  RealmServices() {
    realm = getRealm();
  }

  Realm getRealm() {
    final Configuration config = Configuration.local(
      [
        TodoModel.schema,
        NoteModel.schema,
        FutureModel.schema,
      ],
      schemaVersion: 8,
    );

    return Realm(config);
  }
}
final realmProvider = Provider((ref) => RealmServices());

If I understand correctly, you have already the realm file on your device and it is locked by some reason.

It gets locked if the release build is made with the intent/add_2_calendar package. Otherwise works.

In order to keep your user data you can copy the file to another location.

About this, by default, the file is stored in data/data location which cant be accessed by a normal user. So copying is not possible. I checked the path and its in data/data/packagename/files/default.realm.

  final String realmPath = "${Configuration.defaultStoragePath}\realm\\";
  if (!Directory(realmPath).existsSync()) {
    Directory(realmPath).createSync();
    File("${Configuration.defaultRealmPath}\\${Configuration.defaultRealmName}").copySync(realmPath);
    Configuration.defaultRealmPath = "$realmPath\\${Configuration.defaultRealmName}";
  }
  final realm = Realm(Configuration.local([Task.schema]));

You can use this as an workaround, but it will be good to know why the files are locked. Please let me know, if this helped.

Just tested this and it gives I/flutter (16058): FileSystemException: Creation failed, path = '.

However, i also tried running the production debug variant and it gave this exception:
https://pastebin.com/3LGXbfCY
But doing hot restart reloads the app fine. This is probably not related to the realm issue but anyway/

Also, just to note, I tried changing the default path to storage/emulated/0/android/data/packagename/files/ and in the release build with the intent package added, it worked. But yes, changing the path has its cons, especially in case of android due to scoped storage issue.

So maybe the above code could help figure out why it's locking the realm in the default path?
Could it be that these intent packages work on the condition that the data is stored in local/internal storage instead of a system partition? I hope this is not the case.

@milindgoel15
Copy link
Author

milindgoel15 commented Jul 16, 2023

Some updates to the previous comment:

For stack trace, i tried:
https://stackoverflow.com/questions/73591769/got-a-stack-frame-from-packagestack-trace-where-a-vm-or-web-frame-was-expected

and after that, it gave this error logs:

The following PathNotFoundException was thrown building Home(dirty, dependencies: [UncontrolledProviderScope], state: _HomeState#9a51d):
ealm\', path = '/data/data/packagename/files/default.realm\default.realm' (OS Error: No such file or directory, errno = 2)

This is after i copied the code you provided. But again hot restarting the app works.

@milindgoel15
Copy link
Author

milindgoel15 commented Jul 16, 2023

So I was trying out copying the old data into the new location. I modified the code you provided for copying data as i tested using the android studio to look into the system partition. I found that the realm doesn't use this path: Configuration.defaultStoragePath/realm but uses the /files path.

but it only works in certain conditions. more explained below:

     String appDir = "/storage/emulated/0/Android/data/com.mgprojects.todoify.dev/files/database.realm";

this is my new dir created using storage permission

    final String storagePath = Configuration.defaultStoragePath; // old path

    if (Directory(storagePath).existsSync()) { // this has to run to successfully copy the old data into new location
      Directory(storagePath).createSync();
      File("$storagePath/default.realm").copySync(appDir);
      Configuration.defaultRealmPath = appDir;
      print('old data copied');
    }

when data is copied, when we try to add new todo or anything, it creates an error:

    Exception has occurred.
RealmException (RealmException: Error opening realm at path /storage/emulated/0/Android/data/com.mgprojects.todoify.dev/files/database.realm. Error code: 1021 . Message: Failed to memory buffer:Invalid top array size (ref: 2704, array size: 0) file size: 0, read lock size: some(4096), read lock version: some(4))

To prevent error, we have to reverse the check condition to:

if (!Directory(storagePath).existsSync()) { // now it wont run, making the app work to store new data
      Directory(storagePath).createSync();
      File("$storagePath/default.realm").copySync(appDir);
      Configuration.defaultRealmPath = appDir;
      print('old data copied');
    }

we can use the new directory to store new data by providing it in path

    final Configuration config = Configuration.local(
      [
        TodoModel.schema,
        NoteModel.schema,
        FutureModel.schema,
      ],
      schemaVersion: 8,
      path: appDir,
    );
    return Realm(config);
  }

Do you know how to modify this so as to make the if condition only run once when it has to copy old data?

also, i am getting weird log in console after moving data to new location:

W/1.ui    (17856): type=1400 audit(0.0:48425): avc: denied { create } for name="access_control.new_commit.cv" scontext=u:r:untrusted_app:s0:c86,c257,c512,c768 tcontext=u:object_r:media_rw_data_file:s0:c86,c257,c512,c768 tclass=fifo_file permissive=0 app=com.mgprojects.todoify.dev
W/1.ui    (17856): type=1400 audit(0.0:48426): avc: denied { create } for name="access_control.pick_writer.cv" scontext=u:r:untrusted_app:s0:c86,c257,c512,c768 tcontext=u:object_r:media_rw_data_file:s0:c86,c257,c512,c768 tclass=fifo_file permissive=0 app=com.mgprojects.todoify.dev
W/1.ui    (17856): type=1400 audit(0.0:48427): avc: denied { create } for name="database.realm.note" scontext=u:r:untrusted_app:s0:c86,c257,c512,c768 tcontext=u:object_r:media_rw_data_file:s0:c86,c257,c512,c768 tclass=fifo_file permissive=0 app=com.mgprojects.todoify.dev

Some updates to this:

I was able to add a conditional check for the copy db condition. While it worked fine in the debug variant. The release build was still not able to open the realm even if the new location is in android/data when the intent plugins are added to pubspec.

@milindgoel15
Copy link
Author

Some new tests:
i found out the cause of the issue. In the android/build.gradle file, the code where subProjects are written. If you combine the 2 lines into one single subProjects block, the app will start throwing realm exception that it cant open realm. But reversing them back to separate blocks as they were when you create a fresh project fixes it.

I had them combined since the first build until i added this calendar/intent plugins which starts breaking the build. To test this in your end, go to the same file and combine the 2 blocks:

From:

subprojects {
    project.buildDir = "${rootProject.buildDir}/${project.name}"
}
subprojects {
    project.evaluationDependsOn(':app')
}

to

subprojects {
    project.buildDir = "${rootProject.buildDir}/${project.name}"
    project.evaluationDependsOn(':app')
}

and the app will stop working in the release build. You will also notice the build generated won't be in the usual location, i.e, android/build/outputs and build/app/outputs/flutter-apk.

Because normally, they are generated in build folders only, not in android.

@desistefanova
Copy link
Contributor

Hi @milindgoel15!
Thank you very much about this investigation!
Combining both rows under one subprojects really locked the realm file. The good thing is that it is unlocked if you separate the lines again and restart the app. So, you don't have to copy the realm file, right?
I'm glad to hear that it works for you. We will discuss this in the team, but I don't think we can do anything here to fix that. So I think we can close this issue.

@milindgoel15
Copy link
Author

Yes, we don't need to copy the realm file now.

Although I was wondering if it is possible to implement a backup and restore function? Will the same method of copying the db file be used for such a purpose? Realm doesn't have any built-in function ig.

@desistefanova
Copy link
Contributor

You can use realm.writeCopy(newConfig) to backup and restore. It makes a compact copy of the realm to the path of the new configuration. It has more benefits for the cloud synced realms, but could be used also for local realms. You can check all the test cases here.

originalRealm.writeCopy(configCopy);

@sync-by-unito sync-by-unito bot closed this as completed Jul 18, 2023
@sync-by-unito
Copy link

sync-by-unito bot commented Jul 18, 2023

➤ desistefanova commented:

I'm closing this issue, because the user had found the reason why it happens and it is not something that should be fixed in the realm package.

@milindgoel15
Copy link
Author

You can use realm.writeCopy(newConfig) to backup and restore. It makes a compact copy of the realm to the path of the new configuration. It has more benefits for the cloud synced realms, but could be used also for local realms. You can check all the test cases here.

originalRealm.writeCopy(configCopy);

Alright thanks.

@Shreedhar73
Copy link

Shreedhar73 commented Sep 5, 2023

@milindgoel15 @desistefanova

Hii.

I faced the similar issue few weeks back, and the above solution , separating the subprojects did fixed the issue.

And all of sudden, today same thing is happening, the app is working fine on debug mode, but on release mode it is throwing same Exception.

image

Flutter Doctor output :

[√] Flutter (Channel stable, 3.10.5, on Microsoft Windows [Version 10.0.22622.586], locale en-US)
• Flutter version 3.10.5 on channel stable at C:\Users\Shreedhar\fvm\versions\3.10.5
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 796c8ef792 (3 months ago), 2023-06-13 15:51:02 -0700
• Engine revision 45f6e00911
• Dart version 3.0.5
• DevTools version 2.23.1

[√] Windows Version (Installed version of Windows is version 10 or higher)

[√] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at C:\Users\Shreedhar\AppData\Local\Android\sdk
    • Platform android-33, build-tools 30.0.3
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 11.0.12+7-b1504.28-7817840)
    • All Android licenses accepted.

[X] Chrome - develop for the web (Cannot find Chrome executable at .\Google\Chrome\Application\chrome.exe)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[X] Visual Studio - develop for Windows
    X Visual Studio not installed; this is necessary for Windows development.
      Download at https://visualstudio.microsoft.com/downloads/.
      Please install the "Desktop development with C++" workload, including all of its default components

[√] Android Studio (version 2021.2)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+7-b1504.28-7817840)

[√] VS Code (version 1.81.1)
    • VS Code at C:\Users\Shreedhar\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.70.0

[√] Connected device (3 available)
    • M2101K6G (mobile) • 8e993f7c • android-arm64  • Android 13 (API 33)
    • Windows (desktop) • windows  • windows-x64    • Microsoft Windows [Version 10.0.22622.586]
    • Edge (web)        • edge     • web-javascript • Microsoft Edge 116.0.1938.69

[√] Network resources
    • All expected network resources are available.

@milindgoel15
Copy link
Author

milindgoel15 commented Sep 5, 2023

And all of sudden, today same thing is happening, the app is working fine on debug mode, but on release mode it is throwing same Exception.

Your current error in the screenshot seems to be different from what this issue was for.
However, the error does simply mean that the app fails to create a directory that is not writable.

Try to use the path provider and usage of storage permission. Android doesn't let you write to many folders/directory because of scoped storage so you have to choose your folder path accordingly.

@Shreedhar73
Copy link

@milindgoel15 But I am not trying to write in multiple folders/directory, . I have not setup my app to write in specific directory, so it should use the default location. And it was working till yesterday.

@milindgoel15
Copy link
Author

According to your error, it's trying to create a directory named ./mongodb-realm which is not some default location folder.

@Shreedhar73
Copy link

@milindgoel15 I have not written any code to manually create directory or to create files at any specific directory. I even have to specified the base file path.

image

@milindgoel15
Copy link
Author

check your realm config.

@Shreedhar73
Copy link

image
Realm Config is fine, since it is working fine in debug mode.

@nielsenko
Copy link
Contributor

nielsenko commented Sep 6, 2023

@Shreedhar73 What platform are you running the app on? (OS & version)? Could you tell me the content of config.path after the config is created?

How does your realm import statement look?

Also, in the future, please open new issues pointing to old ones, if you believe you experience a similar issue, instead of continuing on an already closed issue. This works better for our workflow.

@Shreedhar73
Copy link

Shreedhar73 commented Sep 6, 2023

@nielsenko

Android 13,
My Import Statement is as 'import 'package:realm/realm.dart' as realm'

@nielsenko
Copy link
Contributor

@Shreedhar73 And the content of config.path?

@Shreedhar73
Copy link

The app doesnt even reach to the point, where I have called Configuration.FlexibleSync
So I cannot get config.path

@Shreedhar73
Copy link

But if I close the app and restart the app it works fine and the config path here is : /data/data/com.yipl.onlyever/files/mongodb-realm/appID/64af9051276e5ab5f17044b9/default.realm ..

@nielsenko
Copy link
Contributor

@Shreedhar73 What is appConfiguration.baseFilePath before constructing the App?

@nielsenko
Copy link
Contributor

If closing the app, and restarting fixed it, can you then reproduce the issue anymore?

@Shreedhar73
Copy link

@nielsenko yes next time I do clean installation of the app, same issue occurs.

@nielsenko
Copy link
Contributor

Does not compute? 😕 Do you have a minimal reproduction, you can send me?

@Shreedhar73
Copy link

@nielsenko I have not provided the baseFilePath in the appConfiguration but when I print the 'Configuration.defaultRealmPath'
I get , 'default.realm'. and even if I provide the baseFilePath in appConfiguration as Configuration.defaultRealmPath then i get

FileSystemException: Creation failed, path = 'default.realm' (OS Error: Read-only file system, errno = 30)

@nielsenko
Copy link
Contributor

@Shreedhar73 Yes, something goes wrong when determining the default.

Are you running on an emulator, or a real device?

@Shreedhar73
Copy link

@nielsenko Its on real device. Redmi note 10 pro ( Android 13)

@nielsenko
Copy link
Contributor

Thx, as a work-around you could try taking a dependency on path_provider and specify a suitable baseFilePath when constructing the realm.App.

@Shreedhar73
Copy link

Shreedhar73 commented Sep 6, 2023

@nielsenko That worked, although I am not sure if it is the correct way.
Thank you for you time.

@nielsenko
Copy link
Contributor

@Shreedhar73 I'm trying to reproduce on my end. I don't have a Redmi note 10 pro at hand, and I cannot make it fail on my end. Can I ask you to run this project on your end?

Basically just:
https://github.com/nielsenko/default_path_issue/blob/575af1b928f43cff1b6d7c7a6630d05eb0f05403/lib/main.dart#L1-L46

Hmm, not sure why the code-link preview is not working, anyway it is basically just trying to open a sync'ed realm:

import 'package:flutter/material.dart';
import 'package:realm/realm.dart';

part 'main.g.dart';

@RealmModel()
class _Stuff {
  @PrimaryKey()
  @MapTo('_id')
  late ObjectId id;
}

final app = App(AppConfiguration('evil-app-mggjg'));
final userProvider = () async {
  return app.currentUser ?? await app.logIn(Credentials.anonymous());
}();

final realmProvider = () async {
  return Realm(Configuration.flexibleSync(
    await userProvider,
    [Stuff.schema],
  ));
}();

Future<void> main() async {
  final realm = await realmProvider;
  runApp(MyApp(realm: realm));
}

class MyApp extends StatelessWidget {
  final Realm realm;
  const MyApp({super.key, required this.realm});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Material(
        child: Center(
          child: Text(
            'successfully opened realm at:\n${realm.config.path}}',
          ),
        ),
      ),
    );
  }
}

@Shreedhar73
Copy link

@nielsenko Sure just give me few minutes

@Shreedhar73
Copy link

@nielsenko It is working fine. Seems like dependency in my projects are creating the issue.

@nielsenko
Copy link
Contributor

@Shreedhar73 Could you show your pubspec.yaml?

@Shreedhar73
Copy link

Shreedhar73 commented Sep 6, 2023

  bloc: ^8.1.1
  realm: ^1.0.3
  flutter:
    sdk: flutter
  flutter_bloc: ^8.1.2
  flutter_localizations:
    sdk: flutter
  intl: ^0.18.0
  dio: ^5.0.3
  get_it: ^7.2.0
  google_fonts: ^4.0.4
  flutter_html: ^3.0.0-beta.2
  url_launcher: ^6.1.10
  just_audio: ^0.9.32
  
  auto_route: ^5.0.0
  equatable: ^2.0.5
  dartz: ^0.10.1
  shared_preferences: ^2.0.20
  fluttertoast: ^8.2.1
  very_good_infinite_list: ^0.7.0
  youtube_explode_dart: ^2.0.1
  flutter_inappwebview: ^5.7.2+3
  cached_network_image: ^3.2.3
  google_sign_in: ^6.1.2
  flutter_svg: ^2.0.0+1
  firebase_auth: ^4.6.2
  firebase_core: ^2.13.1
  firebase_crashlytics: ^3.3.1
  scroll_to_index: ^3.0.1
  modal_bottom_sheet: ^3.0.0-pre
  visibility_detector: ^0.4.0+2
  image_picker: ^1.0.2
  expandable: 
  collection: 
  video_player: 
  replay_bloc: ^0.2.4
  json_serializable: ^6.6.2
  workmanager: ^0.5.1
  percent_indicator: ^4.2.3
  rounded_background_text: ^0.3.0
  amplify_storage_s3: ^1.0.1
  amplify_flutter: ^1.0.1
  amplify_auth_cognito: ^1.0.1
  flutter_screenutil: ^5.7.0
  google_sign_in_macos:
    path: google_sign_in_macos
  google_sign_in_platform_interface: ^2.2.0
  http:
  flutter_markdown: ^0.6.17+1
  markdown_widget: ^2.2.0
  flutter_math_fork: ^0.7.1
  file_selector: ^1.0.0
  path_provider: ^2.1.1
  desktop_drop: ^0.4.3
  fvp: ^0.4.0
  wakelock_plus: ^1.1.1
  file_picker: ^5.5.0

@nielsenko

@Shreedhar73
Copy link

Shreedhar73 commented Sep 6, 2023

I added all of those packages on the newer project with your code and then the issue is reproducing.
I will try to find if/with which one of them , the issue is occuring

@nielsenko
Copy link
Contributor

Hmm.. I had to update minSdkVersion to 24 on Android to build, but I still don't have any issue on my end (Running on Pixel 7 Pro - Android 14)

@Shreedhar73
Copy link

Shreedhar73 commented Sep 6, 2023

With all those packages ?

Removing fvp works on my side .
I am using fvp for video player on macos

@nielsenko

@nielsenko
Copy link
Contributor

nielsenko commented Sep 6, 2023

Yes, I added them all. But maybe my pubspec.lock ended up in a different state.

I also tried on a Pixel 6 Pro with Android 13, still I see no issue.

What version of fvp and realm was in your pubspec.lock

@Shreedhar73
Copy link

fvp : 0.4.0
realm : 1.4.0

@nielsenko
Copy link
Contributor

@Shreedhar73 What version of flutter are you running currently?

@Shreedhar73
Copy link

@nielsenko 3.10.5

@Shreedhar73
Copy link

@nielsenko Yes so, I think the issue is solved (atleast for my usecase)

Since the OG 'VideoPlayer' doesnt support mac os platform we are using the 'FVP' ,
image.
So the registerWith method for me looks as :
registerWith( options: { 'platforms': ['macos'] }, );

Currently, the registerWith method was being called after the realmAppConfiguration. (on main) Calling "registerWith" before setting up app configuration fixes my issue.

( but that doesnt gives anything why the issue occured on first place and I have to test it on mac application too, I will let you know tommorow how it goes.)

Thanks Again

@nielsenko
Copy link
Contributor

nielsenko commented Sep 6, 2023

Still cannot make it fail on my end.

Tried calling registerWith after realm app configuration. I ensure to use release mode, and always a fresh install, I can not make it fail.

I have pushed a commit with my latest additions to the default_path_issue repo.

Can you make it fail like this on your Redmi?

@Shreedhar73
Copy link

Shreedhar73 commented Sep 6, 2023

@nielsenko with only realm and fvp on pubsec, your code is working fine on my end.

@Shreedhar73
Copy link

@nielsenko

I can replicate the issue (8/10 times) when desktop_drop: ^0.4.3 is on pubspec with fvp and realm.

@nielsenko
Copy link
Contributor

nielsenko commented Sep 6, 2023

Still no success (or rather failure) on my end. I have added the desktop_drop package in the repo. If you have any other suggestions to changes please open a PR on https://github.com/nielsenko/default_path_issue

Right now I have to move on with other work. Happy we got it working for you, but annoyed that I don't understand why it fails in the first place.

Good luck with your project

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 16, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants