Android 11: Cannot access exported GPX files in Android/data

[dbrgn]

🐞 bug report

Description

I recently upgraded my Pixel 3a to Android 11. Yesterday I recorded my track during a race, and today noticed that I cannot access it anymore from the file browser app, or share it from my usual messaging app. The Android/data directory is empty.

The reason for this is that Android 11 changed some storage APIs (including scoped storage). See https://developer.android.com/about/versions/11/privacy/storage for details. With Android 11 being released, a lot more devices will upgrade over the next few days/weeks, rendering GPS recordings inaccessible. (This is a feature that I used almost every day.)

I see two ways to properly fix this: Either move the storage directory to something outside the scoped storage (e.g. ~/Downloads/osmand/) and/or offer a way to export GPX files to a user-defined directory.

(Workaround: I noticed that with some apps like Firefox I can still access the directory. This might be due to the Target API of the app accessing the data directory, I guess Firefox might set the requestLegacyExternalStorage flag.)

🔬 Minimal Reproduction

1. Using the "Trip recording" plugin, record a trace, then save it to storage.
2. Using any file browsing app, try to find the file on the file system (usually Android/data/net.osmand...). On Android 11, the directory is not visible.

🌍 Your Environment

OsmAnd Version: 3.7.4 ((F-Droid)

Device and Android/iOS version: Pixel 3a, Android 11

[vshcherb]

• requestLegacyExternalStorage sounds like an intermediate solution which will be anyway killed in Android 12. This never ending war about limiting file access between apps and it looks like since Android 11/12. None of the apps will be able to share and use common files except standard Share mechanism.

We support export File by File though obviously this is not a great mechanism and we will need to introduce Cloud approach to sync up data with 1 of the cloud data box

[trapicki]

Please do not overdo it with the Cloud stuff.

Many people consider their GPX tracks as personal information: it documents where you where at a certain time, and what you did there. Sharing a GPX track needs to be a deliberate, intentional action, not something that could happen accidentially or automatically.

In the end, "The Cloud" is computers of other people with less control over the data.

Please make sure there is always a non-cloud way of sharing also of multiple tracks, that is offered as default.

[dbrgn]

We support export File by File

Ah, really? How do I find that feature? I tried long-pressing on the GPX traces, but that didn't seem to work...

[vshcherb]

Wait a second, if you click Share button inside Track on top Right.
We also plan to modify "Upload track functionality", now you can upload only to OSM and we plan to have better integration with other cloud services

[dbrgn]

Ahh, I see! I forgot that this detail view exists. Maybe there could be a shortcut so that long-pressing on a track under "Configure map > GPS files" opens the detail view? (Not sure how easy it is to modify the list widget...)

[vshcherb]

We're going to improve for sure for 3.9, I hope so :-)

[sonora]

Side note: The much more powerful way and screen to access track-related functionality is really via Menu/MyPlaces/Tracks. Many users do not know hence #1435.

[scaidermern]

Side note: The much more powerful way and screen to access track-related functionality is really via Menu/MyPlaces/Tracks. Many users do not know hence #1435.

True. However there is still no functionality for sharing multiple tracks at once (#5558).

[g-work]

It is possible to change the data directory of osmand in the settings. When you change it to /storage/emulated/0/osmand (at least on my Pixel 5), you're able to access ist with any file manager.

[LifelessDead]

It is possible to change the data directory of osmand in the settings. When you change it to /storage/emulated/0/osmand, you're able to access ist with any file manager.

The proper workaround in android 11 is:
Osmand Settings -> Data Storage Folder -> Shared memory

The data ends up, accessible, in the folder mentioned above. Note: using "manually specified" no longer works.

[sonora]

See also my comment here #4645 (comment)

... if we have to redesign the storage concept to some degree anyway, it could be beneficial to separate the storage of all map/language/font "data" we allow downloading from OsmAnd servers and the individual data ("documents") which a user personally produces, like tracks, favorites, notes etc.

[vshcherb]

@sonora that's a tricky situation here, if we end up that we have 2 storages: public & private and Public will be very slow to use. We can redesign Public to have different structure and keep replication of some Private folders, but we don't need to redesign private (waste of time & effort).

Anyway, maps are the most heaviest part of storage. So if Public storage won't offer (for now it's only sdcard slow in Android 11) fast access, users will have no option to migrate everything to private. Later we can discuss whether we want to have Public storage for backups or not.

[sonora]

At this time many users are confused if/what they can currently do to mitigate the situation, like use or 'Shared memory' storage path or similar. Probably worth a page in our documentation, but I am currently at loss what to advise ...?

[vshcherb]

Storage needs to be explained here https://docs.osmand.net/en/main@latest/osmand/start-with/storage but I'm also not ready yet to describe all details. Sounds one of the most complicated part of documentation.

[vshcherb]

As I see on Samsung devices with Android 11, access is possible but it is very-very slow (indexing maps etc), it looks like it works but speed performance won't be improved.

[vinc17fr]

"Shared memory" is no longer possible. The only way to export a GPX track file now seems to share it with another application (e.g. "Save to File" to save it into "/storage/emulated/0"), but this can be done for only one track at a time.

[LifelessDead]

To export all tracks at once: OsmAnd -> Settings-> Backup & Restore -> Backup as file Select: My places -> Tracks It stores an Export_dd-mm-yy.osf file, which is just a .zip Open it and all tracks are inside!

[grozin]

I've just tried this. I get Export_22-01-22.osf and a choice of many applications (no Save to File). Choosing, say, Google Disk does not lead to sending this Export file to Google Disk.
Still no way to get my tracks out of this black hole.

[LifelessDead]

Try renaming .osf to .zip first

[nhan000]

I can't extract the zip file after renaming .osf to .zip.

[mariush444]

Use totalcommander, mixplorer, or soli explorer, any other file manager than embeded in android

[faelys]

Hi, I'm sorry for bothering you here, but extracting GPX from osmand+ into my desktop is a very important feature for me, and I can't even make any workaround work. I have an up-to-date Samsung XCover Pro, with Android 11, OsmAnd+ 4.1.11:

• I can select storage on Internal app memory, External storage 1 (on the phone), External storage 2 (on micro SD card), Multiuser storage 1 (on the phone), or Multiuser storage 2 ( on micro SD card), but none of these options seem to make anything available to another app
• Choosing a "manually specified" path asks me whether to move OsmAnd data files to the new destination,
  • choosing "Don't move" fails with "Maps could not be created in the specified directory"
  • chooding to move them fails with "I/O error: /tiles/OsmAnd (online tiles)/.metainfo: open failed: EACCESS (Permission denied)
• The permissions for OsmAnd+ are "Location: only while app is in use" and "Storage: Allow access to media only", but Samsung settings don't seem to allow me to grant a higher level (maybe the app needs to ask for it first?)
• Using the "Share" entry in a specific track option gives me a list of apps, but there is no "Save to File" choice (should I blame Samsung for that?). Among the available apps, I tried WhatsApp, ProtonMail, Mattermost, Signal, and Discord, and they all open a prompt for a message along with the attachment, except there is no attachment. I also tried "FE File Explorer" which asked me a destination directory, but it remained empty without displaying any error message. I have a few other Samsung-specific options that I cannot test because I don't have another Samsung device.
• The full backup as described in the previous comment opens the same "Share" menu, with similar results

To be completely honest, sharing via Bluetooth seems to work with another Android device that I borrowed, but I wasn't planning to acquire another android device myself.

Would you have any suggestion on what else I could try? Maybe another workaround that I missed? Or some other app to check the "share" feature with? Or maybe another open-source GPX recorder android app?

Thanks in advance for your help

[grozin]

Does not work :-( Mainly I use Linux (Gentoo). Have installed mtpfs. After mounting my phone (attached via usb cable) the mountpoint is empty. lsusb shows an mtp device all right.
I have also Windows 10 as a second boot. After plugging my phone the file explorer sees it, but the folder is empty.

[LifelessDead]

This could be a workaround for you. You can get all tracks in a zip in an accessible location.

To export all tracks at once:

OsmAnd -> Settings-> Backup & Restore -> Backup as file

Select: My places -> Tracks

It stores an Export_dd-mm-yy.osf file, which is just a .zip
Open it and all tracks are inside!

[grozin]

I finally found a workaround which works: Osmand Settings -> Data Storage Folder -> manually specified, choose /storage/<sd_card>/Android/media/net.osmand.plus, move all files there. Now tracks are accessible for file managers.

[aknrdureegaesr]

Instead discussing an accessible place for all of Osmand's storage, I think it would tremendously help to simply enable the access to the few files people typically need. Why not simply write just those to an accessible location? In other words: Fix #4645 and a lot of people will be happy (including myself).

[sybrenstuvel]

Much this! I would love to be able to sync my favourites & recorded tracks via SyncThing to other devices (some also running Osmand).

[sonora]

@faelys #13254 (comment)