Open Camera and Scoped Storage

With Open Camera 1.48.3, I've switched to supporting Android 10's scoped storage.

Readers of Android news sites may already know how this has been a pain for developers. Most of the coverage has been for the issue surrounding file managers, so I wanted to share what it meant for Open Camera.

Firstly, a review of the ways that applications can access files, under scoped storage:

• The File API. Yes that's right - whilst the File API can no longer be used for many purposes under scoped storage, it's still required to use the File API when reading/writing to an application's own folders.

• The Mediastore API. This allows saving files without needing any user permission, but is restricted to certain filetypes (images etc), and certain folders (so photos must be in DCIM/ or Pictures/ for example). An application can also read such files, but unless it has the relevant permission, it can only read its own files (and even then, only if they were also saved with the Mediastore API).

• The Storage Access Framework API. This allows reading and writing any file in any folder (subject to system restrictions), but the user must give permission to read or write to a file/folder using the operating system's dialog.

My first criticism is that even if you were writing a brand new scoped-storage-supporting application from scratch, you have to support not one, not two, but three APIs. I mean, if Google really wanted to come up with a new API that they thought was better than the tried and trusted cross-platform standard APIs for files, surely it should at least cover all bases?

But now we have a situation in Open Camera where, by default, an image is saved using Mediastore API (Storage Access Framework isn't suitable for default behaviour, because the user would have to choose a save location using the Storage Access Framework dialog). But if they want to save to a folder that Mediastore doesn't support, Open Camera has to use the Storage Access Framework API. Whilst one can refactor code to support multiple APIs, it's not ideal.

The good news for me is that I was already doing this anyway - we already had this API fragmentation before scoped storage, as Storage Access Framework was the only way to save to SD cards. So in practice, the work required involved switching from the File API usages to the Mediastore API.

In summary, the same action "saving a file" could require an application to implement in up to three different ways, depending on the circumstances such as file location.

Some particular issues I faced were:

• The Mediastore is a database, what's responsible when you look at your images/videos in a gallery app - it stores what you see icons for, their thumbnails and so on. It's why a gallery app won't just show every file in a folder, because unlike File Managers, they are using the Mediastore API. If you use the File API to save a file (pre-scoped storage), or use the Storage Access Framework, the OS will decide whether a thumbnail should be generated for the Mediastore database or not. But if you save using the Mediastore API, a thumbnail will always be generated - because the API to save a file is the same one as inserting into the Mediastore API. Why is this a problem? Because of Open Camera's "Video subtitles" feature. This saves ".SRT" files alongside the video files, but now the .SRT files show up in gallery apps as unplayable files! In my opinion this shows up a fundamental problem in using the Mediastore API (an API that's meant to be for the mediastore database) to save files. There is an assumption of a one-to-one correspondance between "file on disk" and "entry in mediastore database", which is not true here. The only workaround is for users to use Storage Access Framework option. I also wonder how scoped storage will work for video players that support .SRT files - if they use the Mediastore API to read videos, they won't be able to use the File API to load a corresponding .SRT file. They could use Storage Access Framework, but that needs users to give permission to any folder containing videos they want to view. I raised a feature request with Google for this issue.

• In order to display a thumbnail of the most recent photo/video taken, I use the Mediastore API. So in theory this should still work. With Android 10, I no longer request permission to read storage (which is best practice, as it's the whole point of scoped storage), which should mean I can still see the media files "owned" by Open Camera. A problem is that when images are saved using Storage Access Framework, I can't see them in the Mediastore query. I reported this as a possible bug to Google. Google closed this as "Intended Behaviour". In my opinion, the new file ownership/attribution model is fundamentally broken, if saving using Storage Access Framework means the files aren't attributed to the application. I've worked around this by instead using the Storage Access Framework API to query the files in the user's save folder, but this again means two codepaths to do the same thing. (As an aside, I first tried DocumentFile.listFiles() only to find this to be absurdly slow, taking several seconds on a folder with several hundred files; thankfully doing a query on the Storage Access Framework URI was much faster - but that has its own problems due to lack of filtering abilities on the query, see https://stackoverflow.com/questions/56263620/contentresolver-query-on-documentcontract-lists-all-files-disregarding-selection , https://stackoverflow.com/questions/52770188/how-to-filter-the-results-of-a-query-with-buildchilddocumentsuriusingtree .)

So what are the benefits of scoped storage? It does mean on Android 10, Open Camera can now run without needing permission to read/write user files.

But I can't help feeling the same could have been achieved in other ways - it feels to me like Google is using the security argument to force its APIs onto developers. Other possible ideas might have been:

• If it's allowed for an application to write or read a file using Mediastore API (e.g., allowing to write to certain folders like DCIM/ etc, allowing to read your own files), why not still allow this using the File API? It seems that Google have partly allowed this for Android 11, but only for reading, as far as I can tell.

• If a user has given read/write permission to an on-device folder using the Storage Access Framework dialog, then why can't the application then use the File API also for that folder? Or to put it another way, some way to request scoped permission for a folder, that would then allow using any read/write API for that folder.

• The Storage Access Framework dialog shown to users is a bit of a mess in my opinion. It's like it can't decide whether it's being a file dialog, or a request for user permission. It's fine if a user wants to choose a particular folder, but it's clumsy for an application to use this dialog to get user permission for a particular folder. What's needed is a method for an application to say "Can I have permission for folder X?", and then that question be put directly to the user (perhaps with an API option that allows a "Choose alternative folder" button that then goes to the regular dialog). A similar problem is that creating a new folder requires full permission for the containing folder. It would be better if an application could ask permission for "Can I create folder X, and have permission for that folder?" Consider the default behaviour where an application wants to create a sub-folder for itself.

It also doesn't feel like Google eat their own dog food in my opinion. Whilst they made it clear for years that they want users to move to APIs like Storage Access Framework, it took a few more years for APIs like ExifInterface (which Open Camera uses for handling Exif metadata) to fully support non-File APIs. It was also interesting to note that Google's CameraX API initially introduced File-based APIs without equivalent versions for non-File APIs, although they have now fixed that (with ImageCapture.OutputFileOptions). Even the official example CameraXBasic still saves using the File API, which doesn't seem like it's demonstating best practices! (It's able to use the File API because it saves to the "application folder", but that isn't really useful for most real camera apps unless you want the user to lose all their photos when they uninstall...)

What are the downsides? Aside from the outstanding issues I mention above, there is the time spent making the changes, along with testing these. Due to Android 10 now behaving different to older versions, there is also ongoing costs in testing, until most people are on Android 10+. That time is time that would have otherwise been spent fixing bugs or improving or adding new features that users actually want. In general, it seems that more of my time these days is not spent adding user wanted features, but making changes either in response to technical changes in Android, or the policy changes that seem increasingly common (where even if you're not doing anything anyone would consider bad, may still require applications to jump through hoops, and having to deal with it if Google's algorithm decides you got it wrong).

[Anonymous]

Help Please open my camera phone LG4

[Anonymous]

Hi Mark, thank you for this great app which I have been using for a while. Currently on an android 10 (Lineageos) installation on a motorola g5s. I also have ACDSee mobile upload app and stock gallery. I think what you describe in this post explains an issue I am having. If I set OpenCamera to save to the DCIM folder on the SD card, the photos save fine there, but cannot be seen by the stock gallery app or the ACDSee mobile upload app, which I guess both use the Android mediastore. New photos only appear there if I run a manual media scan app or reboot the phone. It wasn't so much of an issue before when I was using Simple Gallery, which did see the new photos right away, but I started getting "no gallery app" errors and there is no way to change the reliance of ACDSee on the Android mediastore and so it is more of an issue now. Everything seems to work as intended if I do not activate the Storage Access Framework and so save only to DCIM on the phone itself. But of course the memory is limited there so this is undesireable. From this post, I understand these issues arise from the multiple APIs google has implemented and it may not be possible to implement a workaround that allows saving to SD card AND immediate addition to the android mediastore. However, I was wondering if there is anything that could be added into the OpenCamera app itself that could trigger a media scan on the relevant file or folder when a new photo or video is added to the SD card? thanks again and best wishes.