#105 MacOS cloud storage changes making GP almost impossible to use

[Anonymous]

Hi, Apple recently changed how cloud storage apps like Dropbox and Google Drive for Desktop (formerly Drive File Stream) work, switching them from being seen by the system as a separate drive ("/Volumes/GoogleDrive") to being stored in ~/Library/CloudStorage. As such, GrandPerspective now tries to scan everything in my google drive, which is close to a terabyte of data, very little of it cached locally, so it tries to download it all first. I tried to scan my MacHD drive, and gave up after 20 minutes or so when I realized what must be happening.
I tried searching to see if there's a way to blacklist a folder so GrandPerspective ignores it, but all I could find were filters for after it's finished scanning. If you could permanently block a folder before you scan, that'd be great!

[Anonymous]

Am facing the same issue whilst tryiing to use GrandPerpective with Google Drive mounted.

[Erwin Bonsma]

The fact that scanning Dropbox data leads to download of offline files was recently submitted in Ticket #104. I was not yet aware that the same happens with Google Drive (I am using neither). Scanning offline iCloud files still works as expected, at least on my machine.

The good news is that you can already use Filter Scans to scan your entire HDD except for specified folders. See:
https://grandperspectiv.sourceforge.net/HelpDocumentation/CreatingViews.html
https://grandperspectiv.sourceforge.net/HelpDocumentation/MasksAndFilters.html

[Erwin Bonsma]

Today I spent some time investigating this further. I tested online cloud storage for both Dropbox and Google Drive. This works as expected.

My Dropbox folder contains three folders which are "online only" and two files that are also available offline. When I scan it using the physical file size measure, it reports a size of 2,92 MB (which is indeed the combined size of both files). See screenshot.

When I next scan using logical size, it reports a size of 104 MB, and a warning that physical size is only 2.4%. See second screenshot. Both scans ran quickly and did not trigger download of stored files.

It could be that for obtaining (up to date) file metadata (such as size and time of last change) macOS behind the scenes performs requests to the cloud storage provider. If so, a scan may indeed be slower.

However, I also disconnected my network and scanned my cloud storage folder, still using logical size. This worked as expected. It reported the size that the files take up in the cloud with again a warning the actual storage on the local drive is (considerably) lower.

To conclude, the statement in the description "so it tries to download it all first" is not correct. Furthermore, I did not see a deterioration in scan performance, but admittedly, I am not storing many files in the cloud.

If you see different behaviour than I am reporting, please carry out the following and share your findings:
a) What is the result of scanning (part of) your online-only cloud storage when using logical size when you are offline (disconnected from the network)? Does it complete the scan? Does the resulting scan show you the expected set of files? Does it show a warning that phyiscal size is smaller?
b) If you repeat the scan when you are online (after you reconnected your network), does this change the results? Does it report exactly the same total size and number of files, or are their differences? Did the scan change status of file, or do all files remain online-only?
c) What is the result of scanning (part of) your online-only cloud storage when using physical size? What is the total size in this case?

Also, when you are using Google Drive the following post may be helpful:
https://support.google.com/drive/answer/12178485

[Erwin Bonsma]

I just came across the following details with regards to OneDrive storage on macOS. The same might apply to other cloud storage provides and could explain why the scan by GrandPerspective was slow:

Why is it sometimes slow to browse folders in my OneDrive?

To save space and system resources, the File Provider platform doesn't actually create the files OneDrive is managing until the first time you need them. The first time you open a OneDrive folder, macOS will create them on-demand. This can sometimes take a moment.
To avoid this delay, you can force the system to pre-create all of these files and folders for you without downloading your content. To do this, open a Terminal window and type "ls -alR ~/OneDrive" (or the path to your OneDrive). This will ensure all of your files and folders are created, but not downloaded, before you browse.

Source: https://techcommunity.microsoft.com/t5/microsoft-onedrive-blog/inside-the-new-files-on-demand-experience-on-macos/ba-p/3058922

Note, if this indeed the reason than it should mean that subsequent scans by GrandPerspective are much faster.

[Erwin Bonsma]

Closing this issue because as far as I can see, the app functions as intended.

Also note that Version 3.1.0, released today, has improved support for cloud storage.
See the new how to scan cloud storage help documentation.

If there are still issues, please create a new issue, providing the details requested above so this can be followed up.