10

I've followed the gsutil install guide following the Google Cloud instructions and I've updated GCloud components to the latest versions. I've just recently installed and configured GCloud to work with my credentials and project. First, I've used the following command to export a single collection I need from my Firestore :

gcloud firestore export gs://my-project-id.appspot.com --collection-ids=theCollectionINeed

Now I see this export is in my Firebase Console Storage section as a folder. Being the project Owner, I'd like to get this export into my local system. For that, I see I need to use gsutil to be able to copy it. Reading the instructions to download the object from your bucket, I've tried with the following command, but I got this error:

$ gsutil cp -r gs://m-project-id.appspot.com/2020-05-22T02:01:06_86154 . Copying gs://lucky-level-dev-6ac34.appspot.com/2020-05-22T02:01:06_86154/2020-05-22T02:01:06_86154.overall_export_metadata... OSError: The filename, directory name, or volume label syntax is incorrect.

I'm running this command using cmd in a Windows 10 environment. I'd like to be able to download this folder from the cloud to my local drive.

Update

After being, I tried to change the bucket folder (object prefix) as suggested:

gsutil mv gs://my-project-id.appspot.com/2020-05-22T02:01:06_86154 gs://my-project-id.appspot.com/2020-06-23_someFolder

But trying again now throws me a new error:

gsutil cp -r gs://my-project-id.appspot.com/2020-05-22_someFolder . Copying gs://my-project-id.appspot.com/2020-05-22_someFolder/2020-05-22T02:01:06_86154.overall_export_metadata... OSError: Invalid argument.9.0 B]

Surely I need to change the name of the file too?

Improve this question
5
  • 2
    I suspect the issue is that GCS object prefix includes colons and these may cause issues for Windows. I don't have Windows. Please try gsutil mv gs://m-project-id.appspot.com/2020-05-22T02:01:06_86154 gs://m-project-id.appspot.com/X to rename the object and then try gsutil cp -r gs://m-project-id.appspot.com/X . again. Commented Jun 23, 2020 at 21:24
  • @DazWilkin Thanks for the suggestion. Now I've surpassed that error; sadly now I got the following: OSError: Invalid argument.9.0 B] I'll update my question. Any ideas? Commented Jun 24, 2020 at 0:44
  • @DazWilkin I also had to rename the inner file that also included colons in its name. Thanks again! Commented Jun 24, 2020 at 1:17
  • 3
    You may wish to file a bug with Google. The command should work on supported OSs and clearly does not with Windows. issuetracker.google.com Commented Jun 24, 2020 at 15:56
  • 1
    For anyone who is interested, this issue is fixed in gcloud storage (github.com/GoogleCloudPlatform/gsutil/issues/1513) Commented Dec 3, 2022 at 20:48

3 Answers 3

Reset to default
11

When doing the cp -r gs:/bucket_name/folder . Gsutil will try to create a folder with "bucket_name" name in the current location, Windows doesn't allow the name of folders to have some special characters including the ':'

You can rename the bucket folder (object prefix) with the command @DazWilkin suggested gsutil mv gs://m-project-id.appspot.com/2020-05-22T02:01:06_86154 gs://m-project-id.appspot.com/new_folder_name and then try again with the new folder name.

Also check that all filenames inside the directory structure don't have the ':' character or any other special one.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Thanks for the suggestion. That really seemed to be the problem, sounds logic but now I've got the following: OSError: Invalid argument.9.0 B] Any ideas?
I also had to rename the inner file that also had colons and the problem was solved. Please, complement your answer with this addition to get a full answer in case someone don't read the comments. Thanks!
Ok I am glad this has been solved! I have updated my answer.
2

(Posted solution from question author to place it in the answer section).

All I had to do was rename the inner file that also had colons in its name. Renaming it solved the problem!

Improve this answer

3 Comments

Do you really think this answers the question by itself?? Don't just over moderate...
@Metafaniel: it is (partial) answer material, so it needs to go in the answer space. I am happy to delete it if you think it is trivial, or close the question as Typo/Unrepro if the answers are unlikely to be useful to future readers, but I do not think this material belongs in the question. Alternatively you can post this answer yourself (minus the attribution statement) as a self-answer and I can delete the CW copy.
Finally, you could ask a question over at Meta Stack Overflow if you want to get second opinions though.
0

In the case of a gcloud firestore export the issue can be avoided by specifying a directory:

# BAD this will auto generate a directory windows doesnt like gcloud firestore export gs://my-bucket # GOOD specify a directory gcloud firestore export gs://my-bucket/firestore-backup # copy as usual gsutil -m cp -r "gs://my-bucket/firestore-backup" .
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.