Google Cloud Storage in GCP : Step by Step Procedure

  • An unified object storage service. It is used as a staging area to move data into other GCP components like Cloud SQL, BigQuery, Cloud Dataproc.
  • It is a blob storage with any format.
  • Rather than storing data on the compute engine (It would be expensive as the compute engine goes away, disk goes away too), store it on Cloud Storage. Cloud Storage is a persistent storage, it is durable, replicated and also made globally available via HTTP URL.

Key Terms

  • Buckets: Basic containers that hold your data. Everything that you store in Google Cloud Storage must be contained in a bucket. You can use buckets to organize your data and control access to your data, but unlike directories and folders, you cannot nest buckets.
  • Bucket names: Should be unique as the name of the buckets stored in single Cloud Storage namespace. Also, bucket names can be used with a CNAME redirect, which means they need to conform to DNS naming conventions.
  • Bucket labels: Bucket labels are key:value metadata pairs that allow you to group your buckets along with other Google Cloud.
  • Objects: Objects are the individual pieces of data that you store in Google Cloud Storage. Objects have two components: object data and object metadata. The object data component is usually a file that you want to store in Google Cloud Storage. The object metadata component is a collection of name-value pairs that describe various object qualities. There is no limit on the number of objects that you can create in a bucket.

Data Handling

  • You can set up a transfer service for ingesting data into Cloud Storage from AWS S3 buckets, our computers etc., for once or recurring .
  • Permissions can be set at project, bucket and object level.

Creating Bucket via GCP Console

  • Click on “CREATE BUCKET”
  • Enter the name and select the storage class according to the requirement. Click on “Create”.

Creating Bucket via gsutil Command Line

  • By installing gcloud sdk and follow the tutorial to initialize to select the project and zone location, then you can execute any command. gsutil will be available.
> gsutil mb gs://simpletext
  • Above command creates a bucket. Go to Storage dashboard.
  • If you haven’t mentioned the storage classes it will by default selects multi-region.
  • Copy some text files to the bucket. If you have a large number of files to transfer you might want to use the gsutil -m option, to perform a parallel (multi-threaded/multi-processing) copy.
> gsutil -m cp test*.txt gs://simpletext
  • Files are available. You can also make files globally available by checking “Share publicly”. Permissions can also be set.
  • There are many other commands available for moving contents to one bucket to other bucket, list contents of the bucket etc.

Setting Up Trasnfer Service

  • On the Cloud Storage dashboard, select “Transfer” and then click on “Create transfer”.
  • Select the source from which you wanted to copy the files, then “Continue”
  • You can select the destination bucket to copy the files to. Check the box according to the requirement, then “Continue”.
  • You can now select either “Run Now” or “Run daily at <time>”. Proceed clicking on “Create”.

Leave a Comment

MFH IT Solutions (Regd No -LIN : AP-03-46-003-03147775)

Consultation & project support organization.


MFH IT Solutions (Regd)
NAD Kotha Road, Opp Bashyam School, Butchurajupalem, Jaya Prakash Nagar Visakhapatnam, Andhra Pradesh – 530027