Build GCP Storage Client (Python)

This post demonstrates how to build a Google Cloud Storage JSON API Python application. The application interacts with the storage service via the JSON API (and the Python client library). It uses a simple interface which allows you to perform tasks such as: list the buckets in a project, list objects in a bucket, create a bucket, create an object and so on. The intent of the application is educational. It should help you to understand the API syntax (and semantics) when interacting with Google Cloud Storage..

JSON API Python Client Library

JSON API Python Client Library

To build the application, you will use Eclipse as shown next. Before you can perform the shown steps, assure that you have satisfied the requirements described here:

  1. Google Cloud Storage Python Applications Prerequisites.
  2. Getting Started with JSON API Client Library in Python

Create the Application Project

  1. Activate Eclipse.
  2. From the File menu, select New->PyDevProject. The PyDev Project window is displayed.
  3. In the Project name: enter PythonJsonApi.
  4. Accept the user default workspace; for example: /Users/[your username]/[your work directory]/PythonJsonApi. This is the directory where the program will be stored.
  5. If you have more than one Python version installed perform the following steps:
    • In the Grammar Version selection list select 2.7.
    • In the Interpreter selection list select the desired Python interpreter. For more information, see Configure PyDev.
    PythonJsonApi Project

    PythonJsonApi Project

  6. Click Finish. The PythonJsonApi project is created and displayed in the Package Explorer.
  7. If the PyDev perspective is not selected yet, click on the Open Perspective icon in the upper right corner of Eclipse. From the pop-up menu select PyDev and click OK.
  8. In the Package Explorer, expand the PythonJsonApi project node.
  9. Right-click the src folder.
  10. Select New->PyDev Module.
  11. In the pop-up dialog window enter the following information:
    • In the Package box enter gcs_examples.
    • In the Name box enter PythonJsonApiMain.
  12. Click Finish.
  13. Select Module Main and  click OK.
    An empty PythonJsonApiMain.py is created. Actually the module contains the standard Python check for main because the template chosen. You’ll enter the code needed to interact with Google Cloud Storage in the next sections.

Download the Application Code

You can download the archived project from this location: storage-xmlapi-python-client. Then import it into Eclipse. The project contains the following modules:

  1. main.py. This is the main entry point of the application which allows you to
    interact with Google Cloud Storage (GCS) using XML API. It contains the main function which is called when the application is executed.
  2. simple_ui.py.  Contains the GCS_SimpleUI class which provides a simple UI to interact with Google Cloud Storage.
  3. commands.py.  Contains the GCS_Command class which provides the entry point for
    processing user’s selection. Based on the user’s selection, the related Google Cloud Storage
    operation is executed.
  4. authentication.py.  It contains the class that generates an authenticated HTTP client object.
  5. bucket_commands.py. Contains the GCS_Bucket class which handles Google Cloud Storage bucket operations.
  6. object_commands.py. Contains the GCS_Object class which handles Google Cloud Storage object operations.
  7. config.py.   Contains data shared by all the modules.

PythonXmlApi Implementation

The PythonXmlApi main function performs the preliminary initialization and starts the application. A set of classes then perform the actual tasks such as: creating a simple UI, processing user’s input, interacting with the storage service and so on. The following picture shows the classes and their hierarchy.

PythonXmlApi Workflow

The first time you start the application, the main function instantiates GCS_SimpleUI andGCS_Command classes then initializes a simple user interface which accepts the user’s input. You will be asked to authenticate the application which uses OAuth2.0 and stores the credentials in a local file called stored_credentials.json. Also, you will be asked to enter the project ID which will be stored in a local file called project.dat.

Every time you make a selection a command call is issued which in turn calls a GCS_Bucket orGCS_Object method. This method executes the actual Google Cloud Storage bucket or object request. The storage service’s response is then displayed for your information. If the request fails, an error is displayed. Finally, depending on the debugging level, the application displays what goes on the “wire”. This is to help you understand the actual HTTP request and response content as explained by the Google Cloud Storage documentation Reference Methods. The following picture depicts the workflow just described.

 

Python

 

Code here

Java

 

Code here

Leave a Reply

Your email address will not be published. Required fields are marked *