OpenStack Cloud Automation using Node-Red

Node-Red API

Introduction

OpenStack is a private cloud computing (or public), that allows customers known as tenants, to provision cloud instances, volumes, and design networks based on tenant needs. During OpenStack operations that are performed by cloud administrators, and due to known limitations in OpenStack development, Automation is not an option anymore for any cloud provider. And being used by one form or another (e.g. scripts)in order to achieve cloud operation excellence.

Node-Red

Node-Red is a flow-based programming tool that started life in early 2013 as a side-project by Nick O’Learny and Dave Conway-Jones of IBM’s Emerging Technology, it is a well-known technology for IOT (Internet of Things) and Home automation.

Why with OpenStack ?

Over the past few months, I’ve been testing Node-Red to Automate some OpenStack operation tasks and found both are a perfect match. in many cases scenarios, Node-Red proved the capability to automate from very simple tasks to complex tasks that involve a lot of components. below is some sample that is not limited to what can be achieved.

  • Generate a Health check Reports for different Cloud operations on Daily basis.
  • Backup Cloud configuration regularly on offside or cloud location.
  • Automate monitoring for tenant components once provisioned and display result into Grafana so the cusotmers can immidialy track performance insights.
  • Read information from Cloud and export the data on target Database for further processing e.g. billing.

The best way to install node-red is by using docker single command or docker-compose. below is an example of a docker-compose file that provisions Node-REd container using port (1880) and creation of nodered-vol that mount nordered /data.

  • Create docker Compose File “docker-compose.yml”
version: "3.9"
services:
  nodered:
    container_name: nodered
    image: nodered
    ports:
      - "1880:1880"
    volumes:
      - type: volume
        source: nodered-vol
        target: /data
volumes:
  nodered-vol: {}
  • Launch Node-Red Container
docker-compose up -d --build
  • Verify
root@nodered:~/infrastructure# docker-compose ps
Name Command State Ports
----------------------------------------------------------------------------------
nodered npm --no-update-notifier - ... Up (healthy) 0.0.0.0:1880->1880/tcp

and Voila! Node-Red is ready to use, just point your browser to the host that installed Node-Red and port 1880

http://<HostIP>:1880

Connecting Node-Red with OpenStack API’s

Now the fun part, we will construct a workflow that Authenticates to OpenStack (generate a Token) and use that Token to Query OpenStack Cinder volume API to list volumes with details. in this post, I will focus only on this point and subsequent Posts will continue on how to proceed further with these data. the possibilities are endless.

First: Begin with an Entry Point

Drag and drop “Inject Node” from the left panel to workspace, Inject node or Timestamp node is a starting point of your flow, Flow will only be executed once you click on the rectangle icon of the Inject node. you can schedule the execution by entering the properties of the Inject Node and change the Repeat property to change the execution based on specific schedule.

Second: Get an Authentication Token

In this step, we need to authenticate to OpenStack in order to get authentication Token. Token is a must in order to query any OpenStack APIs, in Order to build that.

A- Function Node:

Place a Function node and load it with below sample msg.payload in order to define OpenStack domain, Project, Username and Password

  • Replace <Openstack User> with Username, and <Openstack Password> with user password.
msg.payload = {
    "auth": {
        "identity": {
            "methods": [
                "password"
            ],
            "password": {
                "user": {
                    "domain": {
                        "name": "default"
                    },
                    "name": "<Openstack User>",
                    "password": "<OpenStack Password>"
                }
            }
        },
        "scope": {
            "project": {
                "domain": {
                    "name": "default"
                },
                "name": "admin"
            }
        }
    }
}
return msg;




B- HTTP Request Node:

Sends HTTP Request and Returns the Response. Remember the payload of the POST request is already defined in step1 “Function Node”

http://<OpenStack API IP>:35357/v3/auth/tokens

Now, the question time, where is the response?

in order to view the response during constructing the flow and for troubleshooting purposes, attach a “Debug” node to HTTP Request Node and switch to debug in order to view the response. make sure that you switched debug to complete msg object.

Expand Debug message, and under the headers Object, x-subject-token contains the OpenStack Authentication Token that can be used in the rest of the flow. to copy x-subject-token path, just Copy path action

to copy x-subject-token path, just Copy path action

— Each time the flow is executed. a new token is generated, this is good as the token suppose to change in default OpenStack deployments at every hour. so it is best to keep this workflow in every automation of OpenStack.

C- Change Node:

The final step is this stage is to change “x-subject-token” variable to any other name, during experimenting (a lot of time really) this variable name of the token (-) dash causes trouble in the flow, it is advised to use another variable name not containing any special characters. Unfortunately, this is something in the response, help is needed from the Change node. To change the variable name and ignore any special characters.

Change the variable name from “x-subnet-token” to let’s say “ token” as simple as that.

Third and Final: Get OpenStack Data

the final step is to contruct two nodes in order to query Openstack Data like volume list with details.

A- Function Node:

Use a Function node as a payload for POST request handled by HTTP Request.

msg.headers = {};
msg.headers['X-Auth-Token'] = msg.token;
msg.query = "?all_tenants=True"
msg.requestTimeout = 900000;
return msg;+-------------------+--------------------------------------------+
| Payload | Description |
+-------------------+--------------------------------------------+
| msg.headers = {}; | Reset msg Headers (Very Important) |
| msg.query | Query Data for All Cloud Tenant (Optional) |
+-------------------+--------------------------------------------+

B- HTTP Request:

http://<Openstack API>:8776/v3/<Tenant ID>/volumes/detail{{{query}}}

{{{query}}} is substituted with the value of “msg.query” in function node.

  • Attach a Debug node to the result, a json output is displayed in debug pane

an Array of volume is displayed

Below is an example of the final result and the information that can be selected and produced in a report. or other flows like billing.

Share