Using Fog to Interact With an OpenStack Cloud

Posted: January 23, 2016 in Cloud Computing, OpenStack

One of the strengths of OpenStack is that it exposes a very rich API that can be used to control every aspect of your cloud.  Likewise, one of the more intriguing ways of interacting with an OpenStack cloud is programatically.  There is a Ruby Gem named Fog that allows such interaction.  Details on the API methods that Fog provides support for can be found at http://fog.io/ – the website for Fog: The Ruby Cloud Services Library.

One of the coolest features of fog is the number of cloud provides that it can interact with.  At the time of this writing there are 42 of them – yes 42!  However, most of the providers that are listed on the fog website are considered community with support limited to IRC and Google Groups.

To follow this post, ensure that you have Ruby and IRB (Interactive Ruby).  On my system, I have version 2.0.0 of both ruby and ruby-irb installed.  You will also need to an admin user name, password, and the public URL for the OpenStack controller that you are going to work with.

Let’s Get Started

My OpenStack environment is currently based on  Red Hat Enterprise Linux OpenStack Platform 7.  It consists of a single controller node and a single compute node.

The first thing that I want to do is to install the fog gem.  This is actually pretty easy to do with:

gem install fog

Once fog is successfully installed, we can fire up irb and try to connect to our OpenStack environment.  The first line calls for Ruby to utilize the fog gem.  Make sure you enclose it in single quotes.  The 192.168.100.30 IP address is the address of my Controller node.  The user name that you specify must have admin rights – which my user, ted, has.

[tbrunell@tbrunell ~]$ irb
irb(main):001:0> require 'fog';
irb(main):002:0* keystone = Fog::Identity.new :provider                 => 'OpenStack',
irb(main):003:0*                              :openstack_auth_url       => 'http://192.168.100.30:5000/v2.0/tokens',
irb(main):004:0*                              :openstack_username       => 'ted',
irb(main):005:0*                              :openstack_api_key        => 'redhat',
irb(main):006:0*                              :openstack_tenant         => 'admin',
irb(main):007:0*                              :openstack_management_url => 'http://192.168.100.30:35357/v2.0/tokens'

The results should be similar to the below text.  If not, carefully examine the commands that you typed into irb to make sure that your syntax is correct.

=> #<Fog::Identity::OpenStack::V2::Real:26970980 @openstack_auth_url="http://192.168.100.30:5000/v2.0/tokens" @openstack_username="ted" @openstack_api_key="redhat" @openstack_tenant="admin" @openstack_management_url="http://192.168.100.30:35357/v2.0/tokens" @auth_token=nil @openstack_identity_public_endpoint=nil @openstack_auth_uri=#<URI::HTTP:0x00000003371038 URL:http://192.168.100.30:5000/v2.0/tokens&gt; @openstack_must_reauthenticate=false @openstack_endpoint_type="adminURL" @current_user=nil @current_user_id=nil @current_tenant=nil @openstack_service_type=["identity"] @openstack_service_name=nil @identity_prefix=nil @connection_options={:debug_response=>true, :headers=>{"User-Agent"=>"fog/1.37.0 fog-core/1.35.0"}, :persistent=>false} @openstack_management_uri=#<URI::HTTP:0x00000003370c50 URL:http://192.168.100.30:35357/v2.0/tokens&gt; @host="192.168.100.30" @path="/v2.0/tokens" @port=35357 @scheme="http" @identity_connection=#<Fog::Core::Connection:0x00000003370b10 @excon=#<Excon::Connection:33700e8 @data={:chunk_size=>1048576, :ciphers=>"HIGH:!SSLv2:!aNULL:!eNULL:!3DES", :connect_timeout=>60, :debug_request=>false, :debug_response=>true, :headers=>{"User-Agent"=>"fog/1.37.0 fog-core/1.35.0"}, :idempotent=>false, :instrumentor_name=>"excon", :middlewares=>[Excon::Middleware::ResponseParser, Excon::Middleware::Expects, Excon::Middleware::Idempotent, Excon::Middleware::Instrumentor, Excon::Middleware::Mock], :mock=>false, :nonblock=>true, :omit_default_port=>false, :persistent=>false, :read_timeout=>60, :retry_limit=>4, :ssl_verify_peer=>true, :tcp_nodelay=>false, :thread_safe_sockets=>true, :uri_parser=>URI, :versions=>"excon/0.45.4 (x86_64-linux) ruby/2.0.0", :write_timeout=>60, :host=>"192.168.100.30", :hostname=>"192.168.100.30", :path=>"/v2.0/tokens", :port=>35357, :query=>nil, :scheme=>"http"} @socket_key="http://192.168.100.30:35357">> @persistent=false @connection=#<Fog::Core::Connection:0x00000003376d58 @excon=#<Excon::Connection:3376920 @data={:chunk_size=>1048576, :ciphers=>"HIGH:!SSLv2:!aNULL:!eNULL:!3DES", :connect_timeout=>60, :debug_request=>false, :debug_response=>true, :headers=>{"User-Agent"=>"fog/1.37.0 fog-core/1.35.0"}, :idempotent=>false, :instrumentor_name=>"excon", :middlewares=>[Excon::Middleware::ResponseParser, Excon::Middleware::Expects, Excon::Middleware::Idempotent, Excon::Middleware::Instrumentor, Excon::Middleware::Mock], :mock=>false, :nonblock=>true, :omit_default_port=>false, :persistent=>false, :read_timeout=>60, :retry_limit=>4, :ssl_verify_peer=>true, :tcp_nodelay=>false, :thread_safe_sockets=>true, :uri_parser=>URI, :versions=>"excon/0.45.4 (x86_64-linux) ruby/2.0.0", :write_timeout=>60, :host=>"192.168.100.30", :hostname=>"192.168.100.30", :path=>"", :port=>35357, :query=>nil,:scheme=>"http"} @socket_key="http://192.168.100.30:35357">>>

I’ll let you spend time parsing through the above output.  To show what Fog can do, let’s run another command that will return some data that is a bit easier to parse and understand.  From inside of your irb session, run the following command.  I am using a tenant that I have in my OpenStack environment named “demo”, but you can use any other tenant or the admin tenant if you want.

irb(main):008:0> tenant = keystone.tenants.find { |n| n.name == 'demo' }

…and the results are below.  Notice that irb returned the UUID of the tenant along with it’s long description, and short name.

=>   <Fog::Identity::OpenStack::V2::Tenant
    id="70e5573fa7464a33bd27ac2ab25701ab",
    description="Project for demonstrations.",
    enabled=true,
    name="demo"
  >

Pretty cool, eh?  Since we are interacting with the Keystone service right now, lets get a list of all of our users.  Run the following command in irb:

users = keystone.users.find

You should get several objects returned consisting of several fields including the user ID, email address, name, and the ID of the tenant that the user belongs to.  Mixed in will be all of the service accounts that are included in OpenStack.

If you just wanted to locate a single user, you could run the command:

users = keystone.users.find { |n| n.name == 'ted' }

notice the search string that is located in the curly brackets.  The “name” field that we are searching can be swapped out for any of the fields that were included in the objects that were previously returned, like email or tenant_id for example.  I’ll let you try that out though.

Connecting to Other OpenStack Services

Other OpenStack services can be connected to in the same manner that we connected to the Keystone service before.  Here is the snippet of code used to connect to the OpenStack network service (Neutron).

network = Fog::Network.new   :provider                 => 'OpenStack',
                             :openstack_auth_url       => 'http://192.168.100.30:5000/v2.0/tokens',
                             :openstack_username       => 'ted',
                             :openstack_api_key        => 'redhat',
                             :openstack_endpoint_type  => 'publicURL'

Notice the last line – openstack_endpoint_type.  Many times, and depending on how OpenStack was configured, certain API endpoints will not be reachable because they are on isolated network for example.  By specifying the endpoint type, we are essentially overriding the URL values that OpenStack returned.

After running that above command, run the following to get information on your external network.  If you named your external network in OpenStack something other than “external” (and you probably did), simply replace the last instance of external with the name of your network.

external_net = network.networks.find { |n| n.name == 'external' }

Similar output to this should have been received.

=>   <Fog::Network::OpenStack::Network
    id="ced847d3-b71a-4f77-8cf3-a89ea4c8983b",
    name="external",
    subnets=    <Fog::Network::OpenStack::Subnets
      filters={}
      [
              
      ]
    >,
    shared=false,
    status="ACTIVE",
    admin_state_up=true,
    tenant_id="915ec9fcc72646939c4b8e39ab7437a7",
    provider_network_type=nil,
    provider_physical_network=nil,
    provider_segmentation_id=nil,
    router_external=true
  >

If we want to explore irb a bit more, we can run a couple very short commands to grab specific information from the object that was returned.  For example to get the network status and name:

irb(main):020:0> external_net.status
=> "ACTIVE"
irb(main):021:0> external_net.name
=> "external"

Of course, there are still more services that need to be connected to.  They all tend to follow the same pattern as connecting to the network service.  For examples on how to connect and use them from fog check out https://github.com/fog/fog/tree/master/lib/fog/openstack/examples.

What If I Want to Create Something in OpenStack Using Fog?

Great question, and yes, it can be done.

For this part of the demo, we will create a router and connect it to the external network.  If you are still following along, you will need the name of your external network.  In my OpenStack environment, it is named “external”.  Here is a screenshot of what my OpenStack network topology currently looks like.  Notice that there is already an external network, an internal network and a router that I have named “router”.  Yes, I like to be original with my naming conventions.

OpenStack-fog-networkTopology

Let’s go back to irb and this time, authenticate to the OpenStack network service.

[tbrunell@tbrunell ~]$ irb
irb(main):001:0> require 'fog';
irb(main):002:0* network = Fog::Network.new   :provider                 => 'OpenStack',
irb(main):003:0*                              :openstack_auth_url       => 'http://192.168.100.30:5000/v2.0/tokens',
irb(main):004:0*                              :openstack_username       => 'ted',
irb(main):005:0*                              :openstack_api_key        => 'redhat',
irb(main):006:0*                              :openstack_tenant         => 'demo',
irb(main):007:0*                              :openstack_endpoint_type  => 'publicURL'

The command should output a lot of information, much like we saw earlier.  Notice that to authenticate to the network service, all I had to do was specify that in the first line, where a new object is created.

Since we will need information about the tenant that the router is going to be associated with, we need to authenticate with the keystone service as well.  Notice I am using my “demo” tenant this time and not the admin tenant.

keystone = Fog::Identity.new :provider                 => 'OpenStack',
                             :openstack_auth_url       => 'http://192.168.100.30:5000/v2.0/tokens',
                             :openstack_username       => 'ted',
                             :openstack_api_key        => 'redhat',
                             :openstack_tenant         => 'demo',
                             :openstack_management_url => 'http://192.168.100.30:35357/v2.0/tokens'

To get the tenant information simply run the following:

irb(main):014:0> tenant = keystone.tenants.find { |n| n.name == 'demo' }
=> <Fog::Identity::OpenStack::V2::Tenant
id="70e5573fa7464a33bd27ac2ab25701ab",
description="Project for demonstrations.",
enabled=true,
name="demo"
>

Next, we will get the UUID of the external network and put it into the variable, ext_net.

irb(main):015:0> ext_net = network.networks.find { |n| n.name == 'external' }
=> <Fog::Network::OpenStack::Network
id="ced847d3-b71a-4f77-8cf3-a89ea4c8983b",
name="external",
subnets= <Fog::Network::OpenStack::Subnets
filters={}
[
<Fog::Network::OpenStack::Subnet
id="e169880c-1b58-4955-9579-e58f751d9fa5",
name="external_subnet",
network_id="ced847d3-b71a-4f77-8cf3-a89ea4c8983b",
cidr="192.168.100.0/24",
ip_version=4,
gateway_ip="192.168.100.1",
allocation_pools=[{"start"=>"192.168.100.100", "end"=>"192.168.100.250"}],
dns_nameservers=["192.168.100.1", "8.8.8.8"],
host_routes=[],
enable_dhcp=false,
tenant_id="915ec9fcc72646939c4b8e39ab7437a7"
>
]
>,
shared=false,
status="ACTIVE",
admin_state_up=true,
tenant_id="915ec9fcc72646939c4b8e39ab7437a7",
provider_network_type="vxlan",
provider_physical_network=nil,
provider_segmentation_id=1,
router_external=true
>

Now, for the real magic – we create our new router and connect it to the external network in just one short command.

router = network.routers.create :name                  => 'router2',
                                :tenant_id             => tenant.id,
                                :external_gateway_info => ext_net

Notice that I am able to use variables from the tenant and external network information that was retrieved earlier to complete the command.

When I go back to my network topology in OpenStack, I can now see my new router, connected to the external network and ready for further configuration.

OpenStack-fog-networkTopology2

What’s Next?

As you can see, the fog gem is pretty powerful.  You can do a lot of configuration using it including creating security group rules, users, images, etc.  If you are a CloudForms user (or you use it’s open source cousin, ManageIQ), like me, you can create automation scripts to accomplish a variety of things -including provisioning new tenants (with custom quotas) from the service catalog – or use the scripts to gain better control over your cloud.

The possibilities are endless.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s