Protocol

The Stackable API is provided to our customers as a way to interact with their Stackable products via a programmatic interface.  This interface allows users to create, update and destroy objects which in some cases represent billable entities.  Below you will find documented how to interact with this API.

At its core level, the Stackable API is constructed on top of two core technologies.  AMQP (Advanced Message Queuing Protocol), which is used as the base message exchange infrastructure, and Thrift, which is used as a strictly typed message representation framework.  Each of these are explained in detail below.  By combining these two technologies, we get the benefits of open source, highly scalable applications with easy to implement client libraries.

Summary
ProtocolThe Stackable API is provided to our customers as a way to interact with their Stackable products via a programmatic interface.
AMQPThe Advanced Message Queueing Protocol is used by Stackable as an RPC (Remote Procedure Call) message broker.
ThriftThrift http://incubator.apache.org/thrift/ is a binary message encapsulation that is used by Stackable for message serialization of the RPC system.
JSON ProxyAs an alternative to using the Thrift over AMQP system, we also allow customers access to an JSON proxy system which speaks a very simple dialect of JSON.
AuthenticationAuthentication on the Stackable API is accomplished using an HMAC/SHA1 digest with a private API key.
ExampleLet’s look at a specific example.
Functions
AMQP/ThriftHere’s how you’d use the Thrift over AMQP system for this request.
JSON Proxy with plain API keyImplementing this in JSON is fairly easy.
JSON Proxy with signatureThere is an inherent security risk with passing the API key over the wire, which was documented in the method JSON Proxy with plain API key above.

AMQP

The Advanced Message Queueing Protocol is used by Stackable as an RPC (Remote Procedure Call) message broker.  We use version 0-8 of the AMQP specification, and more specifically the RabbitMQ Server 1.6.0 http://www.rabbitmq.com/.

The basic pattern of usage of this protocol is as follows

  • Connect to the AMQP server over SSL (port 5671) to q1.stackable.com, q2.stackable.com or q3.stackable.com
  • Authenticate with the username ‘guest’, password ‘guest’, on virtual host ‘/’
  • Create a temporary reply queue and subscribe to it
  • Publish a Thrift message to the service queue, indicating the temporary reply queue name in the ‘reply to’ field
  • Receive the Thrift response on the temporary reply queue

The details of this AMQP dialog are provided below.  Before moving on, let’s talk about Thrift.

Thrift

Thrift http://incubator.apache.org/thrift/ is a binary message encapsulation that is used by Stackable for message serialization of the RPC system.  We provide a Thrift IDL (Interface Description Language) document which, when compiled by the ‘thrift’ command line tool, can generate code that can be used in C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, Smalltalk, and OCaml to generate binary payloads.

By combining Thrift and AMQP for our API, we set forth language requirements for interacting with our system, although there are alternatives.  Thrift is supported by almost every language you’d want to use, so the more restrictive aspect of this is the AMQP implementation.  For a rather up-to-date list of RabbitMQ and AMQP 0-8 compatible clients, visit http://www.rabbitmq.com/how.html#clients.  Java, Ruby, Python, .NET, PHP, Perl and others are available.

Out of the box, Thrift expects to be in charge of the RPC system, and will need some local modifications to be able to speak over AMQP.  We have written and will provide out of the box code for Perl, and Python users can use txAMQP https://launchpad.net/txamqp, which supports Thrift RPC over AMQP.

The basic pattern of usage of this message type is as follows

  • Obtain the Thrift IDL file
  • Compile native language code using the CLI tool ‘thrift’
  • Write code that, using these native libraries, can create a binary payload and pass it to AMQP for message transmission
  • Write code that, using these native libraries, can receive an expected binary payload response and convert it into native objects

JSON Proxy

As an alternative to using the Thrift over AMQP system, we also allow customers access to an JSON proxy system which speaks a very simple dialect of JSON.  Using it, you may call any method in the API and get a synchronous response.  By using a relaxed JSON syntax, you won’t get the advantage of strict typing and the full expression of values available with binary Thrift.  Without using AMQP, you won’t be able to send the messages asynchronously, and will loose the ability to arbitrarily structure the AMQP system to your own processes.  Finally, without AMQP, you won’t be able to subscribe to events in the API.  For most use cases, however, using the JSON Proxy is sufficient and is less work.

Authentication

Authentication on the Stackable API is accomplished using an HMAC/SHA1 digest with a private API key.  Through the Stackable Control Panel, each customer may obtain an API key to use for the API.  The API key authenticates you on the API infrastructure, and is your own means to gain privileges to perform actions.

Example

Let’s look at a specific example.  Let’s say you’d like to use the Stackable API to create a new container.  After looking over the documentation for <Service.Container.create>, you’ve come up with the following payload you’d like to transmit.

customerId: '6523',
name: 'My first container',
os: 'CentOS_5_x86_64',
stack: 'PHP'
Summary
Functions
AMQP/ThriftHere’s how you’d use the Thrift over AMQP system for this request.
JSON Proxy with plain API keyImplementing this in JSON is fairly easy.
JSON Proxy with signatureThere is an inherent security risk with passing the API key over the wire, which was documented in the method JSON Proxy with plain API key above.

Functions

AMQP/Thrift

Here’s how you’d use the Thrift over AMQP system for this request.

Download the ‘stackable.thrift’ file from http://api.stackable.com/stackable.thrift

Download and install thrift from http://incubator.apache.org/thrift/download/

Using the ‘thrift’ CLI tool, generate static code for your native language.

thrift --gen perl stackable.thrift
 ^ this will generate a directory 'gen-perl/' as follows:
gen-perl/
`-- StackableControl
    |-- Common.pm
    |-- Constants.pm
    |-- Container.pm
    |-- Customer.pm
    |-- DNS.pm
    |-- Database.pm
    |-- Environment.pm
    |-- Loadbalancer.pm
    |-- Puppet.pm
    |-- Site.pm
    |-- Storage.pm
    `-- Types.pm

Using this native library and the Thrift libraries for your language of choice, you can now generate a thrift payload that will be sent over AMQP.  Here’s an example in Perl.

use Thrift::BinaryProtocol;
use Thrift::MemoryBuffer;
use StackableControl::Container;

my $buffer   = Thrift::MemoryBuffer->new(1024);
my $protocol = Thrift::BinaryProtocol->new($buffer);
my $client   = StackableControl::ContainerClient->new($protocol);

$client->send_create(
    6001,  # customerId
    undef, # id
    undef, # uuid
    undef, # address
    'My first container', # name
    StackableControl::ContainerOS::CentOS_5_x86_64, # os
    StackableControl::ContainerStack::PHP, # stack
);

my $payload = $buffer->getBuffer;
print "Thrift binary payload is ".length($payload)." in size\n";

NOTE: The precise method to create a binary payload will differ with each thrift language implementation.

Now that you have the payload, you’re ready to send the request via AMQP and wait for a response.  Firstly, connect to q1.stackable.com, q2.stackable.com or q3.stackable.com on port 5671 with SSL.

<<< Method Connection.Start
>>> Method Connection.StartOk ( mechanism: AMQPLAIN, response: { LOGIN: guest, PASSWORD: guest } )
<<< Method Connection.Tune
>>> Method Connection.TuneOk
<<< Method Connection.Open
>>> Method Connection.OpenOk ( virtual_host: / )

Create a new channel.

>>> Method Channel.Open
<<< Method Channel.OpenOk

Create a temporary queue, which we’ll call the reply queue.

>>> Method Queue.Declare ( queue: null, auto_delete: true, exclusive: false )
<<< Method Queue.DeclareOk ( queue: <reply queue name> )

Subscribe to this reply queue.

>>> Method Basic.Consume ( queue: <reply queue name>, no_ack: true )
<<< Method Basic.ConsumeOk

Publish to the queue named ‘Container’, which implements the ‘Container’ service we’re wanting to access.  Refer to the JSON Proxy with signature instructions for how to create the headers below.  Note that any extra headers you pass will be returned to you, allowing you to store meta data about the request in these headers.

>>> Method Basic.Publish ( routing_key: Container )
>>> Header Basic (
       body_size: <length of thrift payload>,
       content_type: application/x-thrift,
       delivery_mode: 1,
       priority: 1,
       reply_to: <reply queue name>,
       headers: {
          Customer-Key-ID: <api key id>,
          Signature: <signature>,
          Signature-Created: <signature created time>,
          Signature-Method: HMAC/SHA1,
          Signature-Version: 1,
          Your-Custom-Header: passthrough,
       }
    )
>>> Body ( payload: <thrift payload> )

After some time elapses, you should receive a response on your reply queue.

<<< Method Basic.Deliver ( routing_key: <reply queue name> )
<<< Header Basic (
       content_type: application/x-thrift,
       headers: {
          Status-Code: <status code; 200 for success, 4xx for failures>
          Your-Custom-Header: passthrough,
       },
    )
<<< Body ( payload: <thrift response> )

You now have in hand the binary thrift response to your request.  The last step is to decode this.  This is language specific, so here’s a way to do it in Perl, extending the previous code example.

$buffer->resetBuffer();
$buffer->write($thrift_payload);
my $result = eval { $client->recv_create() };
if ($@) {
    print "Failed Container.create call: $@\n";
}
else {
    print "Successful Container.create call; created container id " . $result->id . "\n";
}

JSON Proxy with plain API key

Implementing this in JSON is fairly easy.  Create an HTTP request with the following criteria.

  • Contains the header ‘Stackable-APIKey’ with the 36-character API key
  • Contains the header ‘Stackable-APIKeyID’ with the ID of your API key
  • Contains the header ‘Content-Type’ with the value ‘application/json’
  • Carries the following content, encoded as JSON:
{
    'customerId': '6523',
    'name': 'My first container',
    'os': 'CentOS_5_x86_64',
    'stack': 'PHP'
}

Send the HTTP request using method ‘POST’.  The HTTP response will also be of ContentType ‘application/json’, and will have a JSON-encoded content block.  The response will look something like this.

{
    'success': '1',
    'result': {
        'containerId': 6001,
        ...
    }
}

JSON Proxy with signature

There is an inherent security risk with passing the API key over the wire, which was documented in the method JSON Proxy with plain API key above.  Even though the transmission is hopefully over SSL and secure, since the only thing preventing other people from doing things with your containers is the API key, this key should be treated as a valued secret, and shouldn’t be passed around.  We therefore recommend the following more secure JSON mechanism for API calls.

A request is generated as follows

  • Header ‘Signature’ contains the Base64 HMAC/SHA1 digest
  • Header ‘Signature-Version’ == 1
  • Header ‘Signature-Method’ == ‘HMAC/SHA1’
  • Header ‘Signature-Created’ contains the ISO8601 date used in creation of the ‘Signature’ digest
  • Header ‘Customer-Key-ID’ contains your API key id that corresponds with the API key used in the digest creation

From here the behavior is the same as with JSON Proxy with plain API key above, omitting the ‘Stackable-APIKey’ and ‘Stackable-APIKeyID’ headers.

The Stackable API signature uses the following standards

Here’s the basic process for signature generation

  • Compute the content of the request (encoded JSON opaque binary block)
  • Compute the current combined date and time in UTC as an ISO8601 value
  • Create an HMAC digest with your API key as the key, and SHA1 as the hash function
  • Add the date time and the content of the request to the digest
  • Compute the Base64 value of the digest of the concatenated value

And in pseudo code

var payload = JSON.new( params ).string
var created = DateTime.now.strftime('%Y-%m-%dT%H:%M:%SZ')
var Digest = HMAC.new( APIKey, SHA1.new() )
Digest.add(created)
Digest.add(payload)
var signature = Digest.base64digest
Implementing this in JSON is fairly easy.
The Advanced Message Queueing Protocol is used by Stackable as an RPC (Remote Procedure Call) message broker.
Thrift http://incubator.apache.org/thrift/ is a binary message encapsulation that is used by Stackable for message serialization of the RPC system.
There is an inherent security risk with passing the API key over the wire, which was documented in the method JSON Proxy with plain API key above.
Close