App Engine Plugin Tutorial

Introduction

The App Engine Scaffold allows you to monitor an application you run on Google App Engine using Hyperic HQ. The scaffold is a set of files - a plugin descriptor, some Python scripts, and a shell script - that you can use as a starting point for building a special type HQ plugin that enables HQ components to accept and manage metrics generated by an application running on App Engine

Together, Hyperic HQ and an HQ App Engine Plugin allow you to:

• Chart and trend your application metrics over time, to answer questions like "How many blog posts did I have in June?"
• Execute and apply alerts to synthetic transactions. For instance, you could define an alert on the the time it takes to create a new user, and be notified when the "create user" transaction takes longer than 2 seconds. 
• Apply alerts to metrics, including those that indicate application usage. For instance, given a game application, you could calculate a "games_played_today" metric, push it to HQ, and define an alert so that you will notified if the value is lower than a threshold you define.
• Compare application metrics from different deployment environments. For instance, you could compare how long it takes to run a particular transaction in your development, QA, and production environments.

Implementation Oveview

To use Hyperic HQ to monitor an application running on Google App Engine you will:

• Use the scaffold files as a starting point for building your own HQ App Engine Plugin.
• Deploy some plugin related files to your application environment on App Engine
• Deploy the plugin to HQ

About this Tutorial

This tutorial provides step-by-step instructions for setting up the scaffold with Hyperic HQ to monitor an application running on Google App Engine. The instructions are based on an the example application included with the scaffold. The example application, called 'myfib', calculates a large fibonacci number and returns the elapsed time.

Before You Start

If you are new to Hyperic HQ, these topics provide a brief introduction the product and some of the concepts and terms to which the instructions that follow refer:

• Overview
• HQ Inventory and Access Model
• Auto-Discovery

  Plan Ahead Before you proceed, consider what application runtime behavior you'd like to measure or monitor. The scaffold enables you to push metrics from App Engine to Hyperic HQ, where you'll be able to visualize, monitor, and evaluate the data - and but your application itself must generate the metric values.

Prerequisites

The App Engine devkit must be installed on the machine where you will install the HQ components. The devkit is required by to use the the Python scripts (appcfg.py and dev_appserver.py) that form a portion of the scaffold.

Install Hyperic HQ

The first step is to install Hyperic HQ.

1. Download HQ. Choose an installation package for your environment from the "Download HQ Installer" section of the download page. The full installer allows you to install both the components you need: the HQ Agent and the HQ Server, which includes a built-in database.
2. Install the HQ Server and the HQ Agent. For system requirements and instructions, see [DOC:Installation Overview].
3. After you have started the HQ Server and the HQ Agent, connect to the HQ Portal user interface, verify that the HQ Agent has auto-discovered your machine, and add it your HQ inventory. For instructions, see [DOC:HQ Quick Start].

Download the HQ App Engine Scaffold Bundle

• hqappy-plugin-0.1.tgz
• hqappy-pugin-0.1.zip

Unpack the archive. 

   tar -xzvf hqappy_scaffold-0.1.tgz
   cd hqappy_scaffold

The archive contains:

• /app_engine - contains base files you'll edit in the steps below and copy into your deployed /app_ engine directory.
• myfib_plugin.xml - The HQ Plugin Descriptor File, which defines the metrics to be collected from the sample application and how to collect them.
• getMetrics.sh - Shell script, which is used to test the plugin and metrics.
• getMetrics.properties - Properties for configure getMetrics.sh.

Run the Example

In this step, start the scaffold itself.  getMetrics.sh is a useful utility you can use to dump and test metrics, before you deploy.

• Edit getMetrics.properties, to point pdk.dir at the HQ Agent's pdk (plugin development kit) directory:

    pdk.dir=/opt/hyperic/agent/bundles/agent-3.2.5-776/pdk
  

• Start the example App Engine app:

    $ dev_appserver.py app_engine
  

• Execute ./getMetrics.sh:

  App Engine Fibonacci Availability:
     http:ssl=false,hostname=localhost,port=8080,sotimeout=10,path=/hq-stats/get?pass%3Dshallnot,user=%user%,pass=,realm=%realm%,method=GET,hostheader=%hostheader%,follow=false,pattern=%pattern%,proxy=%proxy%:Availability
     =>100.0%<=
  App Engine Fibonacci Fibonacci 28:
     http:ssl=false,hostname=localhost,port=8080,sotimeout=10,path=/hq-stats/get?pass%3Dshallnot,user=%user%,pass=,realm=%realm%,method=GET,hostheader=%hostheader%,follow=false,pattern=%pattern%,proxy=%proxy%:Fibonacci28
     =>601ms<=
  App Engine Fibonacci Random Number:
     http:ssl=false,hostname=localhost,port=8080,sotimeout=10,path=/hq-stats/get?pass%3Dshallnot,user=%user%,pass=,realm=%realm%,method=GET,hostheader=%hostheader%,follow=false,pattern=%pattern%,proxy=%proxy%:RandomNumber
     =>260.0<=
  

  We can see that the example has 3 metrics. Availability (which is special and defines the red-green light), Fibonacci 28 (the length of time it took us to get fib(28)), and Random Number (random # between 1 and 1000).

Tune the Metrics

If you wish to change the metrics, you need to modify the contents of these files:

1. myfib-plugin.xml -  describes which the metrics to be collected.
2. app_engine/stats.py - the class that will listen for metrics inside the App Engine app.

For example, edit myfib-plugin.xml, to add a new metric, "Party Time", at the end. Setting the unit type to  epoch-seconds enables HQ to format the metric value nicely.

      <metric name="Party Time"
            units="epoch-seconds"
            indicator="true"
            interval="60000"/>

If you run ./getMetrics.sh now, you'll notice that the new 'Party Time' metric is not returned. This is because the App Engine application needs to provide the metric value in order for metric to be returned.

Edit app_engine/stats.py and change get_metrics() to read:

    return {
      'Fibonacci28':   self.time_fib(28),
      'RandomNumber':  random.randrange(0, 1000),
      'PartyTime':     time.time(),  # Wow, it just turned party-o-clock!
    }

Note that the names returned in get_metrics() match the <metric> name, except the spaces are removed.

Now, if you run ./getMetrics.sh you should see the value for the current time:

App Engine Fibonacci Availability:
   http:ssl=false,hostname=localhost,port=8080,sotimeout=10,path=/hq-stats/get?pass%3Dshallnot,user=%user%,pass=,realm=%realm%,method=GET,hostheader=%hostheader%,follow=false,pattern=%pattern%,proxy=%proxy%:Availability
   =>100.0%<=
App Engine Fibonacci Fibonacci 28:
   http:ssl=false,hostname=localhost,port=8080,sotimeout=10,path=/hq-stats/get?pass%3Dshallnot,user=%user%,pass=,realm=%realm%,method=GET,hostheader=%hostheader%,follow=false,pattern=%pattern%,proxy=%proxy%:Fibonacci28
   =>587ms<=
App Engine Fibonacci Random Number:
   http:ssl=false,hostname=localhost,port=8080,sotimeout=10,path=/hq-stats/get?pass%3Dshallnot,user=%user%,pass=,realm=%realm%,method=GET,hostheader=%hostheader%,follow=false,pattern=%pattern%,proxy=%proxy%:RandomNumber
   =>379.0<=
App Engine Fibonacci Party Time:
   http:ssl=false,hostname=localhost,port=8080,sotimeout=10,path=/hq-stats/get?pass%3Dshallnot,user=%user%,pass=,realm=%realm%,method=GET,hostheader=%hostheader%,follow=false,pattern=%pattern%,proxy=%proxy%:PartyTime

   =>8/13/08 8:15:42 PM<=

You can repeat the steps in this section section to refine the metrics you want to collect, and what values they should have. It's great for stubbing out your metrics.

Integrating with You Application

To integrate with your App Engine app, copy app_engine/stats.py and app_engine/statsbase.py into your existing App Engine app directory and make them callable in app.yaml:

  $ cp app_engine/stats.py app_engine/statsbase.py ~/dev/stockquote_app

  $ vi ~/dev/stockquote_app/app.yaml       (make sure you have a handler to stats.py)

handlers:
- url: /hq-stats/.*
  script: stats.py

Now you can deploy your own application and have getMetrics.sh will still be able to pull the same values:

  ./getMetrics.sh

App Engine Fibonacci Availability:
   http:ssl=false,hostname=localhost,port=8080,sotimeout=10,path=/hq-stats/get?pass%3Dshallnot,user=%user%,pass=,realm=%realm%,method=GET,hostheader=%hostheader%,follow=false,pattern=%pattern%,proxy=%proxy%:Availability
   =>100.0%<=
App Engine Fibonacci Fibonacci 28:

...

Rename the Plugin

To change your plugin name from "myfib", replace the string "myfib" in getMetrics.properties and getMetrics.sh with a short name of your application.

Also, rename myfib-plugin.xml, replacing "myfib" with the same application short name.

Now, your App Engine application can export metrics that HQ can pick up.

Deploy the Plugin

Deploy the plugin to the HQ Server and the HQ Agent. These sample commands assume that your application short name is "stockquote"

  cp stockquote-plugin.xml /opt/hyperic/server-3.2.5/hq-engine/server/default/deploy/hq.ear/hq-plugins
  cp stockquote-plugin.xml /opt/hyperic/agent-3.2.5/bundles/agent-3.2.5-672/pdk/plugins

Restart the agent:

  cd /opt/hyperic/agent-3.2.5/bin
  ./hq-agent.sh stop
  ./hq-agent.sh start

Configure the Monitored App as a "Platform Service" in HQ

In this step, you configure the App Engine application that you are monitoring as a "Platform Service" that runs on the "Platform" that maps to the application host.

Login to Hyperic HQ and navigate to the Platform that represents your machine.

Login and navigate to Resources -> Browse -> Platforms

Select your machine, and you should see a screen like this:

Select New Platform Service, and enter your information on the following screen:

At this point, you need to configure your resource to point at the correct web address for the App Engine server. This may differ, depending on your configuration.

Select Configuration Properties (or the Edit button near the bottom), to reconfigure your app:

By default, the App Engine scaffold is shipped with an extremely simple authentication mechanism: Simply checking a cleartext token which is sent as part of the query string. More secure applications should take stricter measures, and all applications should change their default tokens. (change stats.py and the pass= in the path configuration option.

Finally, click the Monitor tab to see your data coming in. You may need to wait for a minute so that the agent is notified and sending metrics back to HQ.

Success

This should give you all the information you need to start pulling in your App Engine data. Please read up on the other HQ tutorials and screencasts for more information on using HQ.