Merge pull request #187 from svrc-pivotal/dynatrace_support_may14

Dynatrace 6.x+ framework support for Java buildpack

This merge adds support for the Dyna Trace application performance monitoring framework. See http://www.dynatrace.com/ . Tests and documentation are included and should be read for full details on how to use this support. Thanks to Stu Charlton (svrc-pivotal) for contributing it.

Author: cgfrost

README.md

@@ -54,6 +54,7 @@ To learn how to configure various properties of the buildpack, follow the "Confi
 * Standard Frameworks
 	* [AppDynamics Agent](docs/framework-app_dynamics_agent.md) ([Configuration](docs/framework-app_dynamics_agent.md#configuration))
 	* [Introscope Agent](docs/framework-introscope_agent.md) ([Configuration](docs/framework-introscope_agent.md#configuration))
+	* [DynaTrace Agent](docs/framework-dyna_trace_agent.md) ([Configuration](docs/framework-dyna_trace_agent.md#configuration))
 	* [Java Options](docs/framework-java_opts.md) ([Configuration](docs/framework-java_opts.md#configuration))
 	* [JRebel Agent](docs/framework-jrebel_agent.md) ([Configuration](docs/framework-jrebel_agent.md#configuration))
 	* [MariaDB JDBC](docs/framework-maria_db_jdbc.md) ([Configuration](docs/framework-maria_db_jdbc.md#configuration))

config/components.yml

@@ -34,6 +34,7 @@ jres:
 frameworks:
   - "JavaBuildpack::Framework::AppDynamicsAgent"
 # - "JavaBuildpack::Framework::IntroscopeAgent"
+# - "JavaBuildpack::Framework::DynaTraceAgent"
   - "JavaBuildpack::Framework::JavaOpts"
   - "JavaBuildpack::Framework::JrebelAgent"
   - "JavaBuildpack::Framework::MariaDbJDBC"

config/dyna_trace_agent.yml

@@ -0,0 +1,19 @@
+# Cloud Foundry Java Buildpack
+# Copyright (c) 2015 the original author or authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Configuration for the Dynatrace framework
+---
+version: 6.1.0_+
+repository_root: ""

docs/framework-dyna_trace_agent.md

@@ -0,0 +1,97 @@
+# DynaTrace Agent Framework
+The DynaTrace Agent Framework causes an application to be automatically configured to work with a bound [DynaTrace Service][] instance (Free trials available).
+
+The Cloud Foundry pushed application name is used as the `agent group` in DynaTrace, and must be pre-configured on the DynaTrace server.
+A system profile may be provided as an optional argument (defaults to `Monitoring`).
+
+**Current Issues:**  
+* The DynaTrace agent slows down app execution significantly at first, but gets faster over time.  You may want to update your CF deployment manifest to set `maximum_health_check_timeout` to 180 or more and/or execute `cf push -t 180` or more when pushing a DynaTrace-monitored application.
+
+* As you `cf push` multiple times, many dead penguins will litter the DynaTrace agent dashboard, as CF launches/disposes application containers.  These can be hidden but will collect in the dynatrace database.
+
+<table>
+  <tr>
+    <td><strong>Detection Criterion</strong></td><td>Existence of a single bound DynaTrace service.
+      <ul>
+        <li>Existence of a DynaTrace service is defined as the <a href="http://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-SERVICES"><code>VCAP_SERVICES</code></a> payload containing a service who's name, label or tag has <code>dynatrace</code> as a substring.</li>
+      </ul>
+    </td>
+  </tr>
+  <tr>
+    <td><strong>Tags</strong></td>
+    <td><tt>dyna-trace-agent=&lt;version&gt;</tt></td>
+  </tr>
+</table>
+Tags are printed to standard output by the buildpack detect script
+
+## User-Provided Service
+Users must provide their own DynaTrace service. A user-provided DynaTrace service must have a name or tag with `dynatrace` in it so that the DynaTrace Agent Framework will automatically configure the application to work with the service.
+
+The credential payload of the service may contain the following entries:
+
+| Name | Description
+| ---- | -----------
+| `server` | The DynaTrace collector hostname to connect to.   Use `host:port` format for a specific port number.
+| `profile` | (optional) The DynaTrace server profile this is associated with.   Uses `Monitoring` by default.
+
+**NOTE** Be sure to open an Application Security Group to your DynaTrace collector prior to starting your application:
+```
+$ cat security.json
+   [
+     {
+       "protocol": "tcp",
+       "destination": "dynatrace_host",
+       "ports": "9998"
+     }
+   ]
+
+$ cf create-security-group dynatrace_group ./security.json
+Creating security group dynatrace_group as admin
+OK
+
+$ cf bind-running-security-group dynatrace_group
+Binding security group dynatrace_group to defaults for running as admin
+OK
+
+TIP: Changes will not apply to existing running applications until they are restarted.
+```
+
+## Configuration
+For general information on configuring the buildpack, refer to [Configuration and Extension][].
+
+The framework can be configured by modifying the [`dyna_trace_agent.yml`][] file in the buildpack fork.  The framework uses the [`Repository` utility support][repositories] and so it supports the [version syntax][] defined there.
+
+| Name | Description
+| ---- | -----------
+| `repository_root` | The URL of the DynaTrace repository index ([details][repositories]).
+| `version` | The version of DynaTrace to use. This buildpack framework has been tested on 6.1.0.
+
+
+**NOTE:**  This framework does not connect to a pre-populated repository.  Instead you will need to create your own repository by:
+
+1.  Downloading the DynaTrace agent unix binary (in JAR format) to an HTTP-accesible location
+1.  Uploading an `index.yml` file with a mapping from the version of the agent to its location to the same HTTP-accessible location
+1.  Configuring the [`dyna_trace_agent.yml`][] file to point to the root of the repository holding both the index and agent binary
+
+Sample **`repository_root`** for [`dyna_trace_agent.yml`][] (under java-buildpack/config) assuming a bosh-lite setup and a local webserver (e.g. `brew install tomcat7`) on port 8080
+
+```
+repository_root: "http://files.192.168.50.1.xip.io:8080/fileserver/dynatrace"
+```
+
+The buildpack would look for an **`index.yml`** file at the specified **repository_root** for obtaining the DynaTrace agent.
+
+The index.yml at the repository_root location should have a entry matching the DynaTrace version and the corresponding DynaTrace agent download JAR
+
+```
+---
+6.1.0.7880: http://files.192.168.50.1.xip.io:8080/fileserver/dynatrace/dynatrace-agent-6.1.0.7880-unix.jar
+```
+
+Ensure the DynaTrace binary is available at the location indicated by the index.yml referred by the DynaTrace repository_root.
+
+[Configuration and Extension]: ../README.md#configuration-and-extension
+[`dyna_trace_agent.yml`]: ../config/dyna_trace_agent.yml
+[DynaTrace Service]: https://dynatrace.com
+[repositories]: extending-repositories.md
+[version syntax]: extending-repositories.md#version-syntax-and-ordering

lib/java_buildpack/component/java_opts.rb

@@ -42,6 +42,16 @@ def add_javaagent(path)
         add_preformatted_options "-javaagent:#{qualify_path path}"
       end
 
+      # Adds a +agentpath+ entry to the +JAVA_OPTS+.  Prepends +$PWD+ to the path (relative to the droplet root) to
+      # ensure that the path is always accurate.
+      #
+      # @param [Pathname] path the path to the +agentpath+ shared library
+      # @param [Properties] properties to append to the agentpath entry
+      # @return [JavaOpts]     +self+ for chaining
+      def add_agentpath_with_props(path, props)
+        add_preformatted_options "-agentpath:#{qualify_path path}=" + props.map { |k, v| "#{k}=#{v}" }.join(',')
+      end
+
       # Adds an +agentpath+ entry to the +JAVA_OPTS+. Prepends +$PWD+ to the path (relative to the droplet root) to
       # ensure that the path is always accurate.
       #

lib/java_buildpack/framework/dyna_trace_agent.rb

@@ -0,0 +1,95 @@
+# Encoding: utf-8
+# Cloud Foundry Java Buildpack
+# Copyright 2015 the original author or authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+require 'fileutils'
+require 'java_buildpack/component/versioned_dependency_component'
+require 'java_buildpack/framework'
+require 'java_buildpack/util/qualify_path'
+
+module JavaBuildpack
+  module Framework
+
+    # Encapsulates the functionality for enabling zero-touch Dynatrace support.
+    class DynaTraceAgent < JavaBuildpack::Component::VersionedDependencyComponent
+      include JavaBuildpack::Util
+
+      # (see JavaBuildpack::Component::BaseComponent#compile)
+      def compile
+        download_zip false
+        @droplet.copy_resources
+        FileUtils.mkdir(home_dir)
+        FileUtils.mv(@droplet.sandbox + 'agent/linux-x86-64/agent', home_dir)
+        delete_extra_files
+      end
+
+      # (see JavaBuildpack::Component::BaseComponent#release)
+      def release
+        @droplet.java_opts
+          .add_agentpath_with_props(agent_dir + 'libdtagent.so',
+                                    name: application_name + '_' + profile_name,
+                                    server: server)
+      end
+
+      protected
+
+      # (see JavaBuildpack::Component::VersionedDependencyComponent#supports?)
+      def supports?
+        @application.services.one_service? FILTER, 'server'
+      end
+
+      private
+
+      FILTER = /dynatrace/.freeze
+
+      private_constant :FILTER
+
+      def application_name
+        @application.details['application_name']
+      end
+
+      def profile_name
+        @application.services.find_service(FILTER)['credentials']['profile'] || 'Monitoring'
+      end
+
+      def agent_dir
+        @droplet.sandbox + 'home/agent/lib64'
+      end
+
+      def delete_extra_files
+        FileUtils.rm_rf(@droplet.sandbox + 'agent')
+        FileUtils.rm_rf(@droplet.sandbox + 'init.d')
+        FileUtils.rm_rf(@droplet.sandbox + 'com')
+        FileUtils.rm_rf(@droplet.sandbox + 'org')
+        FileUtils.rm_rf(@droplet.sandbox + 'META_INF')
+        FileUtils.rm_f(@droplet.sandbox + 'YouShouldNotHaveUnzippedMe.txt')
+      end
+
+      def logs_dir
+        @droplet.sandbox + 'home/log'
+      end
+
+      def home_dir
+        @droplet.sandbox + 'home'
+      end
+
+      def server
+        @application.services.find_service(FILTER)['credentials']['server']
+      end
+
+    end
+
+  end
+end

spec/fixtures/stub-dyna-trace-agent.jar

@@ -29,6 +29,18 @@
     expect(opts).to include('-javaagent:$PWD/.java-buildpack/java_opts/test-java-agent')
   end
 
+  it 'adds a qualified agentpath to the collection' do
+    opts.add_agentpath droplet.sandbox + 'test-agentpath'
+
+    expect(opts).to include('-agentpath:$PWD/.java-buildpack/java_opts/test-agentpath')
+  end
+
+  it 'adds a qualified agentpath with properties to the collection' do
+    opts.add_agentpath_with_props(droplet.sandbox + 'test-agentpath', 'key1' => 'value1', 'key2' => 'value2')
+
+    expect(opts).to include('-agentpath:$PWD/.java-buildpack/java_opts/test-agentpath=key1=value1,key2=value2')
+  end
+
   it 'adds a qualified system property to the collection' do
     opts.add_system_property 'test-key', droplet.sandbox
 

spec/java_buildpack/component/java_opts_spec.rb

@@ -0,0 +1,70 @@
+# Encoding: utf-8
+# Cloud Foundry Java Buildpack
+# Copyright 2015 the original author or authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+require 'spec_helper'
+require 'component_helper'
+require 'java_buildpack/framework/dyna_trace_agent'
+
+describe JavaBuildpack::Framework::DynaTraceAgent do
+  include_context 'component_helper'
+
+  it 'does not detect without dynatrace-n/a service' do
+    expect(component.detect).to be_nil
+  end
+
+  context do
+
+    before do
+      allow(services).to receive(:one_service?).with(/dynatrace/, 'server').and_return(true)
+      allow(services).to receive(:find_service).and_return('credentials' => { 'server' => 'test-host-name' })
+    end
+
+    it 'detects with dynatrace-n/a service' do
+      expect(component.detect).to eq("dyna-trace-agent=#{version}")
+    end
+
+    it 'expands DynaTrace agent zip',
+       cache_fixture: 'stub-dyna-trace-agent.jar' do
+
+      component.compile
+      expect(sandbox + 'home/agent/lib64/libdtagent.so').to exist
+      expect(sandbox + 'YouShouldNotHaveUnzippedMe.txt').not_to exist
+    end
+
+    it 'updates JAVA_OPTS' do
+      component.release
+      expect(java_opts).to include(
+        '-agentpath:$PWD/.java-buildpack/dyna_trace_agent/home/agent/lib64/'\
+        'libdtagent.so=name=test-application-name_Monitoring,server=test-host-name')
+    end
+  end
+
+  context do
+    before do
+      allow(services).to receive(:one_service?).with(/dynatrace/, 'server').and_return(true)
+      allow(services).to receive(:find_service).and_return('credentials' => { 'server' => 'test-host-name',
+                                                                              'profile' => 'test-profile' })
+    end
+
+    it 'updates JAVA_OPTS with custom profile' do
+      component.release
+      expect(java_opts).to include(
+        '-agentpath:$PWD/.java-buildpack/dyna_trace_agent/home/agent/lib64/'\
+        'libdtagent.so=name=test-application-name_test-profile,server=test-host-name')
+    end
+
+  end
+end