~ack/apache2-1

Owner: ack
Status: Approved
Vote: +2 (+2 needed for approval)

CPP?: No
OIL?: No

Support for xenial


Tests

Substrate Status Results Last Updated
lxc RETRY 19 days ago
aws RETRY 19 days ago
gce RETRY 19 days ago

Voted: +0
mbruzek wrote 3 months ago
This is a pretty easy change to approve. The existing charm fails some of our testing and quality guidelines. I would really like someone to take over this charm and rewrite it using reactive in the future.
Voted: +2
mbruzek wrote 3 months ago
Revision 2 passes my review.

Add Comment

Login to comment/vote on this review.


Policy Checklist

Description Unreviewed Pass Fail

General

Must verify that any software installed or utilized is verified as coming from the intended source. mbruzek
  • Any software installed from the Ubuntu or CentOS default archives satisfies this due to the apt and yum sources including cryptographic signing information.
  • Third party repositories must be listed as a configuration option that can be overridden by the user and not hard coded in the charm itself.
  • Launchpad PPAs are acceptable as the add-apt-repository command retrieves the keys securely.
  • Other third party repositories are acceptable if the signing key is embedded in the charm.
Must provide a means to protect users from known security vulnerabilities in a way consistent with best practices as defined by either operating system policies or upstream documentation. mbruzek
Basically, this means there must be instructions on how to apply updates if you use software not from distribution channels.
Must have hooks that are idempotent. mbruzek
Should be built using charm layers.
Should use Juju Resources to deliver required payloads.

Testing and Quality

charm proof must pass without errors or warnings. mbruzek
Must include passing unit, functional, or integration tests. mbruzek
Tests must exercise all relations. mbruzek
Tests must exercise config. mbruzek
set-config, unset-config, and re-set must be tested as a minimum
Must not use anything infrastructure-provider specific (i.e. querying EC2 metadata service). mbruzek
Must be self contained unless the charm is a proxy for an existing cloud service, e.g. ec2-elb charm.
Must not use symlinks. mbruzek
Bundles must only use promulgated charms, they cannot reference charms in personal namespaces. mbruzek
Must call Juju hook tools (relation-*, unit-*, config-*, etc) without a hard coded path. mbruzek
Should include a tests.yaml for all integration tests.

Metadata

Must include a full description of what the software does. mbruzek
Must include a maintainer email address for a team or individual who will be responsive to contact. mbruzek
Must include a license. Call the file 'copyright' and make sure all files' licenses are specified clearly. mbruzek
Must be under a Free license. mbruzek
Must have a well documented and valid README.md. mbruzek
Must describe the service. mbruzek
Must describe how it interacts with other services, if applicable. mbruzek
Must document the interfaces. mbruzek
Must show how to deploy the charm. mbruzek
Must define external dependencies, if applicable. mbruzek
Should link to a recommend production usage bundle and recommended configuration if this differs from the default. mbruzek
Should reference and link to upstream documentation and best practices.

Security

Must not run any network services using default passwords. mbruzek
Must verify and validate any external payload mbruzek
  • Known and understood packaging systems that verify packages like apt, pip, and yum are ok.
  • wget | sh style is not ok.
Should make use of whatever Mandatory Access Control system is provided by the distribution. mbruzek
Should avoid running services as root. mbruzek

All changes | Changes since last revision

Source Diff

Files changed 67

Inline diff comments 0

No comments yet.

Back to file index

Makefile

 1
--- 
 2
+++ Makefile
 3
@@ -0,0 +1,56 @@
 4
+PWD := $(shell pwd)
 5
+SOURCEDEPS_DIR ?= $(shell dirname $(PWD))/.sourcecode
 6
+HOOKS_DIR := $(PWD)/hooks
 7
+TEST_PREFIX := PYTHONPATH=$(HOOKS_DIR)
 8
+TEST_DIR := $(PWD)/hooks/tests
 9
+CHARM_DIR := $(PWD)
10
+PYTHON := /usr/bin/env python
11
+
12
+
13
+build: test lint proof
14
+
15
+revision:
16
+	@test -f revision || echo 0 > revision
17
+
18
+proof: revision
19
+	@echo Proofing charm...
20
+	@(charm proof $(PWD) || [ $$? -eq 100 ]) && echo OK
21
+	@test `cat revision` = 0 && rm revision
22
+
23
+/usr/bin/apt:
24
+	sudo apt-get install -y python-apt
25
+
26
+/usr/bin/virtualenv:
27
+	sudo apt-get install -y python-virtualenv
28
+
29
+/usr/lib/python2.7/dist-packages/jinja2:
30
+	sudo apt-get install -y python-jinja2
31
+
32
+.venv: /usr/bin/apt /usr/bin/virtualenv /usr/lib/python2.7/dist-packages/jinja2
33
+	virtualenv .venv --system-site-packages
34
+	.venv/bin/pip install -I nose testtools mock pyyaml
35
+
36
+test: .venv
37
+	@echo Starting tests...
38
+	@CHARM_DIR=$(CHARM_DIR) $(TEST_PREFIX) .venv/bin/nosetests -s $(TEST_DIR)
39
+
40
+lint:
41
+	@echo Checking for Python syntax...
42
+	@flake8 $(HOOKS_DIR) --ignore=E123,E402 --exclude=$(HOOKS_DIR)/charmhelpers && echo hooks OK
43
+	@flake8 tests --ignore=E123,E402 && echo tests OK
44
+
45
+sourcedeps: $(PWD)/config-manager.txt
46
+	@echo Updating source dependencies...
47
+	@$(PYTHON) cm.py -c $(PWD)/config-manager.txt \
48
+		-p $(SOURCEDEPS_DIR) \
49
+		-t $(PWD)
50
+	@$(PYTHON) build/charm-helpers/tools/charm_helpers_sync/charm_helpers_sync.py \
51
+		-c charm-helpers.yaml \
52
+		-b build/charm-helpers \
53
+		-d hooks/charmhelpers
54
+	@echo Do not forget to commit the updated files if any.
55
+
56
+clean:
57
+	rm -rf .venv
58
+
59
+.PHONY: revision proof test lint sourcedeps charm-payload
Back to file index

README.md

  1
--- 
  2
+++ README.md
  3
@@ -0,0 +1,327 @@
  4
+# Juju charm for Apache
  5
+
  6
+The Apache Software Foundation's goal is to build a secure,
  7
+efficient and extensible HTTP server as standards-compliant open
  8
+source software. The result has long been the number one web server
  9
+on the Internet.  It features support for HTTPS, virtual hosting,
 10
+CGI, SSI, IPv6, easy scripting and database integration,
 11
+request/response filtering, many flexible authentication schemes,
 12
+and more.
 13
+
 14
+## Development
 15
+
 16
+
 17
+The following steps are needed for testing and development of the
 18
+charm, but **not** for deployment:
 19
+
 20
+    sudo apt-get install python-software-properties
 21
+    sudo add-apt-repository ppa:chrisjohnston/flake8
 22
+    sudo apt-get update
 23
+    sudo apt-get install python-flake8 python-nose python-coverage \
 24
+                         python-testtools python-pyasn1 python-pyasn1-modules
 25
+
 26
+To fetch additional source dependencies and run the tests:
 27
+
 28
+    make build
 29
+
 30
+... will run the unit tests, run flake8 over the source to warn
 31
+about formatting issues and output a code coverage summary of the
 32
+'hooks.py' module.
 33
+
 34
+## How to deploy the charm
 35
+
 36
+Assuming you have a copy of the charm into a `charms/$distrocodename/apache2`
 37
+directory relative to your current directory.
 38
+
 39
+... then to perform a deployment execute the following steps:
 40
+
 41
+    juju deploy --repository=charms local:apache2
 42
+    juju set apache2 "vhost_http_template=$(base64 < http_vhost.tmpl)"
 43
+
 44
+
 45
+    # and / or
 46
+    juju set apache2 "vhost_https_template=$(base64 < https_vhost.tmpl)"
 47
+
 48
+If you want a simple `reverseproxy` relation to your services (only
 49
+really useful if you have a single unit on the other side of the
 50
+relation):
 51
+
 52
+    juju add-relation apache2:reverseproxy haproxy:website
 53
+    # and / or
 54
+    juju add-relation apache2:reverseproxy squid-reverseproxy:cached-website
 55
+
 56
+Alternatively, you can use the `balancer` relation so that requests
 57
+are load balanced across multiple units of your services. For more information see the section on `Using the balancer relation`:
 58
+
 59
+    juju add-relation apache2:balancer haproxy:website
 60
+    # and / or
 61
+    juju add-relation apache2:balancer squid-reverseproxy:cached-website
 62
+
 63
+## VirtualHost templates
 64
+
 65
+The charm expects a jinja2 template to be passed in. The variables
 66
+in the template should relate to the services that apache will be
 67
+proxying -- obviously no variables need to be specified if no
 68
+proxying is needed.
 69
+
 70
+Virtual host templates can also be specified via relation.  See the
 71
+vhost-config relation section below for more information.
 72
+
 73
+The vhost_template_vars config allows for further customisation of the vhost
 74
+templates. For example, you can have a single template for a particular
 75
+service, but use vhost_template_vars to customise it slightly for
 76
+devel/staging/production environments.
 77
+
 78
+### Using the reverseproxy relation
 79
+
 80
+The charm will create the service variable, with the `unit_name`,
 81
+when the `reverseproxy` relationship is joined and present this to
 82
+the template at which point the vhost will be generated from the
 83
+template again.  All config settings are also available to the
 84
+template.
 85
+
 86
+For example to access squid then the `{{ squid }}` variable should
 87
+be used.  This will be populated with the hostname:port of the squid
 88
+service. The individual hostname and port can also be accessed via
 89
+`squid_hostname` and `squid_port`.
 90
+
 91
+Note: The service name should be used, not the charm name.  If
 92
+      deploying a charm with a different service name, use that
 93
+      instead.
 94
+
 95
+The joining charm may set an `all_services` variable which
 96
+contains a list of services it provides in yaml format (list of
 97
+associative arrays):
 98
+
 99
+    # ... in haproxy charm, website-relation-joined
100
+    relation-set all_services="
101
+      - {service_name: gunicorn, service_port: 80}
102
+      - {service_name: solr, service_port: 8080}
103
+      - {service_name: my-webapp, service_port: 9090}
104
+    "
105
+
106
+then variables for each service would be available to the jinja2
107
+template in `<juju_service_name>_<sub_service_name>`.  In our example
108
+above haproxy contains stanzas named gunicorn, solr and my-webapp.
109
+These are accessed as `{{ haproxy_gunicorn }}`, `{{ haproxy_solr }}` and
110
+`{{ haproxy_mywebapp }}` respectively.  If any unsupported characters
111
+are in your juju service name or the service names exposed through
112
+"all_services", they will be stripped.
113
+
114
+For example a vhost that will pass all traffic on to an haproxy instance:
115
+
116
+    <VirtualHost *:80>
117
+        ServerName radiotiptop.org.uk
118
+
119
+        CustomLog /var/log/apache2/radiotiptop-access.log combined
120
+        ErrorLog /var/log/apache2/radiotiptop-error.log
121
+
122
+        DocumentRoot /srv/radiotiptop/www/root
123
+
124
+        ProxyRequests off
125
+        <Proxy *>
126
+            Order Allow,Deny
127
+            Allow from All
128
+            ErrorDocument 403 /offline.html
129
+            ErrorDocument 500 /offline.html
130
+            ErrorDocument 502 /offline.html
131
+            ErrorDocument 503 /offline.html
132
+        </Proxy>
133
+
134
+        ProxyPreserveHost off
135
+        ProxyPassReverse / http://{{ haproxy_gunicorn }}/
136
+
137
+        RewriteEngine on
138
+
139
+        RewriteRule ^/$ /index.html [L]
140
+        RewriteRule ^/(.*)$ http://{{ haproxy_gunicorn }}/$1 [P,L]
141
+    </VirtualHost>
142
+
143
+### Using the `balancer` relation
144
+
145
+Using the balancer relation will set up named balancers using
146
+Apache's mod_balancer. Each balancer will be named after the
147
+`sitenames` or `all_services` setting exported from the other side
148
+of the relation. Requests sent through those balancers will have a
149
+`X-Balancer-Name` header set, which can be used by the related
150
+service to appropriatedly route requests internally.
151
+
152
+The joining charm may set an `all_services` variable which
153
+contains a list of services it provides in yaml format (list of
154
+associative arrays):
155
+
156
+    # ... in haproxy charm, website-relation-joined
157
+    relation-set all_services="
158
+      - {service_name: gunicorn, service_port: 80}
159
+      - {service_name: solr, service_port: 8080}
160
+      - {service_name: my-webapp, service_port: 9090}
161
+    "
162
+
163
+Each separate service name will cause a new `balancer` definition to be created on the Apache side, like:
164
+
165
+  <Proxy balancer://gunicorn>
166
+    ProxySet lbmethod=byrequests
167
+    RequestHeader set X-Balancer-Name "gunicorn"
168
+  </Proxy>
169
+
170
+For example a vhost that will pass specific requests to the `gunicorn` service that's defined in haproxy:
171
+
172
+    <VirtualHost *:80>
173
+        ServerName radiotiptop.org.uk
174
+
175
+        CustomLog /var/log/apache2/radiotiptop-access.log combined
176
+        ErrorLog /var/log/apache2/radiotiptop-error.log
177
+
178
+        DocumentRoot /srv/radiotiptop/www/root
179
+
180
+        ProxyRequests off
181
+        <Proxy *>
182
+            Order Allow,Deny
183
+            Allow from All
184
+            ErrorDocument 403 /offline.html
185
+            ErrorDocument 500 /offline.html
186
+            ErrorDocument 502 /offline.html
187
+            ErrorDocument 503 /offline.html
188
+        </Proxy>
189
+
190
+        ProxyPreserveHost on
191
+
192
+        RewriteEngine on
193
+
194
+        RewriteRule ^/$ /index.html [L]
195
+        RewriteRule ^/(.*)$ balancer://gunicorn/$1 [P,L]
196
+    </VirtualHost>
197
+
198
+### Using the vhost-config relation
199
+
200
+The nice thing about this relation, is as long as a charm support it, deploying
201
+apache as a front-end for a web service should be as simple as establishing the
202
+relation.  If you need more details for how to implement this, read on.
203
+
204
+The template files themselves can be specified via this relation.  This makes
205
+deployment of your infrastructure simpler, since users no longer need to
206
+specify a vhosts config option when using apache2 (though they still can).  A
207
+candidate charm should provide a relation on the `apache-vhost-config`
208
+interface.  This charm should simply set the following data when relating:
209
+
210
+    relation-set vhosts="- {port: '443', template: dGVtcGxhdGU=}\n- {port: '80', template: dGVtcGxhdGU=}\n"
211
+
212
+Notice the `vhosts` definition is in yaml, the format is simple. `vhosts`
213
+should contain a yaml encoded data structure of a list of key value hashes, or
214
+dictionaries.  In each dictionary, `port` should be set to the port this vhost
215
+should listen on, `template` should be set to the base64 encoded template file.
216
+You can include as many of these dictionaries as you would like.  If you have
217
+colliding port numbers across your juju infrastructure, the results will be a
218
+bit unpredictable.
219
+
220
+For example, if using python for your relating charm, the code to generate a
221
+yaml_string for a vhost on port `80` would be similar to this:
222
+
223
+    import yaml
224
+    import base64
225
+    template = get_template()
226
+    vhosts = [{"port": "80", "template": base64.b64encode(template)}]
227
+    yaml_string = yaml.dump(vhosts)
228
+
229
+Note, that if you are opening a non-standard port (80 and 443 are opened and
230
+understood by the default install of apache2 in Ubuntu) you will need to
231
+instruct Apache to `Listen` on that port in your vhost file.  Something like the
232
+following will work in your vhost template:
233
+
234
+    Listen 8080
235
+    <VirtualHost *:8080>
236
+    ...
237
+    </VirtualHost>
238
+
239
+
240
+#### Relation settings that apache2 provides
241
+
242
+When your charm relates it will be provided with the following:
243
+
244
+ * `servername` - The Apache2 servername.  This is typically needed by web
245
+   applications so they know how to write URLs.
246
+
247
+ * `ssl_cert` - If you asked for a selfsigned certificate, that cert will
248
+   be available in this setting as a base64 encoded string.
249
+
250
+
251
+### Using the apache-website relation
252
+
253
+The apache-website relation provides a very flexible way of configuring an
254
+Apache2 website, using subordinate charms.  It can support reverse proxies,
255
+static websites, and probably many other forms.
256
+
257
+To support this relation, a charm must set
258
+
259
+ * `domain` - The fully-qualified domain name of the site.
260
+
261
+ * `enabled` - Must be set to 'true' when the web site is ready to be used.
262
+
263
+ * `site_config` - A vhost configuration block.
264
+
265
+ * `site_modules` - A list of modules required by the site.  If any of these
266
+   appear in `disable_modules`, the site will not be enabled.  Otherwise, any
267
+   required modules will be loaded.
268
+
269
+ * `ports` - A space-separated list of ports that the site uses.
270
+
271
+### Using the logs relation
272
+
273
+The logs relation is for use with a logging subordinate charm. The beaver
274
+subordinate can be deployed and related to apache and logstash. Beaver will
275
+tail apache logs and send the logs to logstash.
276
+
277
+## Certs, keys and chains
278
+
279
+`ssl_keylocation`, `ssl_certlocation` and `ssl_chainlocation` are
280
+file names in the charm `/data` directory.  If found, they will be
281
+copied as follows:
282
+
283
+  - /etc/ssl/private/<ssl_keylocation>
284
+  - /etc/ssl/certs/<ssl_certlocation>
285
+  - /etc/ssl/certs/<ssl_chainlocation>
286
+
287
+`ssl_key` and `ssl_cert` can also be specified which are are assumed
288
+to be base64 encoded.  If specified, they will be written to
289
+appropriate directories given the values in ssl_keylocation and
290
+ssl_certlocation as listed above.
291
+
292
+`ssl_cert` may also be set to SELFSIGNED, which will generate a
293
+certificate.  This, of course, is mostly useful for testing and
294
+staging purposes.  The generated certifcate/key will be placed
295
+according to `ssl_certlocation` and `ssl_keylocation` as listed
296
+above.
297
+
298
+`ssl_protocol`, `ssl_honor_cipher_order`, and `ssl_cipher_suite` can
299
+be used to override SSL/TLS version and the cipher suites supported.
300
+These default to what Canonical IS recommends and is using. Before
301
+making any changes, please see the Mozilla Security/Server Side TLS.
302
+
303
+## `{enable,disable}_modules`
304
+
305
+Space separated list of modules to be enabled or disabled. If a module to
306
+be enabled cannot be found then the charm will attempt to install it.
307
+
308
+## OpenId 
309
+
310
+The openid_provider option takes a comma seperated list of OpenID
311
+providers and places them in /etc/apache2/security/allowed-ops.txt. That
312
+file can then be refernced by the allowed-op-list-url option when using
313
+apache_openid
314
+
315
+## TODO:
316
+
317
+  * Document the use of balancer, nrpe, logging and website-cache
318
+
319
+  * Method to deliver site content. This maybe by converting the
320
+    charm to a subordinate and making it the master charms problem
321
+
322
+  * Implement secure method for delivering key.  Juju will likely
323
+    need to provide this.
324
+
325
+  * Tuning. No tuning options are present. Convert apache2.conf to a
326
+    template and expose config options
327
+
328
+  * The all_services variable can be passed as part of the http interface and is
329
+    optional. However its kind of secret and it would be more obvious if a
330
+    seperate interface was used like http-allservices.
Back to file index

charm-helpers.yaml

1
--- 
2
+++ charm-helpers.yaml
3
@@ -0,0 +1,4 @@
4
+include:
5
+    - core.hookenv
6
+    - fetch
7
+    - contrib.charmsupport
Back to file index

cm.py

  1
--- 
  2
+++ cm.py
  3
@@ -0,0 +1,193 @@
  4
+# Copyright 2010-2013 Canonical Ltd. All rights reserved.
  5
+import os
  6
+import re
  7
+import sys
  8
+import errno
  9
+import hashlib
 10
+import subprocess
 11
+import optparse
 12
+
 13
+from os import curdir
 14
+from bzrlib.branch import Branch
 15
+from bzrlib.plugin import load_plugins
 16
+load_plugins()
 17
+from bzrlib.plugins.launchpad import account as lp_account
 18
+
 19
+if 'GlobalConfig' in dir(lp_account):
 20
+    from bzrlib.config import LocationConfig as LocationConfiguration
 21
+    _ = LocationConfiguration
 22
+else:
 23
+    from bzrlib.config import LocationStack as LocationConfiguration
 24
+    _ = LocationConfiguration
 25
+
 26
+
 27
+def get_branch_config(config_file):
 28
+    """
 29
+    Retrieves the sourcedeps configuration for an source dir.
 30
+    Returns a dict of (branch, revspec) tuples, keyed by branch name.
 31
+    """
 32
+    branches = {}
 33
+    with open(config_file, 'r') as stream:
 34
+        for line in stream:
 35
+            line = line.split('#')[0].strip()
 36
+            bzr_match = re.match(r'(\S+)\s+'
 37
+                                 'lp:([^;]+)'
 38
+                                 '(?:;revno=(\d+))?', line)
 39
+            if bzr_match:
 40
+                name, branch, revno = bzr_match.group(1, 2, 3)
 41
+                if revno is None:
 42
+                    revspec = -1
 43
+                else:
 44
+                    revspec = revno
 45
+                branches[name] = (branch, revspec)
 46
+                continue
 47
+            dir_match = re.match(r'(\S+)\s+'
 48
+                                 '\(directory\)', line)
 49
+            if dir_match:
 50
+                name = dir_match.group(1)
 51
+                branches[name] = None
 52
+    return branches
 53
+
 54
+
 55
+def main(config_file, parent_dir, target_dir, verbose):
 56
+    """Do the deed."""
 57
+
 58
+    try:
 59
+        os.makedirs(parent_dir)
 60
+    except OSError, e:
 61
+        if e.errno != errno.EEXIST:
 62
+            raise
 63
+
 64
+    branches = sorted(get_branch_config(config_file).items())
 65
+    for branch_name, spec in branches:
 66
+        if spec is None:
 67
+            # It's a directory, just create it and move on.
 68
+            destination_path = os.path.join(target_dir, branch_name)
 69
+            if not os.path.isdir(destination_path):
 70
+                os.makedirs(destination_path)
 71
+            continue
 72
+
 73
+        (quoted_branch_spec, revspec) = spec
 74
+        revno = int(revspec)
 75
+
 76
+        # qualify mirror branch name with hash of remote repo path to deal
 77
+        # with changes to the remote branch URL over time
 78
+        branch_spec_digest = hashlib.sha1(quoted_branch_spec).hexdigest()
 79
+        branch_directory = branch_spec_digest
 80
+
 81
+        source_path = os.path.join(parent_dir, branch_directory)
 82
+        destination_path = os.path.join(target_dir, branch_name)
 83
+
 84
+        # Remove leftover symlinks/stray files.
 85
+        try:
 86
+            os.remove(destination_path)
 87
+        except OSError, e:
 88
+            if e.errno != errno.EISDIR and e.errno != errno.ENOENT:
 89
+                raise
 90
+
 91
+        lp_url = "lp:" + quoted_branch_spec
 92
+
 93
+        # Create the local mirror branch if it doesn't already exist
 94
+        if verbose:
 95
+            sys.stderr.write('%30s: ' % (branch_name,))
 96
+            sys.stderr.flush()
 97
+
 98
+        fresh = False
 99
+        if not os.path.exists(source_path):
100
+            subprocess.check_call(['bzr', 'branch', '-q', '--no-tree',
101
+                                   '--', lp_url, source_path])
102
+            fresh = True
103
+
104
+        if not fresh:
105
+            source_branch = Branch.open(source_path)
106
+            if revno == -1:
107
+                orig_branch = Branch.open(lp_url)
108
+                fresh = source_branch.revno() == orig_branch.revno()
109
+            else:
110
+                fresh = source_branch.revno() == revno
111
+
112
+        # Freshen the source branch if required.
113
+        if not fresh:
114
+            subprocess.check_call(['bzr', 'pull', '-q', '--overwrite', '-r',
115
+                                   str(revno), '-d', source_path,
116
+                                   '--', lp_url])
117
+
118
+        if os.path.exists(destination_path):
119
+            # Overwrite the destination with the appropriate revision.
120
+            subprocess.check_call(['bzr', 'clean-tree', '--force', '-q',
121
+                                   '--ignored', '-d', destination_path])
122
+            subprocess.check_call(['bzr', 'pull', '-q', '--overwrite',
123
+                                   '-r', str(revno),
124
+                                   '-d', destination_path, '--', source_path])
125
+        else:
126
+            # Create a new branch.
127
+            subprocess.check_call(['bzr', 'branch', '-q', '--hardlink',
128
+                                   '-r', str(revno),
129
+                                   '--', source_path, destination_path])
130
+
131
+        # Check the state of the destination branch.
132
+        destination_branch = Branch.open(destination_path)
133
+        destination_revno = destination_branch.revno()
134
+
135
+        if verbose:
136
+            sys.stderr.write('checked out %4s of %s\n' %
137
+                             ("r" + str(destination_revno), lp_url))
138
+            sys.stderr.flush()
139
+
140
+        if revno != -1 and destination_revno != revno:
141
+            raise RuntimeError("Expected revno %d but got revno %d" %
142
+                               (revno, destination_revno))
143
+
144
+if __name__ == '__main__':
145
+    parser = optparse.OptionParser(
146
+        usage="%prog [options]",
147
+        description=(
148
+            "Add a lightweight checkout in <target> for each "
149
+            "corresponding file in <parent>."),
150
+        add_help_option=False)
151
+    parser.add_option(
152
+        '-p', '--parent', dest='parent',
153
+        default=None,
154
+        help=("The directory of the parent tree."),
155
+        metavar="DIR")
156
+    parser.add_option(
157
+        '-t', '--target', dest='target', default=curdir,
158
+        help=("The directory of the target tree."),
159
+        metavar="DIR")
160
+    parser.add_option(
161
+        '-c', '--config', dest='config', default=None,
162
+        help=("The config file to be used for config-manager."),
163
+        metavar="DIR")
164
+    parser.add_option(
165
+        '-q', '--quiet', dest='verbose', action='store_false',
166
+        help="Be less verbose.")
167
+    parser.add_option(
168
+        '-v', '--verbose', dest='verbose', action='store_true',
169
+        help="Be more verbose.")
170
+    parser.add_option(
171
+        '-h', '--help', action='help',
172
+        help="Show this help message and exit.")
173
+    parser.set_defaults(verbose=True)
174
+
175
+    options, args = parser.parse_args()
176
+
177
+    if options.parent is None:
178
+        options.parent = os.environ.get(
179
+            "SOURCEDEPS_DIR",
180
+            os.path.join(curdir, ".sourcecode"))
181
+
182
+    if options.target is None:
183
+        parser.error(
184
+            "Target directory not specified.")
185
+
186
+    if options.config is None:
187
+        config = [arg for arg in args
188
+                  if arg != "update"]
189
+        if not config or len(config) > 1:
190
+            parser.error("Config not specified")
191
+        options.config = config[0]
192
+
193
+    sys.exit(main(config_file=options.config,
194
+                  parent_dir=options.parent,
195
+                  target_dir=options.target,
196
+                  verbose=options.verbose))
Back to file index

config-manager.txt

1
--- 
2
+++ config-manager.txt
3
@@ -0,0 +1,6 @@
4
+# After making changes to this file, to ensure that your sourcedeps are
5
+# up-to-date do:
6
+#
7
+#   make sourcedeps
8
+
9
+./build/charm-helpers                   lp:charm-helpers;revno=330
Back to file index

config.yaml

  1
--- 
  2
+++ config.yaml
  3
@@ -0,0 +1,208 @@
  4
+options:
  5
+  servername:
  6
+    type: string
  7
+    default: ''
  8
+    description: ServerName for vhost, defaults to the units public-address
  9
+  vhost_http_template:
 10
+    type: string
 11
+    default: ''
 12
+    description: Apache vhost template (base64 encoded).
 13
+  vhost_https_template:
 14
+    type: string
 15
+    default: ''
 16
+    description: Apache vhost template (base64 encoded).
 17
+  vhost_template_vars:
 18
+    type: string
 19
+    default: ''
 20
+    description: Additional custom variables for the vhost templating, in python dict format
 21
+  enable_modules:
 22
+    type: string
 23
+    default: ''
 24
+    description: List of modules to enable
 25
+  disable_modules:
 26
+    type: string
 27
+    default: 'status autoindex'
 28
+    description: List of modules to disable
 29
+  config_change_command:
 30
+    type: string
 31
+    default: "reload"
 32
+    description: |
 33
+       The command to run whenever config has changed. Accepted values are
 34
+       "reload" or "restart" - any other value will mean neither is executed
 35
+       after a config change (which may be desired, if you're running a
 36
+       production server and would rather handle these out of band). Note:
 37
+       some variables like the mpm settings require a restart to go into effect.
 38
+  mpm_type:
 39
+    type: string
 40
+    default: 'worker'
 41
+    description: worker or prefork
 42
+  ssl_keylocation:
 43
+    type: string
 44
+    default: ''
 45
+    description: |
 46
+        Name and location of ssl keyfile in charm/data directory.
 47
+        If not found, will ignore.  Basename of this file will be used
 48
+        as the basename of the key rooted at /etc/ssl/private.  Can
 49
+        be used in conjuntion with the ssl_key parameter to specify
 50
+        the key as a configuration setting.
 51
+  ssl_certlocation:
 52
+    type: string
 53
+    default: ''
 54
+    description: |
 55
+        Name and location of ssl certificate in charm/data directory.
 56
+        If not found, will ignore.  Basename of this file will be used
 57
+        as the basename of the cert rooted at /etc/ssl/certs.  Can
 58
+        be used in conjunction with the ssl_cert parameter to specify
 59
+        the cert as a configuration setting.
 60
+  ssl_chainlocation:
 61
+    type: string
 62
+    default: ''
 63
+    description: |
 64
+        Name and location of the ssl chain file.  Basename of this file
 65
+        will be used as the basename of the chain file rooted at
 66
+        /etc/ssl/certs.
 67
+  lb_balancer_timeout:
 68
+    type: int
 69
+    default: 60
 70
+    description: >
 71
+       How long the backends in mod_proxy_balancer will timeout, in seconds
 72
+  mpm_startservers:
 73
+    type: int
 74
+    default: 2
 75
+    description: Add desc
 76
+  mpm_minsparethreads:
 77
+    type: int
 78
+    default: 25
 79
+    description: Add desc
 80
+  mpm_maxsparethreads:
 81
+    type: int
 82
+    default: 75
 83
+    description: Add desc
 84
+  mpm_threadlimit:
 85
+    type: int
 86
+    default: 64
 87
+    description: Add desc
 88
+  mpm_threadsperchild:
 89
+    type: int
 90
+    default: 64
 91
+    description: Add desc
 92
+  mpm_serverlimit:
 93
+    type: int
 94
+    default: 128
 95
+    description: Add desc
 96
+  mpm_maxclients:
 97
+    type: int
 98
+    default: 2048
 99
+    description: Add desc
100
+  mpm_maxrequestsperchild:
101
+    type: int
102
+    default: 0
103
+    description: Add desc
104
+  nagios_context:
105
+    default: "juju"
106
+    type: string
107
+    description: >
108
+        Used by the nrpe-external-master subordinate charm.
109
+        A string that will be prepended to instance name to set the host name
110
+        in nagios. So for instance the hostname would be something like:
111
+            juju-postgresql-0
112
+        If you're running multiple environments with the same services in them
113
+        this allows you to differentiate between them.
114
+  nagios_servicegroups:
115
+    default: ""
116
+    type: string
117
+    description: >
118
+        A comma-separated list of nagios servicegroups.
119
+        If left empty, the nagios_context will be used as the servicegroup
120
+  nagios_check_http_params:
121
+     default: ""
122
+     type: string
123
+     description: The parameters to pass to the nrpe plugin check_http.
124
+  logrotate_rotate:
125
+    type: string
126
+    description: daily, weekly, monthly, or yearly?
127
+    default: "daily"
128
+  logrotate_count:
129
+    type: int
130
+    description: The number of days we want to retain logs for
131
+    default: 365
132
+  logrotate_dateext:
133
+    type: boolean
134
+    description: >
135
+      Use daily extension like YYYMMDD instead of simply adding a number
136
+    default: True
137
+  package_status:
138
+    default: "install"
139
+    type: "string"
140
+    description: >
141
+      The status of service-affecting packages will be set to this value in the dpkg database.
142
+      Useful valid values are "install" and "hold".
143
+  use_rsyslog:
144
+    type: boolean
145
+    description: >-
146
+      Change logging behaviour to log both access and error logs via rsyslog
147
+    default: False
148
+  ssl_cert:
149
+    type: string
150
+    description: |
151
+        base64 encoded server certificate.  If the keyword 'SELFSIGNED'
152
+        is used, the certificate and key will be autogenerated as
153
+        self-signed.
154
+    default: ''
155
+  ssl_key:
156
+    type: string
157
+    description: |
158
+        base64 encoded server certificate key.  If ssl_cert is
159
+        specified as SELFSIGNED, this will be ignored.
160
+    default: ''
161
+  ssl_chain:
162
+    type: string
163
+    description: |
164
+        base64 encoded chain certificates file.  If ssl_cert is
165
+        specified as SELFSIGNED, this will be ignored.
166
+    default: ''
167
+  ssl_protocol:
168
+    type: string
169
+    description: SSL Protocols to enable.
170
+    default: "ALL -SSLv2 -SSLv3"
171
+  ssl_honor_cipher_order:
172
+    type: string
173
+    description: Enable server cipher suite preference.
174
+    default: "On"
175
+  ssl_cipher_suite:
176
+    type: string
177
+    description: List of server cipher suites.
178
+    default: "EECDH+AESGCM+AES128:EDH+AESGCM+AES128:EECDH+AES128:EDH+AES128:ECDH+AESGCM+AES128:aRSA+AESGCM+AES128:ECDH+AES128:DH+AES128:aRSA+AES128:EECDH+AESGCM:EDH+AESGCM:EECDH:EDH:ECDH+AESGCM:aRSA+AESGCM:ECDH:DH:aRSA:HIGH:!MEDIUM:!aNULL:!NULL:!LOW:!3DES:!DSS:!EXP:!PSK:!SRP"
179
+  server_tokens:
180
+    type: string
181
+    description: Security setting. Set to one of  Full  OS  Minimal  Minor  Major  Prod
182
+    default: "OS"
183
+  server_signature:
184
+    type: string
185
+    description: Security setting. Set to one of  On  Off  EMail
186
+    default: "On"
187
+  trace_enabled:
188
+    type: string
189
+    description: Security setting. Set to one of  On  Off  extended
190
+    default: "On"
191
+  extra_packages:
192
+    type: string
193
+    description: List of extra packages to be installed (e.g. commercial GeoIP package)
194
+    default: ""
195
+  openid_provider:
196
+    type: string
197
+    description: Comma seperated list of OpenID providers for authentication.
198
+    default: ""
199
+  apt-key-id:
200
+    type: string
201
+    default: ""
202
+    description: A PGP key id.  This is used with PPA and the source
203
+      option to import a PGP public key for verifying repository signatures.
204
+      This value must match the PPA for apt-source.
205
+  apt-source:
206
+    type: string
207
+    default: ""
208
+    description: From where to install packages. This is the PPA source line.
209
+      Note that due to a bug in software-properties add-apt-repository cannot
210
+        add the ondrej/apache2 ppa, so the default value here is a full
211
+        sources line.
Back to file index

copyright

 1
--- 
 2
+++ copyright
 3
@@ -0,0 +1,17 @@
 4
+Format: http://dep.debian.net/deps/dep5/
 5
+
 6
+Files: *
 7
+Copyright: Copyright 2012, Canonical Ltd., All Rights Reserved.
 8
+License: GPL-3
 9
+ This program is free software: you can redistribute it and/or modify
10
+ it under the terms of the GNU General Public License as published by
11
+ the Free Software Foundation, either version 3 of the License, or
12
+ (at your option) any later version.
13
+ .
14
+ This program is distributed in the hope that it will be useful,
15
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
16
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
17
+ GNU General Public License for more details.
18
+ .
19
+ You should have received a copy of the GNU General Public License
20
+ along with this program.  If not, see <http://www.gnu.org/licenses/>.
Back to file index

data/balancer.template

 1
--- 
 2
+++ data/balancer.template
 3
@@ -0,0 +1,7 @@
 4
+<Proxy balancer://{{ balancer_name }}>
 5
+{% for host in balancer_addresses -%}
 6
+    BalancerMember http://{{ host }} timeout={{ lb_balancer_timeout }}
 7
+{% endfor %}
 8
+    ProxySet lbmethod=byrequests
 9
+    RequestHeader set X-Balancer-Name "{{ balancer_name }}"
10
+</Proxy>
Back to file index

data/logrotate.conf.template

 1
--- 
 2
+++ data/logrotate.conf.template
 3
@@ -0,0 +1,31 @@
 4
+{{ juju_warning_header }}
 5
+
 6
+/var/log/apache2/*.log {
 7
+        {{ logrotate_rotate }}
 8
+{%- if logrotate_dateext %}
 9
+        dateext
10
+{%- endif %}
11
+        missingok
12
+        rotate {{ logrotate_count }}
13
+        compress
14
+        delaycompress
15
+        notifempty
16
+{%- if use_rsyslog %}
17
+        create 640 syslog adm
18
+        sharedscripts
19
+        postrotate
20
+                reload rsyslog >/dev/null 2>&1 || true
21
+        endscript
22
+{%- else %}
23
+        create 640 root adm
24
+        sharedscripts
25
+        postrotate
26
+                /etc/init.d/apache2 reload > /dev/null
27
+        endscript
28
+        prerotate
29
+                if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
30
+                        run-parts /etc/logrotate.d/httpd-prerotate; \
31
+                fi; \
32
+        endscript
33
+{%- endif %}
34
+}
Back to file index

data/mpm_worker.template

 1
--- 
 2
+++ data/mpm_worker.template
 3
@@ -0,0 +1,10 @@
 4
+<IfModule mpm_worker_module>
 5
+    StartServers          {{ mpm_startservers }}
 6
+    MinSpareThreads       {{ mpm_minsparethreads }}
 7
+    MaxSpareThreads       {{ mpm_maxsparethreads }} 
 8
+    ThreadLimit           {{ mpm_threadlimit }}
 9
+    ThreadsPerChild       {{ mpm_threadsperchild }}
10
+    ServerLimit           {{ mpm_serverlimit }}
11
+    MaxClients            {{ mpm_maxclients }}
12
+    MaxRequestsPerChild   {{ mpm_maxrequestsperchild }}
13
+</IfModule>
Back to file index

data/openssl.cnf

 1
--- 
 2
+++ data/openssl.cnf
 3
@@ -0,0 +1,20 @@
 4
+RANDFILE                = /dev/urandom
 5
+
 6
+[ req ]
 7
+default_bits            = 1024
 8
+default_keyfile         = privkey.pem
 9
+distinguished_name      = req_distinguished_name
10
+prompt                  = no
11
+policy                  = policy_anything
12
+x509_extensions         = v3_ca
13
+
14
+[ req_distinguished_name ]
15
+commonName              = $ENV::OPENSSL_CN
16
+
17
+[ v3_ca ]
18
+# Extensions to add to a certificate request
19
+subjectAltName          = @alt_names
20
+
21
+[alt_names]
22
+DNS.1   = $ENV::OPENSSL_PUBLIC
23
+DNS.2   = $ENV::OPENSSL_PRIVATE
Back to file index

data/security.template

 1
--- 
 2
+++ data/security.template
 3
@@ -0,0 +1,58 @@
 4
+{{ juju_warning_header }}
 5
+#
 6
+# Disable access to the entire file system except for the directories that
 7
+# are explicitly allowed later.
 8
+#
 9
+# This currently breaks the configurations that come with some web application
10
+# Debian packages.
11
+
12
+<Directory />
13
+       AllowOverride None
14
+{%- if is_apache24 %}
15
+       Require all denied
16
+{%- else %}
17
+       Order Deny,Allow
18
+       Deny from all
19
+{%- endif %}
20
+</Directory>
21
+
22
+
23
+
24
+# Changing the following options will not really affect the security of the
25
+# server, but might make attacks slightly more difficult in some cases.
26
+
27
+#
28
+# ServerTokens
29
+# This directive configures what you return as the Server HTTP response
30
+# Header. The default is 'Full' which sends information about the OS-Type
31
+# and compiled in modules.
32
+# Set to one of:  Full | OS | Minimal | Minor | Major | Prod
33
+# where Full conveys the most information, and Prod the least.
34
+#
35
+ServerTokens {{ server_tokens }}
36
+
37
+#
38
+# Optionally add a line containing the server version and virtual host
39
+# name to server-generated pages (internal error documents, FTP directory
40
+# listings, mod_status and mod_info output etc., but not CGI generated
41
+# documents or custom error documents).
42
+# Set to "EMail" to also include a mailto: link to the ServerAdmin.
43
+# Set to one of:  On | Off | EMail
44
+#
45
+ServerSignature {{ server_signature }}
46
+
47
+#
48
+# Allow TRACE method
49
+#
50
+# Set to "extended" to also reflect the request body (only for testing and
51
+# diagnostic purposes).
52
+#
53
+# Set to one of:  On | Off | extended
54
+#
55
+TraceEnable {{ trace_enabled }}
56
+
57
+<IfModule mod_ssl.c>
58
+	SSLProtocol {{ ssl_protocol }}
59
+	SSLHonorCipherOrder {{ ssl_honor_cipher_order }}
60
+	SSLCipherSuite {{ ssl_cipher_suite }}
61
+</IfModule>
Back to file index

data/syslog-apache.conf

 1
--- 
 2
+++ data/syslog-apache.conf
 3
@@ -0,0 +1,13 @@
 4
+#
 5
+#    "             "
 6
+#  mmm   m   m   mmm   m   m
 7
+#    #   #   #     #   #   #
 8
+#    #   #   #     #   #   #
 9
+#    #   "mm"#     #   "mm"#
10
+#    #             #
11
+#  ""            ""
12
+# This file is managed by Juju. Do not make local changes.
13
+#
14
+
15
+ErrorLog syslog
16
+CustomLog "|/usr/bin/logger -p local0.info -t apache2" vhost_combined
Back to file index

data/syslog-rsyslog.conf

 1
--- 
 2
+++ data/syslog-rsyslog.conf
 3
@@ -0,0 +1,26 @@
 4
+#
 5
+#    "             "
 6
+#  mmm   m   m   mmm   m   m
 7
+#    #   #   #     #   #   #
 8
+#    #   #   #     #   #   #
 9
+#    #   "mm"#     #   "mm"#
10
+#    #             #
11
+#  ""            ""
12
+# This file is managed by Juju. Do not make local changes.
13
+#
14
+
15
+# Create a template to print just the raw message to avoid duplicate timestamps
16
+# and other info not needed. Also, as documented on the rsyslog website
17
+# http://www.rsyslog.com/log-normalization-and-the-leading-space/
18
+# msg has a leading whitespace so strip that.
19
+$template ApacheLogFormat,"%msg:2:10000%\n"
20
+
21
+# We want all access entries, even if rsyslog deems them as repeated msgs.
22
+$RepeatedMsgReduction off
23
+
24
+# Error logs
25
+if $syslogfacility-text == 'local0' and $syslogseverity == 3 and $syslogtag == "apache2:" then /var/log/apache2/error.log;ApacheLogFormat
26
+& ~
27
+# Access logs
28
+if $syslogfacility-text == 'local0' and $syslogseverity == 6 and $syslogtag == "apache2:" then /var/log/apache2/access.log;ApacheLogFormat
29
+& ~
Back to file index

hooks/apache-website-relation-changed

   1
--- 
   2
+++ hooks/apache-website-relation-changed
   3
@@ -0,0 +1,1088 @@
   4
+#!/usr/bin/env python
   5
+
   6
+import errno
   7
+import os
   8
+import re
   9
+import socket
  10
+import subprocess
  11
+import sys
  12
+import yaml
  13
+import base64
  14
+import grp
  15
+import pwd
  16
+import shutil
  17
+import os.path
  18
+import ast
  19
+
  20
+from charmhelpers.core.hookenv import (
  21
+    open_port,
  22
+    close_port,
  23
+    log,
  24
+    config as orig_config_get,
  25
+    relations_of_type,
  26
+    relation_set,
  27
+    relation_ids,
  28
+    unit_get
  29
+)
  30
+from charmhelpers.contrib.charmsupport import nrpe
  31
+from charmhelpers.fetch import apt_update, add_source
  32
+
  33
+###############################################################################
  34
+# Global variables
  35
+###############################################################################
  36
+default_apache2_service_config_dir = "/var/run/apache2"
  37
+service_affecting_packages = ['apache2']
  38
+default_apache22_config_dir = "/etc/apache2/conf.d"
  39
+default_apache24_config_dir = "/etc/apache2/conf-available"
  40
+default_apache_base_dir = "/etc/apache2"
  41
+
  42
+juju_warning_header = """#
  43
+#    "             "
  44
+#  mmm   m   m   mmm   m   m
  45
+#    #   #   #     #   #   #
  46
+#    #   #   #     #   #   #
  47
+#    #   "mm"#     #   "mm"#
  48
+#    #             #
  49
+#  ""            ""
  50
+# This file is managed by Juju. Do not make local changes.
  51
+#"""
  52
+
  53
+
  54
+###############################################################################
  55
+# Supporting functions
  56
+###############################################################################
  57
+
  58
+def apt_get_install(package=None):
  59
+    """Install a package."""
  60
+    if package is None:
  61
+        return False
  62
+    cmd_line = ['apt-get', '-y', 'install', '-qq']
  63
+    cmd_line.append(package)
  64
+    return subprocess.call(cmd_line)
  65
+
  66
+
  67
+def ensure_package_status(packages, status):
  68
+    if status in ['install', 'hold']:
  69
+        selections = ''.join(['{} {}\n'.format(package, status)
  70
+                              for package in packages])
  71
+        dpkg = subprocess.Popen(['dpkg', '--set-selections'],
  72
+                                stdin=subprocess.PIPE)
  73
+        dpkg.communicate(input=selections)
  74
+
  75
+
  76
+# -----------------------------------------------------------------------------
  77
+# apt_get_purge( package ):  Purges a package
  78
+# -----------------------------------------------------------------------------
  79
+def apt_get_purge(packages=None):
  80
+    if packages is None:
  81
+        return False
  82
+    cmd_line = ['apt-get', '-y', 'purge', '-qq']
  83
+    cmd_line.append(packages)
  84
+    return subprocess.call(cmd_line)
  85
+
  86
+
  87
+# -----------------------------------------------------------------------------
  88
+# service_apache2:  Convenience function to start/stop/restart/reload
  89
+#                   the apache2 service
  90
+# -----------------------------------------------------------------------------
  91
+def service_apache2(action=None):
  92
+    if action is None:
  93
+        return
  94
+    elif action == "check":
  95
+        args = ['/usr/sbin/apache2ctl', 'configtest']
  96
+    else:
  97
+        args = ['service', 'apache2', action]
  98
+    ret_val = subprocess.call(args)
  99
+    return ret_val == 0
 100
+
 101
+
 102
+def run(command, *args, **kwargs):
 103
+    try:
 104
+        output = subprocess.check_output(command, *args, **kwargs)
 105
+        return output
 106
+    except Exception, e:
 107
+        print e
 108
+        raise
 109
+
 110
+
 111
+def enable_module(module=None):
 112
+    if module is None:
 113
+        return True
 114
+    if os.path.exists("/etc/apache2/mods-enabled/%s.load" % (module)):
 115
+        log("Module already loaded: %s" % module)
 116
+        return True
 117
+    if not os.path.exists("/etc/apache2/mods-available/%s.load" % (module)):
 118
+        return_value = apt_get_install("libapache2-mod-%s" % (module))
 119
+        if return_value != 0:
 120
+            log("Installing module %s failed" % (module))
 121
+            return False
 122
+    return_value = subprocess.call(['/usr/sbin/a2enmod', module])
 123
+    if return_value != 0:
 124
+        return False
 125
+    if service_apache2("check"):
 126
+        service_apache2("reload")
 127
+        return True
 128
+
 129
+
 130
+def disable_module(module=None):
 131
+    if module is None:
 132
+        return True
 133
+    if not os.path.exists("/etc/apache2/mods-enabled/%s.load" % (module)):
 134
+        log("Module already disabled: %s" % module)
 135
+        return True
 136
+    return_value = subprocess.call(['/usr/sbin/a2dismod', module])
 137
+    if return_value != 0:
 138
+        return False
 139
+    if service_apache2("check"):
 140
+        service_apache2("reload")
 141
+        return True
 142
+
 143
+
 144
+def is_apache24():
 145
+    return os.path.exists("/usr/sbin/a2enconf")
 146
+
 147
+
 148
+def site_filename(name, enabled=False):
 149
+    if enabled:
 150
+        sites_dir = "%s/sites-enabled" % default_apache_base_dir
 151
+    else:
 152
+        sites_dir = "%s/sites-available" % default_apache_base_dir
 153
+
 154
+    if is_apache24():
 155
+        return "{}/{}.conf".format(sites_dir, name)
 156
+    else:
 157
+        return "{}/{}".format(sites_dir, name)
 158
+
 159
+
 160
+def conf_filename(name):
 161
+    """Return an apache2 config filename path, as:
 162
+      2.4: /etc/apache2/conf-available/foo.conf
 163
+      2.2: /etc/apache2/conf.d/foo
 164
+    """
 165
+    if is_apache24():
 166
+        return "{}/{}.conf".format(default_apache24_config_dir, name)
 167
+    else:
 168
+        return "{}/{}".format(default_apache22_config_dir, name)
 169
+
 170
+
 171
+def conf_enable(name):
 172
+    "Enable apache2 config without reloading service"
 173
+    if is_apache24():
 174
+        return subprocess.call(['/usr/sbin/a2enconf', name]) == 0
 175
+    # no-op otherwise
 176
+    return True
 177
+
 178
+
 179
+def conf_disable(name):
 180
+    "Disable apache2 config without reloading service"
 181
+    if is_apache24():
 182
+        return subprocess.call(['/usr/sbin/a2disconf', name]) == 0
 183
+    # no-op otherwise
 184
+    return True
 185
+
 186
+
 187
+def gen_selfsigned_cert(config, cert_file, key_file):
 188
+    """
 189
+    Create a self-signed certificate.
 190
+
 191
+    @param config: charm data from config-get
 192
+    @param cert_file: destination path of generated certificate
 193
+    @param key_file: destination path of generated private key
 194
+    """
 195
+    os.environ['OPENSSL_CN'] = config['servername']
 196
+    os.environ['OPENSSL_PUBLIC'] = unit_get("public-address")
 197
+    os.environ['OPENSSL_PRIVATE'] = unit_get("private-address")
 198
+    run(
 199
+        ['openssl', 'req', '-new', '-x509', '-nodes',
 200
+         '-days', '3650', '-config',
 201
+         os.path.join(os.environ['CHARM_DIR'], 'data', 'openssl.cnf'),
 202
+         '-keyout', key_file, '-out', cert_file])
 203
+
 204
+
 205
+def is_selfsigned_cert_stale(config, cert_file, key_file):
 206
+    """
 207
+    Do we need to generate a new self-signed cert?
 208
+
 209
+    @param config: charm data from config-get
 210
+    @param cert_file: destination path of generated certificate
 211
+    @param key_file: destination path of generated private key
 212
+    """
 213
+    # Basic Existence Checks
 214
+    if not os.path.exists(cert_file):
 215
+        return True
 216
+    if not os.path.exists(key_file):
 217
+        return True
 218
+
 219
+    # Common Name
 220
+    from OpenSSL import crypto
 221
+    cert = crypto.load_certificate(
 222
+        crypto.FILETYPE_PEM, file(cert_file).read())
 223
+    cn = cert.get_subject().commonName
 224
+    if config['servername'] != cn:
 225
+        return True
 226
+
 227
+    # Subject Alternate Name -- only trusty+ support this
 228
+    try:
 229
+        from pyasn1.codec.der import decoder
 230
+        from pyasn1_modules import rfc2459
 231
+    except ImportError:
 232
+        log("Cannot check subjAltName on <= 12.04, skipping.")
 233
+        return False
 234
+    cert_addresses = set()
 235
+    unit_addresses = set(
 236
+        [unit_get("public-address"), unit_get("private-address")])
 237
+    for i in range(0, cert.get_extension_count()):
 238
+        extension = cert.get_extension(i)
 239
+        try:
 240
+            names = decoder.decode(
 241
+                extension.get_data(), asn1Spec=rfc2459.SubjectAltName())[0]
 242
+            for name in names:
 243
+                cert_addresses.add(str(name.getComponent()))
 244
+        except:
 245
+            pass
 246
+    if cert_addresses != unit_addresses:
 247
+        log("subjAltName: Cert (%s) != Unit (%s), assuming stale" % (
 248
+            cert_addresses, unit_addresses))
 249
+        return True
 250
+
 251
+    return False
 252
+
 253
+
 254
+def _get_key_file_location(config_data):
 255
+    """Look at the config, generate the key file location."""
 256
+    key_file = None
 257
+    if config_data['ssl_keylocation']:
 258
+        key_file = '/etc/ssl/private/%s' % \
 259
+            (config_data['ssl_keylocation'].rpartition('/')[2])
 260
+    return key_file
 261
+
 262
+
 263
+def _get_cert_file_location(config_data):
 264
+    """Look at the config, generate the cert file location."""
 265
+    cert_file = None
 266
+    if config_data['ssl_certlocation']:
 267
+        cert_file = '/etc/ssl/certs/%s' % \
 268
+            (config_data['ssl_certlocation'].rpartition('/')[2])
 269
+    return cert_file
 270
+
 271
+
 272
+def _get_chain_file_location(config_data):
 273
+    """Look at the config, generate the chain file location."""
 274
+    chain_file = None
 275
+    if config_data['ssl_chainlocation']:
 276
+        chain_file = '/etc/ssl/certs/%s' % \
 277
+            (config_data['ssl_chainlocation'].rpartition('/')[2])
 278
+    return chain_file
 279
+
 280
+
 281
+def config_get(scope=None):
 282
+    """
 283
+    Wrapper around charm helper's config_get to replace an empty servername
 284
+    with the public-address.
 285
+    """
 286
+    result = orig_config_get(scope)
 287
+    if scope == "servername" and len(result) == 0:
 288
+        result = unit_get("public-address")
 289
+    elif isinstance(result, dict) and result.get("servername", "") == "":
 290
+        result["servername"] = unit_get("public-address")
 291
+    return result
 292
+
 293
+
 294
+def install_hook():
 295
+    apt_source = config_get('apt-source') or ''
 296
+    apt_key_id = config_get('apt-key-id') or False
 297
+    if apt_source and apt_key_id:
 298
+        print apt_source + " and " + apt_key_id
 299
+        add_source(apt_source, apt_key_id)
 300
+        open('config.apt-source', 'w').write(apt_source)
 301
+    if not os.path.exists(default_apache2_service_config_dir):
 302
+        os.mkdir(default_apache2_service_config_dir, 0600)
 303
+    apt_update(fatal=True)
 304
+    apt_get_install("python-jinja2")
 305
+    apt_get_install("python-pyasn1")
 306
+    apt_get_install("python-pyasn1-modules")
 307
+    apt_get_install("python-yaml")
 308
+    install_status = apt_get_install("apache2")
 309
+    if install_status == 0:
 310
+        ensure_package_status(service_affecting_packages,
 311
+                              config_get('package_status'))
 312
+    ensure_extra_packages()
 313
+    # the apache2 deb does not yet have http2 module in mods-available. Add it.
 314
+    open('/etc/apache2/mods-available/http2.load', 'w').write(
 315
+        'LoadModule http2_module /usr/lib/apache2/modules/mod_http2.so')
 316
+    open('/etc/apache2/mods-available/http2.conf', 'w').write(
 317
+        '''<IfModule http2_module>
 318
+  ProtocolsHonorOrder On
 319
+  Protocols h2 http/1.1
 320
+</IfModule>
 321
+''')
 322
+    return install_status
 323
+
 324
+
 325
+def ensure_extra_packages():
 326
+    extra = str(config_get('extra_packages'))
 327
+    if extra:
 328
+        install_status = apt_get_install(extra)
 329
+        if install_status == 0:
 330
+            ensure_package_status(filter(None, extra.split(' ')),
 331
+                                  config_get('package_status'))
 332
+
 333
+
 334
+def dump_data(data2dump, log_prefix):
 335
+    log_file = '/tmp/pprint-%s.log' % (log_prefix)
 336
+    if data2dump is not None:
 337
+        logFile = open(log_file, 'w')
 338
+        import pprint
 339
+        pprint.pprint(data2dump, logFile)
 340
+        logFile.close()
 341
+
 342
+
 343
+def get_reverseproxy_data(relation='reverseproxy'):
 344
+    relation_data = relations_of_type(relation)
 345
+    reverseproxy_data = {}
 346
+    if relation_data is None or len(relation_data) == 0:
 347
+        return reverseproxy_data
 348
+    for unit_data in relation_data:
 349
+        unit_name = unit_data["__unit__"]
 350
+        if 'port' not in unit_data:
 351
+            return reverseproxy_data
 352
+        # unit_name: <service-name>-<unit_number>
 353
+        # jinja2 templates require python-type variables, remove all characters
 354
+        # that do not comply
 355
+        unit_type = re.sub(r'(.*)/[0-9]*', r'\1', unit_name)
 356
+        unit_type = re.sub('[^a-zA-Z0-9_]*', '', unit_type)
 357
+        log('unit_type: %s' % unit_type)
 358
+
 359
+        host = unit_data['private-address']
 360
+        if unit_type in reverseproxy_data:
 361
+            continue
 362
+        for config_setting in unit_data.keys():
 363
+            if config_setting in ("__unit__", "__relid__"):
 364
+                continue
 365
+            config_key = '%s_%s' % (unit_type,
 366
+                                    config_setting.replace("-", "_"))
 367
+            config_key = re.sub('[^a-zA-Z0-9_]*', '', config_key)
 368
+            reverseproxy_data[config_key] = unit_data[
 369
+                config_setting]
 370
+            reverseproxy_data[unit_type] = '%s:%s' % (
 371
+                host, unit_data['port'])
 372
+        if 'all_services' in unit_data:
 373
+            service_data = yaml.safe_load(unit_data['all_services'])
 374
+            for service_item in service_data:
 375
+                service_name = service_item['service_name']
 376
+                service_port = service_item['service_port']
 377
+                service_key = '%s_%s' % (unit_type, service_name)
 378
+                service_key = re.sub('[^a-zA-Z0-9_]*', '', service_key)
 379
+                reverseproxy_data[service_key] = '%s:%s' % (host, service_port)
 380
+    return reverseproxy_data
 381
+
 382
+
 383
+def update_balancers():
 384
+    relation_data = relations_of_type('balancer')
 385
+    if relation_data is None or len(relation_data) == 0:
 386
+        log("No relation data, exiting.")
 387
+        return
 388
+
 389
+    unit_dict = {}
 390
+    for unit_data in relation_data:
 391
+        unit_name = unit_data["__unit__"]
 392
+        if "port" not in unit_data:
 393
+            log("No port in relation data for '%s', skipping." % unit_name)
 394
+            continue
 395
+        port = unit_data["port"]
 396
+        if "private-address" not in unit_data:
 397
+            log("No private-address in relation data for '%s', skipping." %
 398
+                unit_name)
 399
+            continue
 400
+        host = unit_data['private-address']
 401
+
 402
+        if "all_services" in unit_data:
 403
+            service_data = yaml.safe_load(unit_data[
 404
+                "all_services"])
 405
+            for service_item in service_data:
 406
+                service_port = service_item["service_port"]
 407
+                current_units = unit_dict.setdefault(
 408
+                    service_item["service_name"], [])
 409
+                current_units.append("%s:%s" % (host, service_port))
 410
+        else:
 411
+            if "sitenames" in unit_data:
 412
+                unit_types = unit_data["sitenames"].split()
 413
+            else:
 414
+                unit_types = (re.sub(r"(.*)/[0-9]*", r"\1", unit_name),)
 415
+
 416
+            for unit_type in unit_types:
 417
+                current_units = unit_dict.setdefault(unit_type, [])
 418
+                current_units.append("%s:%s" % (host, port))
 419
+
 420
+    if not unit_dict:
 421
+        return
 422
+
 423
+    write_balancer_config(unit_dict)
 424
+    return unit_dict
 425
+
 426
+
 427
+def write_balancer_config(unit_dict):
 428
+    config_data = config_get()
 429
+
 430
+    from jinja2 import Environment, FileSystemLoader
 431
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 432
+        os.environ['CHARM_DIR'], 'data')))
 433
+    for balancer_name in unit_dict.keys():
 434
+        balancer_host_file = conf_filename('{}.balancer'.format(balancer_name))
 435
+        templ_vars = {
 436
+            'balancer_name': balancer_name,
 437
+            'balancer_addresses': unit_dict[balancer_name],
 438
+            'lb_balancer_timeout': config_data['lb_balancer_timeout'],
 439
+        }
 440
+        template = template_env.get_template(
 441
+            'balancer.template').render(templ_vars)
 442
+        log("Writing file: %s with data: %s" % (balancer_host_file,
 443
+                                                templ_vars))
 444
+        with open(balancer_host_file, 'w') as balancer_config:
 445
+            balancer_config.write(str(template))
 446
+        conf_enable('{}.balancer'.format(balancer_name))
 447
+
 448
+
 449
+def update_nrpe_checks():
 450
+    nrpe_compat = nrpe.NRPE()
 451
+    conf = nrpe_compat.config
 452
+    check_http_params = conf.get('nagios_check_http_params')
 453
+    if check_http_params:
 454
+        nrpe_compat.add_check(
 455
+            shortname='vhost',
 456
+            description='Check Virtual Host',
 457
+            check_cmd='check_http %s' % check_http_params
 458
+        )
 459
+    nrpe_compat.write()
 460
+
 461
+
 462
+def create_mpm_workerfile():
 463
+    config_data = config_get()
 464
+    mpm_workerfile = conf_filename('000mpm-worker')
 465
+    from jinja2 import Environment, FileSystemLoader
 466
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 467
+        os.environ['CHARM_DIR'], 'data')))
 468
+    templ_vars = {
 469
+        'mpm_type': config_data['mpm_type'],
 470
+        'mpm_startservers': config_data['mpm_startservers'],
 471
+        'mpm_minsparethreads': config_data['mpm_minsparethreads'],
 472
+        'mpm_maxsparethreads': config_data['mpm_maxsparethreads'],
 473
+        'mpm_threadlimit': config_data['mpm_threadlimit'],
 474
+        'mpm_threadsperchild': config_data['mpm_threadsperchild'],
 475
+        'mpm_serverlimit': config_data['mpm_serverlimit'],
 476
+        'mpm_maxclients': config_data['mpm_maxclients'],
 477
+        'mpm_maxrequestsperchild': config_data['mpm_maxrequestsperchild'],
 478
+    }
 479
+    template = \
 480
+        template_env.get_template('mpm_worker.template').render(templ_vars)
 481
+    with open(mpm_workerfile, 'w') as mpm_config:
 482
+        mpm_config.write(str(template))
 483
+    conf_enable('000mpm-worker')
 484
+
 485
+
 486
+def create_security():
 487
+    config_data = config_get()
 488
+    securityfile = conf_filename('security')
 489
+    from jinja2 import Environment, FileSystemLoader
 490
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 491
+        os.environ['CHARM_DIR'], 'data')))
 492
+    templ_vars = {
 493
+        'juju_warning_header': juju_warning_header,
 494
+        'server_tokens': config_data['server_tokens'],
 495
+        'server_signature': config_data['server_signature'],
 496
+        'trace_enabled': config_data['trace_enabled'],
 497
+        'ssl_protocol': config_data['ssl_protocol'],
 498
+        'ssl_honor_cipher_order': config_data['ssl_honor_cipher_order'],
 499
+        'ssl_cipher_suite': config_data['ssl_cipher_suite'],
 500
+        'is_apache24': is_apache24(),
 501
+    }
 502
+    template = \
 503
+        template_env.get_template('security.template').render(templ_vars)
 504
+    with open(securityfile, 'w') as security_config:
 505
+        security_config.write(str(template))
 506
+    conf_enable('security')
 507
+
 508
+
 509
+def ship_logrotate_conf():
 510
+    config_data = config_get()
 511
+    logrotate_file = '/etc/logrotate.d/apache2'
 512
+    from jinja2 import Environment, FileSystemLoader
 513
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 514
+        os.environ['CHARM_DIR'], 'data')))
 515
+    templ_vars = {
 516
+        'juju_warning_header': juju_warning_header,
 517
+        'logrotate_rotate': config_data['logrotate_rotate'],
 518
+        'logrotate_count': config_data['logrotate_count'],
 519
+        'logrotate_dateext': config_data['logrotate_dateext'],
 520
+    }
 521
+    template = template_env.get_template('logrotate.conf.template').render(
 522
+        templ_vars)
 523
+    with open(logrotate_file, 'w') as logrotate_conf:
 524
+        logrotate_conf.write(str(template))
 525
+
 526
+
 527
+def create_vhost(port, protocol=None, config_key=None, template_str=None,
 528
+                 config_data={}, relationship_data={}):
 529
+    """
 530
+    Create and enable a vhost in apache.
 531
+
 532
+    @param port: port on which to listen (int)
 533
+    @param protocol: used to name the vhost file intelligently.  If not
 534
+        specified the port will be used instead. (ex: http, https)
 535
+    @param config_key: key in the configuration to look up to
 536
+        retrieve the template.
 537
+    @param template_str: The template itself as a string.
 538
+    @param config_data: juju get-config configuration data.
 539
+    @param relationship_data: if in a relationship, pass in the appropriate
 540
+        structure.  This will be used to inform the template.
 541
+    """
 542
+    if protocol is None:
 543
+        protocol = str(port)
 544
+    if template_str is None:
 545
+        if not config_key or not config_data[config_key]:
 546
+            log("Vhost Template not provided, not configuring: %s" % port)
 547
+            return False
 548
+        template_str = config_data[config_key]
 549
+    from jinja2 import Template
 550
+    template = Template(str(base64.b64decode(template_str)))
 551
+    template_data = dict(config_data.items() + relationship_data.items())
 552
+    if config_data.get('vhost_template_vars'):
 553
+        extra_vars = ast.literal_eval(config_data['vhost_template_vars'])
 554
+        template_data.update(extra_vars)
 555
+    vhost_name = '%s_%s' % (config_data['servername'], protocol)
 556
+    vhost_file = site_filename(vhost_name)
 557
+    log("Writing file %s with config and relation data" % vhost_file)
 558
+    with open(vhost_file, 'w') as vhost:
 559
+        vhost.write(str(template.render(template_data)))
 560
+    subprocess.call(['/usr/sbin/a2ensite', vhost_name])
 561
+    return True
 562
+
 563
+
 564
+MPM_TYPES = ['mpm_worker', 'mpm_prefork', 'mpm_event']
 565
+
 566
+
 567
+def enable_mpm(config):
 568
+    """Enables a particular mpm module.
 569
+
 570
+    Different from simply enabling a module, as one and only one mpm module
 571
+    *must* be enabled.
 572
+    """
 573
+    # only do anything if value has changed, to avoid a needless restart
 574
+    if not config.changed('mpm_type'):
 575
+        return
 576
+
 577
+    mpm_type = config.get('mpm_type', '')
 578
+    name = 'mpm_' + mpm_type
 579
+    if name not in MPM_TYPES:
 580
+        log('bad mpm_type: %s. Falling back to mpm_worker' % mpm_type)
 581
+        name = 'mpm_worker'
 582
+
 583
+    # disable all other mpm modules
 584
+    for mpm in MPM_TYPES:
 585
+        if mpm != name:
 586
+            return_value = subprocess.call(['/usr/sbin/a2dismod', mpm])
 587
+            if return_value != 0:
 588
+                return False
 589
+
 590
+    return_value = subprocess.call(['/usr/sbin/a2enmod', name])
 591
+    if return_value != 0:
 592
+        return False
 593
+
 594
+    if service_apache2("check"):
 595
+        log("Switching mpm module to {}".format(name))
 596
+        service_apache2("restart")  # must be a restart to switch mpm
 597
+        return True
 598
+    else:
 599
+        log("Failed to switch mpm module to {}".format(name))
 600
+        return False
 601
+
 602
+
 603
+def config_changed():
 604
+    relationship_data = {}
 605
+    config_data = config_get()
 606
+
 607
+    apt_source = config_data['apt-source']
 608
+    old_apt_source = ''
 609
+    try:
 610
+        old_apt_source = open('config.apt-source', 'r').read()
 611
+    except IOError:
 612
+        pass
 613
+    if old_apt_source != apt_source:
 614
+        subprocess.check_call(['add-apt-repository', '--yes', '-r',
 615
+                               old_apt_source])
 616
+        add_source(apt_source, config_data['apt-key-id'])
 617
+        open('config.apt-source', 'w').write(apt_source)
 618
+
 619
+    ensure_package_status(service_affecting_packages,
 620
+                          config_data['package_status'])
 621
+    ensure_extra_packages()
 622
+
 623
+    relationship_data.update(get_reverseproxy_data(relation='reverseproxy'))
 624
+    relationship_data.update(get_reverseproxy_data(relation='website-cache'))
 625
+    if update_balancers():
 626
+        # apache 2.4 has lbmethods split, needs to enable specific module(s)
 627
+        if is_apache24():
 628
+            enable_module('lbmethod_byrequests')
 629
+
 630
+    disabled_modules = config_data['disable_modules'].split()
 631
+    apache_websites = ApacheWebsites.from_config(
 632
+        relations_of_type("apache-website"), disabled_modules)
 633
+    enabled_modules = config_data.get('enable_modules', '').split()
 634
+    enabled_modules = apache_websites.list_enabled_modules(enabled_modules)
 635
+    for module in enabled_modules:
 636
+        enable_module(module)
 637
+
 638
+    if config_data['disable_modules']:
 639
+        for module in disabled_modules:
 640
+            disable_module(module)
 641
+
 642
+    apache_websites.disable_sites()
 643
+    apache_websites.write_configs()
 644
+    apache_websites.enable_sites()
 645
+    apache_websites.configure_extra_ports()
 646
+    all_ports = apache_websites.list_enabled_ports()
 647
+    enable_mpm(config_data)
 648
+    # XXX we only configure the worker mpm?
 649
+    create_mpm_workerfile()
 650
+    create_security()
 651
+
 652
+    ports = {'http': 80, 'https': 443}
 653
+    for protocol, port in ports.iteritems():
 654
+        if create_vhost(
 655
+                port,
 656
+                protocol=protocol,
 657
+                config_key="vhost_%s_template" % protocol,
 658
+                config_data=config_data,
 659
+                relationship_data=relationship_data):
 660
+            all_ports.add(port)
 661
+
 662
+    cert_file = _get_cert_file_location(config_data)
 663
+    key_file = _get_key_file_location(config_data)
 664
+    chain_file = _get_chain_file_location(config_data)
 665
+
 666
+    if cert_file is not None and key_file is not None:
 667
+        # ssl_cert is SELFSIGNED so generate self-signed certificate for use.
 668
+        if config_data['ssl_cert'] and config_data['ssl_cert'] == "SELFSIGNED":
 669
+            if is_selfsigned_cert_stale(config_data, cert_file, key_file):
 670
+                gen_selfsigned_cert(config_data, cert_file, key_file)
 671
+
 672
+        # Use SSL certificate and key provided either as a base64 string or
 673
+        # shipped out with the charm.
 674
+        else:
 675
+            # Certificate provided as base64-encoded string.
 676
+            if config_data['ssl_cert']:
 677
+                log("Writing cert from config ssl_cert: %s" % cert_file)
 678
+                with open(cert_file, 'w') as f:
 679
+                    f.write(str(base64.b64decode(config_data['ssl_cert'])))
 680
+            # Use certificate file shipped out with charm.
 681
+            else:
 682
+                source = os.path.join(os.environ['CHARM_DIR'], 'data',
 683
+                                      config_data['ssl_certlocation'])
 684
+                if os.path.exists(source):
 685
+                    shutil.copy(source, cert_file)
 686
+                else:
 687
+                    log("Certificate not found, ignoring: %s" % source)
 688
+
 689
+            # Private key provided as base64-encoded string.
 690
+            if config_data['ssl_key']:
 691
+                log("Writing key from config ssl_key: %s" % key_file)
 692
+                with open(key_file, 'w') as f:
 693
+                    f.write(str(base64.b64decode(config_data['ssl_key'])))
 694
+            # Use private key shipped out with charm.
 695
+            else:
 696
+                source = os.path.join(os.environ['CHARM_DIR'], 'data',
 697
+                                      config_data['ssl_keylocation'])
 698
+                if os.path.exists(source):
 699
+                    shutil.copy(source, key_file)
 700
+                else:
 701
+                    log("Key file not found, ignoring: %s" % source)
 702
+
 703
+            if chain_file is not None:
 704
+                # Chain certificates provided as base64-encoded string.
 705
+                if config_data['ssl_chain']:
 706
+                    log("Writing chain certificates file from"
 707
+                        "config ssl_chain: %s" % chain_file)
 708
+                    with open(chain_file, 'w') as f:
 709
+                        f.write(str(base64.b64decode(
 710
+                            config_data['ssl_chain'])))
 711
+                # Use chain certificates shipped out with charm.
 712
+                else:
 713
+                    source = os.path.join(os.environ['CHARM_DIR'], 'data',
 714
+                                          config_data['ssl_chainlocation'])
 715
+                    if os.path.exists(source):
 716
+                        shutil.copy(source, chain_file)
 717
+                    else:
 718
+                        log("Chain certificates not found, "
 719
+                            "ignoring: %s" % source)
 720
+
 721
+        # Tighten permissions on private key file.
 722
+        if os.path.exists(key_file):
 723
+            os.chmod(key_file, 0440)
 724
+            os.chown(key_file, pwd.getpwnam('root').pw_uid,
 725
+                     grp.getgrnam('ssl-cert').gr_gid)
 726
+
 727
+    apache_syslog_conf = conf_filename("syslog")
 728
+    rsyslog_apache_conf = "/etc/rsyslog.d/45-apache2.conf"
 729
+    if config_data['use_rsyslog']:
 730
+        shutil.copy2("data/syslog-apache.conf", apache_syslog_conf)
 731
+        conf_enable("syslog")
 732
+        shutil.copy2("data/syslog-rsyslog.conf", rsyslog_apache_conf)
 733
+        # Fix permissions of access.log and error.log to allow syslog user to
 734
+        # write to
 735
+        os.chown("/var/log/apache2/access.log", pwd.getpwnam('syslog').pw_uid,
 736
+                 pwd.getpwnam('syslog').pw_gid)
 737
+        os.chown("/var/log/apache2/error.log", pwd.getpwnam('syslog').pw_uid,
 738
+                 pwd.getpwnam('syslog').pw_gid)
 739
+    else:
 740
+        conf_disable("syslog")
 741
+        if os.path.exists(apache_syslog_conf):
 742
+            os.unlink(apache_syslog_conf)
 743
+        if os.path.exists(rsyslog_apache_conf):
 744
+            os.unlink(rsyslog_apache_conf)
 745
+    run(["/usr/sbin/service", "rsyslog", "restart"])
 746
+
 747
+    # Disable the default website because we don't want people to see the
 748
+    # "It works!" page on production services and remove the
 749
+    # conf.d/other-vhosts-access-log conf.
 750
+    ensure_disabled(["000-default"])
 751
+    conf_disable("other-vhosts-access-log")
 752
+    if os.path.exists(conf_filename("other-vhosts-access-log")):
 753
+        os.unlink(conf_filename("other-vhosts-access-log"))
 754
+
 755
+    if service_apache2("check"):
 756
+        if config_data["config_change_command"] in ["reload", "restart"]:
 757
+            service_apache2(config_data["config_change_command"])
 758
+
 759
+    if config_data['openid_provider']:
 760
+        if not os.path.exists('/etc/apache2/security'):
 761
+            os.mkdir('/etc/apache2/security', 0755)
 762
+        with open('/etc/apache2/security/allowed-ops.txt', 'w') as f:
 763
+            f.write(config_data['openid_provider'].replace(',', '\n'))
 764
+            f.write('\n')
 765
+            os.chmod(key_file, 0444)
 766
+
 767
+    all_ports.update(update_vhost_config_relation())
 768
+    ensure_ports(all_ports)
 769
+    update_nrpe_checks()
 770
+    ship_logrotate_conf()
 771
+    if config_get().changed('servername'):
 772
+        logs_relation_joined()
 773
+
 774
+
 775
+def ensure_disabled(sites):
 776
+    to_disable = [s for s in sites if os.path.exists(site_filename(s, True))]
 777
+    if len(to_disable) == 0:
 778
+        return
 779
+    run(["/usr/sbin/a2dissite"] + to_disable)
 780
+
 781
+
 782
+def ensure_removed(filename):
 783
+    try:
 784
+        os.unlink(filename)
 785
+    except OSError as e:
 786
+        if e.errno != errno.ENOENT:
 787
+            raise
 788
+
 789
+
 790
+class ApacheWebsites:
 791
+
 792
+    @classmethod
 793
+    def from_config(cls, relations, disabled_modules):
 794
+        """Return an ApacheWebsites with information about all sites."""
 795
+        if relations is None:
 796
+            relations = []
 797
+        self_relations = {}
 798
+        for relation in relations:
 799
+            self_relation = {'domain': relation.get('domain')}
 800
+            enabled = bool(relation.get('enabled', 'False').lower() == 'true')
 801
+            site_modules = relation.get('site_modules', '').split()
 802
+            for module in site_modules:
 803
+                if module in disabled_modules:
 804
+                    enabled = False
 805
+                    log('site {} requires disabled_module {}'.format(
 806
+                        relation['__relid__'], module))
 807
+                break
 808
+            self_relation['site_modules'] = site_modules
 809
+            self_relation['enabled'] = enabled
 810
+            self_relation['site_config'] = relation.get('site_config')
 811
+            self_relation['ports'] = [
 812
+                int(p) for p in relation.get('ports', '').split()]
 813
+            self_relations[relation['__relid__']] = self_relation
 814
+        return cls(self_relations)
 815
+
 816
+    def __init__(self, relations):
 817
+        self.relations = relations
 818
+
 819
+    def write_configs(self):
 820
+        for key, relation in self.relations.items():
 821
+            config_file = site_filename(key)
 822
+            site_config = relation['site_config']
 823
+            if site_config is None:
 824
+                ensure_removed(config_file)
 825
+            else:
 826
+                with open(config_file, 'w') as output:
 827
+                    output.write(site_config)
 828
+
 829
+    def iter_enabled_sites(self):
 830
+        return ((k, v) for k, v in self.relations.items() if v['enabled'])
 831
+
 832
+    def enable_sites(self):
 833
+        enabled_sites = [k for k, v in self.iter_enabled_sites()]
 834
+        enabled_sites.sort()
 835
+        if len(enabled_sites) == 0:
 836
+            return
 837
+        subprocess.check_call(['/usr/sbin/a2ensite'] + enabled_sites)
 838
+
 839
+    def disable_sites(self):
 840
+        disabled_sites = [k for k, v in self.relations.items()
 841
+                          if not v['enabled']]
 842
+        disabled_sites.sort()
 843
+        if len(disabled_sites) == 0:
 844
+            return
 845
+        ensure_disabled(disabled_sites)
 846
+
 847
+    def list_enabled_modules(self, enabled_modules):
 848
+        enabled_modules = set(enabled_modules)
 849
+        for key, relation in self.iter_enabled_sites():
 850
+            enabled_modules.update(relation['site_modules'])
 851
+        return enabled_modules
 852
+
 853
+    def list_enabled_ports(self):
 854
+        enabled_ports = set()
 855
+        for key, relation in self.iter_enabled_sites():
 856
+            enabled_ports.update(relation['ports'])
 857
+        return enabled_ports
 858
+
 859
+    def configure_extra_ports(self):
 860
+        extra_ports = self.list_enabled_ports()
 861
+        extra_ports.discard(80)
 862
+        extra_ports.discard(443)
 863
+        extra_ports_conf = conf_filename('extra_ports')
 864
+        if len(extra_ports) > 0:
 865
+            with file(extra_ports_conf, 'w') as f:
 866
+                for port in sorted(extra_ports):
 867
+                    f.write('Listen {}\n'.format(port))
 868
+            conf_enable('extra_ports')
 869
+        else:
 870
+            conf_disable('extra_ports')
 871
+            ensure_removed(extra_ports_conf)
 872
+
 873
+
 874
+def update_vhost_config_relation():
 875
+    """
 876
+    Update the vhost file and include the certificate in the relation
 877
+    if it is self-signed.
 878
+    """
 879
+    vhost_ports = set()
 880
+    relation_data = relations_of_type("vhost-config")
 881
+    config_data = config_get()
 882
+    if relation_data is None:
 883
+        return vhost_ports
 884
+
 885
+    for unit_data in relation_data:
 886
+        if "vhosts" in unit_data:
 887
+            all_relation_data = {}
 888
+            all_relation_data.update(
 889
+                get_reverseproxy_data(relation='reverseproxy'))
 890
+            all_relation_data.update(
 891
+                get_reverseproxy_data(relation='website-cache'))
 892
+            try:
 893
+                vhosts = yaml.safe_load(unit_data["vhosts"])
 894
+                for vhost in vhosts:
 895
+                    port = vhost["port"]
 896
+                    if create_vhost(
 897
+                            port,
 898
+                            template_str=vhost["template"],
 899
+                            config_data=config_data,
 900
+                            relationship_data=all_relation_data):
 901
+                        vhost_ports.add(port)
 902
+            except Exception as e:
 903
+                log("Error reading configuration data from relation! %s" % e)
 904
+                raise
 905
+
 906
+    if service_apache2("check"):
 907
+        service_apache2("reload")
 908
+
 909
+    vhost_relation_settings = {
 910
+        "servername": config_data["servername"]}
 911
+
 912
+    cert_file = _get_cert_file_location(config_data)
 913
+    key_file = _get_key_file_location(config_data)
 914
+
 915
+    if cert_file is not None and key_file is not None:
 916
+        if config_data['ssl_cert'] and config_data['ssl_cert'] == "SELFSIGNED":
 917
+            with open(cert_file, 'r') as f:
 918
+                cert = base64.b64encode(f.read())
 919
+            vhost_relation_settings["ssl_cert"] = cert
 920
+    for id in relation_ids("vhost-config"):
 921
+        relation_set(relation_id=id, relation_settings=vhost_relation_settings)
 922
+    return vhost_ports
 923
+
 924
+
 925
+def start_hook():
 926
+    if service_apache2("status"):
 927
+        return(service_apache2("restart"))
 928
+    else:
 929
+        return(service_apache2("start"))
 930
+
 931
+
 932
+def stop_hook():
 933
+    if service_apache2("status"):
 934
+        return(service_apache2("stop"))
 935
+
 936
+
 937
+def reverseproxy_interface(hook_name=None):
 938
+    if hook_name is None:
 939
+        return(None)
 940
+    if hook_name == "changed":
 941
+        config_changed()
 942
+
 943
+
 944
+def website_interface(hook_name=None):
 945
+    if hook_name is None:
 946
+        return(None)
 947
+    my_host = socket.getfqdn(socket.gethostname())
 948
+    if my_host == "localhost":
 949
+        my_host = socket.gethostname()
 950
+    default_port = 80
 951
+    subprocess.call([
 952
+        'relation-set',
 953
+        'port=%d' % default_port,
 954
+        'hostname=%s' % my_host,
 955
+        'servername=%s' % config_get('servername')
 956
+    ])
 957
+
 958
+
 959
+def ensure_ports(ports):
 960
+    """Ensure that only the desired ports are open."""
 961
+    open_ports = set(get_open_ports())
 962
+    ports = set(ports)
 963
+    wanted_closed = ports.difference(open_ports)
 964
+    for port in sorted(wanted_closed):
 965
+        open_port(port)
 966
+    unwanted_open = open_ports.difference(ports)
 967
+    for port in sorted(unwanted_open):
 968
+        close_port(port)
 969
+    set_open_ports(list(sorted(ports)))
 970
+
 971
+
 972
+def get_open_ports():
 973
+    """Get the list of open ports from the standard file."""
 974
+    try:
 975
+        pfile = open(os.path.join(os.environ['CHARM_DIR'], 'ports.yaml'))
 976
+    except IOError as e:
 977
+        if e.errno == errno.ENOENT:
 978
+            return []
 979
+        else:
 980
+            raise
 981
+    with pfile:
 982
+        return yaml.safe_load(pfile)
 983
+
 984
+
 985
+def set_open_ports(ports):
 986
+    """Write the list of open ports to the standard file."""
 987
+    ports_path = os.path.join(os.environ['CHARM_DIR'], 'ports.yaml')
 988
+    with open(ports_path, 'w') as pfile:
 989
+        yaml.safe_dump(ports, pfile)
 990
+
 991
+
 992
+def get_log_files():
 993
+    """
 994
+    Read all of the apache config files from __ and get ErrorLog and AccessLog
 995
+    values.
 996
+
 997
+    Returns a tuple with first value list of access log files and second value
 998
+    list of error log files.
 999
+    """
1000
+    access_logs = []
1001
+    error_logs = []
1002
+    for protocol in ['http', 'https']:
1003
+        vhost_name = '%s_%s' % (config_get()['servername'], protocol)
1004
+        vhost_file = site_filename(vhost_name)
1005
+        try:
1006
+            # Using read().split('\n') here to work around a mocks open_mock
1007
+            # inadequacy: http://bugs.python.org/issue17467
1008
+            for line in open(vhost_file, 'r').read().split('\n'):
1009
+                if 'CustomLog' in line:
1010
+                    access_logs.append(line.split()[1])
1011
+                elif 'ErrorLog' in line:
1012
+                    error_logs.append(line.split()[1])
1013
+        except:
1014
+            pass
1015
+    return access_logs, error_logs
1016
+
1017
+
1018
+def logs_relation_joined():
1019
+    """
1020
+    Sets relation value with filenames
1021
+    """
1022
+    access_log_files, error_log_files = get_log_files()
1023
+    log_files = access_log_files[:]
1024
+    log_files.extend(error_log_files)
1025
+    types = ['apache_access' for a in access_log_files]
1026
+    types.extend(['apache_error' for a in error_log_files])
1027
+    data = {'files': '\n'.join(log_files),
1028
+            'types': '\n'.join(types),
1029
+            }
1030
+    _relation_ids = relation_ids('logs')
1031
+    for _relation_id in _relation_ids:
1032
+        log("logs-relation-joined setting relation data for {} to {}".format(
1033
+            _relation_id, data))
1034
+        relation_set(
1035
+            relation_id=_relation_id,
1036
+            relation_settings=data)
1037
+
1038
+
1039
+###############################################################################
1040
+# Main section
1041
+###############################################################################
1042
+def main(hook_name):
1043
+    if hook_name == "install":
1044
+        install_hook()
1045
+    elif hook_name == "config-changed" or hook_name == "upgrade-charm":
1046
+        config_changed()
1047
+    elif hook_name == "start":
1048
+        start_hook()
1049
+    elif hook_name == "stop":
1050
+        stop_hook()
1051
+    elif hook_name == "reverseproxy-relation-broken":
1052
+        config_changed()
1053
+    elif hook_name == "reverseproxy-relation-changed":
1054
+        config_changed()
1055
+    elif hook_name == "reverseproxy-relation-joined":
1056
+        config_changed()
1057
+    elif hook_name == "balancer-relation-broken":
1058
+        config_changed()
1059
+    elif hook_name == "balancer-relation-changed":
1060
+        config_changed()
1061
+    elif hook_name == "balancer-relation-joined":
1062
+        config_changed()
1063
+    elif hook_name == "website-cache-relation-broken":
1064
+        config_changed()
1065
+    elif hook_name == "website-cache-relation-changed":
1066
+        config_changed()
1067
+    elif hook_name == "website-cache-relation-joined":
1068
+        config_changed()
1069
+    elif hook_name == "website-relation-joined":
1070
+        website_interface("joined")
1071
+    elif hook_name == 'apache-website-relation-changed':
1072
+        config_changed()
1073
+    elif hook_name in ("nrpe-external-master-relation-changed",
1074
+                       "local-monitors-relation-changed"):
1075
+        update_nrpe_checks()
1076
+    elif hook_name == "vhost-config-relation-changed":
1077
+        config_changed()
1078
+    elif hook_name == "logs-relation-joined":
1079
+        logs_relation_joined()
1080
+    else:
1081
+        print "Unknown hook"
1082
+        sys.exit(1)
1083
+
1084
+if __name__ == "__main__":
1085
+    hook_name = os.path.basename(sys.argv[0])
1086
+    # Also support being invoked directly with hook as argument name.
1087
+    if hook_name == "hooks.py":
1088
+        if len(sys.argv) < 2:
1089
+            sys.exit("Missing required hook name argument.")
1090
+        hook_name = sys.argv[1]
1091
+    main(hook_name)
Back to file index

hooks/balancer-relation-broken

   1
--- 
   2
+++ hooks/balancer-relation-broken
   3
@@ -0,0 +1,1088 @@
   4
+#!/usr/bin/env python
   5
+
   6
+import errno
   7
+import os
   8
+import re
   9
+import socket
  10
+import subprocess
  11
+import sys
  12
+import yaml
  13
+import base64
  14
+import grp
  15
+import pwd
  16
+import shutil
  17
+import os.path
  18
+import ast
  19
+
  20
+from charmhelpers.core.hookenv import (
  21
+    open_port,
  22
+    close_port,
  23
+    log,
  24
+    config as orig_config_get,
  25
+    relations_of_type,
  26
+    relation_set,
  27
+    relation_ids,
  28
+    unit_get
  29
+)
  30
+from charmhelpers.contrib.charmsupport import nrpe
  31
+from charmhelpers.fetch import apt_update, add_source
  32
+
  33
+###############################################################################
  34
+# Global variables
  35
+###############################################################################
  36
+default_apache2_service_config_dir = "/var/run/apache2"
  37
+service_affecting_packages = ['apache2']
  38
+default_apache22_config_dir = "/etc/apache2/conf.d"
  39
+default_apache24_config_dir = "/etc/apache2/conf-available"
  40
+default_apache_base_dir = "/etc/apache2"
  41
+
  42
+juju_warning_header = """#
  43
+#    "             "
  44
+#  mmm   m   m   mmm   m   m
  45
+#    #   #   #     #   #   #
  46
+#    #   #   #     #   #   #
  47
+#    #   "mm"#     #   "mm"#
  48
+#    #             #
  49
+#  ""            ""
  50
+# This file is managed by Juju. Do not make local changes.
  51
+#"""
  52
+
  53
+
  54
+###############################################################################
  55
+# Supporting functions
  56
+###############################################################################
  57
+
  58
+def apt_get_install(package=None):
  59
+    """Install a package."""
  60
+    if package is None:
  61
+        return False
  62
+    cmd_line = ['apt-get', '-y', 'install', '-qq']
  63
+    cmd_line.append(package)
  64
+    return subprocess.call(cmd_line)
  65
+
  66
+
  67
+def ensure_package_status(packages, status):
  68
+    if status in ['install', 'hold']:
  69
+        selections = ''.join(['{} {}\n'.format(package, status)
  70
+                              for package in packages])
  71
+        dpkg = subprocess.Popen(['dpkg', '--set-selections'],
  72
+                                stdin=subprocess.PIPE)
  73
+        dpkg.communicate(input=selections)
  74
+
  75
+
  76
+# -----------------------------------------------------------------------------
  77
+# apt_get_purge( package ):  Purges a package
  78
+# -----------------------------------------------------------------------------
  79
+def apt_get_purge(packages=None):
  80
+    if packages is None:
  81
+        return False
  82
+    cmd_line = ['apt-get', '-y', 'purge', '-qq']
  83
+    cmd_line.append(packages)
  84
+    return subprocess.call(cmd_line)
  85
+
  86
+
  87
+# -----------------------------------------------------------------------------
  88
+# service_apache2:  Convenience function to start/stop/restart/reload
  89
+#                   the apache2 service
  90
+# -----------------------------------------------------------------------------
  91
+def service_apache2(action=None):
  92
+    if action is None:
  93
+        return
  94
+    elif action == "check":
  95
+        args = ['/usr/sbin/apache2ctl', 'configtest']
  96
+    else:
  97
+        args = ['service', 'apache2', action]
  98
+    ret_val = subprocess.call(args)
  99
+    return ret_val == 0
 100
+
 101
+
 102
+def run(command, *args, **kwargs):
 103
+    try:
 104
+        output = subprocess.check_output(command, *args, **kwargs)
 105
+        return output
 106
+    except Exception, e:
 107
+        print e
 108
+        raise
 109
+
 110
+
 111
+def enable_module(module=None):
 112
+    if module is None:
 113
+        return True
 114
+    if os.path.exists("/etc/apache2/mods-enabled/%s.load" % (module)):
 115
+        log("Module already loaded: %s" % module)
 116
+        return True
 117
+    if not os.path.exists("/etc/apache2/mods-available/%s.load" % (module)):
 118
+        return_value = apt_get_install("libapache2-mod-%s" % (module))
 119
+        if return_value != 0:
 120
+            log("Installing module %s failed" % (module))
 121
+            return False
 122
+    return_value = subprocess.call(['/usr/sbin/a2enmod', module])
 123
+    if return_value != 0:
 124
+        return False
 125
+    if service_apache2("check"):
 126
+        service_apache2("reload")
 127
+        return True
 128
+
 129
+
 130
+def disable_module(module=None):
 131
+    if module is None:
 132
+        return True
 133
+    if not os.path.exists("/etc/apache2/mods-enabled/%s.load" % (module)):
 134
+        log("Module already disabled: %s" % module)
 135
+        return True
 136
+    return_value = subprocess.call(['/usr/sbin/a2dismod', module])
 137
+    if return_value != 0:
 138
+        return False
 139
+    if service_apache2("check"):
 140
+        service_apache2("reload")
 141
+        return True
 142
+
 143
+
 144
+def is_apache24():
 145
+    return os.path.exists("/usr/sbin/a2enconf")
 146
+
 147
+
 148
+def site_filename(name, enabled=False):
 149
+    if enabled:
 150
+        sites_dir = "%s/sites-enabled" % default_apache_base_dir
 151
+    else:
 152
+        sites_dir = "%s/sites-available" % default_apache_base_dir
 153
+
 154
+    if is_apache24():
 155
+        return "{}/{}.conf".format(sites_dir, name)
 156
+    else:
 157
+        return "{}/{}".format(sites_dir, name)
 158
+
 159
+
 160
+def conf_filename(name):
 161
+    """Return an apache2 config filename path, as:
 162
+      2.4: /etc/apache2/conf-available/foo.conf
 163
+      2.2: /etc/apache2/conf.d/foo
 164
+    """
 165
+    if is_apache24():
 166
+        return "{}/{}.conf".format(default_apache24_config_dir, name)
 167
+    else:
 168
+        return "{}/{}".format(default_apache22_config_dir, name)
 169
+
 170
+
 171
+def conf_enable(name):
 172
+    "Enable apache2 config without reloading service"
 173
+    if is_apache24():
 174
+        return subprocess.call(['/usr/sbin/a2enconf', name]) == 0
 175
+    # no-op otherwise
 176
+    return True
 177
+
 178
+
 179
+def conf_disable(name):
 180
+    "Disable apache2 config without reloading service"
 181
+    if is_apache24():
 182
+        return subprocess.call(['/usr/sbin/a2disconf', name]) == 0
 183
+    # no-op otherwise
 184
+    return True
 185
+
 186
+
 187
+def gen_selfsigned_cert(config, cert_file, key_file):
 188
+    """
 189
+    Create a self-signed certificate.
 190
+
 191
+    @param config: charm data from config-get
 192
+    @param cert_file: destination path of generated certificate
 193
+    @param key_file: destination path of generated private key
 194
+    """
 195
+    os.environ['OPENSSL_CN'] = config['servername']
 196
+    os.environ['OPENSSL_PUBLIC'] = unit_get("public-address")
 197
+    os.environ['OPENSSL_PRIVATE'] = unit_get("private-address")
 198
+    run(
 199
+        ['openssl', 'req', '-new', '-x509', '-nodes',
 200
+         '-days', '3650', '-config',
 201
+         os.path.join(os.environ['CHARM_DIR'], 'data', 'openssl.cnf'),
 202
+         '-keyout', key_file, '-out', cert_file])
 203
+
 204
+
 205
+def is_selfsigned_cert_stale(config, cert_file, key_file):
 206
+    """
 207
+    Do we need to generate a new self-signed cert?
 208
+
 209
+    @param config: charm data from config-get
 210
+    @param cert_file: destination path of generated certificate
 211
+    @param key_file: destination path of generated private key
 212
+    """
 213
+    # Basic Existence Checks
 214
+    if not os.path.exists(cert_file):
 215
+        return True
 216
+    if not os.path.exists(key_file):
 217
+        return True
 218
+
 219
+    # Common Name
 220
+    from OpenSSL import crypto
 221
+    cert = crypto.load_certificate(
 222
+        crypto.FILETYPE_PEM, file(cert_file).read())
 223
+    cn = cert.get_subject().commonName
 224
+    if config['servername'] != cn:
 225
+        return True
 226
+
 227
+    # Subject Alternate Name -- only trusty+ support this
 228
+    try:
 229
+        from pyasn1.codec.der import decoder
 230
+        from pyasn1_modules import rfc2459
 231
+    except ImportError:
 232
+        log("Cannot check subjAltName on <= 12.04, skipping.")
 233
+        return False
 234
+    cert_addresses = set()
 235
+    unit_addresses = set(
 236
+        [unit_get("public-address"), unit_get("private-address")])
 237
+    for i in range(0, cert.get_extension_count()):
 238
+        extension = cert.get_extension(i)
 239
+        try:
 240
+            names = decoder.decode(
 241
+                extension.get_data(), asn1Spec=rfc2459.SubjectAltName())[0]
 242
+            for name in names:
 243
+                cert_addresses.add(str(name.getComponent()))
 244
+        except:
 245
+            pass
 246
+    if cert_addresses != unit_addresses:
 247
+        log("subjAltName: Cert (%s) != Unit (%s), assuming stale" % (
 248
+            cert_addresses, unit_addresses))
 249
+        return True
 250
+
 251
+    return False
 252
+
 253
+
 254
+def _get_key_file_location(config_data):
 255
+    """Look at the config, generate the key file location."""
 256
+    key_file = None
 257
+    if config_data['ssl_keylocation']:
 258
+        key_file = '/etc/ssl/private/%s' % \
 259
+            (config_data['ssl_keylocation'].rpartition('/')[2])
 260
+    return key_file
 261
+
 262
+
 263
+def _get_cert_file_location(config_data):
 264
+    """Look at the config, generate the cert file location."""
 265
+    cert_file = None
 266
+    if config_data['ssl_certlocation']:
 267
+        cert_file = '/etc/ssl/certs/%s' % \
 268
+            (config_data['ssl_certlocation'].rpartition('/')[2])
 269
+    return cert_file
 270
+
 271
+
 272
+def _get_chain_file_location(config_data):
 273
+    """Look at the config, generate the chain file location."""
 274
+    chain_file = None
 275
+    if config_data['ssl_chainlocation']:
 276
+        chain_file = '/etc/ssl/certs/%s' % \
 277
+            (config_data['ssl_chainlocation'].rpartition('/')[2])
 278
+    return chain_file
 279
+
 280
+
 281
+def config_get(scope=None):
 282
+    """
 283
+    Wrapper around charm helper's config_get to replace an empty servername
 284
+    with the public-address.
 285
+    """
 286
+    result = orig_config_get(scope)
 287
+    if scope == "servername" and len(result) == 0:
 288
+        result = unit_get("public-address")
 289
+    elif isinstance(result, dict) and result.get("servername", "") == "":
 290
+        result["servername"] = unit_get("public-address")
 291
+    return result
 292
+
 293
+
 294
+def install_hook():
 295
+    apt_source = config_get('apt-source') or ''
 296
+    apt_key_id = config_get('apt-key-id') or False
 297
+    if apt_source and apt_key_id:
 298
+        print apt_source + " and " + apt_key_id
 299
+        add_source(apt_source, apt_key_id)
 300
+        open('config.apt-source', 'w').write(apt_source)
 301
+    if not os.path.exists(default_apache2_service_config_dir):
 302
+        os.mkdir(default_apache2_service_config_dir, 0600)
 303
+    apt_update(fatal=True)
 304
+    apt_get_install("python-jinja2")
 305
+    apt_get_install("python-pyasn1")
 306
+    apt_get_install("python-pyasn1-modules")
 307
+    apt_get_install("python-yaml")
 308
+    install_status = apt_get_install("apache2")
 309
+    if install_status == 0:
 310
+        ensure_package_status(service_affecting_packages,
 311
+                              config_get('package_status'))
 312
+    ensure_extra_packages()
 313
+    # the apache2 deb does not yet have http2 module in mods-available. Add it.
 314
+    open('/etc/apache2/mods-available/http2.load', 'w').write(
 315
+        'LoadModule http2_module /usr/lib/apache2/modules/mod_http2.so')
 316
+    open('/etc/apache2/mods-available/http2.conf', 'w').write(
 317
+        '''<IfModule http2_module>
 318
+  ProtocolsHonorOrder On
 319
+  Protocols h2 http/1.1
 320
+</IfModule>
 321
+''')
 322
+    return install_status
 323
+
 324
+
 325
+def ensure_extra_packages():
 326
+    extra = str(config_get('extra_packages'))
 327
+    if extra:
 328
+        install_status = apt_get_install(extra)
 329
+        if install_status == 0:
 330
+            ensure_package_status(filter(None, extra.split(' ')),
 331
+                                  config_get('package_status'))
 332
+
 333
+
 334
+def dump_data(data2dump, log_prefix):
 335
+    log_file = '/tmp/pprint-%s.log' % (log_prefix)
 336
+    if data2dump is not None:
 337
+        logFile = open(log_file, 'w')
 338
+        import pprint
 339
+        pprint.pprint(data2dump, logFile)
 340
+        logFile.close()
 341
+
 342
+
 343
+def get_reverseproxy_data(relation='reverseproxy'):
 344
+    relation_data = relations_of_type(relation)
 345
+    reverseproxy_data = {}
 346
+    if relation_data is None or len(relation_data) == 0:
 347
+        return reverseproxy_data
 348
+    for unit_data in relation_data:
 349
+        unit_name = unit_data["__unit__"]
 350
+        if 'port' not in unit_data:
 351
+            return reverseproxy_data
 352
+        # unit_name: <service-name>-<unit_number>
 353
+        # jinja2 templates require python-type variables, remove all characters
 354
+        # that do not comply
 355
+        unit_type = re.sub(r'(.*)/[0-9]*', r'\1', unit_name)
 356
+        unit_type = re.sub('[^a-zA-Z0-9_]*', '', unit_type)
 357
+        log('unit_type: %s' % unit_type)
 358
+
 359
+        host = unit_data['private-address']
 360
+        if unit_type in reverseproxy_data:
 361
+            continue
 362
+        for config_setting in unit_data.keys():
 363
+            if config_setting in ("__unit__", "__relid__"):
 364
+                continue
 365
+            config_key = '%s_%s' % (unit_type,
 366
+                                    config_setting.replace("-", "_"))
 367
+            config_key = re.sub('[^a-zA-Z0-9_]*', '', config_key)
 368
+            reverseproxy_data[config_key] = unit_data[
 369
+                config_setting]
 370
+            reverseproxy_data[unit_type] = '%s:%s' % (
 371
+                host, unit_data['port'])
 372
+        if 'all_services' in unit_data:
 373
+            service_data = yaml.safe_load(unit_data['all_services'])
 374
+            for service_item in service_data:
 375
+                service_name = service_item['service_name']
 376
+                service_port = service_item['service_port']
 377
+                service_key = '%s_%s' % (unit_type, service_name)
 378
+                service_key = re.sub('[^a-zA-Z0-9_]*', '', service_key)
 379
+                reverseproxy_data[service_key] = '%s:%s' % (host, service_port)
 380
+    return reverseproxy_data
 381
+
 382
+
 383
+def update_balancers():
 384
+    relation_data = relations_of_type('balancer')
 385
+    if relation_data is None or len(relation_data) == 0:
 386
+        log("No relation data, exiting.")
 387
+        return
 388
+
 389
+    unit_dict = {}
 390
+    for unit_data in relation_data:
 391
+        unit_name = unit_data["__unit__"]
 392
+        if "port" not in unit_data:
 393
+            log("No port in relation data for '%s', skipping." % unit_name)
 394
+            continue
 395
+        port = unit_data["port"]
 396
+        if "private-address" not in unit_data:
 397
+            log("No private-address in relation data for '%s', skipping." %
 398
+                unit_name)
 399
+            continue
 400
+        host = unit_data['private-address']
 401
+
 402
+        if "all_services" in unit_data:
 403
+            service_data = yaml.safe_load(unit_data[
 404
+                "all_services"])
 405
+            for service_item in service_data:
 406
+                service_port = service_item["service_port"]
 407
+                current_units = unit_dict.setdefault(
 408
+                    service_item["service_name"], [])
 409
+                current_units.append("%s:%s" % (host, service_port))
 410
+        else:
 411
+            if "sitenames" in unit_data:
 412
+                unit_types = unit_data["sitenames"].split()
 413
+            else:
 414
+                unit_types = (re.sub(r"(.*)/[0-9]*", r"\1", unit_name),)
 415
+
 416
+            for unit_type in unit_types:
 417
+                current_units = unit_dict.setdefault(unit_type, [])
 418
+                current_units.append("%s:%s" % (host, port))
 419
+
 420
+    if not unit_dict:
 421
+        return
 422
+
 423
+    write_balancer_config(unit_dict)
 424
+    return unit_dict
 425
+
 426
+
 427
+def write_balancer_config(unit_dict):
 428
+    config_data = config_get()
 429
+
 430
+    from jinja2 import Environment, FileSystemLoader
 431
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 432
+        os.environ['CHARM_DIR'], 'data')))
 433
+    for balancer_name in unit_dict.keys():
 434
+        balancer_host_file = conf_filename('{}.balancer'.format(balancer_name))
 435
+        templ_vars = {
 436
+            'balancer_name': balancer_name,
 437
+            'balancer_addresses': unit_dict[balancer_name],
 438
+            'lb_balancer_timeout': config_data['lb_balancer_timeout'],
 439
+        }
 440
+        template = template_env.get_template(
 441
+            'balancer.template').render(templ_vars)
 442
+        log("Writing file: %s with data: %s" % (balancer_host_file,
 443
+                                                templ_vars))
 444
+        with open(balancer_host_file, 'w') as balancer_config:
 445
+            balancer_config.write(str(template))
 446
+        conf_enable('{}.balancer'.format(balancer_name))
 447
+
 448
+
 449
+def update_nrpe_checks():
 450
+    nrpe_compat = nrpe.NRPE()
 451
+    conf = nrpe_compat.config
 452
+    check_http_params = conf.get('nagios_check_http_params')
 453
+    if check_http_params:
 454
+        nrpe_compat.add_check(
 455
+            shortname='vhost',
 456
+            description='Check Virtual Host',
 457
+            check_cmd='check_http %s' % check_http_params
 458
+        )
 459
+    nrpe_compat.write()
 460
+
 461
+
 462
+def create_mpm_workerfile():
 463
+    config_data = config_get()
 464
+    mpm_workerfile = conf_filename('000mpm-worker')
 465
+    from jinja2 import Environment, FileSystemLoader
 466
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 467
+        os.environ['CHARM_DIR'], 'data')))
 468
+    templ_vars = {
 469
+        'mpm_type': config_data['mpm_type'],
 470
+        'mpm_startservers': config_data['mpm_startservers'],
 471
+        'mpm_minsparethreads': config_data['mpm_minsparethreads'],
 472
+        'mpm_maxsparethreads': config_data['mpm_maxsparethreads'],
 473
+        'mpm_threadlimit': config_data['mpm_threadlimit'],
 474
+        'mpm_threadsperchild': config_data['mpm_threadsperchild'],
 475
+        'mpm_serverlimit': config_data['mpm_serverlimit'],
 476
+        'mpm_maxclients': config_data['mpm_maxclients'],
 477
+        'mpm_maxrequestsperchild': config_data['mpm_maxrequestsperchild'],
 478
+    }
 479
+    template = \
 480
+        template_env.get_template('mpm_worker.template').render(templ_vars)
 481
+    with open(mpm_workerfile, 'w') as mpm_config:
 482
+        mpm_config.write(str(template))
 483
+    conf_enable('000mpm-worker')
 484
+
 485
+
 486
+def create_security():
 487
+    config_data = config_get()
 488
+    securityfile = conf_filename('security')
 489
+    from jinja2 import Environment, FileSystemLoader
 490
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 491
+        os.environ['CHARM_DIR'], 'data')))
 492
+    templ_vars = {
 493
+        'juju_warning_header': juju_warning_header,
 494
+        'server_tokens': config_data['server_tokens'],
 495
+        'server_signature': config_data['server_signature'],
 496
+        'trace_enabled': config_data['trace_enabled'],
 497
+        'ssl_protocol': config_data['ssl_protocol'],
 498
+        'ssl_honor_cipher_order': config_data['ssl_honor_cipher_order'],
 499
+        'ssl_cipher_suite': config_data['ssl_cipher_suite'],
 500
+        'is_apache24': is_apache24(),
 501
+    }
 502
+    template = \
 503
+        template_env.get_template('security.template').render(templ_vars)
 504
+    with open(securityfile, 'w') as security_config:
 505
+        security_config.write(str(template))
 506
+    conf_enable('security')
 507
+
 508
+
 509
+def ship_logrotate_conf():
 510
+    config_data = config_get()
 511
+    logrotate_file = '/etc/logrotate.d/apache2'
 512
+    from jinja2 import Environment, FileSystemLoader
 513
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 514
+        os.environ['CHARM_DIR'], 'data')))
 515
+    templ_vars = {
 516
+        'juju_warning_header': juju_warning_header,
 517
+        'logrotate_rotate': config_data['logrotate_rotate'],
 518
+        'logrotate_count': config_data['logrotate_count'],
 519
+        'logrotate_dateext': config_data['logrotate_dateext'],
 520
+    }
 521
+    template = template_env.get_template('logrotate.conf.template').render(
 522
+        templ_vars)
 523
+    with open(logrotate_file, 'w') as logrotate_conf:
 524
+        logrotate_conf.write(str(template))
 525
+
 526
+
 527
+def create_vhost(port, protocol=None, config_key=None, template_str=None,
 528
+                 config_data={}, relationship_data={}):
 529
+    """
 530
+    Create and enable a vhost in apache.
 531
+
 532
+    @param port: port on which to listen (int)
 533
+    @param protocol: used to name the vhost file intelligently.  If not
 534
+        specified the port will be used instead. (ex: http, https)
 535
+    @param config_key: key in the configuration to look up to
 536
+        retrieve the template.
 537
+    @param template_str: The template itself as a string.
 538
+    @param config_data: juju get-config configuration data.
 539
+    @param relationship_data: if in a relationship, pass in the appropriate
 540
+        structure.  This will be used to inform the template.
 541
+    """
 542
+    if protocol is None:
 543
+        protocol = str(port)
 544
+    if template_str is None:
 545
+        if not config_key or not config_data[config_key]:
 546
+            log("Vhost Template not provided, not configuring: %s" % port)
 547
+            return False
 548
+        template_str = config_data[config_key]
 549
+    from jinja2 import Template
 550
+    template = Template(str(base64.b64decode(template_str)))
 551
+    template_data = dict(config_data.items() + relationship_data.items())
 552
+    if config_data.get('vhost_template_vars'):
 553
+        extra_vars = ast.literal_eval(config_data['vhost_template_vars'])
 554
+        template_data.update(extra_vars)
 555
+    vhost_name = '%s_%s' % (config_data['servername'], protocol)
 556
+    vhost_file = site_filename(vhost_name)
 557
+    log("Writing file %s with config and relation data" % vhost_file)
 558
+    with open(vhost_file, 'w') as vhost:
 559
+        vhost.write(str(template.render(template_data)))
 560
+    subprocess.call(['/usr/sbin/a2ensite', vhost_name])
 561
+    return True
 562
+
 563
+
 564
+MPM_TYPES = ['mpm_worker', 'mpm_prefork', 'mpm_event']
 565
+
 566
+
 567
+def enable_mpm(config):
 568
+    """Enables a particular mpm module.
 569
+
 570
+    Different from simply enabling a module, as one and only one mpm module
 571
+    *must* be enabled.
 572
+    """
 573
+    # only do anything if value has changed, to avoid a needless restart
 574
+    if not config.changed('mpm_type'):
 575
+        return
 576
+
 577
+    mpm_type = config.get('mpm_type', '')
 578
+    name = 'mpm_' + mpm_type
 579
+    if name not in MPM_TYPES:
 580
+        log('bad mpm_type: %s. Falling back to mpm_worker' % mpm_type)
 581
+        name = 'mpm_worker'
 582
+
 583
+    # disable all other mpm modules
 584
+    for mpm in MPM_TYPES:
 585
+        if mpm != name:
 586
+            return_value = subprocess.call(['/usr/sbin/a2dismod', mpm])
 587
+            if return_value != 0:
 588
+                return False
 589
+
 590
+    return_value = subprocess.call(['/usr/sbin/a2enmod', name])
 591
+    if return_value != 0:
 592
+        return False
 593
+
 594
+    if service_apache2("check"):
 595
+        log("Switching mpm module to {}".format(name))
 596
+        service_apache2("restart")  # must be a restart to switch mpm
 597
+        return True
 598
+    else:
 599
+        log("Failed to switch mpm module to {}".format(name))
 600
+        return False
 601
+
 602
+
 603
+def config_changed():
 604
+    relationship_data = {}
 605
+    config_data = config_get()
 606
+
 607
+    apt_source = config_data['apt-source']
 608
+    old_apt_source = ''
 609
+    try:
 610
+        old_apt_source = open('config.apt-source', 'r').read()
 611
+    except IOError:
 612
+        pass
 613
+    if old_apt_source != apt_source:
 614
+        subprocess.check_call(['add-apt-repository', '--yes', '-r',
 615
+                               old_apt_source])
 616
+        add_source(apt_source, config_data['apt-key-id'])
 617
+        open('config.apt-source', 'w').write(apt_source)
 618
+
 619
+    ensure_package_status(service_affecting_packages,
 620
+                          config_data['package_status'])
 621
+    ensure_extra_packages()
 622
+
 623
+    relationship_data.update(get_reverseproxy_data(relation='reverseproxy'))
 624
+    relationship_data.update(get_reverseproxy_data(relation='website-cache'))
 625
+    if update_balancers():
 626
+        # apache 2.4 has lbmethods split, needs to enable specific module(s)
 627
+        if is_apache24():
 628
+            enable_module('lbmethod_byrequests')
 629
+
 630
+    disabled_modules = config_data['disable_modules'].split()
 631
+    apache_websites = ApacheWebsites.from_config(
 632
+        relations_of_type("apache-website"), disabled_modules)
 633
+    enabled_modules = config_data.get('enable_modules', '').split()
 634
+    enabled_modules = apache_websites.list_enabled_modules(enabled_modules)
 635
+    for module in enabled_modules:
 636
+        enable_module(module)
 637
+
 638
+    if config_data['disable_modules']:
 639
+        for module in disabled_modules:
 640
+            disable_module(module)
 641
+
 642
+    apache_websites.disable_sites()
 643
+    apache_websites.write_configs()
 644
+    apache_websites.enable_sites()
 645
+    apache_websites.configure_extra_ports()
 646
+    all_ports = apache_websites.list_enabled_ports()
 647
+    enable_mpm(config_data)
 648
+    # XXX we only configure the worker mpm?
 649
+    create_mpm_workerfile()
 650
+    create_security()
 651
+
 652
+    ports = {'http': 80, 'https': 443}
 653
+    for protocol, port in ports.iteritems():
 654
+        if create_vhost(
 655
+                port,
 656
+                protocol=protocol,
 657
+                config_key="vhost_%s_template" % protocol,
 658
+                config_data=config_data,
 659
+                relationship_data=relationship_data):
 660
+            all_ports.add(port)
 661
+
 662
+    cert_file = _get_cert_file_location(config_data)
 663
+    key_file = _get_key_file_location(config_data)
 664
+    chain_file = _get_chain_file_location(config_data)
 665
+
 666
+    if cert_file is not None and key_file is not None:
 667
+        # ssl_cert is SELFSIGNED so generate self-signed certificate for use.
 668
+        if config_data['ssl_cert'] and config_data['ssl_cert'] == "SELFSIGNED":
 669
+            if is_selfsigned_cert_stale(config_data, cert_file, key_file):
 670
+                gen_selfsigned_cert(config_data, cert_file, key_file)
 671
+
 672
+        # Use SSL certificate and key provided either as a base64 string or
 673
+        # shipped out with the charm.
 674
+        else:
 675
+            # Certificate provided as base64-encoded string.
 676
+            if config_data['ssl_cert']:
 677
+                log("Writing cert from config ssl_cert: %s" % cert_file)
 678
+                with open(cert_file, 'w') as f:
 679
+                    f.write(str(base64.b64decode(config_data['ssl_cert'])))
 680
+            # Use certificate file shipped out with charm.
 681
+            else:
 682
+                source = os.path.join(os.environ['CHARM_DIR'], 'data',
 683
+                                      config_data['ssl_certlocation'])
 684
+                if os.path.exists(source):
 685
+                    shutil.copy(source, cert_file)
 686
+                else:
 687
+                    log("Certificate not found, ignoring: %s" % source)
 688
+
 689
+            # Private key provided as base64-encoded string.
 690
+            if config_data['ssl_key']:
 691
+                log("Writing key from config ssl_key: %s" % key_file)
 692
+                with open(key_file, 'w') as f:
 693
+                    f.write(str(base64.b64decode(config_data['ssl_key'])))
 694
+            # Use private key shipped out with charm.
 695
+            else:
 696
+                source = os.path.join(os.environ['CHARM_DIR'], 'data',
 697
+                                      config_data['ssl_keylocation'])
 698
+                if os.path.exists(source):
 699
+                    shutil.copy(source, key_file)
 700
+                else:
 701
+                    log("Key file not found, ignoring: %s" % source)
 702
+
 703
+            if chain_file is not None:
 704
+                # Chain certificates provided as base64-encoded string.
 705
+                if config_data['ssl_chain']:
 706
+                    log("Writing chain certificates file from"
 707
+                        "config ssl_chain: %s" % chain_file)
 708
+                    with open(chain_file, 'w') as f:
 709
+                        f.write(str(base64.b64decode(
 710
+                            config_data['ssl_chain'])))
 711
+                # Use chain certificates shipped out with charm.
 712
+                else:
 713
+                    source = os.path.join(os.environ['CHARM_DIR'], 'data',
 714
+                                          config_data['ssl_chainlocation'])
 715
+                    if os.path.exists(source):
 716
+                        shutil.copy(source, chain_file)
 717
+                    else:
 718
+                        log("Chain certificates not found, "
 719
+                            "ignoring: %s" % source)
 720
+
 721
+        # Tighten permissions on private key file.
 722
+        if os.path.exists(key_file):
 723
+            os.chmod(key_file, 0440)
 724
+            os.chown(key_file, pwd.getpwnam('root').pw_uid,
 725
+                     grp.getgrnam('ssl-cert').gr_gid)
 726
+
 727
+    apache_syslog_conf = conf_filename("syslog")
 728
+    rsyslog_apache_conf = "/etc/rsyslog.d/45-apache2.conf"
 729
+    if config_data['use_rsyslog']:
 730
+        shutil.copy2("data/syslog-apache.conf", apache_syslog_conf)
 731
+        conf_enable("syslog")
 732
+        shutil.copy2("data/syslog-rsyslog.conf", rsyslog_apache_conf)
 733
+        # Fix permissions of access.log and error.log to allow syslog user to
 734
+        # write to
 735
+        os.chown("/var/log/apache2/access.log", pwd.getpwnam('syslog').pw_uid,
 736
+                 pwd.getpwnam('syslog').pw_gid)
 737
+        os.chown("/var/log/apache2/error.log", pwd.getpwnam('syslog').pw_uid,
 738
+                 pwd.getpwnam('syslog').pw_gid)
 739
+    else:
 740
+        conf_disable("syslog")
 741
+        if os.path.exists(apache_syslog_conf):
 742
+            os.unlink(apache_syslog_conf)
 743
+        if os.path.exists(rsyslog_apache_conf):
 744
+            os.unlink(rsyslog_apache_conf)
 745
+    run(["/usr/sbin/service", "rsyslog", "restart"])
 746
+
 747
+    # Disable the default website because we don't want people to see the
 748
+    # "It works!" page on production services and remove the
 749
+    # conf.d/other-vhosts-access-log conf.
 750
+    ensure_disabled(["000-default"])
 751
+    conf_disable("other-vhosts-access-log")
 752
+    if os.path.exists(conf_filename("other-vhosts-access-log")):
 753
+        os.unlink(conf_filename("other-vhosts-access-log"))
 754
+
 755
+    if service_apache2("check"):
 756
+        if config_data["config_change_command"] in ["reload", "restart"]:
 757
+            service_apache2(config_data["config_change_command"])
 758
+
 759
+    if config_data['openid_provider']:
 760
+        if not os.path.exists('/etc/apache2/security'):
 761
+            os.mkdir('/etc/apache2/security', 0755)
 762
+        with open('/etc/apache2/security/allowed-ops.txt', 'w') as f:
 763
+            f.write(config_data['openid_provider'].replace(',', '\n'))
 764
+            f.write('\n')
 765
+            os.chmod(key_file, 0444)
 766
+
 767
+    all_ports.update(update_vhost_config_relation())
 768
+    ensure_ports(all_ports)
 769
+    update_nrpe_checks()
 770
+    ship_logrotate_conf()
 771
+    if config_get().changed('servername'):
 772
+        logs_relation_joined()
 773
+
 774
+
 775
+def ensure_disabled(sites):
 776
+    to_disable = [s for s in sites if os.path.exists(site_filename(s, True))]
 777
+    if len(to_disable) == 0:
 778
+        return
 779
+    run(["/usr/sbin/a2dissite"] + to_disable)
 780
+
 781
+
 782
+def ensure_removed(filename):
 783
+    try:
 784
+        os.unlink(filename)
 785
+    except OSError as e:
 786
+        if e.errno != errno.ENOENT:
 787
+            raise
 788
+
 789
+
 790
+class ApacheWebsites:
 791
+
 792
+    @classmethod
 793
+    def from_config(cls, relations, disabled_modules):
 794
+        """Return an ApacheWebsites with information about all sites."""
 795
+        if relations is None:
 796
+            relations = []
 797
+        self_relations = {}
 798
+        for relation in relations:
 799
+            self_relation = {'domain': relation.get('domain')}
 800
+            enabled = bool(relation.get('enabled', 'False').lower() == 'true')
 801
+            site_modules = relation.get('site_modules', '').split()
 802
+            for module in site_modules:
 803
+                if module in disabled_modules:
 804
+                    enabled = False
 805
+                    log('site {} requires disabled_module {}'.format(
 806
+                        relation['__relid__'], module))
 807
+                break
 808
+            self_relation['site_modules'] = site_modules
 809
+            self_relation['enabled'] = enabled
 810
+            self_relation['site_config'] = relation.get('site_config')
 811
+            self_relation['ports'] = [
 812
+                int(p) for p in relation.get('ports', '').split()]
 813
+            self_relations[relation['__relid__']] = self_relation
 814
+        return cls(self_relations)
 815
+
 816
+    def __init__(self, relations):
 817
+        self.relations = relations
 818
+
 819
+    def write_configs(self):
 820
+        for key, relation in self.relations.items():
 821
+            config_file = site_filename(key)
 822
+            site_config = relation['site_config']
 823
+            if site_config is None:
 824
+                ensure_removed(config_file)
 825
+            else:
 826
+                with open(config_file, 'w') as output:
 827
+                    output.write(site_config)
 828
+
 829
+    def iter_enabled_sites(self):
 830
+        return ((k, v) for k, v in self.relations.items() if v['enabled'])
 831
+
 832
+    def enable_sites(self):
 833
+        enabled_sites = [k for k, v in self.iter_enabled_sites()]
 834
+        enabled_sites.sort()
 835
+        if len(enabled_sites) == 0:
 836
+            return
 837
+        subprocess.check_call(['/usr/sbin/a2ensite'] + enabled_sites)
 838
+
 839
+    def disable_sites(self):
 840
+        disabled_sites = [k for k, v in self.relations.items()
 841
+                          if not v['enabled']]
 842
+        disabled_sites.sort()
 843
+        if len(disabled_sites) == 0:
 844
+            return
 845
+        ensure_disabled(disabled_sites)
 846
+
 847
+    def list_enabled_modules(self, enabled_modules):
 848
+        enabled_modules = set(enabled_modules)
 849
+        for key, relation in self.iter_enabled_sites():
 850
+            enabled_modules.update(relation['site_modules'])
 851
+        return enabled_modules
 852
+
 853
+    def list_enabled_ports(self):
 854
+        enabled_ports = set()
 855
+        for key, relation in self.iter_enabled_sites():
 856
+            enabled_ports.update(relation['ports'])
 857
+        return enabled_ports
 858
+
 859
+    def configure_extra_ports(self):
 860
+        extra_ports = self.list_enabled_ports()
 861
+        extra_ports.discard(80)
 862
+        extra_ports.discard(443)
 863
+        extra_ports_conf = conf_filename('extra_ports')
 864
+        if len(extra_ports) > 0:
 865
+            with file(extra_ports_conf, 'w') as f:
 866
+                for port in sorted(extra_ports):
 867
+                    f.write('Listen {}\n'.format(port))
 868
+            conf_enable('extra_ports')
 869
+        else:
 870
+            conf_disable('extra_ports')
 871
+            ensure_removed(extra_ports_conf)
 872
+
 873
+
 874
+def update_vhost_config_relation():
 875
+    """
 876
+    Update the vhost file and include the certificate in the relation
 877
+    if it is self-signed.
 878
+    """
 879
+    vhost_ports = set()
 880
+    relation_data = relations_of_type("vhost-config")
 881
+    config_data = config_get()
 882
+    if relation_data is None:
 883
+        return vhost_ports
 884
+
 885
+    for unit_data in relation_data:
 886
+        if "vhosts" in unit_data:
 887
+            all_relation_data = {}
 888
+            all_relation_data.update(
 889
+                get_reverseproxy_data(relation='reverseproxy'))
 890
+            all_relation_data.update(
 891
+                get_reverseproxy_data(relation='website-cache'))
 892
+            try:
 893
+                vhosts = yaml.safe_load(unit_data["vhosts"])
 894
+                for vhost in vhosts:
 895
+                    port = vhost["port"]
 896
+                    if create_vhost(
 897
+                            port,
 898
+                            template_str=vhost["template"],
 899
+                            config_data=config_data,
 900
+                            relationship_data=all_relation_data):
 901
+                        vhost_ports.add(port)
 902
+            except Exception as e:
 903
+                log("Error reading configuration data from relation! %s" % e)
 904
+                raise
 905
+
 906
+    if service_apache2("check"):
 907
+        service_apache2("reload")
 908
+
 909
+    vhost_relation_settings = {
 910
+        "servername": config_data["servername"]}
 911
+
 912
+    cert_file = _get_cert_file_location(config_data)
 913
+    key_file = _get_key_file_location(config_data)
 914
+
 915
+    if cert_file is not None and key_file is not None:
 916
+        if config_data['ssl_cert'] and config_data['ssl_cert'] == "SELFSIGNED":
 917
+            with open(cert_file, 'r') as f:
 918
+                cert = base64.b64encode(f.read())
 919
+            vhost_relation_settings["ssl_cert"] = cert
 920
+    for id in relation_ids("vhost-config"):
 921
+        relation_set(relation_id=id, relation_settings=vhost_relation_settings)
 922
+    return vhost_ports
 923
+
 924
+
 925
+def start_hook():
 926
+    if service_apache2("status"):
 927
+        return(service_apache2("restart"))
 928
+    else:
 929
+        return(service_apache2("start"))
 930
+
 931
+
 932
+def stop_hook():
 933
+    if service_apache2("status"):
 934
+        return(service_apache2("stop"))
 935
+
 936
+
 937
+def reverseproxy_interface(hook_name=None):
 938
+    if hook_name is None:
 939
+        return(None)
 940
+    if hook_name == "changed":
 941
+        config_changed()
 942
+
 943
+
 944
+def website_interface(hook_name=None):
 945
+    if hook_name is None:
 946
+        return(None)
 947
+    my_host = socket.getfqdn(socket.gethostname())
 948
+    if my_host == "localhost":
 949
+        my_host = socket.gethostname()
 950
+    default_port = 80
 951
+    subprocess.call([
 952
+        'relation-set',
 953
+        'port=%d' % default_port,
 954
+        'hostname=%s' % my_host,
 955
+        'servername=%s' % config_get('servername')
 956
+    ])
 957
+
 958
+
 959
+def ensure_ports(ports):
 960
+    """Ensure that only the desired ports are open."""
 961
+    open_ports = set(get_open_ports())
 962
+    ports = set(ports)
 963
+    wanted_closed = ports.difference(open_ports)
 964
+    for port in sorted(wanted_closed):
 965
+        open_port(port)
 966
+    unwanted_open = open_ports.difference(ports)
 967
+    for port in sorted(unwanted_open):
 968
+        close_port(port)
 969
+    set_open_ports(list(sorted(ports)))
 970
+
 971
+
 972
+def get_open_ports():
 973
+    """Get the list of open ports from the standard file."""
 974
+    try:
 975
+        pfile = open(os.path.join(os.environ['CHARM_DIR'], 'ports.yaml'))
 976
+    except IOError as e:
 977
+        if e.errno == errno.ENOENT:
 978
+            return []
 979
+        else:
 980
+            raise
 981
+    with pfile:
 982
+        return yaml.safe_load(pfile)
 983
+
 984
+
 985
+def set_open_ports(ports):
 986
+    """Write the list of open ports to the standard file."""
 987
+    ports_path = os.path.join(os.environ['CHARM_DIR'], 'ports.yaml')
 988
+    with open(ports_path, 'w') as pfile:
 989
+        yaml.safe_dump(ports, pfile)
 990
+
 991
+
 992
+def get_log_files():
 993
+    """
 994
+    Read all of the apache config files from __ and get ErrorLog and AccessLog
 995
+    values.
 996
+
 997
+    Returns a tuple with first value list of access log files and second value
 998
+    list of error log files.
 999
+    """
1000
+    access_logs = []
1001
+    error_logs = []
1002
+    for protocol in ['http', 'https']:
1003
+        vhost_name = '%s_%s' % (config_get()['servername'], protocol)
1004
+        vhost_file = site_filename(vhost_name)
1005
+        try:
1006
+            # Using read().split('\n') here to work around a mocks open_mock
1007
+            # inadequacy: http://bugs.python.org/issue17467
1008
+            for line in open(vhost_file, 'r').read().split('\n'):
1009
+                if 'CustomLog' in line:
1010
+                    access_logs.append(line.split()[1])
1011
+                elif 'ErrorLog' in line:
1012
+                    error_logs.append(line.split()[1])
1013
+        except:
1014
+            pass
1015
+    return access_logs, error_logs
1016
+
1017
+
1018
+def logs_relation_joined():
1019
+    """
1020
+    Sets relation value with filenames
1021
+    """
1022
+    access_log_files, error_log_files = get_log_files()
1023
+    log_files = access_log_files[:]
1024
+    log_files.extend(error_log_files)
1025
+    types = ['apache_access' for a in access_log_files]
1026
+    types.extend(['apache_error' for a in error_log_files])
1027
+    data = {'files': '\n'.join(log_files),
1028
+            'types': '\n'.join(types),
1029
+            }
1030
+    _relation_ids = relation_ids('logs')
1031
+    for _relation_id in _relation_ids:
1032
+        log("logs-relation-joined setting relation data for {} to {}".format(
1033
+            _relation_id, data))
1034
+        relation_set(
1035
+            relation_id=_relation_id,
1036
+            relation_settings=data)
1037
+
1038
+
1039
+###############################################################################
1040
+# Main section
1041
+###############################################################################
1042
+def main(hook_name):
1043
+    if hook_name == "install":
1044
+        install_hook()
1045
+    elif hook_name == "config-changed" or hook_name == "upgrade-charm":
1046
+        config_changed()
1047
+    elif hook_name == "start":
1048
+        start_hook()
1049
+    elif hook_name == "stop":
1050
+        stop_hook()
1051
+    elif hook_name == "reverseproxy-relation-broken":
1052
+        config_changed()
1053
+    elif hook_name == "reverseproxy-relation-changed":
1054
+        config_changed()
1055
+    elif hook_name == "reverseproxy-relation-joined":
1056
+        config_changed()
1057
+    elif hook_name == "balancer-relation-broken":
1058
+        config_changed()
1059
+    elif hook_name == "balancer-relation-changed":
1060
+        config_changed()
1061
+    elif hook_name == "balancer-relation-joined":
1062
+        config_changed()
1063
+    elif hook_name == "website-cache-relation-broken":
1064
+        config_changed()
1065
+    elif hook_name == "website-cache-relation-changed":
1066
+        config_changed()
1067
+    elif hook_name == "website-cache-relation-joined":
1068
+        config_changed()
1069
+    elif hook_name == "website-relation-joined":
1070
+        website_interface("joined")
1071
+    elif hook_name == 'apache-website-relation-changed':
1072
+        config_changed()
1073
+    elif hook_name in ("nrpe-external-master-relation-changed",
1074
+                       "local-monitors-relation-changed"):
1075
+        update_nrpe_checks()
1076
+    elif hook_name == "vhost-config-relation-changed":
1077
+        config_changed()
1078
+    elif hook_name == "logs-relation-joined":
1079
+        logs_relation_joined()
1080
+    else:
1081
+        print "Unknown hook"
1082
+        sys.exit(1)
1083
+
1084
+if __name__ == "__main__":
1085
+    hook_name = os.path.basename(sys.argv[0])
1086
+    # Also support being invoked directly with hook as argument name.
1087
+    if hook_name == "hooks.py":
1088
+        if len(sys.argv) < 2:
1089
+            sys.exit("Missing required hook name argument.")
1090
+        hook_name = sys.argv[1]
1091
+    main(hook_name)
Back to file index

hooks/balancer-relation-changed

   1
--- 
   2
+++ hooks/balancer-relation-changed
   3
@@ -0,0 +1,1088 @@
   4
+#!/usr/bin/env python
   5
+
   6
+import errno
   7
+import os
   8
+import re
   9
+import socket
  10
+import subprocess
  11
+import sys
  12
+import yaml
  13
+import base64
  14
+import grp
  15
+import pwd
  16
+import shutil
  17
+import os.path
  18
+import ast
  19
+
  20
+from charmhelpers.core.hookenv import (
  21
+    open_port,
  22
+    close_port,
  23
+    log,
  24
+    config as orig_config_get,
  25
+    relations_of_type,
  26
+    relation_set,
  27
+    relation_ids,
  28
+    unit_get
  29
+)
  30
+from charmhelpers.contrib.charmsupport import nrpe
  31
+from charmhelpers.fetch import apt_update, add_source
  32
+
  33
+###############################################################################
  34
+# Global variables
  35
+###############################################################################
  36
+default_apache2_service_config_dir = "/var/run/apache2"
  37
+service_affecting_packages = ['apache2']
  38
+default_apache22_config_dir = "/etc/apache2/conf.d"
  39
+default_apache24_config_dir = "/etc/apache2/conf-available"
  40
+default_apache_base_dir = "/etc/apache2"
  41
+
  42
+juju_warning_header = """#
  43
+#    "             "
  44
+#  mmm   m   m   mmm   m   m
  45
+#    #   #   #     #   #   #
  46
+#    #   #   #     #   #   #
  47
+#    #   "mm"#     #   "mm"#
  48
+#    #             #
  49
+#  ""            ""
  50
+# This file is managed by Juju. Do not make local changes.
  51
+#"""
  52
+
  53
+
  54
+###############################################################################
  55
+# Supporting functions
  56
+###############################################################################
  57
+
  58
+def apt_get_install(package=None):
  59
+    """Install a package."""
  60
+    if package is None:
  61
+        return False
  62
+    cmd_line = ['apt-get', '-y', 'install', '-qq']
  63
+    cmd_line.append(package)
  64
+    return subprocess.call(cmd_line)
  65
+
  66
+
  67
+def ensure_package_status(packages, status):
  68
+    if status in ['install', 'hold']:
  69
+        selections = ''.join(['{} {}\n'.format(package, status)
  70
+                              for package in packages])
  71
+        dpkg = subprocess.Popen(['dpkg', '--set-selections'],
  72
+                                stdin=subprocess.PIPE)
  73
+        dpkg.communicate(input=selections)
  74
+
  75
+
  76
+# -----------------------------------------------------------------------------
  77
+# apt_get_purge( package ):  Purges a package
  78
+# -----------------------------------------------------------------------------
  79
+def apt_get_purge(packages=None):
  80
+    if packages is None:
  81
+        return False
  82
+    cmd_line = ['apt-get', '-y', 'purge', '-qq']
  83
+    cmd_line.append(packages)
  84
+    return subprocess.call(cmd_line)
  85
+
  86
+
  87
+# -----------------------------------------------------------------------------
  88
+# service_apache2:  Convenience function to start/stop/restart/reload
  89
+#                   the apache2 service
  90
+# -----------------------------------------------------------------------------
  91
+def service_apache2(action=None):
  92
+    if action is None:
  93
+        return
  94
+    elif action == "check":
  95
+        args = ['/usr/sbin/apache2ctl', 'configtest']
  96
+    else:
  97
+        args = ['service', 'apache2', action]
  98
+    ret_val = subprocess.call(args)
  99
+    return ret_val == 0
 100
+
 101
+
 102
+def run(command, *args, **kwargs):
 103
+    try:
 104
+        output = subprocess.check_output(command, *args, **kwargs)
 105
+        return output
 106
+    except Exception, e:
 107
+        print e
 108
+        raise
 109
+
 110
+
 111
+def enable_module(module=None):
 112
+    if module is None:
 113
+        return True
 114
+    if os.path.exists("/etc/apache2/mods-enabled/%s.load" % (module)):
 115
+        log("Module already loaded: %s" % module)
 116
+        return True
 117
+    if not os.path.exists("/etc/apache2/mods-available/%s.load" % (module)):
 118
+        return_value = apt_get_install("libapache2-mod-%s" % (module))
 119
+        if return_value != 0:
 120
+            log("Installing module %s failed" % (module))
 121
+            return False
 122
+    return_value = subprocess.call(['/usr/sbin/a2enmod', module])
 123
+    if return_value != 0:
 124
+        return False
 125
+    if service_apache2("check"):
 126
+        service_apache2("reload")
 127
+        return True
 128
+
 129
+
 130
+def disable_module(module=None):
 131
+    if module is None:
 132
+        return True
 133
+    if not os.path.exists("/etc/apache2/mods-enabled/%s.load" % (module)):
 134
+        log("Module already disabled: %s" % module)
 135
+        return True
 136
+    return_value = subprocess.call(['/usr/sbin/a2dismod', module])
 137
+    if return_value != 0:
 138
+        return False
 139
+    if service_apache2("check"):
 140
+        service_apache2("reload")
 141
+        return True
 142
+
 143
+
 144
+def is_apache24():
 145
+    return os.path.exists("/usr/sbin/a2enconf")
 146
+
 147
+
 148
+def site_filename(name, enabled=False):
 149
+    if enabled:
 150
+        sites_dir = "%s/sites-enabled" % default_apache_base_dir
 151
+    else:
 152
+        sites_dir = "%s/sites-available" % default_apache_base_dir
 153
+
 154
+    if is_apache24():
 155
+        return "{}/{}.conf".format(sites_dir, name)
 156
+    else:
 157
+        return "{}/{}".format(sites_dir, name)
 158
+
 159
+
 160
+def conf_filename(name):
 161
+    """Return an apache2 config filename path, as:
 162
+      2.4: /etc/apache2/conf-available/foo.conf
 163
+      2.2: /etc/apache2/conf.d/foo
 164
+    """
 165
+    if is_apache24():
 166
+        return "{}/{}.conf".format(default_apache24_config_dir, name)
 167
+    else:
 168
+        return "{}/{}".format(default_apache22_config_dir, name)
 169
+
 170
+
 171
+def conf_enable(name):
 172
+    "Enable apache2 config without reloading service"
 173
+    if is_apache24():
 174
+        return subprocess.call(['/usr/sbin/a2enconf', name]) == 0
 175
+    # no-op otherwise
 176
+    return True
 177
+
 178
+
 179
+def conf_disable(name):
 180
+    "Disable apache2 config without reloading service"
 181
+    if is_apache24():
 182
+        return subprocess.call(['/usr/sbin/a2disconf', name]) == 0
 183
+    # no-op otherwise
 184
+    return True
 185
+
 186
+
 187
+def gen_selfsigned_cert(config, cert_file, key_file):
 188
+    """
 189
+    Create a self-signed certificate.
 190
+
 191
+    @param config: charm data from config-get
 192
+    @param cert_file: destination path of generated certificate
 193
+    @param key_file: destination path of generated private key
 194
+    """
 195
+    os.environ['OPENSSL_CN'] = config['servername']
 196
+    os.environ['OPENSSL_PUBLIC'] = unit_get("public-address")
 197
+    os.environ['OPENSSL_PRIVATE'] = unit_get("private-address")
 198
+    run(
 199
+        ['openssl', 'req', '-new', '-x509', '-nodes',
 200
+         '-days', '3650', '-config',
 201
+         os.path.join(os.environ['CHARM_DIR'], 'data', 'openssl.cnf'),
 202
+         '-keyout', key_file, '-out', cert_file])
 203
+
 204
+
 205
+def is_selfsigned_cert_stale(config, cert_file, key_file):
 206
+    """
 207
+    Do we need to generate a new self-signed cert?
 208
+
 209
+    @param config: charm data from config-get
 210
+    @param cert_file: destination path of generated certificate
 211
+    @param key_file: destination path of generated private key
 212
+    """
 213
+    # Basic Existence Checks
 214
+    if not os.path.exists(cert_file):
 215
+        return True
 216
+    if not os.path.exists(key_file):
 217
+        return True
 218
+
 219
+    # Common Name
 220
+    from OpenSSL import crypto
 221
+    cert = crypto.load_certificate(
 222
+        crypto.FILETYPE_PEM, file(cert_file).read())
 223
+    cn = cert.get_subject().commonName
 224
+    if config['servername'] != cn:
 225
+        return True
 226
+
 227
+    # Subject Alternate Name -- only trusty+ support this
 228
+    try:
 229
+        from pyasn1.codec.der import decoder
 230
+        from pyasn1_modules import rfc2459
 231
+    except ImportError:
 232
+        log("Cannot check subjAltName on <= 12.04, skipping.")
 233
+        return False
 234
+    cert_addresses = set()
 235
+    unit_addresses = set(
 236
+        [unit_get("public-address"), unit_get("private-address")])
 237
+    for i in range(0, cert.get_extension_count()):
 238
+        extension = cert.get_extension(i)
 239
+        try:
 240
+            names = decoder.decode(
 241
+                extension.get_data(), asn1Spec=rfc2459.SubjectAltName())[0]
 242
+            for name in names:
 243
+                cert_addresses.add(str(name.getComponent()))
 244
+        except:
 245
+            pass
 246
+    if cert_addresses != unit_addresses:
 247
+        log("subjAltName: Cert (%s) != Unit (%s), assuming stale" % (
 248
+            cert_addresses, unit_addresses))
 249
+        return True
 250
+
 251
+    return False
 252
+
 253
+
 254
+def _get_key_file_location(config_data):
 255
+    """Look at the config, generate the key file location."""
 256
+    key_file = None
 257
+    if config_data['ssl_keylocation']:
 258
+        key_file = '/etc/ssl/private/%s' % \
 259
+            (config_data['ssl_keylocation'].rpartition('/')[2])
 260
+    return key_file
 261
+
 262
+
 263
+def _get_cert_file_location(config_data):
 264
+    """Look at the config, generate the cert file location."""
 265
+    cert_file = None
 266
+    if config_data['ssl_certlocation']:
 267
+        cert_file = '/etc/ssl/certs/%s' % \
 268
+            (config_data['ssl_certlocation'].rpartition('/')[2])
 269
+    return cert_file
 270
+
 271
+
 272
+def _get_chain_file_location(config_data):
 273
+    """Look at the config, generate the chain file location."""
 274
+    chain_file = None
 275
+    if config_data['ssl_chainlocation']:
 276
+        chain_file = '/etc/ssl/certs/%s' % \
 277
+            (config_data['ssl_chainlocation'].rpartition('/')[2])
 278
+    return chain_file
 279
+
 280
+
 281
+def config_get(scope=None):
 282
+    """
 283
+    Wrapper around charm helper's config_get to replace an empty servername
 284
+    with the public-address.
 285
+    """
 286
+    result = orig_config_get(scope)
 287
+    if scope == "servername" and len(result) == 0:
 288
+        result = unit_get("public-address")
 289
+    elif isinstance(result, dict) and result.get("servername", "") == "":
 290
+        result["servername"] = unit_get("public-address")
 291
+    return result
 292
+
 293
+
 294
+def install_hook():
 295
+    apt_source = config_get('apt-source') or ''
 296
+    apt_key_id = config_get('apt-key-id') or False
 297
+    if apt_source and apt_key_id:
 298
+        print apt_source + " and " + apt_key_id
 299
+        add_source(apt_source, apt_key_id)
 300
+        open('config.apt-source', 'w').write(apt_source)
 301
+    if not os.path.exists(default_apache2_service_config_dir):
 302
+        os.mkdir(default_apache2_service_config_dir, 0600)
 303
+    apt_update(fatal=True)
 304
+    apt_get_install("python-jinja2")
 305
+    apt_get_install("python-pyasn1")
 306
+    apt_get_install("python-pyasn1-modules")
 307
+    apt_get_install("python-yaml")
 308
+    install_status = apt_get_install("apache2")
 309
+    if install_status == 0:
 310
+        ensure_package_status(service_affecting_packages,
 311
+                              config_get('package_status'))
 312
+    ensure_extra_packages()
 313
+    # the apache2 deb does not yet have http2 module in mods-available. Add it.
 314
+    open('/etc/apache2/mods-available/http2.load', 'w').write(
 315
+        'LoadModule http2_module /usr/lib/apache2/modules/mod_http2.so')
 316
+    open('/etc/apache2/mods-available/http2.conf', 'w').write(
 317
+        '''<IfModule http2_module>
 318
+  ProtocolsHonorOrder On
 319
+  Protocols h2 http/1.1
 320
+</IfModule>
 321
+''')
 322
+    return install_status
 323
+
 324
+
 325
+def ensure_extra_packages():
 326
+    extra = str(config_get('extra_packages'))
 327
+    if extra:
 328
+        install_status = apt_get_install(extra)
 329
+        if install_status == 0:
 330
+            ensure_package_status(filter(None, extra.split(' ')),
 331
+                                  config_get('package_status'))
 332
+
 333
+
 334
+def dump_data(data2dump, log_prefix):
 335
+    log_file = '/tmp/pprint-%s.log' % (log_prefix)
 336
+    if data2dump is not None:
 337
+        logFile = open(log_file, 'w')
 338
+        import pprint
 339
+        pprint.pprint(data2dump, logFile)
 340
+        logFile.close()
 341
+
 342
+
 343
+def get_reverseproxy_data(relation='reverseproxy'):
 344
+    relation_data = relations_of_type(relation)
 345
+    reverseproxy_data = {}
 346
+    if relation_data is None or len(relation_data) == 0:
 347
+        return reverseproxy_data
 348
+    for unit_data in relation_data:
 349
+        unit_name = unit_data["__unit__"]
 350
+        if 'port' not in unit_data:
 351
+            return reverseproxy_data
 352
+        # unit_name: <service-name>-<unit_number>
 353
+        # jinja2 templates require python-type variables, remove all characters
 354
+        # that do not comply
 355
+        unit_type = re.sub(r'(.*)/[0-9]*', r'\1', unit_name)
 356
+        unit_type = re.sub('[^a-zA-Z0-9_]*', '', unit_type)
 357
+        log('unit_type: %s' % unit_type)
 358
+
 359
+        host = unit_data['private-address']
 360
+        if unit_type in reverseproxy_data:
 361
+            continue
 362
+        for config_setting in unit_data.keys():
 363
+            if config_setting in ("__unit__", "__relid__"):
 364
+                continue
 365
+            config_key = '%s_%s' % (unit_type,
 366
+                                    config_setting.replace("-", "_"))
 367
+            config_key = re.sub('[^a-zA-Z0-9_]*', '', config_key)
 368
+            reverseproxy_data[config_key] = unit_data[
 369
+                config_setting]
 370
+            reverseproxy_data[unit_type] = '%s:%s' % (
 371
+                host, unit_data['port'])
 372
+        if 'all_services' in unit_data:
 373
+            service_data = yaml.safe_load(unit_data['all_services'])
 374
+            for service_item in service_data:
 375
+                service_name = service_item['service_name']
 376
+                service_port = service_item['service_port']
 377
+                service_key = '%s_%s' % (unit_type, service_name)
 378
+                service_key = re.sub('[^a-zA-Z0-9_]*', '', service_key)
 379
+                reverseproxy_data[service_key] = '%s:%s' % (host, service_port)
 380
+    return reverseproxy_data
 381
+
 382
+
 383
+def update_balancers():
 384
+    relation_data = relations_of_type('balancer')
 385
+    if relation_data is None or len(relation_data) == 0:
 386
+        log("No relation data, exiting.")
 387
+        return
 388
+
 389
+    unit_dict = {}
 390
+    for unit_data in relation_data:
 391
+        unit_name = unit_data["__unit__"]
 392
+        if "port" not in unit_data:
 393
+            log("No port in relation data for '%s', skipping." % unit_name)
 394
+            continue
 395
+        port = unit_data["port"]
 396
+        if "private-address" not in unit_data:
 397
+            log("No private-address in relation data for '%s', skipping." %
 398
+                unit_name)
 399
+            continue
 400
+        host = unit_data['private-address']
 401
+
 402
+        if "all_services" in unit_data:
 403
+            service_data = yaml.safe_load(unit_data[
 404
+                "all_services"])
 405
+            for service_item in service_data:
 406
+                service_port = service_item["service_port"]
 407
+                current_units = unit_dict.setdefault(
 408
+                    service_item["service_name"], [])
 409
+                current_units.append("%s:%s" % (host, service_port))
 410
+        else:
 411
+            if "sitenames" in unit_data:
 412
+                unit_types = unit_data["sitenames"].split()
 413
+            else:
 414
+                unit_types = (re.sub(r"(.*)/[0-9]*", r"\1", unit_name),)
 415
+
 416
+            for unit_type in unit_types:
 417
+                current_units = unit_dict.setdefault(unit_type, [])
 418
+                current_units.append("%s:%s" % (host, port))
 419
+
 420
+    if not unit_dict:
 421
+        return
 422
+
 423
+    write_balancer_config(unit_dict)
 424
+    return unit_dict
 425
+
 426
+
 427
+def write_balancer_config(unit_dict):
 428
+    config_data = config_get()
 429
+
 430
+    from jinja2 import Environment, FileSystemLoader
 431
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 432
+        os.environ['CHARM_DIR'], 'data')))
 433
+    for balancer_name in unit_dict.keys():
 434
+        balancer_host_file = conf_filename('{}.balancer'.format(balancer_name))
 435
+        templ_vars = {
 436
+            'balancer_name': balancer_name,
 437
+            'balancer_addresses': unit_dict[balancer_name],
 438
+            'lb_balancer_timeout': config_data['lb_balancer_timeout'],
 439
+        }
 440
+        template = template_env.get_template(
 441
+            'balancer.template').render(templ_vars)
 442
+        log("Writing file: %s with data: %s" % (balancer_host_file,
 443
+                                                templ_vars))
 444
+        with open(balancer_host_file, 'w') as balancer_config:
 445
+            balancer_config.write(str(template))
 446
+        conf_enable('{}.balancer'.format(balancer_name))
 447
+
 448
+
 449
+def update_nrpe_checks():
 450
+    nrpe_compat = nrpe.NRPE()
 451
+    conf = nrpe_compat.config
 452
+    check_http_params = conf.get('nagios_check_http_params')
 453
+    if check_http_params:
 454
+        nrpe_compat.add_check(
 455
+            shortname='vhost',
 456
+            description='Check Virtual Host',
 457
+            check_cmd='check_http %s' % check_http_params
 458
+        )
 459
+    nrpe_compat.write()
 460
+
 461
+
 462
+def create_mpm_workerfile():
 463
+    config_data = config_get()
 464
+    mpm_workerfile = conf_filename('000mpm-worker')
 465
+    from jinja2 import Environment, FileSystemLoader
 466
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 467
+        os.environ['CHARM_DIR'], 'data')))
 468
+    templ_vars = {
 469
+        'mpm_type': config_data['mpm_type'],
 470
+        'mpm_startservers': config_data['mpm_startservers'],
 471
+        'mpm_minsparethreads': config_data['mpm_minsparethreads'],
 472
+        'mpm_maxsparethreads': config_data['mpm_maxsparethreads'],
 473
+        'mpm_threadlimit': config_data['mpm_threadlimit'],
 474
+        'mpm_threadsperchild': config_data['mpm_threadsperchild'],
 475
+        'mpm_serverlimit': config_data['mpm_serverlimit'],
 476
+        'mpm_maxclients': config_data['mpm_maxclients'],
 477
+        'mpm_maxrequestsperchild': config_data['mpm_maxrequestsperchild'],
 478
+    }
 479
+    template = \
 480
+        template_env.get_template('mpm_worker.template').render(templ_vars)
 481
+    with open(mpm_workerfile, 'w') as mpm_config:
 482
+        mpm_config.write(str(template))
 483
+    conf_enable('000mpm-worker')
 484
+
 485
+
 486
+def create_security():
 487
+    config_data = config_get()
 488
+    securityfile = conf_filename('security')
 489
+    from jinja2 import Environment, FileSystemLoader
 490
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 491
+        os.environ['CHARM_DIR'], 'data')))
 492
+    templ_vars = {
 493
+        'juju_warning_header': juju_warning_header,
 494
+        'server_tokens': config_data['server_tokens'],
 495
+        'server_signature': config_data['server_signature'],
 496
+        'trace_enabled': config_data['trace_enabled'],
 497
+        'ssl_protocol': config_data['ssl_protocol'],
 498
+        'ssl_honor_cipher_order': config_data['ssl_honor_cipher_order'],
 499
+        'ssl_cipher_suite': config_data['ssl_cipher_suite'],
 500
+        'is_apache24': is_apache24(),
 501
+    }
 502
+    template = \
 503
+        template_env.get_template('security.template').render(templ_vars)
 504
+    with open(securityfile, 'w') as security_config:
 505
+        security_config.write(str(template))
 506
+    conf_enable('security')
 507
+
 508
+
 509
+def ship_logrotate_conf():
 510
+    config_data = config_get()
 511
+    logrotate_file = '/etc/logrotate.d/apache2'
 512
+    from jinja2 import Environment, FileSystemLoader
 513
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 514
+        os.environ['CHARM_DIR'], 'data')))
 515
+    templ_vars = {
 516
+        'juju_warning_header': juju_warning_header,
 517
+        'logrotate_rotate': config_data['logrotate_rotate'],
 518
+        'logrotate_count': config_data['logrotate_count'],
 519
+        'logrotate_dateext': config_data['logrotate_dateext'],
 520
+    }
 521
+    template = template_env.get_template('logrotate.conf.template').render(
 522
+        templ_vars)
 523
+    with open(logrotate_file, 'w') as logrotate_conf:
 524
+        logrotate_conf.write(str(template))
 525
+
 526
+
 527
+def create_vhost(port, protocol=None, config_key=None, template_str=None,
 528
+                 config_data={}, relationship_data={}):
 529
+    """
 530
+    Create and enable a vhost in apache.
 531
+
 532
+    @param port: port on which to listen (int)
 533
+    @param protocol: used to name the vhost file intelligently.  If not
 534
+        specified the port will be used instead. (ex: http, https)
 535
+    @param config_key: key in the configuration to look up to
 536
+        retrieve the template.
 537
+    @param template_str: The template itself as a string.
 538
+    @param config_data: juju get-config configuration data.
 539
+    @param relationship_data: if in a relationship, pass in the appropriate
 540
+        structure.  This will be used to inform the template.
 541
+    """
 542
+    if protocol is None:
 543
+        protocol = str(port)
 544
+    if template_str is None:
 545
+        if not config_key or not config_data[config_key]:
 546
+            log("Vhost Template not provided, not configuring: %s" % port)
 547
+            return False
 548
+        template_str = config_data[config_key]
 549
+    from jinja2 import Template
 550
+    template = Template(str(base64.b64decode(template_str)))
 551
+    template_data = dict(config_data.items() + relationship_data.items())
 552
+    if config_data.get('vhost_template_vars'):
 553
+        extra_vars = ast.literal_eval(config_data['vhost_template_vars'])
 554
+        template_data.update(extra_vars)
 555
+    vhost_name = '%s_%s' % (config_data['servername'], protocol)
 556
+    vhost_file = site_filename(vhost_name)
 557
+    log("Writing file %s with config and relation data" % vhost_file)
 558
+    with open(vhost_file, 'w') as vhost:
 559
+        vhost.write(str(template.render(template_data)))
 560
+    subprocess.call(['/usr/sbin/a2ensite', vhost_name])
 561
+    return True
 562
+
 563
+
 564
+MPM_TYPES = ['mpm_worker', 'mpm_prefork', 'mpm_event']
 565
+
 566
+
 567
+def enable_mpm(config):
 568
+    """Enables a particular mpm module.
 569
+
 570
+    Different from simply enabling a module, as one and only one mpm module
 571
+    *must* be enabled.
 572
+    """
 573
+    # only do anything if value has changed, to avoid a needless restart
 574
+    if not config.changed('mpm_type'):
 575
+        return
 576
+
 577
+    mpm_type = config.get('mpm_type', '')
 578
+    name = 'mpm_' + mpm_type
 579
+    if name not in MPM_TYPES:
 580
+        log('bad mpm_type: %s. Falling back to mpm_worker' % mpm_type)
 581
+        name = 'mpm_worker'
 582
+
 583
+    # disable all other mpm modules
 584
+    for mpm in MPM_TYPES:
 585
+        if mpm != name:
 586
+            return_value = subprocess.call(['/usr/sbin/a2dismod', mpm])
 587
+            if return_value != 0:
 588
+                return False
 589
+
 590
+    return_value = subprocess.call(['/usr/sbin/a2enmod', name])
 591
+    if return_value != 0:
 592
+        return False
 593
+
 594
+    if service_apache2("check"):
 595
+        log("Switching mpm module to {}".format(name))
 596
+        service_apache2("restart")  # must be a restart to switch mpm
 597
+        return True
 598
+    else:
 599
+        log("Failed to switch mpm module to {}".format(name))
 600
+        return False
 601
+
 602
+
 603
+def config_changed():
 604
+    relationship_data = {}
 605
+    config_data = config_get()
 606
+
 607
+    apt_source = config_data['apt-source']
 608
+    old_apt_source = ''
 609
+    try:
 610
+        old_apt_source = open('config.apt-source', 'r').read()
 611
+    except IOError:
 612
+        pass
 613
+    if old_apt_source != apt_source:
 614
+        subprocess.check_call(['add-apt-repository', '--yes', '-r',
 615
+                               old_apt_source])
 616
+        add_source(apt_source, config_data['apt-key-id'])
 617
+        open('config.apt-source', 'w').write(apt_source)
 618
+
 619
+    ensure_package_status(service_affecting_packages,
 620
+                          config_data['package_status'])
 621
+    ensure_extra_packages()
 622
+
 623
+    relationship_data.update(get_reverseproxy_data(relation='reverseproxy'))
 624
+    relationship_data.update(get_reverseproxy_data(relation='website-cache'))
 625
+    if update_balancers():
 626
+        # apache 2.4 has lbmethods split, needs to enable specific module(s)
 627
+        if is_apache24():
 628
+            enable_module('lbmethod_byrequests')
 629
+
 630
+    disabled_modules = config_data['disable_modules'].split()
 631
+    apache_websites = ApacheWebsites.from_config(
 632
+        relations_of_type("apache-website"), disabled_modules)
 633
+    enabled_modules = config_data.get('enable_modules', '').split()
 634
+    enabled_modules = apache_websites.list_enabled_modules(enabled_modules)
 635
+    for module in enabled_modules:
 636
+        enable_module(module)
 637
+
 638
+    if config_data['disable_modules']:
 639
+        for module in disabled_modules:
 640
+            disable_module(module)
 641
+
 642
+    apache_websites.disable_sites()
 643
+    apache_websites.write_configs()
 644
+    apache_websites.enable_sites()
 645
+    apache_websites.configure_extra_ports()
 646
+    all_ports = apache_websites.list_enabled_ports()
 647
+    enable_mpm(config_data)
 648
+    # XXX we only configure the worker mpm?
 649
+    create_mpm_workerfile()
 650
+    create_security()
 651
+
 652
+    ports = {'http': 80, 'https': 443}
 653
+    for protocol, port in ports.iteritems():
 654
+        if create_vhost(
 655
+                port,
 656
+                protocol=protocol,
 657
+                config_key="vhost_%s_template" % protocol,
 658
+                config_data=config_data,
 659
+                relationship_data=relationship_data):
 660
+            all_ports.add(port)
 661
+
 662
+    cert_file = _get_cert_file_location(config_data)
 663
+    key_file = _get_key_file_location(config_data)
 664
+    chain_file = _get_chain_file_location(config_data)
 665
+
 666
+    if cert_file is not None and key_file is not None:
 667
+        # ssl_cert is SELFSIGNED so generate self-signed certificate for use.
 668
+        if config_data['ssl_cert'] and config_data['ssl_cert'] == "SELFSIGNED":
 669
+            if is_selfsigned_cert_stale(config_data, cert_file, key_file):
 670
+                gen_selfsigned_cert(config_data, cert_file, key_file)
 671
+
 672
+        # Use SSL certificate and key provided either as a base64 string or
 673
+        # shipped out with the charm.
 674
+        else:
 675
+            # Certificate provided as base64-encoded string.
 676
+            if config_data['ssl_cert']:
 677
+                log("Writing cert from config ssl_cert: %s" % cert_file)
 678
+                with open(cert_file, 'w') as f:
 679
+                    f.write(str(base64.b64decode(config_data['ssl_cert'])))
 680
+            # Use certificate file shipped out with charm.
 681
+            else:
 682
+                source = os.path.join(os.environ['CHARM_DIR'], 'data',
 683
+                                      config_data['ssl_certlocation'])
 684
+                if os.path.exists(source):
 685
+                    shutil.copy(source, cert_file)
 686
+                else:
 687
+                    log("Certificate not found, ignoring: %s" % source)
 688
+
 689
+            # Private key provided as base64-encoded string.
 690
+            if config_data['ssl_key']:
 691
+                log("Writing key from config ssl_key: %s" % key_file)
 692
+                with open(key_file, 'w') as f:
 693
+                    f.write(str(base64.b64decode(config_data['ssl_key'])))
 694
+            # Use private key shipped out with charm.
 695
+            else:
 696
+                source = os.path.join(os.environ['CHARM_DIR'], 'data',
 697
+                                      config_data['ssl_keylocation'])
 698
+                if os.path.exists(source):
 699
+                    shutil.copy(source, key_file)
 700
+                else:
 701
+                    log("Key file not found, ignoring: %s" % source)
 702
+
 703
+            if chain_file is not None:
 704
+                # Chain certificates provided as base64-encoded string.
 705
+                if config_data['ssl_chain']:
 706
+                    log("Writing chain certificates file from"
 707
+                        "config ssl_chain: %s" % chain_file)
 708
+                    with open(chain_file, 'w') as f:
 709
+                        f.write(str(base64.b64decode(
 710
+                            config_data['ssl_chain'])))
 711
+                # Use chain certificates shipped out with charm.
 712
+                else:
 713
+                    source = os.path.join(os.environ['CHARM_DIR'], 'data',
 714
+                                          config_data['ssl_chainlocation'])
 715
+                    if os.path.exists(source):
 716
+                        shutil.copy(source, chain_file)
 717
+                    else:
 718
+                        log("Chain certificates not found, "
 719
+                            "ignoring: %s" % source)
 720
+
 721
+        # Tighten permissions on private key file.
 722
+        if os.path.exists(key_file):
 723
+            os.chmod(key_file, 0440)
 724
+            os.chown(key_file, pwd.getpwnam('root').pw_uid,
 725
+                     grp.getgrnam('ssl-cert').gr_gid)
 726
+
 727
+    apache_syslog_conf = conf_filename("syslog")
 728
+    rsyslog_apache_conf = "/etc/rsyslog.d/45-apache2.conf"
 729
+    if config_data['use_rsyslog']:
 730
+        shutil.copy2("data/syslog-apache.conf", apache_syslog_conf)
 731
+        conf_enable("syslog")
 732
+        shutil.copy2("data/syslog-rsyslog.conf", rsyslog_apache_conf)
 733
+        # Fix permissions of access.log and error.log to allow syslog user to
 734
+        # write to
 735
+        os.chown("/var/log/apache2/access.log", pwd.getpwnam('syslog').pw_uid,
 736
+                 pwd.getpwnam('syslog').pw_gid)
 737
+        os.chown("/var/log/apache2/error.log", pwd.getpwnam('syslog').pw_uid,
 738
+                 pwd.getpwnam('syslog').pw_gid)
 739
+    else:
 740
+        conf_disable("syslog")
 741
+        if os.path.exists(apache_syslog_conf):
 742
+            os.unlink(apache_syslog_conf)
 743
+        if os.path.exists(rsyslog_apache_conf):
 744
+            os.unlink(rsyslog_apache_conf)
 745
+    run(["/usr/sbin/service", "rsyslog", "restart"])
 746
+
 747
+    # Disable the default website because we don't want people to see the
 748
+    # "It works!" page on production services and remove the
 749
+    # conf.d/other-vhosts-access-log conf.
 750
+    ensure_disabled(["000-default"])
 751
+    conf_disable("other-vhosts-access-log")
 752
+    if os.path.exists(conf_filename("other-vhosts-access-log")):
 753
+        os.unlink(conf_filename("other-vhosts-access-log"))
 754
+
 755
+    if service_apache2("check"):
 756
+        if config_data["config_change_command"] in ["reload", "restart"]:
 757
+            service_apache2(config_data["config_change_command"])
 758
+
 759
+    if config_data['openid_provider']:
 760
+        if not os.path.exists('/etc/apache2/security'):
 761
+            os.mkdir('/etc/apache2/security', 0755)
 762
+        with open('/etc/apache2/security/allowed-ops.txt', 'w') as f:
 763
+            f.write(config_data['openid_provider'].replace(',', '\n'))
 764
+            f.write('\n')
 765
+            os.chmod(key_file, 0444)
 766
+
 767
+    all_ports.update(update_vhost_config_relation())
 768
+    ensure_ports(all_ports)
 769
+    update_nrpe_checks()
 770
+    ship_logrotate_conf()
 771
+    if config_get().changed('servername'):
 772
+        logs_relation_joined()
 773
+
 774
+
 775
+def ensure_disabled(sites):
 776
+    to_disable = [s for s in sites if os.path.exists(site_filename(s, True))]
 777
+    if len(to_disable) == 0:
 778
+        return
 779
+    run(["/usr/sbin/a2dissite"] + to_disable)
 780
+
 781
+
 782
+def ensure_removed(filename):
 783
+    try:
 784
+        os.unlink(filename)
 785
+    except OSError as e:
 786
+        if e.errno != errno.ENOENT:
 787
+            raise
 788
+
 789
+
 790
+class ApacheWebsites:
 791
+
 792
+    @classmethod
 793
+    def from_config(cls, relations, disabled_modules):
 794
+        """Return an ApacheWebsites with information about all sites."""
 795
+        if relations is None:
 796
+            relations = []
 797
+        self_relations = {}
 798
+        for relation in relations:
 799
+            self_relation = {'domain': relation.get('domain')}
 800
+            enabled = bool(relation.get('enabled', 'False').lower() == 'true')
 801
+            site_modules = relation.get('site_modules', '').split()
 802
+            for module in site_modules:
 803
+                if module in disabled_modules:
 804
+                    enabled = False
 805
+                    log('site {} requires disabled_module {}'.format(
 806
+                        relation['__relid__'], module))
 807
+                break
 808
+            self_relation['site_modules'] = site_modules
 809
+            self_relation['enabled'] = enabled
 810
+            self_relation['site_config'] = relation.get('site_config')
 811
+            self_relation['ports'] = [
 812
+                int(p) for p in relation.get('ports', '').split()]
 813
+            self_relations[relation['__relid__']] = self_relation
 814
+        return cls(self_relations)
 815
+
 816
+    def __init__(self, relations):
 817
+        self.relations = relations
 818
+
 819
+    def write_configs(self):
 820
+        for key, relation in self.relations.items():
 821
+            config_file = site_filename(key)
 822
+            site_config = relation['site_config']
 823
+            if site_config is None:
 824
+                ensure_removed(config_file)
 825
+            else:
 826
+                with open(config_file, 'w') as output:
 827
+                    output.write(site_config)
 828
+
 829
+    def iter_enabled_sites(self):
 830
+        return ((k, v) for k, v in self.relations.items() if v['enabled'])
 831
+
 832
+    def enable_sites(self):
 833
+        enabled_sites = [k for k, v in self.iter_enabled_sites()]
 834
+        enabled_sites.sort()
 835
+        if len(enabled_sites) == 0:
 836
+            return
 837
+        subprocess.check_call(['/usr/sbin/a2ensite'] + enabled_sites)
 838
+
 839
+    def disable_sites(self):
 840
+        disabled_sites = [k for k, v in self.relations.items()
 841
+                          if not v['enabled']]
 842
+        disabled_sites.sort()
 843
+        if len(disabled_sites) == 0:
 844
+            return
 845
+        ensure_disabled(disabled_sites)
 846
+
 847
+    def list_enabled_modules(self, enabled_modules):
 848
+        enabled_modules = set(enabled_modules)
 849
+        for key, relation in self.iter_enabled_sites():
 850
+            enabled_modules.update(relation['site_modules'])
 851
+        return enabled_modules
 852
+
 853
+    def list_enabled_ports(self):
 854
+        enabled_ports = set()
 855
+        for key, relation in self.iter_enabled_sites():
 856
+            enabled_ports.update(relation['ports'])
 857
+        return enabled_ports
 858
+
 859
+    def configure_extra_ports(self):
 860
+        extra_ports = self.list_enabled_ports()
 861
+        extra_ports.discard(80)
 862
+        extra_ports.discard(443)
 863
+        extra_ports_conf = conf_filename('extra_ports')
 864
+        if len(extra_ports) > 0:
 865
+            with file(extra_ports_conf, 'w') as f:
 866
+                for port in sorted(extra_ports):
 867
+                    f.write('Listen {}\n'.format(port))
 868
+            conf_enable('extra_ports')
 869
+        else:
 870
+            conf_disable('extra_ports')
 871
+            ensure_removed(extra_ports_conf)
 872
+
 873
+
 874
+def update_vhost_config_relation():
 875
+    """
 876
+    Update the vhost file and include the certificate in the relation
 877
+    if it is self-signed.
 878
+    """
 879
+    vhost_ports = set()
 880
+    relation_data = relations_of_type("vhost-config")
 881
+    config_data = config_get()
 882
+    if relation_data is None:
 883
+        return vhost_ports
 884
+
 885
+    for unit_data in relation_data:
 886
+        if "vhosts" in unit_data:
 887
+            all_relation_data = {}
 888
+            all_relation_data.update(
 889
+                get_reverseproxy_data(relation='reverseproxy'))
 890
+            all_relation_data.update(
 891
+                get_reverseproxy_data(relation='website-cache'))
 892
+            try:
 893
+                vhosts = yaml.safe_load(unit_data["vhosts"])
 894
+                for vhost in vhosts:
 895
+                    port = vhost["port"]
 896
+                    if create_vhost(
 897
+                            port,
 898
+                            template_str=vhost["template"],
 899
+                            config_data=config_data,
 900
+                            relationship_data=all_relation_data):
 901
+                        vhost_ports.add(port)
 902
+            except Exception as e:
 903
+                log("Error reading configuration data from relation! %s" % e)
 904
+                raise
 905
+
 906
+    if service_apache2("check"):
 907
+        service_apache2("reload")
 908
+
 909
+    vhost_relation_settings = {
 910
+        "servername": config_data["servername"]}
 911
+
 912
+    cert_file = _get_cert_file_location(config_data)
 913
+    key_file = _get_key_file_location(config_data)
 914
+
 915
+    if cert_file is not None and key_file is not None:
 916
+        if config_data['ssl_cert'] and config_data['ssl_cert'] == "SELFSIGNED":
 917
+            with open(cert_file, 'r') as f:
 918
+                cert = base64.b64encode(f.read())
 919
+            vhost_relation_settings["ssl_cert"] = cert
 920
+    for id in relation_ids("vhost-config"):
 921
+        relation_set(relation_id=id, relation_settings=vhost_relation_settings)
 922
+    return vhost_ports
 923
+
 924
+
 925
+def start_hook():
 926
+    if service_apache2("status"):
 927
+        return(service_apache2("restart"))
 928
+    else:
 929
+        return(service_apache2("start"))
 930
+
 931
+
 932
+def stop_hook():
 933
+    if service_apache2("status"):
 934
+        return(service_apache2("stop"))
 935
+
 936
+
 937
+def reverseproxy_interface(hook_name=None):
 938
+    if hook_name is None:
 939
+        return(None)
 940
+    if hook_name == "changed":
 941
+        config_changed()
 942
+
 943
+
 944
+def website_interface(hook_name=None):
 945
+    if hook_name is None:
 946
+        return(None)
 947
+    my_host = socket.getfqdn(socket.gethostname())
 948
+    if my_host == "localhost":
 949
+        my_host = socket.gethostname()
 950
+    default_port = 80
 951
+    subprocess.call([
 952
+        'relation-set',
 953
+        'port=%d' % default_port,
 954
+        'hostname=%s' % my_host,
 955
+        'servername=%s' % config_get('servername')
 956
+    ])
 957
+
 958
+
 959
+def ensure_ports(ports):
 960
+    """Ensure that only the desired ports are open."""
 961
+    open_ports = set(get_open_ports())
 962
+    ports = set(ports)
 963
+    wanted_closed = ports.difference(open_ports)
 964
+    for port in sorted(wanted_closed):
 965
+        open_port(port)
 966
+    unwanted_open = open_ports.difference(ports)
 967
+    for port in sorted(unwanted_open):
 968
+        close_port(port)
 969
+    set_open_ports(list(sorted(ports)))
 970
+
 971
+
 972
+def get_open_ports():
 973
+    """Get the list of open ports from the standard file."""
 974
+    try:
 975
+        pfile = open(os.path.join(os.environ['CHARM_DIR'], 'ports.yaml'))
 976
+    except IOError as e:
 977
+        if e.errno == errno.ENOENT:
 978
+            return []
 979
+        else:
 980
+            raise
 981
+    with pfile:
 982
+        return yaml.safe_load(pfile)
 983
+
 984
+
 985
+def set_open_ports(ports):
 986
+    """Write the list of open ports to the standard file."""
 987
+    ports_path = os.path.join(os.environ['CHARM_DIR'], 'ports.yaml')
 988
+    with open(ports_path, 'w') as pfile:
 989
+        yaml.safe_dump(ports, pfile)
 990
+
 991
+
 992
+def get_log_files():
 993
+    """
 994
+    Read all of the apache config files from __ and get ErrorLog and AccessLog
 995
+    values.
 996
+
 997
+    Returns a tuple with first value list of access log files and second value
 998
+    list of error log files.
 999
+    """
1000
+    access_logs = []
1001
+    error_logs = []
1002
+    for protocol in ['http', 'https']:
1003
+        vhost_name = '%s_%s' % (config_get()['servername'], protocol)
1004
+        vhost_file = site_filename(vhost_name)
1005
+        try:
1006
+            # Using read().split('\n') here to work around a mocks open_mock
1007
+            # inadequacy: http://bugs.python.org/issue17467
1008
+            for line in open(vhost_file, 'r').read().split('\n'):
1009
+                if 'CustomLog' in line:
1010
+                    access_logs.append(line.split()[1])
1011
+                elif 'ErrorLog' in line:
1012
+                    error_logs.append(line.split()[1])
1013
+        except:
1014
+            pass
1015
+    return access_logs, error_logs
1016
+
1017
+
1018
+def logs_relation_joined():
1019
+    """
1020
+    Sets relation value with filenames
1021
+    """
1022
+    access_log_files, error_log_files = get_log_files()
1023
+    log_files = access_log_files[:]
1024
+    log_files.extend(error_log_files)
1025
+    types = ['apache_access' for a in access_log_files]
1026
+    types.extend(['apache_error' for a in error_log_files])
1027
+    data = {'files': '\n'.join(log_files),
1028
+            'types': '\n'.join(types),
1029
+            }
1030
+    _relation_ids = relation_ids('logs')
1031
+    for _relation_id in _relation_ids:
1032
+        log("logs-relation-joined setting relation data for {} to {}".format(
1033
+            _relation_id, data))
1034
+        relation_set(
1035
+            relation_id=_relation_id,
1036
+            relation_settings=data)
1037
+
1038
+
1039
+###############################################################################
1040
+# Main section
1041
+###############################################################################
1042
+def main(hook_name):
1043
+    if hook_name == "install":
1044
+        install_hook()
1045
+    elif hook_name == "config-changed" or hook_name == "upgrade-charm":
1046
+        config_changed()
1047
+    elif hook_name == "start":
1048
+        start_hook()
1049
+    elif hook_name == "stop":
1050
+        stop_hook()
1051
+    elif hook_name == "reverseproxy-relation-broken":
1052
+        config_changed()
1053
+    elif hook_name == "reverseproxy-relation-changed":
1054
+        config_changed()
1055
+    elif hook_name == "reverseproxy-relation-joined":
1056
+        config_changed()
1057
+    elif hook_name == "balancer-relation-broken":
1058
+        config_changed()
1059
+    elif hook_name == "balancer-relation-changed":
1060
+        config_changed()
1061
+    elif hook_name == "balancer-relation-joined":
1062
+        config_changed()
1063
+    elif hook_name == "website-cache-relation-broken":
1064
+        config_changed()
1065
+    elif hook_name == "website-cache-relation-changed":
1066
+        config_changed()
1067
+    elif hook_name == "website-cache-relation-joined":
1068
+        config_changed()
1069
+    elif hook_name == "website-relation-joined":
1070
+        website_interface("joined")
1071
+    elif hook_name == 'apache-website-relation-changed':
1072
+        config_changed()
1073
+    elif hook_name in ("nrpe-external-master-relation-changed",
1074
+                       "local-monitors-relation-changed"):
1075
+        update_nrpe_checks()
1076
+    elif hook_name == "vhost-config-relation-changed":
1077
+        config_changed()
1078
+    elif hook_name == "logs-relation-joined":
1079
+        logs_relation_joined()
1080
+    else:
1081
+        print "Unknown hook"
1082
+        sys.exit(1)
1083
+
1084
+if __name__ == "__main__":
1085
+    hook_name = os.path.basename(sys.argv[0])
1086
+    # Also support being invoked directly with hook as argument name.
1087
+    if hook_name == "hooks.py":
1088
+        if len(sys.argv) < 2:
1089
+            sys.exit("Missing required hook name argument.")
1090
+        hook_name = sys.argv[1]
1091
+    main(hook_name)
Back to file index

hooks/balancer-relation-joined

   1
--- 
   2
+++ hooks/balancer-relation-joined
   3
@@ -0,0 +1,1088 @@
   4
+#!/usr/bin/env python
   5
+
   6
+import errno
   7
+import os
   8
+import re
   9
+import socket
  10
+import subprocess
  11
+import sys
  12
+import yaml
  13
+import base64
  14
+import grp
  15
+import pwd
  16
+import shutil
  17
+import os.path
  18
+import ast
  19
+
  20
+from charmhelpers.core.hookenv import (
  21
+    open_port,
  22
+    close_port,
  23
+    log,
  24
+    config as orig_config_get,
  25
+    relations_of_type,
  26
+    relation_set,
  27
+    relation_ids,
  28
+    unit_get
  29
+)
  30
+from charmhelpers.contrib.charmsupport import nrpe
  31
+from charmhelpers.fetch import apt_update, add_source
  32
+
  33
+###############################################################################
  34
+# Global variables
  35
+###############################################################################
  36
+default_apache2_service_config_dir = "/var/run/apache2"
  37
+service_affecting_packages = ['apache2']
  38
+default_apache22_config_dir = "/etc/apache2/conf.d"
  39
+default_apache24_config_dir = "/etc/apache2/conf-available"
  40
+default_apache_base_dir = "/etc/apache2"
  41
+
  42
+juju_warning_header = """#
  43
+#    "             "
  44
+#  mmm   m   m   mmm   m   m
  45
+#    #   #   #     #   #   #
  46
+#    #   #   #     #   #   #
  47
+#    #   "mm"#     #   "mm"#
  48
+#    #             #
  49
+#  ""            ""
  50
+# This file is managed by Juju. Do not make local changes.
  51
+#"""
  52
+
  53
+
  54
+###############################################################################
  55
+# Supporting functions
  56
+###############################################################################
  57
+
  58
+def apt_get_install(package=None):
  59
+    """Install a package."""
  60
+    if package is None:
  61
+        return False
  62
+    cmd_line = ['apt-get', '-y', 'install', '-qq']
  63
+    cmd_line.append(package)
  64
+    return subprocess.call(cmd_line)
  65
+
  66
+
  67
+def ensure_package_status(packages, status):
  68
+    if status in ['install', 'hold']:
  69
+        selections = ''.join(['{} {}\n'.format(package, status)
  70
+                              for package in packages])
  71
+        dpkg = subprocess.Popen(['dpkg', '--set-selections'],
  72
+                                stdin=subprocess.PIPE)
  73
+        dpkg.communicate(input=selections)
  74
+
  75
+
  76
+# -----------------------------------------------------------------------------
  77
+# apt_get_purge( package ):  Purges a package
  78
+# -----------------------------------------------------------------------------
  79
+def apt_get_purge(packages=None):
  80
+    if packages is None:
  81
+        return False
  82
+    cmd_line = ['apt-get', '-y', 'purge', '-qq']
  83
+    cmd_line.append(packages)
  84
+    return subprocess.call(cmd_line)
  85
+
  86
+
  87
+# -----------------------------------------------------------------------------
  88
+# service_apache2:  Convenience function to start/stop/restart/reload
  89
+#                   the apache2 service
  90
+# -----------------------------------------------------------------------------
  91
+def service_apache2(action=None):
  92
+    if action is None:
  93
+        return
  94
+    elif action == "check":
  95
+        args = ['/usr/sbin/apache2ctl', 'configtest']
  96
+    else:
  97
+        args = ['service', 'apache2', action]
  98
+    ret_val = subprocess.call(args)
  99
+    return ret_val == 0
 100
+
 101
+
 102
+def run(command, *args, **kwargs):
 103
+    try:
 104
+        output = subprocess.check_output(command, *args, **kwargs)
 105
+        return output
 106
+    except Exception, e:
 107
+        print e
 108
+        raise
 109
+
 110
+
 111
+def enable_module(module=None):
 112
+    if module is None:
 113
+        return True
 114
+    if os.path.exists("/etc/apache2/mods-enabled/%s.load" % (module)):
 115
+        log("Module already loaded: %s" % module)
 116
+        return True
 117
+    if not os.path.exists("/etc/apache2/mods-available/%s.load" % (module)):
 118
+        return_value = apt_get_install("libapache2-mod-%s" % (module))
 119
+        if return_value != 0:
 120
+            log("Installing module %s failed" % (module))
 121
+            return False
 122
+    return_value = subprocess.call(['/usr/sbin/a2enmod', module])
 123
+    if return_value != 0:
 124
+        return False
 125
+    if service_apache2("check"):
 126
+        service_apache2("reload")
 127
+        return True
 128
+
 129
+
 130
+def disable_module(module=None):
 131
+    if module is None:
 132
+        return True
 133
+    if not os.path.exists("/etc/apache2/mods-enabled/%s.load" % (module)):
 134
+        log("Module already disabled: %s" % module)
 135
+        return True
 136
+    return_value = subprocess.call(['/usr/sbin/a2dismod', module])
 137
+    if return_value != 0:
 138
+        return False
 139
+    if service_apache2("check"):
 140
+        service_apache2("reload")
 141
+        return True
 142
+
 143
+
 144
+def is_apache24():
 145
+    return os.path.exists("/usr/sbin/a2enconf")
 146
+
 147
+
 148
+def site_filename(name, enabled=False):
 149
+    if enabled:
 150
+        sites_dir = "%s/sites-enabled" % default_apache_base_dir
 151
+    else:
 152
+        sites_dir = "%s/sites-available" % default_apache_base_dir
 153
+
 154
+    if is_apache24():
 155
+        return "{}/{}.conf".format(sites_dir, name)
 156
+    else:
 157
+        return "{}/{}".format(sites_dir, name)
 158
+
 159
+
 160
+def conf_filename(name):
 161
+    """Return an apache2 config filename path, as:
 162
+      2.4: /etc/apache2/conf-available/foo.conf
 163
+      2.2: /etc/apache2/conf.d/foo
 164
+    """
 165
+    if is_apache24():
 166
+        return "{}/{}.conf".format(default_apache24_config_dir, name)
 167
+    else:
 168
+        return "{}/{}".format(default_apache22_config_dir, name)
 169
+
 170
+
 171
+def conf_enable(name):
 172
+    "Enable apache2 config without reloading service"
 173
+    if is_apache24():
 174
+        return subprocess.call(['/usr/sbin/a2enconf', name]) == 0
 175
+    # no-op otherwise
 176
+    return True
 177
+
 178
+
 179
+def conf_disable(name):
 180
+    "Disable apache2 config without reloading service"
 181
+    if is_apache24():
 182
+        return subprocess.call(['/usr/sbin/a2disconf', name]) == 0
 183
+    # no-op otherwise
 184
+    return True
 185
+
 186
+
 187
+def gen_selfsigned_cert(config, cert_file, key_file):
 188
+    """
 189
+    Create a self-signed certificate.
 190
+
 191
+    @param config: charm data from config-get
 192
+    @param cert_file: destination path of generated certificate
 193
+    @param key_file: destination path of generated private key
 194
+    """
 195
+    os.environ['OPENSSL_CN'] = config['servername']
 196
+    os.environ['OPENSSL_PUBLIC'] = unit_get("public-address")
 197
+    os.environ['OPENSSL_PRIVATE'] = unit_get("private-address")
 198
+    run(
 199
+        ['openssl', 'req', '-new', '-x509', '-nodes',
 200
+         '-days', '3650', '-config',
 201
+         os.path.join(os.environ['CHARM_DIR'], 'data', 'openssl.cnf'),
 202
+         '-keyout', key_file, '-out', cert_file])
 203
+
 204
+
 205
+def is_selfsigned_cert_stale(config, cert_file, key_file):
 206
+    """
 207
+    Do we need to generate a new self-signed cert?
 208
+
 209
+    @param config: charm data from config-get
 210
+    @param cert_file: destination path of generated certificate
 211
+    @param key_file: destination path of generated private key
 212
+    """
 213
+    # Basic Existence Checks
 214
+    if not os.path.exists(cert_file):
 215
+        return True
 216
+    if not os.path.exists(key_file):
 217
+        return True
 218
+
 219
+    # Common Name
 220
+    from OpenSSL import crypto
 221
+    cert = crypto.load_certificate(
 222
+        crypto.FILETYPE_PEM, file(cert_file).read())
 223
+    cn = cert.get_subject().commonName
 224
+    if config['servername'] != cn:
 225
+        return True
 226
+
 227
+    # Subject Alternate Name -- only trusty+ support this
 228
+    try:
 229
+        from pyasn1.codec.der import decoder
 230
+        from pyasn1_modules import rfc2459
 231
+    except ImportError:
 232
+        log("Cannot check subjAltName on <= 12.04, skipping.")
 233
+        return False
 234
+    cert_addresses = set()
 235
+    unit_addresses = set(
 236
+        [unit_get("public-address"), unit_get("private-address")])
 237
+    for i in range(0, cert.get_extension_count()):
 238
+        extension = cert.get_extension(i)
 239
+        try:
 240
+            names = decoder.decode(
 241
+                extension.get_data(), asn1Spec=rfc2459.SubjectAltName())[0]
 242
+            for name in names:
 243
+                cert_addresses.add(str(name.getComponent()))
 244
+        except:
 245
+            pass
 246
+    if cert_addresses != unit_addresses:
 247
+        log("subjAltName: Cert (%s) != Unit (%s), assuming stale" % (
 248
+            cert_addresses, unit_addresses))
 249
+        return True
 250
+
 251
+    return False
 252
+
 253
+
 254
+def _get_key_file_location(config_data):
 255
+    """Look at the config, generate the key file location."""
 256
+    key_file = None
 257
+    if config_data['ssl_keylocation']:
 258
+        key_file = '/etc/ssl/private/%s' % \
 259
+            (config_data['ssl_keylocation'].rpartition('/')[2])
 260
+    return key_file
 261
+
 262
+
 263
+def _get_cert_file_location(config_data):
 264
+    """Look at the config, generate the cert file location."""
 265
+    cert_file = None
 266
+    if config_data['ssl_certlocation']:
 267
+        cert_file = '/etc/ssl/certs/%s' % \
 268
+            (config_data['ssl_certlocation'].rpartition('/')[2])
 269
+    return cert_file
 270
+
 271
+
 272
+def _get_chain_file_location(config_data):
 273
+    """Look at the config, generate the chain file location."""
 274
+    chain_file = None
 275
+    if config_data['ssl_chainlocation']:
 276
+        chain_file = '/etc/ssl/certs/%s' % \
 277
+            (config_data['ssl_chainlocation'].rpartition('/')[2])
 278
+    return chain_file
 279
+
 280
+
 281
+def config_get(scope=None):
 282
+    """
 283
+    Wrapper around charm helper's config_get to replace an empty servername
 284
+    with the public-address.
 285
+    """
 286
+    result = orig_config_get(scope)
 287
+    if scope == "servername" and len(result) == 0:
 288
+        result = unit_get("public-address")
 289
+    elif isinstance(result, dict) and result.get("servername", "") == "":
 290
+        result["servername"] = unit_get("public-address")
 291
+    return result
 292
+
 293
+
 294
+def install_hook():
 295
+    apt_source = config_get('apt-source') or ''
 296
+    apt_key_id = config_get('apt-key-id') or False
 297
+    if apt_source and apt_key_id:
 298
+        print apt_source + " and " + apt_key_id
 299
+        add_source(apt_source, apt_key_id)
 300
+        open('config.apt-source', 'w').write(apt_source)
 301
+    if not os.path.exists(default_apache2_service_config_dir):
 302
+        os.mkdir(default_apache2_service_config_dir, 0600)
 303
+    apt_update(fatal=True)
 304
+    apt_get_install("python-jinja2")
 305
+    apt_get_install("python-pyasn1")
 306
+    apt_get_install("python-pyasn1-modules")
 307
+    apt_get_install("python-yaml")
 308
+    install_status = apt_get_install("apache2")
 309
+    if install_status == 0:
 310
+        ensure_package_status(service_affecting_packages,
 311
+                              config_get('package_status'))
 312
+    ensure_extra_packages()
 313
+    # the apache2 deb does not yet have http2 module in mods-available. Add it.
 314
+    open('/etc/apache2/mods-available/http2.load', 'w').write(
 315
+        'LoadModule http2_module /usr/lib/apache2/modules/mod_http2.so')
 316
+    open('/etc/apache2/mods-available/http2.conf', 'w').write(
 317
+        '''<IfModule http2_module>
 318
+  ProtocolsHonorOrder On
 319
+  Protocols h2 http/1.1
 320
+</IfModule>
 321
+''')
 322
+    return install_status
 323
+
 324
+
 325
+def ensure_extra_packages():
 326
+    extra = str(config_get('extra_packages'))
 327
+    if extra:
 328
+        install_status = apt_get_install(extra)
 329
+        if install_status == 0:
 330
+            ensure_package_status(filter(None, extra.split(' ')),
 331
+                                  config_get('package_status'))
 332
+
 333
+
 334
+def dump_data(data2dump, log_prefix):
 335
+    log_file = '/tmp/pprint-%s.log' % (log_prefix)
 336
+    if data2dump is not None:
 337
+        logFile = open(log_file, 'w')
 338
+        import pprint
 339
+        pprint.pprint(data2dump, logFile)
 340
+        logFile.close()
 341
+
 342
+
 343
+def get_reverseproxy_data(relation='reverseproxy'):
 344
+    relation_data = relations_of_type(relation)
 345
+    reverseproxy_data = {}
 346
+    if relation_data is None or len(relation_data) == 0:
 347
+        return reverseproxy_data
 348
+    for unit_data in relation_data:
 349
+        unit_name = unit_data["__unit__"]
 350
+        if 'port' not in unit_data:
 351
+            return reverseproxy_data
 352
+        # unit_name: <service-name>-<unit_number>
 353
+        # jinja2 templates require python-type variables, remove all characters
 354
+        # that do not comply
 355
+        unit_type = re.sub(r'(.*)/[0-9]*', r'\1', unit_name)
 356
+        unit_type = re.sub('[^a-zA-Z0-9_]*', '', unit_type)
 357
+        log('unit_type: %s' % unit_type)
 358
+
 359
+        host = unit_data['private-address']
 360
+        if unit_type in reverseproxy_data:
 361
+            continue
 362
+        for config_setting in unit_data.keys():
 363
+            if config_setting in ("__unit__", "__relid__"):
 364
+                continue
 365
+            config_key = '%s_%s' % (unit_type,
 366
+                                    config_setting.replace("-", "_"))
 367
+            config_key = re.sub('[^a-zA-Z0-9_]*', '', config_key)
 368
+            reverseproxy_data[config_key] = unit_data[
 369
+                config_setting]
 370
+            reverseproxy_data[unit_type] = '%s:%s' % (
 371
+                host, unit_data['port'])
 372
+        if 'all_services' in unit_data:
 373
+            service_data = yaml.safe_load(unit_data['all_services'])
 374
+            for service_item in service_data:
 375
+                service_name = service_item['service_name']
 376
+                service_port = service_item['service_port']
 377
+                service_key = '%s_%s' % (unit_type, service_name)
 378
+                service_key = re.sub('[^a-zA-Z0-9_]*', '', service_key)
 379
+                reverseproxy_data[service_key] = '%s:%s' % (host, service_port)
 380
+    return reverseproxy_data
 381
+
 382
+
 383
+def update_balancers():
 384
+    relation_data = relations_of_type('balancer')
 385
+    if relation_data is None or len(relation_data) == 0:
 386
+        log("No relation data, exiting.")
 387
+        return
 388
+
 389
+    unit_dict = {}
 390
+    for unit_data in relation_data:
 391
+        unit_name = unit_data["__unit__"]
 392
+        if "port" not in unit_data:
 393
+            log("No port in relation data for '%s', skipping." % unit_name)
 394
+            continue
 395
+        port = unit_data["port"]
 396
+        if "private-address" not in unit_data:
 397
+            log("No private-address in relation data for '%s', skipping." %
 398
+                unit_name)
 399
+            continue
 400
+        host = unit_data['private-address']
 401
+
 402
+        if "all_services" in unit_data:
 403
+            service_data = yaml.safe_load(unit_data[
 404
+                "all_services"])
 405
+            for service_item in service_data:
 406
+                service_port = service_item["service_port"]
 407
+                current_units = unit_dict.setdefault(
 408
+                    service_item["service_name"], [])
 409
+                current_units.append("%s:%s" % (host, service_port))
 410
+        else:
 411
+            if "sitenames" in unit_data:
 412
+                unit_types = unit_data["sitenames"].split()
 413
+            else:
 414
+                unit_types = (re.sub(r"(.*)/[0-9]*", r"\1", unit_name),)
 415
+
 416
+            for unit_type in unit_types:
 417
+                current_units = unit_dict.setdefault(unit_type, [])
 418
+                current_units.append("%s:%s" % (host, port))
 419
+
 420
+    if not unit_dict:
 421
+        return
 422
+
 423
+    write_balancer_config(unit_dict)
 424
+    return unit_dict
 425
+
 426
+
 427
+def write_balancer_config(unit_dict):
 428
+    config_data = config_get()
 429
+
 430
+    from jinja2 import Environment, FileSystemLoader
 431
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 432
+        os.environ['CHARM_DIR'], 'data')))
 433
+    for balancer_name in unit_dict.keys():
 434
+        balancer_host_file = conf_filename('{}.balancer'.format(balancer_name))
 435
+        templ_vars = {
 436
+            'balancer_name': balancer_name,
 437
+            'balancer_addresses': unit_dict[balancer_name],
 438
+            'lb_balancer_timeout': config_data['lb_balancer_timeout'],
 439
+        }
 440
+        template = template_env.get_template(
 441
+            'balancer.template').render(templ_vars)
 442
+        log("Writing file: %s with data: %s" % (balancer_host_file,
 443
+                                                templ_vars))
 444
+        with open(balancer_host_file, 'w') as balancer_config:
 445
+            balancer_config.write(str(template))
 446
+        conf_enable('{}.balancer'.format(balancer_name))
 447
+
 448
+
 449
+def update_nrpe_checks():
 450
+    nrpe_compat = nrpe.NRPE()
 451
+    conf = nrpe_compat.config
 452
+    check_http_params = conf.get('nagios_check_http_params')
 453
+    if check_http_params:
 454
+        nrpe_compat.add_check(
 455
+            shortname='vhost',
 456
+            description='Check Virtual Host',
 457
+            check_cmd='check_http %s' % check_http_params
 458
+        )
 459
+    nrpe_compat.write()
 460
+
 461
+
 462
+def create_mpm_workerfile():
 463
+    config_data = config_get()
 464
+    mpm_workerfile = conf_filename('000mpm-worker')
 465
+    from jinja2 import Environment, FileSystemLoader
 466
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 467
+        os.environ['CHARM_DIR'], 'data')))
 468
+    templ_vars = {
 469
+        'mpm_type': config_data['mpm_type'],
 470
+        'mpm_startservers': config_data['mpm_startservers'],
 471
+        'mpm_minsparethreads': config_data['mpm_minsparethreads'],
 472
+        'mpm_maxsparethreads': config_data['mpm_maxsparethreads'],
 473
+        'mpm_threadlimit': config_data['mpm_threadlimit'],
 474
+        'mpm_threadsperchild': config_data['mpm_threadsperchild'],
 475
+        'mpm_serverlimit': config_data['mpm_serverlimit'],
 476
+        'mpm_maxclients': config_data['mpm_maxclients'],
 477
+        'mpm_maxrequestsperchild': config_data['mpm_maxrequestsperchild'],
 478
+    }
 479
+    template = \
 480
+        template_env.get_template('mpm_worker.template').render(templ_vars)
 481
+    with open(mpm_workerfile, 'w') as mpm_config:
 482
+        mpm_config.write(str(template))
 483
+    conf_enable('000mpm-worker')
 484
+
 485
+
 486
+def create_security():
 487
+    config_data = config_get()
 488
+    securityfile = conf_filename('security')
 489
+    from jinja2 import Environment, FileSystemLoader
 490
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 491
+        os.environ['CHARM_DIR'], 'data')))
 492
+    templ_vars = {
 493
+        'juju_warning_header': juju_warning_header,
 494
+        'server_tokens': config_data['server_tokens'],
 495
+        'server_signature': config_data['server_signature'],
 496
+        'trace_enabled': config_data['trace_enabled'],
 497
+        'ssl_protocol': config_data['ssl_protocol'],
 498
+        'ssl_honor_cipher_order': config_data['ssl_honor_cipher_order'],
 499
+        'ssl_cipher_suite': config_data['ssl_cipher_suite'],
 500
+        'is_apache24': is_apache24(),
 501
+    }
 502
+    template = \
 503
+        template_env.get_template('security.template').render(templ_vars)
 504
+    with open(securityfile, 'w') as security_config:
 505
+        security_config.write(str(template))
 506
+    conf_enable('security')
 507
+
 508
+
 509
+def ship_logrotate_conf():
 510
+    config_data = config_get()
 511
+    logrotate_file = '/etc/logrotate.d/apache2'
 512
+    from jinja2 import Environment, FileSystemLoader
 513
+    template_env = Environment(loader=FileSystemLoader(os.path.join(
 514
+        os.environ['CHARM_DIR'], 'data')))
 515
+    templ_vars = {
 516
+        'juju_warning_header': juju_warning_header,
 517
+        'logrotate_rotate': config_data['logrotate_rotate'],
 518
+        'logrotate_count': config_data['logrotate_count'],
 519
+        'logrotate_dateext': config_data['logrotate_dateext'],
 520
+    }
 521
+    template = template_env.get_template('logrotate.conf.template').render(
 522
+        templ_vars)
 523
+    with open(logrotate_file, 'w') as logrotate_conf:
 524
+        logrotate_conf.write(str(template))
 525
+
 526
+
 527
+def create_vhost(port, protocol=None, config_key=None, template_str=None,
 528
+                 config_data={}, relationship_data={}):
 529
+    """
 530
+    Create and enable a vhost in apache.
 531
+
 532
+    @param port: port on which to listen (int)
 533
+    @param protocol: used to name the vhost file intelligently.  If not
 534
+        specified the port will be used instead. (ex: http, https)
 535
+    @param config_key: key in the configuration to look up to
 536
+        retrieve the template.
 537
+    @param template_str: The template itself as a string.
 538
+    @param config_data: juju get-config configuration data.
 539
+    @param relationship_data: if in a relationship, pass in the appropriate
 540
+        structure.  This will be used to inform the template.
 541
+    """
 542
+    if protocol is None:
 543
+        protocol = str(port)
 544
+    if template_str is None:
 545
+        if not config_key or not config_data[config_key]:
 546
+            log("Vhost Template not provided, not configuring: %s" % port)
 547
+            return False
 548
+        template_str = config_data[config_key]
 549
+    from jinja2 import Template
 550
+    template = Template(str(base64.b64decode(template_str)))
 551
+    template_data = dict(config_data.items() + relationship_data.items())
 552
+    if config_data.get('vhost_template_vars'):
 553
+        extra_vars = ast.literal_eval(config_data['vhost_template_vars'])
 554
+        template_data.update(extra_vars)
 555
+    vhost_name = '%s_%s' % (config_data['servername'], protocol)
 556
+    vhost_file = site_filename(vhost_name)
 557
+    log("Writing file %s with config and relation data" % vhost_file)
 558
+    with open(vhost_file, 'w') as vhost:
 559
+        vhost.write(str(template.render(template_data)))
 560
+    subprocess.call(['/usr/sbin/a2ensite', vhost_name])
 561
+    return True
 562
+
 563
+
 564
+MPM_TYPES = ['mpm_worker', 'mpm_prefork', 'mpm_event']
 565
+
 566
+
 567
+def enable_mpm(config):
 568
+    """Enables a particular mpm module.
 569
+
 570
+    Different from simply enabling a module, as one and only one mpm module
 571
+    *must* be enabled.
 572
+    """
 573
+    # only do anything if value has changed, to avoid a needless restart
 574
+    if not config.changed('mpm_type'):
 575
+        return
 576
+
 577
+    mpm_type = config.get('mpm_type', '')
 578
+    name = 'mpm_' + mpm_type
 579
+    if name not in MPM_TYPES:
 580
+        log('bad mpm_type: %s. Falling back to mpm_worker' % mpm_type)
 581
+        name = 'mpm_worker'
 582
+
 583
+    # disable all other mpm modules
 584
+    for mpm in MPM_TYPES:
 585
+        if mpm != name:
 586
+            return_value = subprocess.call(['/usr/sbin/a2dismod', mpm])
 587
+            if return_value != 0:
 588
+                return False
 589
+
 590
+    return_value = subprocess.call(['/usr/sbin/a2enmod', name])
 591
+    if return_value != 0:
 592
+        return False
 593
+
 594
+    if service_apache2("check"):
 595
+        log("Switching mpm module to {}".format(name))
 596
+        service_apache2("restart")  # must be a restart to switch mpm
 597
+        return True
 598
+    else:
 599
+        log("Failed to switch mpm module to {}".format(name))
 600
+        return False
 601
+
 602
+
 603
+def config_changed():
 604
+    relationship_data = {}
 605
+    config_data = config_get()
 606
+
 607
+    apt_source = config_data['apt-source']
 608
+    old_apt_source = ''
 609
+    try:
 610
+        old_apt_source = open('config.apt-source', 'r').read()
 611
+    except IOError:
 612
+        pass
 613
+    if old_apt_source != apt_source:
 614
+        subprocess.check_call(['add-apt-repository', '--yes', '-r',
 615
+                               old_apt_source])
 616
+        add_source(apt_source, config_data['apt-key-id'])
 617
+        open('config.apt-source', 'w').write(apt_source)
 618
+
 619
+    ensure_package_status(service_affecting_packages,
 620
+                          config_data['package_status'])
 621
+    ensure_extra_packages()
 622
+
 623
+    relationship_data.update(get_reverseproxy_data(relation='reverseproxy'))
 624
+    relationship_data.update(get_reverseproxy_data(relation='website-cache'))
 625
+    if update_balancers():
 626
+        # apache 2.4 has lbmethods split, needs to enable specific module(s)
 627
+        if is_apache24():
 628
+            enable_module('lbmethod_byrequests')
 629
+
 630
+    disabled_modules = config_data['disable_modules'].split()
 631
+    apache_websites = ApacheWebsites.from_config(
 632
+        relations_of_type("apache-website"), disabled_modules)
 633
+    enabled_modules = config_data.get('enable_modules', '').split()
 634
+    enabled_modules = apache_websites.list_enabled_modules(enabled_modules)
 635
+    for module in enabled_modules:
 636
+        enable_module(module)
 637
+
 638
+    if config_data['disable_modules']:
 639
+        for module in disabled_modules:
 640
+            disable_module(module)
 641
+
 642
+    apache_websites.disable_sites()
 643
+    apache_websites.write_configs()
 644
+    apache_websites.enable_sites()
 645
+    apache_websites.configure_extra_ports()
 646
+    all_ports = apache_websites.list_enabled_ports()
 647
+    enable_mpm(config_data)
 648
+    # XXX we only configure the worker mpm?
 649
+    create_mpm_workerfile()
 650
+    create_security()
 651
+
 652
+    ports = {'http': 80, 'https': 443}
 653
+    for protocol, port in ports.iteritems():
 654
+        if create_vhost(
 655
+                port,
 656
+                protocol=protocol,
 657
+                config_key="vhost_%s_template" % protocol,
 658
+                config_data=config_data,
 659
+                relationship_data=relationship_data):
 660
+            all_ports.add(port)
 661
+
 662
+    cert_file = _get_cert_file_location(config_data)
 663
+    key_file = _get_key_file_location(config_data)
 664
+    chain_file = _get_chain_file_location(config_data)
 665
+
 666
+    if cert_file is not None and key_file is not None:
 667
+        # ssl_cert is SELFSIGNED so generate self-signed certificate for use.
 668
+        if config_data['ssl_cert'] and config_data['ssl_cert'] == "SELFSIGNED":
 669
+            if is_selfsigned_cert_stale(config_data, cert_file, key_file):
 670
+                gen_selfsigned_cert(config_data, cert_file, key_file)
 671
+
 672
+        # Use SSL certificate and key provided either as a base64 string or
 673
+        # shipped out with the charm.
 674
+        else:
 675
+            # Certificate provided as base64-encoded string.
 676
+            if config_data['ssl_cert']:
 677
+                log("Writing cert from config ssl_cert: %s" % cert_file)
 678
+                with open(cert_file, 'w') as f:
 679
+                    f.write(str(base64.b64decode(config_data['ssl_cert'])))
 680
+            # Use certificate file shipped out with charm.
 681
+            else:
 682
+                source = os.path.join(os.environ['CHARM_DIR'], 'data',
 683
+                                      config_data['ssl_certlocation'])
 684
+                if os.path.exists(source):
 685
+                    shutil.copy(source, cert_file)
 686
+                else:
 687
+                    log("Certificate not found, ignoring: %s" % source)
 688
+
 689
+            # Private key provided as base64-encoded string.
 690
+            if config_data['ssl_key']:
 691
+                log("Writing key from config ssl_key: %s" % key_file)
 692
+                with open(key_file, 'w') as f:
 693
+                    f.write(str(base64.b64decode(config_data['ssl_key'])))
 694
+            # Use private key shipped out with charm.
 695
+            else:
 696
+                source = os.path.join(os.environ['CHARM_DIR'], 'data',
 697
+                                      config_data['ssl_keylocation'])
 698
+                if os.path.exists(source):
 699
+                    shutil.copy(source, key_file)
 700
+                else:
 701
+                    log("Key file not found, ignoring: %s" % source)
 702
+
 703
+            if chain_file is not None:
 704
+                # Chain certificates provided as base64-encoded string.
 705
+                if config_data['ssl_chain']:
 706
+                    log("Writing chain certificates file from"
 707
+                        "config ssl_chain: %s" % chain_file)
 708
+                    with open(chain_file, 'w') as f:
 709
+                        f.write(str(base64.b64decode(
 710
+                            config_data['ssl_chain'])))
 711
+                # Use chain certificates shipped out with charm.
 712
+                else:
 713
+                    source = os.path.join(os.environ['CHARM_DIR'], 'data',
 714
+                                          config_data['ssl_chainlocation'])
 715
+                    if os.path.exists(source):
 716
+                        shutil.copy(source, chain_file)
 717
+                    else:
 718
+                        log("Chain certificates not found, "
 719
+                            "ignoring: %s" % source)
 720
+
 721
+        # Tighten permissions on private key file.
 722
+        if os.path.exists(key_file):
 723
+            os.chmod(key_file, 0440)
 724
+            os.chown(key_file, pwd.getpwnam('root').pw_uid,
 725
+                     grp.getgrnam('ssl-cert').gr_gid)
 726
+
 727
+    apache_syslog_conf = conf_filename("syslog")
 728
+    rsyslog_apache_conf = "/etc/rsyslog.d/45-apache2.conf"
 729
+    if config_data['use_rsyslog']:
 730
+        shutil.copy2("data/syslog-apache.conf", apache_syslog_conf)
 731
+        conf_enable("syslog")
 732
+        shutil.copy2("data/syslog-rsyslog.conf", rsyslog_apache_conf)
 733
+        # Fix permissions of access.log and error.log to allow syslog user to
 734
+        # write to
 735
+        os.chown("/var/log/apache2/access.log", pwd.getpwnam('syslog').pw_uid,
 736
+                 pwd.getpwnam('syslog').pw_gid)
 737
+        os.chown("/var/log/apache2/error.log", pwd.getpwnam('syslog').pw_uid,
 738
+                 pwd.getpwnam('syslog').pw_gid)
 739
+    else:
 740
+        conf_disable("syslog")
 741
+        if os.path.exists(apache_syslog_conf):
 742
+            os.unlink(apache_syslog_conf)
 743
+        if os.path.exists(rsyslog_apache_conf):
 744
+            os.unlink(rsyslog_apache_conf)
 745
+    run(["/usr/sbin/service", "rsyslog", "restart"])
 746
+
 747
+    # Disable the default website because we don't want people to see the
 748
+    # "It works!" page on production services and remove the
 749
+    # conf.d/other-vhosts-access-log conf.
 750
+    ensure_disabled(["000-default"])
 751
+    conf_disable("other-vhosts-access-log")
 752
+    if os.path.exists(conf_filename("other-vhosts-access-log")):
 753
+        os.unlink(conf_filename("other-vhosts-access-log"))
 754
+
 755
+    if service_apache2("check"):
 756
+        if config_data["config_change_command"] in ["reload", "restart"]:
 757
+            service_apache2(config_data["config_change_command"])
 758
+
 759
+    if config_data['openid_provider']:
 760
+        if not os.path.exists('/etc/apache2/security'):
 761
+            os.mkdir('/etc/apache2/security', 0755)
 762
+        with open('/etc/apache2/security/allowed-ops.txt', 'w') as f:
 763
+            f.write(config_data['openid_provider'].replace(',', '\n'))
 764
+            f.write('\n')
 765
+            os.chmod(key_file, 0444)
 766
+
 767
+    all_ports.update(update_vhost_config_relation())
 768
+    ensure_ports(all_ports)
 769
+    update_nrpe_checks()
 770
+    ship_logrotate_conf()
 771
+    if config_get().changed('servername'):
 772
+        logs_relation_joined()
 773
+
 774
+
 775
+def ensure_disabled(sites):
 776
+    to_disable = [s for s in sites if os.path.exists(site_filename(s, True))]
 777
+    if len(to_disable) == 0:
 778
+        return
 779
+    run(["/usr/sbin/a2dissite"] + to_disable)
 780
+
 781
+
 782
+def ensure_removed(filename):
 783
+    try:
 784
+        os.unlink(filename)
 785
+    except OSError as e:
 786
+        if e.errno != errno.ENOENT:
 787
+            raise
 788
+
 789
+
 790
+class ApacheWebsites:
 791
+
 792
+    @classmethod
 793
+    def from_config(cls, relations, disabled_modules):
 794
+        """Return an ApacheWebsites with information about all sites."""
 795
+        if relations is None:
 796
+            relations = []
 797
+        self_relations = {}
 798
+        for relation in relations:
 799
+            self_relation = {'domain': relation.get('domain')}
 800
+            enabled = bool(relation.get('enabled', 'False').lower() == 'true')
 801
+            site_modules = relation.get('site_modules', '').split()
 802
+            for module in site_modules:
 803
+                if module in disabled_modules:
 804
+                    enabled = False
 805
+                    log('site {} requires disabled_module {}'.format(
 806
+                        relation['__relid__'], module))
 807
+                break
 808
+            self_relation['site_modules'] = site_modules
 809
+            self_relation['enabled'] = enabled
 810
+            self_relation['site_config'] = relation.get('site_config')
 811
+            self_relation['ports'] = [
 812
+                int(p) for p in relation.get('ports', '').split()]
 813
+            self_relations[relation['__relid__']] = self_relation
 814
+        return cls(self_relations)
 815
+
 816
+    def __init__(self, relations):
 817
+        self.relations = relations
 818
+
 819
+    def write_configs(self):
 820
+        for key, relation in self.relations.items():
 821
+            config_file = site_filename(key)
 822
+            site_config = relation['site_config']
 823
+            if site_config is None:
 824
+                ensure_removed(config_file)
 825
+            else:
 826
+                with open(config_file, 'w') as output:
 827
+                    output.write(site_config)
 828
+
 829
+    def iter_enabled_sites(self):
 830
+        return ((k, v) for k, v in self.relations.items() if v['enabled'])
 831
+
 832
+    def enable_sites(self):
 833
+        enabled_sites = [k for k, v in self.iter_enabled_sites()]
 834
+        enabled_sites.sort()
 835
+        if len(enabled_sites) == 0:
 836
+            return
 837
+        subprocess.check_call(['/usr/sbin/a2ensite'] + enabled_sites)
 838
+
 839
+    def disable_sites(self):
 840
+        disabled_sites = [k for k, v in self.relations.items()
 841
+                          if not v['enabled']]
 842
+        disabled_sites.sort()
 843
+        if len(disabled_sites) == 0:
 844
+            return
 845
+        ensure_disabled(disabled_sites)
 846
+
 847
+    def list_enabled_modules(self, enabled_modules):
 848
+        enabled_modules = set(enabled_modules)
 849
+        for key, relation in self.iter_enabled_sites():
 850
+            enabled_modules.update(relation['site_modules'])
 851
+        return enabled_modules
 852
+
 853
+    def list_enabled_ports(self):
 854
+        enabled_ports = set()
 855
+        for key, relation in self.iter_enabled_sites():
 856
+            enabled_ports.update(relation['ports'])
 857
+        return enabled_ports
 858
+
 859
+    def configure_extra_ports(self):
 860
+        extra_ports = self.list_enabled_ports()
 861
+        extra_ports.discard(80)
 862
+        extra_ports.discard(443)
 863
+        extra_ports_conf = conf_filename('extra_ports')
 864
+        if len(extra_ports) > 0:
 865
+            with file(extra_ports_conf, 'w') as f:
 866
+                for port in sorted(extra_ports):
 867
+                    f.write('Listen {}\n'.format(port))
 868
+            conf_enable('extra_ports')
 869
+        else:
 870
+            conf_disable('extra_ports')
 871
+            ensure_removed(extra_ports_conf)
 872
+
 873
+
 874
+def update_vhost_config_relation():
 875
+    """
 876
+    Update the vhost file and include the certificate in the relation
 877
+    if it is self-signed.
 878
+    """
 879
+    vhost_ports = set()
 880
+    relation_data = relations_of_type("vhost-config")
 881
+    config_data = config_get()
 882
+    if relation_data is None:
 883
+        return vhost_ports
 884
+
 885
+    for unit_data in relation_data:
 886
+        if "vhosts" in unit_data:
 887
+            all_relation_data = {}
 888
+            all_relation_data.update(
 889
+                get_reverseproxy_data(relation='reverseproxy'))
 890
+            all_relation_data.update(
 891
+                get_reverseproxy_data(relation='website-cache'))
 892
+            try:
 893
+                vhosts = yaml.safe_load(unit_data["vhosts"])
 894
+                for vhost in vhosts:
 895
+                    port = vhost["port"]
 896
+                    if create_vhost(
 897
+                            port,
 898
+                            template_str=vhost["template"],
 899
+                            config_data=config_data,
 900
+                            relationship_data=all_relation_data):
 901
+                        vhost_ports.add(port)
 902
+            except Exception as e:
 903
+                log("Error reading configuration data from relation! %s" % e)
 904
+                raise
 905
+
 906
+    if service_apache2("check"):
 907
+        service_apache2("reload")
 908
+
 909
+    vhost_relation_settings = {
 910
+        "servername": config_data["servername"]}
 911
+
 912
+    cert_file = _get_cert_file_location(config_data)
 913
+    key_file = _get_key_file_location(config_data)
 914
+
 915
+    if cert_file is not None and key_file is not None:
 916
+        if config_data['ssl_cert'] and config_data['ssl_cert'] == "SELFSIGNED":
 917
+            with open(cert_file, 'r') as f:
 918
+                cert = base64.b64encode(f.read())
 919
+            vhost_relation_settings["ssl_cert"] = cert
 920
+    for id in relation_ids("vhost-config"):
 921
+        relation_set(relation_id=id, relation_settings=vhost_relation_settings)
 922
+    return vhost_ports
 923
+
 924
+
 925
+def start_hook():
 926
+    if service_apache2("status"):
 927
+        return(service_apache2("restart"))
 928
+    else:
 929
+        return(service_apache2("start"))
 930
+
 931
+
 932
+def stop_hook():
 933
+    if service_apache2("status"):
 934
+        return(service_apache2("stop"))
 935
+
 936
+
 937
+def reverseproxy_interface(hook_name=None):
 938
+    if hook_name is None:
 939
+        return(None)
 940
+    if hook_name == "changed":
 941
+        config_changed()
 942
+
 943
+
 944
+def website_interface(hook_name=None):
 945
+    if hook_name is None:
 946
+        return(None)
 947
+    my_host = socket.getfqdn(socket.gethostname())
 948
+    if my_host == "localhost":
 949
+        my_host = socket.gethostname()
 950
+    default_port = 80
 951
+    subprocess.call([
 952
+        'relation-set',
 953
+        'port=%d' % default_port,
 954
+        'hostname=%s' % my_host,
 955
+        'servername=%s' % config_get('servername')
 956
+    ])
 957
+
 958
+
 959
+def ensure_ports(ports):
 960
+    """Ensure that only the desired ports are open."""
 961
+    open_ports = set(get_open_ports())
 962
+    ports = set(ports)
 963
+    wanted_closed = ports.difference(open_ports)
 964
+    for port in sorted(wanted_closed):
 965
+        open_port(port)
 966
+    unwanted_open = open_ports.difference(ports)
 967
+    for port in sorted(unwanted_open):
 968
+        close_port(port)
 969
+    set_open_ports(list(sorted(ports)))
 970
+
 971
+
 972
+def get_open_ports():
 973
+    """Get the list of open ports from the standard file."""
 974
+    try:
 975
+        pfile = open(os.path.join(os.environ['CHARM_DIR'], 'ports.yaml'))
 976
+    except IOError as e:
 977
+        if e.errno == errno.ENOENT:
 978
+            return []
 979
+        else:
 980
+            raise
 981
+    with pfile:
 982
+        return yaml.safe_load(pfile)
 983
+
 984
+
 985
+def set_open_ports(ports):
 986
+    """Write the list of open ports to the standard file."""
 987
+    ports_path = os.path.join(os.environ['CHARM_DIR'], 'ports.yaml')
 988
+    with open(ports_path, 'w') as pfile:
 989
+        yaml.safe_dump(ports, pfile)
 990
+
 991
+
 992
+def get_log_files():
 993
+    """
 994
+    Read all of the apache config files from __ and get ErrorLog and AccessLog
 995
+    values.
 996
+
 997
+    Returns a tuple with first value list of access log files and second value
 998
+    list of error log files.
 999
+    """
1000
+    access_logs = []
1001
+    error_logs = []
1002
+    for protocol in ['http', 'https']:
1003
+        vhost_name = '%s_%s' % (config_get()['servername'], protocol)
1004
+        vhost_file = site_filename(vhost_name)
1005
+        try:
1006
+            # Using read().split('\n') here to work around a mocks open_mock
1007
+            # inadequacy: http://bugs.python.org/issue17467
1008
+            for line in open(vhost_file, 'r').read().split('\n'):
1009
+                if 'CustomLog' in line:
1010
+                    access_logs.append(line.split()[1])
1011
+                elif 'ErrorLog' in line:
1012
+                    error_logs.append(line.split()[1])
1013
+        except:
1014
+            pass
1015
+    return access_logs, error_logs
1016
+
1017
+
1018
+def logs_relation_joined():
1019
+    """
1020
+    Sets relation value with filenames
1021
+    """
1022
+    access_log_files, error_log_files = get_log_files()
1023
+    log_files = access_log_files[:]
1024
+    log_files.extend(error_log_files)
1025
+    types = ['apache_access' for a in access_log_files]
1026
+    types.extend(['apache_error' for a in error_log_files])
1027
+    data = {'files': '\n'.join(log_files),
1028
+            'types': '\n'.join(types),
1029
+            }
1030
+    _relation_ids = relation_ids('logs')
1031
+    for _relation_id in _relation_ids:
1032
+        log("logs-relation-joined setting relation data for {} to {}".format(
1033
+            _relation_id, data))
1034
+        relation_set(
1035
+            relation_id=_relation_id,
1036
+            relation_settings=data)
1037
+
1038
+
1039
+###############################################################################
1040
+# Main section
1041
+###############################################################################
1042
+def main(hook_name):
1043
+    if hook_name == "install":
1044
+        install_hook()
1045
+    elif hook_name == "config-changed" or hook_name == "upgrade-charm":
1046
+        config_changed()
1047
+    elif hook_name == "start":
1048
+        start_hook()
1049
+    elif hook_name == "stop":
1050
+        stop_hook()
1051
+    elif hook_name == "reverseproxy-relation-broken":
1052
+        config_changed()
1053
+    elif hook_name == "reverseproxy-relation-changed":
1054
+        config_changed()
1055
+    elif hook_name == "reverseproxy-relation-joined":
1056
+        config_changed()
1057
+    elif hook_name == "balancer-relation-broken":
1058
+        config_changed()
1059
+    elif hook_name == "balancer-relation-changed":
1060
+        config_changed()
1061
+    elif hook_name == "balancer-relation-joined":
1062
+        config_changed()
1063
+    elif hook_name == "website-cache-relation-broken":
1064
+        config_changed()
1065
+    elif hook_name == "website-cache-relation-changed":
1066
+        config_changed()
1067
+    elif hook_name == "website-cache-relation-joined":
1068
+        config_changed()
1069
+    elif hook_name == "website-relation-joined":
1070
+        website_interface("joined")
1071
+    elif hook_name == 'apache-website-relation-changed':
1072
+        config_changed()
1073
+    elif hook_name in ("nrpe-external-master-relation-changed",
1074
+                       "local-monitors-relation-changed"):
1075
+        update_nrpe_checks()
1076
+    elif hook_name == "vhost-config-relation-changed":
1077
+        config_changed()
1078
+    elif hook_name == "logs-relation-joined":
1079
+        logs_relation_joined()
1080
+    else:
1081
+        print "Unknown hook"
1082
+        sys.exit(1)
1083
+
1084
+if __name__ == "__main__":
1085
+    hook_name = os.path.basename(sys.argv[0])
1086
+    # Also support being invoked directly with hook as argument name.
1087
+    if hook_name == "hooks.py":
1088
+        if len(sys.argv) < 2:
1089
+            sys.exit("Missing required hook name argument.")
1090
+        hook_name = sys.argv[1]
1091
+    main(hook_name)
Back to file index

hooks/charmhelpers/__init__.py

 1
--- 
 2
+++ hooks/charmhelpers/__init__.py
 3
@@ -0,0 +1,38 @@
 4
+# Copyright 2014-2015 Canonical Limited.
 5
+#
 6
+# This file is part of charm-helpers.
 7
+#
 8
+# charm-helpers is free software: you can redistribute it and/or modify
 9
+# it under the terms of the GNU Lesser General Public License version 3 as
10
+# published by the Free Software Foundation.
11
+#
12
+# charm-helpers is distributed in the hope that it will be useful,
13
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
14
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
15
+# GNU Lesser General Public License for more details.
16
+#
17
+# You should have received a copy of the GNU Lesser General Public License
18
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
19
+
20
+# Bootstrap charm-helpers, installing its dependencies if necessary using
21
+# only standard libraries.
22
+import subprocess
23
+import sys
24
+
25
+try:
26
+    import six  # flake8: noqa
27
+except ImportError:
28
+    if sys.version_info.major == 2:
29
+        subprocess.check_call(['apt-get', 'install', '-y', 'python-six'])
30
+    else:
31
+        subprocess.check_call(['apt-get', 'install', '-y', 'python3-six'])
32
+    import six  # flake8: noqa
33
+
34
+try:
35
+    import yaml  # flake8: noqa
36
+except ImportError:
37
+    if sys.version_info.major == 2:
38
+        subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml'])
39
+    else:
40
+        subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml'])
41
+    import yaml  # flake8: noqa
Back to file index

hooks/charmhelpers/contrib/__init__.py

 1
--- 
 2
+++ hooks/charmhelpers/contrib/__init__.py
 3
@@ -0,0 +1,15 @@
 4
+# Copyright 2014-2015 Canonical Limited.
 5
+#
 6
+# This file is part of charm-helpers.
 7
+#
 8
+# charm-helpers is free software: you can redistribute it and/or modify
 9
+# it under the terms of the GNU Lesser General Public License version 3 as
10
+# published by the Free Software Foundation.
11
+#
12
+# charm-helpers is distributed in the hope that it will be useful,
13
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
14
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
15
+# GNU Lesser General Public License for more details.
16
+#
17
+# You should have received a copy of the GNU Lesser General Public License
18
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
Back to file index

hooks/charmhelpers/contrib/charmsupport/__init__.py

 1
--- 
 2
+++ hooks/charmhelpers/contrib/charmsupport/__init__.py
 3
@@ -0,0 +1,15 @@
 4
+# Copyright 2014-2015 Canonical Limited.
 5
+#
 6
+# This file is part of charm-helpers.
 7
+#
 8
+# charm-helpers is free software: you can redistribute it and/or modify
 9
+# it under the terms of the GNU Lesser General Public License version 3 as
10
+# published by the Free Software Foundation.
11
+#
12
+# charm-helpers is distributed in the hope that it will be useful,
13
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
14
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
15
+# GNU Lesser General Public License for more details.
16
+#
17
+# You should have received a copy of the GNU Lesser General Public License
18
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
Back to file index

hooks/charmhelpers/contrib/charmsupport/nrpe.py

  1
--- 
  2
+++ hooks/charmhelpers/contrib/charmsupport/nrpe.py
  3
@@ -0,0 +1,358 @@
  4
+# Copyright 2014-2015 Canonical Limited.
  5
+#
  6
+# This file is part of charm-helpers.
  7
+#
  8
+# charm-helpers is free software: you can redistribute it and/or modify
  9
+# it under the terms of the GNU Lesser General Public License version 3 as
 10
+# published by the Free Software Foundation.
 11
+#
 12
+# charm-helpers is distributed in the hope that it will be useful,
 13
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
 14
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 15
+# GNU Lesser General Public License for more details.
 16
+#
 17
+# You should have received a copy of the GNU Lesser General Public License
 18
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
 19
+
 20
+"""Compatibility with the nrpe-external-master charm"""
 21
+# Copyright 2012 Canonical Ltd.
 22
+#
 23
+# Authors:
 24
+#  Matthew Wedgwood <matthew.wedgwood@canonical.com>
 25
+
 26
+import subprocess
 27
+import pwd
 28
+import grp
 29
+import os
 30
+import glob
 31
+import shutil
 32
+import re
 33
+import shlex
 34
+import yaml
 35
+
 36
+from charmhelpers.core.hookenv import (
 37
+    config,
 38
+    local_unit,
 39
+    log,
 40
+    relation_ids,
 41
+    relation_set,
 42
+    relations_of_type,
 43
+)
 44
+
 45
+from charmhelpers.core.host import service
 46
+
 47
+# This module adds compatibility with the nrpe-external-master and plain nrpe
 48
+# subordinate charms. To use it in your charm:
 49
+#
 50
+# 1. Update metadata.yaml
 51
+#
 52
+#   provides:
 53
+#     (...)
 54
+#     nrpe-external-master:
 55
+#       interface: nrpe-external-master
 56
+#       scope: container
 57
+#
 58
+#   and/or
 59
+#
 60
+#   provides:
 61
+#     (...)
 62
+#     local-monitors:
 63
+#       interface: local-monitors
 64
+#       scope: container
 65
+
 66
+#
 67
+# 2. Add the following to config.yaml
 68
+#
 69
+#    nagios_context:
 70
+#      default: "juju"
 71
+#      type: string
 72
+#      description: |
 73
+#        Used by the nrpe subordinate charms.
 74
+#        A string that will be prepended to instance name to set the host name
 75
+#        in nagios. So for instance the hostname would be something like:
 76
+#            juju-myservice-0
 77
+#        If you're running multiple environments with the same services in them
 78
+#        this allows you to differentiate between them.
 79
+#    nagios_servicegroups:
 80
+#      default: ""
 81
+#      type: string
 82
+#      description: |
 83
+#        A comma-separated list of nagios servicegroups.
 84
+#        If left empty, the nagios_context will be used as the servicegroup
 85
+#
 86
+# 3. Add custom checks (Nagios plugins) to files/nrpe-external-master
 87
+#
 88
+# 4. Update your hooks.py with something like this:
 89
+#
 90
+#    from charmsupport.nrpe import NRPE
 91
+#    (...)
 92
+#    def update_nrpe_config():
 93
+#        nrpe_compat = NRPE()
 94
+#        nrpe_compat.add_check(
 95
+#            shortname = "myservice",
 96
+#            description = "Check MyService",
 97
+#            check_cmd = "check_http -w 2 -c 10 http://localhost"
 98
+#            )
 99
+#        nrpe_compat.add_check(
100
+#            "myservice_other",
101
+#            "Check for widget failures",
102
+#            check_cmd = "/srv/myapp/scripts/widget_check"
103
+#            )
104
+#        nrpe_compat.write()
105
+#
106
+#    def config_changed():
107
+#        (...)
108
+#        update_nrpe_config()
109
+#
110
+#    def nrpe_external_master_relation_changed():
111
+#        update_nrpe_config()
112
+#
113
+#    def local_monitors_relation_changed():
114
+#        update_nrpe_config()
115
+#
116
+# 5. ln -s hooks.py nrpe-external-master-relation-changed
117
+#    ln -s hooks.py local-monitors-relation-changed
118
+
119
+
120
+class CheckException(Exception):
121
+    pass
122
+
123
+
124
+class Check(object):
125
+    shortname_re = '[A-Za-z0-9-_]+$'
126
+    service_template = ("""
127
+#---------------------------------------------------
128
+# This file is Juju managed
129
+#---------------------------------------------------
130
+define service {{
131
+    use                             active-service
132
+    host_name                       {nagios_hostname}
133
+    service_description             {nagios_hostname}[{shortname}] """
134
+                        """{description}
135
+    check_command                   check_nrpe!{command}
136
+    servicegroups                   {nagios_servicegroup}
137
+}}
138
+""")
139
+
140
+    def __init__(self, shortname, description, check_cmd):
141
+        super(Check, self).__init__()
142
+        # XXX: could be better to calculate this from the service name
143
+        if not re.match(self.shortname_re, shortname):
144
+            raise CheckException("shortname must match {}".format(
145
+                Check.shortname_re))
146
+        self.shortname = shortname
147
+        self.command = "check_{}".format(shortname)
148
+        # Note: a set of invalid characters is defined by the
149
+        # Nagios server config
150
+        # The default is: illegal_object_name_chars=`~!$%^&*"|'<>?,()=
151
+        self.description = description
152
+        self.check_cmd = self._locate_cmd(check_cmd)
153
+
154
+    def _locate_cmd(self, check_cmd):
155
+        search_path = (
156
+            '/usr/lib/nagios/plugins',
157
+            '/usr/local/lib/nagios/plugins',
158
+        )
159
+        parts = shlex.split(check_cmd)
160
+        for path in search_path:
161
+            if os.path.exists(os.path.join(path, parts[0])):
162
+                command = os.path.join(path, parts[0])
163
+                if len(parts) > 1:
164
+                    command += " " + " ".join(parts[1:])
165
+                return command
166
+        log('Check command not found: {}'.format(parts[0]))
167
+        return ''
168
+
169
+    def write(self, nagios_context, hostname, nagios_servicegroups):
170
+        nrpe_check_file = '/etc/nagios/nrpe.d/{}.cfg'.format(
171
+            self.command)
172
+        with open(nrpe_check_file, 'w') as nrpe_check_config:
173
+            nrpe_check_config.write("# check {}\n".format(self.shortname))
174
+            nrpe_check_config.write("command[{}]={}\n".format(
175
+                self.command, self.check_cmd))
176
+
177
+        if not os.path.exists(NRPE.nagios_exportdir):
178
+            log('Not writing service config as {} is not accessible'.format(
179
+                NRPE.nagios_exportdir))
180
+        else:
181
+            self.write_service_config(nagios_context, hostname,
182
+                                      nagios_servicegroups)
183
+
184
+    def write_service_config(self, nagios_context, hostname,
185
+                             nagios_servicegroups):
186
+        for f in os.listdir(NRPE.nagios_exportdir):
187
+            if re.search('.*{}.cfg'.format(self.command), f):
188
+                os.remove(os.path.join(NRPE.nagios_exportdir, f))
189
+
190
+        templ_vars = {
191
+            'nagios_hostname': hostname,
192
+            'nagios_servicegroup': nagios_servicegroups,
193
+            'description': self.description,
194
+            'shortname': self.shortname,
195
+            'command': self.command,
196
+        }
197
+        nrpe_service_text = Check.service_template.format(**templ_vars)
198
+        nrpe_service_file = '{}/service__{}_{}.cfg'.format(
199
+            NRPE.nagios_exportdir, hostname, self.command)
200
+        with open(nrpe_service_file, 'w') as nrpe_service_config:
201
+            nrpe_service_config.write(str(nrpe_service_text))
202
+
203
+    def run(self):
204
+        subprocess.call(self.check_cmd)
205
+
206
+
207
+class NRPE(object):
208
+    nagios_logdir = '/var/log/nagios'
209
+    nagios_exportdir = '/var/lib/nagios/export'
210
+    nrpe_confdir = '/etc/nagios/nrpe.d'
211
+
212
+    def __init__(self, hostname=None):
213
+        super(NRPE, self).__init__()
214
+        self.config = config()
215
+        self.nagios_context = self.config['nagios_context']
216
+        if 'nagios_servicegroups' in self.config and self.config['nagios_servicegroups']:
217
+            self.nagios_servicegroups = self.config['nagios_servicegroups']
218
+        else:
219
+            self.nagios_servicegroups = self.nagios_context
220
+        self.unit_name = local_unit().replace('/', '-')
221
+        if hostname:
222
+            self.hostname = hostname
223
+        else:
224
+            self.hostname = "{}-{}".format(self.nagios_context, self.unit_name)
225
+        self.checks = []
226
+
227
+    def add_check(self, *args, **kwargs):
228
+        self.checks.append(Check(*args, **kwargs))
229
+
230
+    def write(self):
231
+        try:
232
+            nagios_uid = pwd.getpwnam('nagios').pw_uid
233
+            nagios_gid = grp.getgrnam('nagios').gr_gid
234
+        except:
235
+            log("Nagios user not set up, nrpe checks not updated")
236
+            return
237
+
238
+        if not os.path.exists(NRPE.nagios_logdir):
239
+            os.mkdir(NRPE.nagios_logdir)
240
+            os.chown(NRPE.nagios_logdir, nagios_uid, nagios_gid)
241
+
242
+        nrpe_monitors = {}
243
+        monitors = {"monitors": {"remote": {"nrpe": nrpe_monitors}}}
244
+        for nrpecheck in self.checks:
245
+            nrpecheck.write(self.nagios_context, self.hostname,
246
+                            self.nagios_servicegroups)
247
+            nrpe_monitors[nrpecheck.shortname] = {
248
+                "command": nrpecheck.command,
249
+            }
250
+
251
+        service('restart', 'nagios-nrpe-server')
252
+
253
+        for rid in relation_ids("local-monitors"):
254
+            relation_set(relation_id=rid, monitors=yaml.dump(monitors))
255
+
256
+
257
+def get_nagios_hostcontext(relation_name='nrpe-external-master'):
258
+    """
259
+    Query relation with nrpe subordinate, return the nagios_host_context
260
+
261
+    :param str relation_name: Name of relation nrpe sub joined to
262
+    """
263
+    for rel in relations_of_type(relation_name):
264
+        if 'nagios_hostname' in rel:
265
+            return rel['nagios_host_context']
266
+
267
+
268
+def get_nagios_hostname(relation_name='nrpe-external-master'):
269
+    """
270
+    Query relation with nrpe subordinate, return the nagios_hostname
271
+
272
+    :param str relation_name: Name of relation nrpe sub joined to
273
+    """
274
+    for rel in relations_of_type(relation_name):
275
+        if 'nagios_hostname' in rel:
276
+            return rel['nagios_hostname']
277
+
278
+
279
+def get_nagios_unit_name(relation_name='nrpe-external-master'):
280
+    """
281
+    Return the nagios unit name prepended with host_context if needed
282
+
283
+    :param str relation_name: Name of relation nrpe sub joined to
284
+    """
285
+    host_context = get_nagios_hostcontext(relation_name)
286
+    if host_context:
287
+        unit = "%s:%s" % (host_context, local_unit())
288
+    else:
289
+        unit = local_unit()
290
+    return unit
291
+
292
+
293
+def add_init_service_checks(nrpe, services, unit_name):
294
+    """
295
+    Add checks for each service in list
296
+
297
+    :param NRPE nrpe: NRPE object to add check to
298
+    :param list services: List of services to check
299
+    :param str unit_name: Unit name to use in check description
300
+    """
301
+    for svc in services:
302
+        upstart_init = '/etc/init/%s.conf' % svc
303
+        sysv_init = '/etc/init.d/%s' % svc
304
+        if os.path.exists(upstart_init):
305
+            nrpe.add_check(
306
+                shortname=svc,
307
+                description='process check {%s}' % unit_name,
308
+                check_cmd='check_upstart_job %s' % svc
309
+            )
310
+        elif os.path.exists(sysv_init):
311
+            cronpath = '/etc/cron.d/nagios-service-check-%s' % svc
312
+            cron_file = ('*/5 * * * * root '
313
+                         '/usr/local/lib/nagios/plugins/check_exit_status.pl '
314
+                         '-s /etc/init.d/%s status > '
315
+                         '/var/lib/nagios/service-check-%s.txt\n' % (svc,
316
+                                                                     svc)
317
+                         )
318
+            f = open(cronpath, 'w')
319
+            f.write(cron_file)
320
+            f.close()
321
+            nrpe.add_check(
322
+                shortname=svc,
323
+                description='process check {%s}' % unit_name,
324
+                check_cmd='check_status_file.py -f '
325
+                          '/var/lib/nagios/service-check-%s.txt' % svc,
326
+            )
327
+
328
+
329
+def copy_nrpe_checks():
330
+    """
331
+    Copy the nrpe checks into place
332
+
333
+    """
334
+    NAGIOS_PLUGINS = '/usr/local/lib/nagios/plugins'
335
+    nrpe_files_dir = os.path.join(os.getenv('CHARM_DIR'), 'hooks',
336
+                                  'charmhelpers', 'contrib', 'openstack',
337
+                                  'files')
338
+
339
+    if not os.path.exists(NAGIOS_PLUGINS):
340
+        os.makedirs(NAGIOS_PLUGINS)
341
+    for fname in glob.glob(os.path.join(nrpe_files_dir, "check_*")):
342
+        if os.path.isfile(fname):
343
+            shutil.copy2(fname,
344
+                         os.path.join(NAGIOS_PLUGINS, os.path.basename(fname)))
345
+
346
+
347
+def add_haproxy_checks(nrpe, unit_name):
348
+    """
349
+    Add checks for each service in list
350
+
351
+    :param NRPE nrpe: NRPE object to add check to
352
+    :param str unit_name: Unit name to use in check description
353
+    """
354
+    nrpe.add_check(
355
+        shortname='haproxy_servers',
356
+        description='Check HAProxy {%s}' % unit_name,
357
+        check_cmd='check_haproxy.sh')
358
+    nrpe.add_check(
359
+        shortname='haproxy_queue',
360
+        description='Check HAProxy queue depth {%s}' % unit_name,
361
+        check_cmd='check_haproxy_queue_depth.sh')
Back to file index

hooks/charmhelpers/contrib/charmsupport/volumes.py

  1
--- 
  2
+++ hooks/charmhelpers/contrib/charmsupport/volumes.py
  3
@@ -0,0 +1,175 @@
  4
+# Copyright 2014-2015 Canonical Limited.
  5
+#
  6
+# This file is part of charm-helpers.
  7
+#
  8
+# charm-helpers is free software: you can redistribute it and/or modify
  9
+# it under the terms of the GNU Lesser General Public License version 3 as
 10
+# published by the Free Software Foundation.
 11
+#
 12
+# charm-helpers is distributed in the hope that it will be useful,
 13
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
 14
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 15
+# GNU Lesser General Public License for more details.
 16
+#
 17
+# You should have received a copy of the GNU Lesser General Public License
 18
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
 19
+
 20
+'''
 21
+Functions for managing volumes in juju units. One volume is supported per unit.
 22
+Subordinates may have their own storage, provided it is on its own partition.
 23
+
 24
+Configuration stanzas::
 25
+
 26
+  volume-ephemeral:
 27
+    type: boolean
 28
+    default: true
 29
+    description: >
 30
+      If false, a volume is mounted as sepecified in "volume-map"
 31
+      If true, ephemeral storage will be used, meaning that log data
 32
+         will only exist as long as the machine. YOU HAVE BEEN WARNED.
 33
+  volume-map:
 34
+    type: string
 35
+    default: {}
 36
+    description: >
 37
+      YAML map of units to device names, e.g:
 38
+        "{ rsyslog/0: /dev/vdb, rsyslog/1: /dev/vdb }"
 39
+      Service units will raise a configure-error if volume-ephemeral
 40
+      is 'true' and no volume-map value is set. Use 'juju set' to set a
 41
+      value and 'juju resolved' to complete configuration.
 42
+
 43
+Usage::
 44
+
 45
+    from charmsupport.volumes import configure_volume, VolumeConfigurationError
 46
+    from charmsupport.hookenv import log, ERROR
 47
+    def post_mount_hook():
 48
+        stop_service('myservice')
 49
+    def post_mount_hook():
 50
+        start_service('myservice')
 51
+
 52
+    if __name__ == '__main__':
 53
+        try:
 54
+            configure_volume(before_change=pre_mount_hook,
 55
+                             after_change=post_mount_hook)
 56
+        except VolumeConfigurationError:
 57
+            log('Storage could not be configured', ERROR)
 58
+
 59
+'''
 60
+
 61
+# XXX: Known limitations
 62
+# - fstab is neither consulted nor updated
 63
+
 64
+import os
 65
+from charmhelpers.core import hookenv
 66
+from charmhelpers.core import host
 67
+import yaml
 68
+
 69
+
 70
+MOUNT_BASE = '/srv/juju/volumes'
 71
+
 72
+
 73
+class VolumeConfigurationError(Exception):
 74
+    '''Volume configuration data is missing or invalid'''
 75
+    pass
 76
+
 77
+
 78
+def get_config():
 79
+    '''Gather and sanity-check volume configuration data'''
 80
+    volume_config = {}
 81
+    config = hookenv.config()
 82
+
 83
+    errors = False
 84
+
 85
+    if config.get('volume-ephemeral') in (True, 'True', 'true', 'Yes', 'yes'):
 86
+        volume_config['ephemeral'] = True
 87
+    else:
 88
+        volume_config['ephemeral'] = False
 89
+
 90
+    try:
 91
+        volume_map = yaml.safe_load(config.get('volume-map', '{}'))
 92
+    except yaml.YAMLError as e:
 93
+        hookenv.log("Error parsing YAML volume-map: {}".format(e),
 94
+                    hookenv.ERROR)
 95
+        errors = True
 96
+    if volume_map is None:
 97
+        # probably an empty string
 98
+        volume_map = {}
 99
+    elif not isinstance(volume_map, dict):
100
+        hookenv.log("Volume-map should be a dictionary, not {}".format(
101
+            type(volume_map)))
102
+        errors = True
103
+
104
+    volume_config['device'] = volume_map.get(os.environ['JUJU_UNIT_NAME'])
105
+    if volume_config['device'] and volume_config['ephemeral']:
106
+        # asked for ephemeral storage but also defined a volume ID
107
+        hookenv.log('A volume is defined for this unit, but ephemeral '
108
+                    'storage was requested', hookenv.ERROR)
109
+        errors = True
110
+    elif not volume_config['device'] and not volume_config['ephemeral']:
111
+        # asked for permanent storage but did not define volume ID
112
+        hookenv.log('Ephemeral storage was requested, but there is no volume '
113
+                    'defined for this unit.', hookenv.ERROR)
114
+        errors = True
115
+
116
+    unit_mount_name = hookenv.local_unit().replace('/', '-')
117
+    volume_config['mountpoint'] = os.path.join(MOUNT_BASE, unit_mount_name)
118
+
119
+    if errors:
120
+        return None
121
+    return volume_config
122
+
123
+
124
+def mount_volume(config):
125
+    if os.path.exists(config['mountpoint']):
126
+        if not os.path.isdir(config['mountpoint']):
127
+            hookenv.log('Not a directory: {}'.format(config['mountpoint']))
128
+            raise VolumeConfigurationError()
129
+    else:
130
+        host.mkdir(config['mountpoint'])
131
+    if os.path.ismount(config['mountpoint']):
132
+        unmount_volume(config)
133
+    if not host.mount(config['device'], config['mountpoint'], persist=True):
134
+        raise VolumeConfigurationError()
135
+
136
+
137
+def unmount_volume(config):
138
+    if os.path.ismount(config['mountpoint']):
139
+        if not host.umount(config['mountpoint'], persist=True):
140
+            raise VolumeConfigurationError()
141
+
142
+
143
+def managed_mounts():
144
+    '''List of all mounted managed volumes'''
145
+    return filter(lambda mount: mount[0].startswith(MOUNT_BASE), host.mounts())
146
+
147
+
148
+def configure_volume(before_change=lambda: None, after_change=lambda: None):
149
+    '''Set up storage (or don't) according to the charm's volume configuration.
150
+       Returns the mount point or "ephemeral". before_change and after_change
151
+       are optional functions to be called if the volume configuration changes.
152
+    '''
153
+
154
+    config = get_config()
155
+    if not config:
156
+        hookenv.log('Failed to read volume configuration', hookenv.CRITICAL)
157
+        raise VolumeConfigurationError()
158
+
159
+    if config['ephemeral']:
160
+        if os.path.ismount(config['mountpoint']):
161
+            before_change()
162
+            unmount_volume(config)
163
+            after_change()
164
+        return 'ephemeral'
165
+    else:
166
+        # persistent storage
167
+        if os.path.ismount(config['mountpoint']):
168
+            mounts = dict(managed_mounts())
169
+            if mounts.get(config['mountpoint']) != config['device']:
170
+                before_change()
171
+                unmount_volume(config)
172
+                mount_volume(config)
173
+                after_change()
174
+        else:
175
+            before_change()
176
+            mount_volume(config)
177
+            after_change()
178
+        return config['mountpoint']
Back to file index

hooks/charmhelpers/core/__init__.py

 1
--- 
 2
+++ hooks/charmhelpers/core/__init__.py
 3
@@ -0,0 +1,15 @@
 4
+# Copyright 2014-2015 Canonical Limited.
 5
+#
 6
+# This file is part of charm-helpers.
 7
+#
 8
+# charm-helpers is free software: you can redistribute it and/or modify
 9
+# it under the terms of the GNU Lesser General Public License version 3 as
10
+# published by the Free Software Foundation.
11
+#
12
+# charm-helpers is distributed in the hope that it will be useful,
13
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
14
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
15
+# GNU Lesser General Public License for more details.
16
+#
17
+# You should have received a copy of the GNU Lesser General Public License
18
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
Back to file index

hooks/charmhelpers/core/hookenv.py

  1
--- 
  2
+++ hooks/charmhelpers/core/hookenv.py
  3
@@ -0,0 +1,568 @@
  4
+# Copyright 2014-2015 Canonical Limited.
  5
+#
  6
+# This file is part of charm-helpers.
  7
+#
  8
+# charm-helpers is free software: you can redistribute it and/or modify
  9
+# it under the terms of the GNU Lesser General Public License version 3 as
 10
+# published by the Free Software Foundation.
 11
+#
 12
+# charm-helpers is distributed in the hope that it will be useful,
 13
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
 14
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 15
+# GNU Lesser General Public License for more details.
 16
+#
 17
+# You should have received a copy of the GNU Lesser General Public License
 18
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
 19
+
 20
+"Interactions with the Juju environment"
 21
+# Copyright 2013 Canonical Ltd.
 22
+#
 23
+# Authors:
 24
+#  Charm Helpers Developers <juju@lists.ubuntu.com>
 25
+
 26
+import os
 27
+import json
 28
+import yaml
 29
+import subprocess
 30
+import sys
 31
+from subprocess import CalledProcessError
 32
+
 33
+import six
 34
+if not six.PY3:
 35
+    from UserDict import UserDict
 36
+else:
 37
+    from collections import UserDict
 38
+
 39
+CRITICAL = "CRITICAL"
 40
+ERROR = "ERROR"
 41
+WARNING = "WARNING"
 42
+INFO = "INFO"
 43
+DEBUG = "DEBUG"
 44
+MARKER = object()
 45
+
 46
+cache = {}
 47
+
 48
+
 49
+def cached(func):
 50
+    """Cache return values for multiple executions of func + args
 51
+
 52
+    For example::
 53
+
 54
+        @cached
 55
+        def unit_get(attribute):
 56
+            pass
 57
+
 58
+        unit_get('test')
 59
+
 60
+    will cache the result of unit_get + 'test' for future calls.
 61
+    """
 62
+    def wrapper(*args, **kwargs):
 63
+        global cache
 64
+        key = str((func, args, kwargs))
 65
+        try:
 66
+            return cache[key]
 67
+        except KeyError:
 68
+            res = func(*args, **kwargs)
 69
+            cache[key] = res
 70
+            return res
 71
+    return wrapper
 72
+
 73
+
 74
+def flush(key):
 75
+    """Flushes any entries from function cache where the
 76
+    key is found in the function+args """
 77
+    flush_list = []
 78
+    for item in cache:
 79
+        if key in item:
 80
+            flush_list.append(item)
 81
+    for item in flush_list:
 82
+        del cache[item]
 83
+
 84
+
 85
+def log(message, level=None):
 86
+    """Write a message to the juju log"""
 87
+    command = ['juju-log']
 88
+    if level:
 89
+        command += ['-l', level]
 90
+    if not isinstance(message, six.string_types):
 91
+        message = repr(message)
 92
+    command += [message]
 93
+    subprocess.call(command)
 94
+
 95
+
 96
+class Serializable(UserDict):
 97
+    """Wrapper, an object that can be serialized to yaml or json"""
 98
+
 99
+    def __init__(self, obj):
100
+        # wrap the object
101
+        UserDict.__init__(self)
102
+        self.data = obj
103
+
104
+    def __getattr__(self, attr):
105
+        # See if this object has attribute.
106
+        if attr in ("json", "yaml", "data"):
107
+            return self.__dict__[attr]
108
+        # Check for attribute in wrapped object.
109
+        got = getattr(self.data, attr, MARKER)
110
+        if got is not MARKER:
111
+            return got
112
+        # Proxy to the wrapped object via dict interface.
113
+        try:
114
+            return self.data[attr]
115
+        except KeyError:
116
+            raise AttributeError(attr)
117
+
118
+    def __getstate__(self):
119
+        # Pickle as a standard dictionary.
120
+        return self.data
121
+
122
+    def __setstate__(self, state):
123
+        # Unpickle into our wrapper.
124
+        self.data = state
125
+
126
+    def json(self):
127
+        """Serialize the object to json"""
128
+        return json.dumps(self.data)
129
+
130
+    def yaml(self):
131
+        """Serialize the object to yaml"""
132
+        return yaml.dump(self.data)
133
+
134
+
135
+def execution_environment():
136
+    """A convenient bundling of the current execution context"""
137
+    context = {}
138
+    context['conf'] = config()
139
+    if relation_id():
140
+        context['reltype'] = relation_type()
141
+        context['relid'] = relation_id()
142
+        context['rel'] = relation_get()
143
+    context['unit'] = local_unit()
144
+    context['rels'] = relations()
145
+    context['env'] = os.environ
146
+    return context
147
+
148
+
149
+def in_relation_hook():
150
+    """Determine whether we're running in a relation hook"""
151
+    return 'JUJU_RELATION' in os.environ
152
+
153
+
154
+def relation_type():
155
+    """The scope for the current relation hook"""
156
+    return os.environ.get('JUJU_RELATION', None)
157
+
158
+
159
+def relation_id():
160
+    """The relation ID for the current relation hook"""
161
+    return os.environ.get('JUJU_RELATION_ID', None)
162
+
163
+
164
+def local_unit():
165
+    """Local unit ID"""
166
+    return os.environ['JUJU_UNIT_NAME']
167
+
168
+
169
+def remote_unit():
170
+    """The remote unit for the current relation hook"""
171
+    return os.environ['JUJU_REMOTE_UNIT']
172
+
173
+
174
+def service_name():
175
+    """The name service group this unit belongs to"""
176
+    return local_unit().split('/')[0]
177
+
178
+
179
+def hook_name():
180
+    """The name of the currently executing hook"""
181
+    return os.path.basename(sys.argv[0])
182
+
183
+
184
+class Config(dict):
185
+    """A dictionary representation of the charm's config.yaml, with some
186
+    extra features:
187
+
188
+    - See which values in the dictionary have changed since the previous hook.
189
+    - For values that have changed, see what the previous value was.
190
+    - Store arbitrary data for use in a later hook.
191
+
192
+    NOTE: Do not instantiate this object directly - instead call
193
+    ``hookenv.config()``, which will return an instance of :class:`Config`.
194
+
195
+    Example usage::
196
+
197
+        >>> # inside a hook
198
+        >>> from charmhelpers.core import hookenv
199
+        >>> config = hookenv.config()
200
+        >>> config['foo']
201
+        'bar'
202
+        >>> # store a new key/value for later use
203
+        >>> config['mykey'] = 'myval'
204
+
205
+
206
+        >>> # user runs `juju set mycharm foo=baz`
207
+        >>> # now we're inside subsequent config-changed hook
208
+        >>> config = hookenv.config()
209
+        >>> config['foo']
210
+        'baz'
211
+        >>> # test to see if this val has changed since last hook
212
+        >>> config.changed('foo')
213
+        True
214
+        >>> # what was the previous value?
215
+        >>> config.previous('foo')
216
+        'bar'
217
+        >>> # keys/values that we add are preserved across hooks
218
+        >>> config['mykey']
219
+        'myval'
220
+
221
+    """
222
+    CONFIG_FILE_NAME = '.juju-persistent-config'
223
+
224
+    def __init__(self, *args, **kw):
225
+        super(Config, self).__init__(*args, **kw)
226
+        self.implicit_save = True
227
+        self._prev_dict = None
228
+        self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME)
229
+        if os.path.exists(self.path):
230
+            self.load_previous()
231
+
232
+    def __getitem__(self, key):
233
+        """For regular dict lookups, check the current juju config first,
234
+        then the previous (saved) copy. This ensures that user-saved values
235
+        will be returned by a dict lookup.
236
+
237
+        """
238
+        try:
239
+            return dict.__getitem__(self, key)
240
+        except KeyError:
241
+            return (self._prev_dict or {})[key]
242
+
243
+    def keys(self):
244
+        prev_keys = []
245
+        if self._prev_dict is not None:
246
+            prev_keys = self._prev_dict.keys()
247
+        return list(set(prev_keys + list(dict.keys(self))))
248
+
249
+    def load_previous(self, path=None):
250
+        """Load previous copy of config from disk.
251
+
252
+        In normal usage you don't need to call this method directly - it
253
+        is called automatically at object initialization.
254
+
255
+        :param path:
256
+
257
+            File path from which to load the previous config. If `None`,
258
+            config is loaded from the default location. If `path` is
259
+            specified, subsequent `save()` calls will write to the same
260
+            path.
261
+
262
+        """
263
+        self.path = path or self.path
264
+        with open(self.path) as f:
265
+            self._prev_dict = json.load(f)
266
+
267
+    def changed(self, key):
268
+        """Return True if the current value for this key is different from
269
+        the previous value.
270
+
271
+        """
272
+        if self._prev_dict is None:
273
+            return True
274
+        return self.previous(key) != self.get(key)
275
+
276
+    def previous(self, key):
277
+        """Return previous value for this key, or None if there
278
+        is no previous value.
279
+
280
+        """
281
+        if self._prev_dict:
282
+            return self._prev_dict.get(key)
283
+        return None
284
+
285
+    def save(self):
286
+        """Save this config to disk.
287
+
288
+        If the charm is using the :mod:`Services Framework <services.base>`
289
+        or :meth:'@hook <Hooks.hook>' decorator, this
290
+        is called automatically at the end of successful hook execution.
291
+        Otherwise, it should be called directly by user code.
292
+
293
+        To disable automatic saves, set ``implicit_save=False`` on this
294
+        instance.
295
+
296
+        """
297
+        if self._prev_dict:
298
+            for k, v in six.iteritems(self._prev_dict):
299
+                if k not in self:
300
+                    self[k] = v
301
+        with open(self.path, 'w') as f:
302
+            json.dump(self, f)
303
+
304
+
305
+@cached
306
+def config(scope=None):
307
+    """Juju charm configuration"""
308
+    config_cmd_line = ['config-get']
309
+    if scope is not None:
310
+        config_cmd_line.append(scope)
311
+    config_cmd_line.append('--format=json')
312
+    try:
313
+        config_data = json.loads(
314
+            subprocess.check_output(config_cmd_line).decode('UTF-8'))
315
+        if scope is not None:
316
+            return config_data
317
+        return Config(config_data)
318
+    except ValueError:
319
+        return None
320
+
321
+
322
+@cached
323
+def relation_get(attribute=None, unit=None, rid=None):
324
+    """Get relation information"""
325
+    _args = ['relation-get', '--format=json']
326
+    if rid:
327
+        _args.append('-r')
328
+        _args.append(rid)
329
+    _args.append(attribute or '-')
330
+    if unit:
331
+        _args.append(unit)
332
+    try:
333
+        return json.loads(subprocess.check_output(_args).decode('UTF-8'))
334
+    except ValueError:
335
+        return None
336
+    except CalledProcessError as e:
337
+        if e.returncode == 2:
338
+            return None
339
+        raise
340
+
341
+
342
+def relation_set(relation_id=None, relation_settings=None, **kwargs):
343
+    """Set relation information for the current unit"""
344
+    relation_settings = relation_settings if relation_settings else {}
345
+    relation_cmd_line = ['relation-set']
346
+    if relation_id is not None:
347
+        relation_cmd_line.extend(('-r', relation_id))
348
+    for k, v in (list(relation_settings.items()) + list(kwargs.items())):
349
+        if v is None:
350
+            relation_cmd_line.append('{}='.format(k))
351
+        else:
352
+            relation_cmd_line.append('{}={}'.format(k, v))
353
+    subprocess.check_call(relation_cmd_line)
354
+    # Flush cache of any relation-gets for local unit
355
+    flush(local_unit())
356
+