Skip to content

H
helm3
Commit 09588677 authored by Brendan Melville's avatar Brendan Melville
Browse files
Options

Merge pull request #76 from jackgr/doc-updates 

Update documentation to rationalize the terminology
parents 11f5996c e9abaf53
Hide whitespace changes
Inline Side-by-side
Showing with 493 additions and 304 deletions
README.md
View file @ 09588677
......@@ -2,27 +2,28 @@
[![Go Report Card](http://goreportcard.com/badge/kubernetes/deployment-manager)](http://goreportcard.com/report/kubernetes/deployment-manager)
Deployment Manager (DM) provides parameterized templates for Kubernetes clusters.
Deployment Manager (DM) provides parameterized templates for Kubernetes resources,
such as:
You can use it deploy ready-to-use types, such as:
* [Replicated Service](types/replicatedservice/v1)
* [Redis](types/redis/v1)
* [Replicated Service](templates/replicatedservice/v1)
* [Redis](templates/redis/v1)
Types live in ordinary Github repositories. This repository contains the DM
code, but also acts as a DM type registry.
Templates live in ordinary Github repositories called template registries. This
Github repository contains a template registry, as well as the DM source code.
You can also use DM to deploy simple templates that use types, such as:
You can use DM to deploy simple configurations that use templates, such as:
* [Guestbook](examples/guestbook/guestbook.yaml)
* [Deployment Manager](examples/bootstrap/bootstrap.yaml)
A template is just a `YAML` file that supplies parameters. (Yes, you're reading
that second example correctly. It uses DM to deploy itself.
See [examples/bootstrap/README.md](examples/bootstrap/README.md) for more information.)
A configuration is just a `YAML` file that supplies parameters. (Yes,
you're reading that second example correctly. It uses DM to deploy itself. See
[examples/bootstrap/README.md](examples/bootstrap/README.md) for more information.)
DM runs server side, in your Kubernetes cluster, so it can tell you what types
you've instantiated there, what instances you've created of a given type, and even
how the instances are organized. So, you can ask questions like:
you've instantiated there, including both primitive types and templates, what
instances you've created of a given type, and even how the instances are organized.
So, you can ask questions like:
* What Redis instances are running in this cluster?
* What Redis master and slave services are part of this Redis instance?
......@@ -31,10 +32,12 @@ how the instances are organized. So, you can ask questions like:
Because DM stores its state in the cluster, not on your workstation, you can ask
those questions from any client at any time.
For more information about types, including both primitive types and templates,
see the [design document](../design/design.md#types).
Please hang out with us in
[the Slack chat room](https://kubernetes.slack.com/messages/sig-configuration/)
and/or
[the Google Group](https://groups.google.com/forum/#!forum/kubernetes-sig-config)
and/or [the Google Group](https://groups.google.com/forum/#!forum/kubernetes-sig-config)
for the Kubernetes configuration SIG. Your feedback and contributions are welcome.
## Installing Deployment Manager
......@@ -44,8 +47,8 @@ Follow these 3 steps to install DM:
1. Make sure your Kubernetes cluster is up and running, and that you can run
`kubectl` commands against it.
1. Clone this repository into the src folder of your GOPATH, if you haven't already.
See the [Kubernetes docs](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md)
for how to setup Go and the repos.
See the [Kubernetes developer documentation](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md)
for information on how to setup Go and use the repository.
1. Use `kubectl` to install DM into your cluster `kubectl create -f
install.yaml`
......@@ -64,9 +67,9 @@ is up and running!
### Setting up the client
The easiest way to interact with Deployment Manager is through the `dm` tool
hitting a`kubectl` proxy. To set that up:
hitting a `kubectl` proxy. To set that up:
1. Build the tool by running `make` from the deployment-manager repo.
1. Build the tool by running `make` in the deployment-manager repository.
1. Run `kubectl proxy --port=8001 &` to start a proxy that lets you interact
with the Kubernetes API server through port 8001 on localhost. `dm` uses
`http://localhost:8001/api/v1/proxy/namespaces/default/services/manager-service:manager`
......@@ -74,85 +77,81 @@ as the default service address for DM.
### Using the client
#### Deploying from a type registry
The DM client, `dm`, can deploy configurations from the command line. It can also
pull templates from a template registry, generate configurations from them using
parameters supplied on the command line, and deploy the resulting configurations.
#### Deploying a configuration
This command deploys a redis cluster with two workers from the type definition
in this repository:
`dm` can deploy a configuration from a file, or read one from `stdin`. This
command deploys the canonical Guestbook example from the examples directory:
```
dm deploy redis:v1
dm deploy examples/guestbook/guestbook.yaml
```
When you deploy a type, you can optionally supply values for input parameters,
like this:
You can now use `kubectl` to see Guestbook running:
```
dm --properties workers=3 deploy redis:v1
kubectl get service
```
When you deploy a type, `dm` generates a template from the type and input
parameters, and then deploys it.
Look for frontend-service. If your cluster supports external load balancing, it
will have an external IP assigned to it, and you can navigate to it in your browser
to see the guestbook in action.
#### Deploying from a template
For more information about this example, see [examples/guestbook/README.md](examples/guestbook/README.md)
#### Deploying a template directly
You can also deploy an existing template, or read one from `stdin`. This command
deploys the canonical Guestbook example from the examples directory:
You can also deploy a template directly, without a configuration. This command
deploys a redis cluster with two workers from the redis template in this repository:
```
dm deploy examples/guestbook/guestbook.yaml
dm deploy redis:v1
```
You can now use `kubectl` to see Guestbook running:
You can optionally supply values for template parameters on the command line,
like this:
```
kubectl get service
dm --properties workers=3 deploy redis:v1
```
Look for frontend-service. If your cluster supports external load balancing, it
will have an external IP assigned to it, and you can navigate to it in your browser
to see the guestbook in action.
When you deploy a template directly, without a configuration, `dm` generates a
configuration from the template and any supplied parameters, and then deploys the
generated configuration.
For more information about this example, see [examples/guestbook/README.md](examples/guestbook/README.md)
For more information about deploying templates from a template registry or adding
types to a template registry, see [the template registry documentation](docs/templates/registry.md).
### Additional commands
The command line tool makes it easy to configure a cluster from a set of predefined
types. Here's a list of available commands:
`dm` makes it easy to configure a cluster from a set of predefined templates.
Here's a list of available `dm` commands:
```
expand Expands the supplied template(s)
deploy Deploys the supplied type or template(s)
expand Expands the supplied configuration(s)
deploy Deploys the named template or the supplied configuration(s)
list Lists the deployments in the cluster
get Retrieves the supplied deployment
delete Deletes the supplied deployment
update Updates a deployment using the supplied template(s)
get Retrieves the named deployment
delete Deletes the named deployment
update Updates a deployment using the supplied configuration(s)
deployed-types Lists the types deployed in the cluster
deployed-instances Lists the instances of the supplied type deployed in the cluster
types Lists the types in the current registry
describe Describes the supplied type in the current registry
deployed-instances Lists the instances of the named type deployed in the cluster
templates Lists the templates in a given template registry
describe Describes the named template in a given template registry
```
## Uninstalling Deployment Manager
You can uninstall Deployment Manager using the same configuration file:
You can uninstall Deployment Manager using the same configuration:
```
kubectl delete -f install.yaml
```
## Creating a type registry
All you need to create a type registry is a Github repository with top level file
named `registry.yaml`, and a top level folder named `types` that contains type definitions.
A type definition is just a folder that contains one or more versions, like `/v1`,
`/v2`, etc.
A version is just a folder that contains a type definition. As you can see from the
examples above, a type definition is just a Python or [Jinja](http://jinja.pocoo.org/)
file plus an optional schema.
## Building the container images
## Building the Container Images
This project runs Deployment Manager on Kubernetes as three replicated services.
By default, install.yaml uses prebuilt images stored in Google Container Registry
......@@ -167,7 +166,7 @@ GCloud.
There is a more detailed [design document](docs/design/design.md) available.
## Status of the project
## Status of the Project
This project is still under active development, so you might run into issues. If
you do, please don't be shy about letting us know, or better yet, contribute a
......
docs/design/design.md
View file @ 09588677
# Deployment Manager Design
## Overview
Deployment Manager is a service that runs in a Kubernetes cluster. It
provides a declarative configuration language to describe Kubernetes
resources, and a mechanism for deploying, updating, and deleting configurations.
This document describes the configuration language, object model, and
architecture of the service in detail.
Deployment Manager (DM) is a service that runs in a Kubernetes cluster,
supported by a command line interface. It provides a declarative `YAML`-based
language for configuring Kubernetes resources, and a mechanism for deploying,
updating, and deleting configurations. This document describes the configuration
language, the API model, and the service architecture in detail.
## Configuration Language
The configuration language in Deployment Manager consists of two parts: a
YAML-based language for describing resources, and a templating mechanism for
creating abstract parameterizable types.
A configuration consists of a list of resources in YAML. Resources have three
properties:
DM uses a `YAML`-based configuration language with a templating mechanism. A
configuration is a `YAML` file that describes a list of resources. A resource has
three properties:
* name: the name to use when managing the resource
* type: the type of the resource being managed
* properties: the configuration properties of the resource
* `name`: the name to use when managing the resource
* `type`: the type of the resource being configured
* `properties`: the configuration properties of the resource
An example snippet of a configuration looks like:
Here's a snippet from a typical configuration file:
```
resources:
......@@ -37,51 +36,63 @@ resources:
...
```
### References
Resources can reference values from other resources. The version of Deployment
Manager running in the Google Cloud Platform uses references to understand
dependencies between resources and properly order the operations it performs on
a configuration. (This version of DM doesn't yet order operations to satisfy
dependencies, but it will soon.)
It describes two resources:
A reference follows this syntax: **$(ref.NAME.PATH)**, where _NAME_ is the name
of the resource being referenced, and _PATH_ is a JSON path to the value in the
resource object.
* A replication controller named `my-rc`, and
* A service named `my-service`
For example:
## Types
```
$(ref.my-service.metadata.name)
```
Resource types are either primitives or templates.
In this case, _my-service_ is the name of the resource, and _metadata.name_ is
the JSON path to the value being referenced.
### Primitives
### Configurable Resources
Primitives are types implemented by the Kubernetes runtime, such as:
Configurable resources are the primitive resources that can be configured in
Deployment Manager, including:
* `Pod`
* `ReplicationController`
* `Service`
* `Namespace`
* `Secret`
* Pod
* ReplicationController
* Service
DM processes primitive resources by passing their properties directly to
`kubectl` to create, update, or delete the corresponding objects in the cluster.
Deployment Manager processes configurable resources by passing their
configuration properties directly to kubectl to create, update, or delete them
in the cluster.
(Note that DM runs `kubectl` server side, in a container.)
### Templates
Templates are abstract types that can be created using Python or
Templates are abstract types created using Python or
[Jinja](http://jinja.pocoo.org/). A template takes a set of properties as input,
and must output a valid YAML configuration string. Properties are bound to
values when a template is instantiated in a configuration.
and must output a valid `YAML` configuration. Properties are bound to values when
a template is instantiated by a configuration.
Templates are expanded before primitive resources are processed. The
configuration produced by expanding a template may contain primitive resources
and/or additional template invocations. All template invocations are expanded
recursively until the resulting configuration is a list of primitive resources.
(Note, however, that DM preserves the template hierarchy and any dependencies
between resources in a layout that can be used to reason programmatically about
the structure of the resulting collection of resources created in the cluster,
as described in greater detail below.)
Here's an example of a template written in [Jinja](http://jinja.pocoo.org/):
```
resources:
- name: {{ env['name'] }}-service
type: Service
properties:
prop1: {{ properties['prop1'] }}
...
```
Templates are expanded in a pre-processing step before configurable resources
are processed. They can output configurations containing configurable resources,
or additional nested templates. Nested templates are processed recursively.
As you can see, it's just a `YAML` file that contains expansion directives. For
more information about the kinds of things you can do in a Jinja based template,
see [the Jina documentation](http://jinja.pocoo.org/docs/).
An example of a template in python is:
Here's an example of a template written in Python:
```
import yaml
......@@ -99,39 +110,33 @@ def GenerateConfig(context):
return yaml.dump({'resources': resources})
```
and in Jinja:
```
resources:
- name: {{ env['name'] }}-service
type: Service
properties:
prop1: {{ properties['prop1'] }}
...
```
Of course, you can do a lot more in Python than in Jinja, but basic things, such
as simple parameter substitution, may be easier to implement and easier to read in
Jinja than in Python.
Templates provide access to multiple sets of data, which can be used for
parameterizing or further customizing configurations:
Templates provide access to multiple sets of data that can be used to
parameterize or further customize configurations:
* env: a map of key/value pairs from the environment, including pairs
defined by Deployment Manager, such as _deployment_, _name_, and _type_
* properties: a map of the key/value pairs passed in the properties section when
instantiating the template
* imports: a map of import file names to file contents of all imports
* `env`: a map of key/value pairs from the environment, including pairs
defined by Deployment Manager, such as `deployment`, `name`, and `type`
* `properties`: a map of the key/value pairs passed in the properties section
of the template invocation
* `imports`: a map of import file names to file contents for all imports
originally specified for the configuration
In Python, this data is available from the _context_ object passed into the
_GenerateConfig_ method.
In Jinja, these variables are available in the global scope. In Python, they are
available as properties of the `context` object passed into the `GenerateConfig`
method.
### Template Schemas
A schema can be optionally provided for a template. The schema describes
the template in more detail, including:
### Template schemas
* info: more information about the template, including long description and
title
* imports: any sub-imports used by this template (may be relative path or URL)
* required: properties which are required when instantiating the template
* properties: JSON Schema descriptions of each property the template accepts
A template can optionally be accompanied by a schema that describes it in more
detail, including:
* `info`: more information about the template, including long description and title
* `imports`: any files imported by this template (may be relative paths or URLs)
* `required`: properties that must have values when the template is expanded
* `properties`: A `JSON Schema` description of each property the template accepts
Here's an example of a template schema:
......@@ -153,21 +158,23 @@ properties:
default: prop-value
```
Schemas are used by Deployment Manager to validate properties during
template instantiation, and to provide default values.
When a schema is provided for a template, DM uses it to validate properties
passed to the template by its invocation, and to provide default values for
properties that were not given values.
Schemas must be supplied to DM along with the templates they describe.
Schemas must be imported with the templates they describe, when passing
configuration to Deployment Manager.
### Supplying templates
### Instantiating Templates
Templates can be supplied to DM in two different ways:
Templates can be used in two different ways: either passed to the API as an
imported file, or used from a public HTTP endpoint.
* They can be passed to DM along with configurations that import them, or
* They can be retrieved by DM from public HTTP endpoints for configurations that
reference them.
#### Imported Templates
#### Template imports
Templates can be imported as part of the target configuration, and used
directly, for example:
Configurations can import templates using path declarations. For example:
```
imports:
......@@ -180,50 +187,85 @@ resources:
prop1: prop-value
```
The _imports_ list is not understood by the Deployment Manager service.
The `imports` list is not understood by the Deployment Manager service.
It's a directive used by client-side tools to specify what additional files
should be included when passing a configuration to the API.
should be included when passing the configuration to the API.
Using the Deployment Manager API, these templates can be included in the
imports section of the _configuration_.
If you are calling the Deployment Manager service directly, you must embed the
imported templates in the configuration passed to the API.
#### External Templates
#### Template references
Templates can also be used from a public HTTP endpoint. For example:
Configurations can also reference templates using URLs for public HTTP endpoints.
DM will attempt to resolve template references during expansion. For example:
```
resources:
- name: example
type: https://raw.githubusercontent.com/example/example.py
- name: my-template
type: https://raw.githubusercontent.com/my-template/my-template.py
properties:
prop1: prop-value
```
The service will process external templates as follows:
When resolving template references, DM assumes that templates are stored in
directories, which may also contain schemas, examples and other supporting files.
It therefore processes template references as follows:
1. Attempt to fetch the template, and treat it as an import.
1. Attempt to fetch the schema for the template from
`<base path>/<template name>.schema`
1. Attempt to fetch files imported by the schema from `<base path>/<import path>`
Referring to the previous example,
* the base path is `https://raw.githubusercontent.com/my-template`,
* the template name is `my-template`, and
* the schema name is `my-template.schema`
If we include a configuration that uses the template as an example, then the
directory that contains `my-template` might look like this:
```
example.yaml
my-template.py
my-template.py.schema
helper.py
```
### Value references
Resources can reference values from other resources. The version of Deployment
Manager running in the Google Cloud Platform uses references to understand
dependencies between resources and properly order the operations it performs on
a configuration.
(Note that this version of DM doesn't yet order operations to satisfy
dependencies, but it will soon.)
1. Fetch the external template as an import
1. Attempt to fetch the schema for the template, using
_<full template path>.schema_ as the schema path
1. Repeat for any sub-imports found in the schema file
A reference follows this syntax: `$(ref.NAME.PATH)`, where `NAME` is the name
of the resource being referenced, and `PATH` is a `JSON` path to the value in the
resource object.
When fetching schema files and sub-imports, the base path of the external
template is used for relative paths.
For example:
```
$(ref.my-service.metadata.name)
```
In this case, `my-service` is the name of the resource, and `metadata.name` is
the `JSON` path to the value being referenced.
## API Model
Deployment Manager exposes a set of RESTful collections over HTTP/JSON.
DM exposes a set of RESTful collections over HTTP/JSON.
### Deployments
Deployments are the primary resource in the Deployment Manager service. The
inputs to a deployment are:
Deployments are the primary resources managed by the Deployment Manager service.
The inputs to a deployment are:
* name
* configuration
When creating a deployment, users pass their configuration,
as well as any import files (templates, datafiles, etc.), all encoded in `YAML`,
in as the _configuration_.
* `name`: the name by which the deployment can be referenced once created
* `configuration`: the configuration file, plus any imported files (templates,
schemas, helper files used by the templates, etc.).
Creating, updating or deleting a deployment creates a new manifest for the
deployment. When deleting a deployment, the deployment is first updated to
......@@ -240,10 +282,9 @@ http://manager-service/deployments
A manifest is created for a deployment every time it is changed. It contains
three key components:
* inputConfig: the original input configuration for the manifest
* expandedConfig: the expanded configuration to be used when processing resources
for the manifest
* layout: the hierarchical structure of the manifest
* `inputConfig`: the original input configuration for the manifest
* `expandedConfig`: the expanded configuration describing only primitive resources
* `layout`: the hierarchical structure of the configuration
Manifests are available at the HTTP endpoint:
......@@ -251,31 +292,31 @@ Manifests are available at the HTTP endpoint:
http://manager-service/deployments/<deployment>/manifests
```
#### Expanded Configuration
#### Expanded configuration
Given a new _inputConfig_, Deployment Manager expands all template
instantiations recursively until there is a flat set of configurable resources.
This final set is stored as the _expandedConfig_ and is used during resource
processing.
Given a new `inputConfig`, DM expands all template invocations recursively,
until the result is a flat set of primitive resources. This final set is stored
as the `expandedConfig` and is used to instantiate the primitive resources.
#### Layout
Users can use templates to build a rich, deep hierarchical architecture in their
configuration. Expansion flattens this hierarchy and removes the template
relationships from the configuration to create a format optimized for the process
of instantiating the resources. However, the structural information contained in
the original configuration has many uses, so rather than discard it, Deployment
Manager preserves it in the form of a _layout_.
Using templates, callers can build rich, deeply hierarchical architectures in
their configurations. Expansion flattens these hierarchies to simplify the process
of instantiating the primitive resources. However, the structural information
contained in the original configuration has many potential uses, so rather than
discard it, DM preserves it in the form of a `layout`.
The _layout_ looks very much like an input configuration. It is a YAML list of
resources, where each resource contains the following information:
The `layout` looks a lot like the original configuration. It is a `YAML` file
that describes a list of resources. Each resource contains the `name`, `type`
and `properties` from the original configuration, plus a list of nested resources
discovered during expansion. The resulting structure looks like this:
* name: name of the resource
* type: type of the resource
* properties: properties of the resource, set only for templates
* resources: sub-resources from expansion, set only for templates
An example layout is:
Here's an example of a layout:
```
resources:
......@@ -290,26 +331,35 @@ resources:
type: Service
```
The layout can be used for visualizing the architecture of resources, including
their hierarchical structure and reference relationships.
In this example, the top level resource is a replicated service named `rs`,
defined by the template named `replicatedservice.py`. Expansion produced the
two nested resources: a replication controller named `rs-rc`, and a service
named `rs-service`.
Using the layout, callers can discover that `rs-rc` and `rs-service` are part
of the replicated service named `rs`. More importantly, if `rs` was created by
the expansion of a larger configuration, such as one that described a complete
application, callers could discover that `rs-rc` and `rs-service` were part of
the application, and perhaps even that they were part of a RabbitMQ cluster in
the application's mid-tier.
### Types
The types API provides information about existing types being used the cluster.
The types API provides information about types used in the cluster.
It can be used to list all known types that are in use in existing deployments:
It can be used to list all known types used by active deployments:
```
http://manager-service/types
```
It can be used to list all active instances of a specific type in the cluster:
Or to list all active instances of a specific type in the cluster:
```
http://manager-service/types/<type>/instances
```
Passing _all_ as the type shows all instances of all types in the cluster. Type
instances include the following information:
Passing `all` as the type name shows all instances of all types in the
cluster. The following information is reported for type instances:
* name: name of resource
* type: type of resource
......@@ -318,58 +368,67 @@ instances include the following information:
* path: JSON path to the entry for the resource in the manifest layout
## Architecture
The Deployment Manager service is built to run as a service within a Kubernetes
cluster. It has three major components to manage deployments. The following
diagram illustrates the relationships between the components, which are described
in more detail below.
The Deployment Manager service is manages deployments within a Kubernetes
cluster. It has three major components. The following diagram illustrates the
components and the relationships between them.
![Architecture Diagram](architecture.png "Architecture Diagram")
Currently there are two caveats in the design of the service:
Currently, there are two caveats in the service implementation:
* Synchronous API: the API currently blocks on all processing for
a deployment request. In the future, this design will change to an
asynchronous operation-based mode.
* Non-persistence: the service currently stores all metadata in memory,
so it will lose all knowledge of deployments and their metadata on restart.
In the future, the service will persist all deployment metadata.
* In-memory state: the service currently stores all state in memory,
so it will lose all knowledge of deployments and related objects on restart.
In the future, the service will persist all state in the cluster.
### Manager
The **manager** service acts as both the API server and the workflow engine for
processing deployments. It uses the following process:
The `manager` service acts as both the API server and the workflow engine for
processing deployments. It handles a `POST` to the `/deployments` collection as
follows:
1. Create a new deployment with a manifest containing _inputConfig_ from the
1. Create a new deployment with a manifest containing `inputConfig` from the
user request
1. Call out to the **expandybird** service to expand the _inputConfig_
1. Store the resulting _expandedConfig_ and _layout_
1. Call out to the **resourcifier** service to perform processing on resources
from the _expandedConfig_
1. Call out to the `expandybird` service to expand the `inputConfig`
1. Store the resulting `expandedConfig` and `layout`
1. Call out to the `resourcifier` service to instantiate the primitive resources
described by the `expandedConfig`
1. Respond with success or error messages to the original API request
The manager is responsible for saving the metadata associated with
`GET`, `PUT` and `DELETE` operations are processed in a similar manner, except
that:
* No expansion is performed for `GET` or `DELETE`
* The primitive resources are updated for `PUT` and deleted for `DELETE`
The manager is responsible for saving the information associated with
deployments, manifests, type instances, and other resources in the Deployment
Manager model.
### Expandybird
The **expandybird** service takes in input configurations, performs all template
expansions, and returns the resulting flat configuration and layout. It is completely
stateless.
The `expandybird` service takes in a configuration, performs all necesary
template expansions, and returns the resulting flat configuration and layout.
It is completely stateless.
Because templates are written in Python or Jinja, the actual expansion process
is performed in a sub-process that runs a Python interpreter. A new sub-process
is created for every request to expandybird.
is created for every request to `expandybird`.
Currently, expansion is not sandboxed, but templates should be reproducable,
hermetically sealed entities. Future designs may therefore, introduce a sandbox to
limit external interaction, such as network or disk access, during expansion.
hermetically sealed entities. Future designs may therefore introduce a sandbox to
limit external interaction, such as network or disk access, during expansion.
### Resourcifier
The **resourcifier** service takes in flat expanded configurations containing
only configurable resources, and makes the respective kubectl calls to process
each resource. It is totally stateless, and handles requests synchronously.
The `resourcifier` service takes in a flat expanded configuration describing
only primitive resources, and makes the necessary `kubectl` calls to process
them. It is totally stateless, and handles requests synchronously.
The `resourcifier` runs `kubectl` in a sub-process within its container. A new
sub-process is created for every request to `resourcifier`.
The resourcifier returns either success or error messages encountered during
resource processing.
It returns either success or error messages encountered during resource processing.
docs/types/registry.md
View file @ 09588677
# Type Registry
# Template Registries
The Deployment Manager client allows you to deploy
[template types](https://github.com/kubernetes/deployment-manager/blob/master/docs/design/design.md#templates)
directly from a Github repository. You can use types from existing repositories
or integrate with your own repository.
DM lets configurations instantiate [templates](../design/design.md#templates)
using both [imports](../design/design.md#template-imports) and
[references](../design/design.md#template-references).
In order for a Github repository to integrate with Deployment Manager, it must
store Deployment Manager templates in a manner that conforms to the required
**Type Registry** structure detailed in this document.
Because template references can use any public HTTP endpoint, they provide
a way to share templates. While you can store templates anywhere you want and
organize them any way you want, you may not be able to share them effectively
without some organizing principles. This document defines conventions for
template registries that store templates in Github and organize them by name
and by version to make sharing easier.
## File structure
The repository must use the following file structure to store Deployment
Manager template types:
## Template Versions
Since templates referenced by configurations and by other templates may change
over time, we need a versioning scheme, so that template references can be reliably
resolved to specific template versions.
Every template must therefore carry a version based on the
[Semantic Versioning](http://semver.org/) specification. A template version
consists of a MAJOR version, a MINOR version and a PATCH version, and can
be represented as a three part string starting with the letter `v` and using
dot delimiters between the parts. For example `v1.1.0`.
Parts may be omitted from left to right, up to but not include the MAJOR
version. All omitted parts default to zero. So, for example:
* `v1.1` is equivalent to `v1.1.0`, and
* `v2` is equivalent to `v2.0.0`
As required by Semantic Versioning:
* The MAJOR version must be incremented for incompatible changes
* The MINOR version must be incremented functionality is added in a backwards-compatible
manner, and
* The PATCH version must be incremented for backwards-compatible bug fixes.
When resolving a template reference, DM will attempt to fetch the template with
the highest available PATCH version that has the same MAJOR and MINOR versions as
the referenced version.
## Template Validation
Every template version should include a configuration named `example.yaml`
that can be used to deploy an instance of the template. This file may be used,
along with any supporting files it requires, to validate the template.
## Template Organization
Technically, all you need to reference a template is a directory at a public
HTTP endpoint that contains a template file named either `<template-name>.py`
or `<template-name>.jinja`, depending on the implementation language, along
with any supporting files it might require, such as an optional schema file
named `<template-name>.py.schema` or `<template-name>.jinja.schema`, respectively,
helper files used by the implementation, files imported by the schema, and so on.
These constraints impose a basic level of organization on the template definition
by ensuring that the template and all of its supporting files at least live in the
same directory, and that the template and schema files follow well-defined naming
conventions.
They do not, however, provide any encapsulation. Without additional constraints,
there is nothing to prevent template publishers from putting multiple templates,
or multiple versions of the same template, in the same directory. While there
might be some benefits in allowing templates to share a directory, such as avoiding
the duplication of helper files, the cost of discovering and maintaining templates
would quickly outweigh them as the number of templates in the directory increased.
Every template version must therefore live in its own directory, and that
directory must contain one and only one top-level template file and supporting
files for one and only template version.
Since it may reduce management overhead to store many different templates,
and/or many versions of the same template, in a single repository, we need a way
to organize templates within a repository.
A template repository must therefore place all of the versions of a given
template in directories named for the template versions under a directory named
for the template.
For example:
```
templateA/
v1/
example.yaml
templateA.py
templateA.py.schema
v1.0.1/
example.yaml
templateA.py
templateA.py.schema
v1.1/
example.yaml
templateA.py
templateA.py.schema
helper.py
```
<repository-root>/
types/
<type-1>/
<version-1>/
<type-files>
<version-2>/
...
<type-2>/
The template directories may be organized in any way that makes sense to the
repository maintainers.
For example, this flat list of template directories is valid:
```
templates/
templateA/
v1/
...
templateB/
v2/
...
```
This example, where template directories are organized by category, is also valid:
```
templates/
big-data/
templateA/
v1/
...
templateB/
v2/
...
signals
templateC/
v1/
...
templateD/
v1.1/
...
```
## Versions
Types are versioned based on [Semantic Versioning](http://semver.org/), for
example, *v1.1.0*. A type may have any number of versions.
## Template Registries
## Types
Each type version must contain a top-level Deployment Manager template
[Deployment Manager template](https://github.com/kubernetes/deployment-manager/blob/master/docs/design/design.md#templates)
named either `<type-name>.py` or `<type-name>.jinja`, depending on the
templating language used for the type.
Github is a convenient place to store and manage templates. A template registry
is a Github repository that conforms to the requirements detailed in this document.
A
[template schema](https://github.com/kubernetes/deployment-manager/blob/master/docs/design/design.md#template-schemas)
must also be present, named `<template>.schema` (e.g., `my-template.py.schema`).
Other files may exist as part of the type and imported through the schema,
including sub-templates, data files, or other metadata used by the template.
For a working example of a template registry, please see the
[Kubernetes Template Registry](https://github.com/kubernetes/deployment-manager/tree/master/templates).
## Test Configuration
Each type version should include an example YAML configuration called
`example.yaml` to be used for deploying an instance of the type. This is useful
for development purposes.
### Accessing a template registry
## Sample Registry
An example of a valid type registry repository looks like:
The Deployment Manager client, `dm`, can deploy templates directly from a registry
using the following command:
```
/
types/
redis/
v1/
example.yaml
redis.jinja
redis.jinja.schema
replicatedservice/
v3/
example.yaml
replicatedservice.py
replicatedservice.py.schema
$ dm deploy <template-name>:<version>
```
For a working example of a type registry, please see the
[kubernetes/deployment-manager registry](https://github.com/kubernetes/deployment-manager/tree/master/types).
To resolve the template reference, `dm` looks for a template version directory
with the given version in the template directory with the given template name.
## Using Types
The Deployment Manager client can deploy types directly from a registry with
the following command:
By default, it uses the Kubernetes Template Registry. However, you can set a
different default using the `--registry` flag:
```
$ dm deploy <type-name>:<version>
$ dm --registry my-org/my-repo/my-root-directory deploy <template-name>:<version>
```
This will default to the type registry in the kubernetes/deployment-manager
Github repository. You can change this to another repository that contains a
registry with the `--registry` flag:
Alternatively, you can qualify the template name with the path to the template
directory within the registry, like this:
```
$ dm --registry my-repo/registry deploy <type-name>:<version>
$ dm deploy my-org/my-repo/my-root-directory/<template-name>:<version>
```
For types that require properties:
Specifying the path to the template directory this way doesn't change the default.
For templates that require properties, you can provide them on the command line:
```
$ dm --properties prop1=value1,prop2=value2 deploy <template-name>:<version>
```
$ dm --properties prop1=value1,prop2=value2 deploy <type-name>:<version>
### Changing a template registry
DM relies on Github to provide the tools and processes needed to add, modify or
delete the contents of a template registry. Conventions for changing a template
registry are defined by the registry maintainers, and should be published in the
top level README.md or a file it references, following usual Github practices.
The [Kubernetes Template Registry](https://github.com/kubernetes/deployment-manager/tree/master/templates)
follows the [git setup](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md#git-setup)
used by Kubernetes.
examples/guestbook/README.md
View file @ 09588677
......@@ -9,9 +9,9 @@ First, make sure DM is installed in your Kubernetes cluster and that the
Guestbook example is deployed by following the instructions in the top level
[README.md](../../README.md).
## Understanding the Guestbook example template
## Understanding the Guestbook example
Let's take a closer look at the template used by the Guestbook example.
Let's take a closer look at the configuration used by the Guestbook example.
### Replicated services
......@@ -19,20 +19,22 @@ The typical design pattern for microservices in Kubernetes is to create a
replication controller and a service with the same selector, so that the service
exposes ports from the pods managed by the replication controller.
We have created a parameterized type for this kind of replicated service called
[Replicated Service](../../types/replicatedservice/v1), and we use it three times in this
example.
We have created a parameterized template for this kind of replicated service
called [Replicated Service](../../templates/replicatedservice/v1), and we use it
three times in the Guestbook example.
Note that the type is defined by a
[python script](../../types/replicatedservice/v1/replicatedservice.py). It also has a
[schema](../../types/replicatedservice/v1/replicatedservice.py.schema). Schemas are
optional. If present in the type definition, they are used to validate uses of the
type that appear in DM templates.
The template is defined by a
[Python script](../../templates/replicatedservice/v1/replicatedservice.py). It
also has a [schema](../../templates/replicatedservice/v1/replicatedservice.py.schema).
Schemas are optional. If provided, they are used to validate template invocations
that appear in configurations.
For more information about types and templates, see the [design document](../../docs/design/design.md).
For more information about templates and schemas, see the
[design document](../../docs/design/design.md#templates).
### The Guestbook application
The Guestbook application consists of 2 microservices: a front end and a Redis cluster.
The Guestbook application consists of 2 microservices: a front end and a Redis
cluster.
#### The front end
......@@ -40,7 +42,7 @@ The front end is a replicated service with 3 replicas:
```
- name: frontend
type: https://raw.githubusercontent.com/kubernetes/deployment-manager/master/types/replicatedservice/v1/replicatedservice.py
type: https://raw.githubusercontent.com/kubernetes/deployment-manager/master/templates/replicatedservice/v1/replicatedservice.py
properties:
service_port: 80
container_port: 80
......@@ -49,13 +51,14 @@ The front end is a replicated service with 3 replicas:
image: gcr.io/google_containers/example-guestbook-php-redis:v3
```
(Note that we use the URL for the type replicatedservice.py, not just the type name.)
(Note that we use the URL for a specific version of the template replicatedservice.py,
not just the template name.)
#### The Redis cluster
The Redis cluster consists of two replicated services: a master with a single replica
and the slaves with 2 replicas. It's defined by [this composite type](../../types/redis/v1/redis.jinja),
which is a [Jinja](http://jinja.pocoo.org/) template with a [schema](../../types/redis/v1/redis.jinja.schema).
and the slaves with 2 replicas. It's defined by [this template](../../templates/redis/v1/redis.jinja),
which is a [Jinja](http://jinja.pocoo.org/) file with a [schema](../../templates/redis/v1/redis.jinja.schema).
```
{% set REDIS_PORT = 6379 %}
......@@ -63,7 +66,7 @@ which is a [Jinja](http://jinja.pocoo.org/) template with a [schema](../../types
resources:
- name: redis-master
type: https://raw.githubusercontent.com/kubernetes/deployment-manager/master/types/replicatedservice/v1/replicatedservice.py
type: https://raw.githubusercontent.com/kubernetes/deployment-manager/master/templates/replicatedservice/v1/replicatedservice.py
properties:
# This has to be overwritten since service names are hard coded in the code
service_name: redis-master
......@@ -75,7 +78,7 @@ resources:
image: redis
- name: redis-slave
type: https://raw.githubusercontent.com/kubernetes/deployment-manager/master/types/replicatedservice/v1/replicatedservice.py
type: https://raw.githubusercontent.com/kubernetes/deployment-manager/master/templates/replicatedservice/v1/replicatedservice.py
properties:
# This has to be overwritten since service names are hard coded in the code
service_name: redis-slave
......@@ -94,16 +97,17 @@ resources:
### Displaying types
You can see the types you deployed to the cluster using the `deployed-types` command:
You can see both the both primitive types and the templates you deployed to the
cluster using the `deployed-types` command:
```
dm deployed-types
["Service","ReplicationController","redis.jinja","https://raw.githubusercontent.com/kubernetes/deployment-manager/master/types/replicatedservice/v1/replicatedservice.py"]
["Service","ReplicationController","redis.jinja","https://raw.githubusercontent.com/kubernetes/deployment-manager/master/templates/replicatedservice/v1/replicatedservice.py"]
```
This output shows 2 primitive types (Service and ReplicationController), and 2
composite types (redis.jinja and one imported from github (replicatedservice.py)).
templates (redis.jinja and one imported from github named replicatedservice.py).
You can also see where a specific type is being used with the `deployed-instances` command:
......@@ -115,6 +119,7 @@ dm deployed-instances Service
This output describes the deployment and manifest, as well as the JSON paths to
the instances of the type within the layout.
For more information about deployments, manifests and layouts, see the [design document](../../docs/design/design.md).
For more information about deployments, manifests and layouts, see the
[design document](../../docs/design/design.md#api-model).
types/README.md 0 → 100644
View file @ 09588677
# Kubernetes Template Registry
Welcome to the Kubernetes Template Registry!
This registry holds Deployment Manager
[templates](https://github.com/kubernetes/deployment-manager/tree/master/docs/design/design.md#templates)
that you can use to deploy Kubernetes resources.
For more information about installing and using Deployment Manager, see its
[README.md](https://github.com/kubernetes/deployment-manager/tree/master/README.md).
## Organization
The Kubernetes Template Registry is organized as a flat list of template
directories. Templates are versioned. The versions of a template live in version
directories under its template directory.
For more information about Deployment Manager template registries, including
directory structure and template versions, see the
[template registry documentation](https://github.com/kubernetes/deployment-manager/tree/master/docs/templates/registry.md)
## Contributing
The Kubernetes Template Registry follows the
[git setup](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md#git-setup)
used by Kubernetes.
You will also need to have a Contributor License Agreement on file, as described
in [CONTRIBUTING.md](../CONTRIBUTING.md).
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment