Tuesday 8 November 2011

Puppet Enterprise Server Installation and Configuration

What is Puppet Enterprise?
Puppet Enterprise is the commercially supported, packaged release of Puppet, the leading open source solution for enterprise systems management, including data centre automation and configuration management. Puppet automates the provisioning, patching, and configuration of operating system and application components across enterprise and cloud infrastructure. Puppet Enterprise packages the underlying components required, and bundles the requisite support and services necessary to support enterprise deployments. Puppet Enterprise pricing is based on number of nodes it manages and you can find detail information here - Puppet Enterprise Pricing

How is it different than the open source distribution of Puppet?
Simplified Integration & Installation: Get up to speed immediately with a pre-built integration of Puppet, Puppet Master, Dashboard, Facter and all Puppet dependencies such as Ruby, Passenger, and Apache.

Scalable: Pre-configured to offer out-of-the-box scalability and the performance levels required in large installations.
Tested & Approved: Packaged, tested and QA’d by Puppet Labs engineers
Support: Enterprise-class support is included: issue resolution, feature enhancement priority requests, and best practices advice.
Always Current: Maintenance is included in your license. You always have access to the latest and greatest releases of the integrated platform.

What are the specific components and dependencies included with Puppet Enterprise?
Puppet Enterprise includes the following in a single installer:

·   PUPPET COMPONENTS:
-  Puppet Master
-  Puppet Agent
-  Puppet Dashboard
-  Puppet Module Tool
-  Puppet Compliance
-  User Account Management
-  Facter

·   THIRD-PARTY PRODUCTS:
-  Ruby
-  Apache
-  Phusion Passenger

.   THIRD-PARTY LIBRARIES:
-  Ruby on Rails
-  RubyGems
-  Rack
-  Rake
-  Haml
-  jQuery
-  ActiveMQ

Proposed Puppet architecture for our environment

Puppet Enterprise Installation
Login as root and then run following,

#mkdir /usr/local/download && cd /usr/local/download
#export http_proxy=http://proxy:3128 (Only if you are using proxy server)
#wget --proxy-user={username} --proxy-password={password} http://pm.puppetlabs.com/puppet-enterprise/1.2.3/puppet-enterprise-1.2.3-all.tar.gz
#tar zxvf puppet-enterprise-1.2.3-all.tar.gz (This was the latest as 8 November 2011)
#cd puppet-enterprise-1.2.3-all
#./puppet-enterprise-installer

By default all Puppet clients search for Puppet master as Puppet, therefore it’s required to update
the DNS with CANME, do following (You may update /etc/hosts in both server and the node but it’s recommended to use CNAME),

puppet   IN   CNAME  bakingcake.cake.com.

Start Puppet master using following command and make it to start when system reboots,
service pe-puppet start
chkconfig pe-puppet on

Also you can start Puppet enterprise Dashboard by,
service pe-puppet-dashboard-workers restart
chkconfig pe-puppet-dashboard-workers on

Here are the lists of ports used by Puppet and Puppet related applications,
Puppet Master – 8140 (Eg: puppet:8140)
Puppet Dashboard – 3000 (Eg: http://puppet:3000)
MCollective - 61613

Add following entry in to iptables in Puppet Master server where client nodes can connect,
-A INPUT -p tcp -m state --state NEW --dport 8140 -j ACCEPT
-A INPUT -p tcp -m state --state NEW --dport 3000 -j ACCEPT
-A INPUT -p tcp -m state --state NEW --dport 61613 -j ACCEPT

Puppet Client Installation
Login as root and then run following,

#export http_proxy=http://proxy:3128
#rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm
#yum install ruby ruby-libs ruby-shadow -y
#yum install puppet puppet-server facter -y

Start Puppet using following commands,
chkconfig puppet on
service puppet start

Send first message to Puppet Master,
#puppetd --server puppet --waitforcert 60 –test

Once above message send to Puppet Master, Administrator should authorised the Public key for the node to be able to initiate connection. It is preventive mechanism used by Puppet to restrict unauthorised communication with Puppet Master. Please do following,

puppet cert –list (This gives list of nodes waiting to be authenticated)
puppet cert --sign “name of the client”
Puppet Configuration

Puppet Master Configuration
Below visio diagram shows the file structure in puppet master,

Following scenario has been selected in order to explain the end to end configuration to be done on puppet master and the client.
Scenario – We want to create a central ntp.conf file and it needed to be pushed to puppet client. Also we want to make sure once file get copied to the client successfully reboot the ntp daemon. Assume puppet client already authenticated with the puppet master.
 Below servers have selected to do the testing,
-       Puppet master - node_name: puppet client (FQDN - puppet.mycake.org.au)
-       Puppet client – node name: client (FQDN - client.mycake.org.au)
Do the following as root,
[root@puppet ]# cd /etc/puppetlabs/puppet
[root@puppet /etc/puppetlabs/puppet]#

Allow puppet client to access file server on puppet master,
[root@puppet /etc/puppetlabs/puppet]# more fileserver.conf
# This file consists of arbitrarily named sections/modules
# defining where files are served from and to whom

# Define a section 'files'
# Adapt the allow/deny settings to your needs. Order
# for allow/deny does not matter, allow always takes precedence
# over deny
[files]
  path /etc/puppetlabs/puppet/files
  allow node_name
  allow *.example.com

Check whether puppet client is allowed to connect to puppet master,
[root@puppet /etc/puppetlabs/puppet/manifests]# pwd
/etc/puppetlabs/puppet/manifests
[root@puppet manifests]# ls
 nodes.pp
 site.pp

[root@puppet manifests]# more nodes.pp
node 'client.my.bakary.org.au' {
include sudo
}

Specify remote file bucket,
[root@puppet /etc/puppetlabs/puppet/manifests]# more site.pp
# specify remote filebucket
filebucket { 'main':
  server => 'puppet.mycake.org.au ',
  path => false,
}

Create sample ntp directory and sample ntp.config file -> /etc/puppet/files
[root@puppet /etc/puppetlabs/files]# mkdir ntp && cd ntp
[root@puppet /etc/puppetlabs/files/ntp]# cp /etc/ntp.conf ntp.conf

Add following to site.pp,
-       Create class and specify the file to be updated or created and the location to be inserted in puppet node
-       Ensure the file is resides
-       Set file permissions
-       Set users and groups it belongs
-       Source location on puppet master
-       Tell puppet which client it should be run

[root@puppet /etc/puppetlabs/puppet/manifests]# vi site.pp
class ntp_class {
        # create the ntp file
        file { "/etc/ntp.conf":
                ensure => present,
                owner => root,
                group => root,
                mode => 0440,
                source => "puppet:///files/ntp/ntp.conf"
        }
}

# tell puppet on which client to run the class
node client.mycake.org.au {
    include ntp_class
}

File { backup => 'main' }

Puppet Dashboard Configuration
Edit /etc/puppetlabs/puppet/puppet.conf and include following in the [master] section,
[master]
reports = puppet_dashboard,store
reportdir = /var/lib/puppet/reports
reporturl = http://puppet:3000/reports

Restart puppet master and start puppet-dashboard
[root@puppet ]# service pe-puppet restart
Stopping puppetmaster:                                     [  OK  ]
Starting puppetmaster:                                     [  OK  ]
[root@puppet ]#  service pe-puppet-dashboard-workers start
Starting puppet-dashboard:                                 [  OK  ]

In order to test web GUI go to the following link in your browser
http://puppet:3000/

In order to add puppet clients to dashboard do following in the client /etc/puppet/puppet.conf
 [agent]
    report = true

Run puppet in noop mode on the client
[root@client]# puppetd --noop --test

What Next
·         Identify basic modules to be managed by Puppet
·         Select at least 3 Puppet clients
·         Based on above define re-usable component classes on Puppet
·         Setup svn repository to manage all editable files used by Puppet
·         Manage above clients via Puppet
·         Extend the master puppet configuration
·         Setup Cobbler to manage Kickstart
·         Configure to manage Nagios NRPE, LDAP and SUDO via Puppet

I will post most of the very soon.

Please let me know what you think and any obvious mistakes and improvements. Love to know what you guys think. Thanks

References
[1]. http://www.engineyard.com/blog/2011/why-puppet-should-manage-your-infrastructure/
[2]. http://bhuga.net/rapid-and-incremental-infrastructure-development-puppet
[3]. http://docs.puppetlabs.com/guides/from_source.html
[4]. http://docs.puppetlabs.com/mcollective/reference/basic/configuration.html
[5]. http://projects.puppetlabs.com/projects/puppet/wiki/Yum_Server_Build_Patterns
[6]. http://projects.puppetlabs.com/projects/puppet/wiki/Sudo_Patterns
[7]. http://www.howtoforge.com/installing_puppet_on_ubuntu
[8]. http://docs.puppetlabs.com/learning/
[9]. http://itservices.stanford.edu/strategy/sysadmin/automation
[10]. http://docs.puppetlabs.com/guides/language_guide.html
[11]. http://projects.puppetlabs.com/projects/1/wiki/Ldap_Nodes
[12]. http://blog.gurski.org/index.php/2010/01/28/automatic-monitoring-with-puppet-and-nagios/
[13]. http://docs.puppetlabs.com/guides/exported_resources.html
[14]. http://docs.puppetlabs.com/guides/setting_up.html
[15]. http://projects.puppetlabs.com/projects/1/wiki/Puppet_Scalability
[16]. http://projects.puppetlabs.com/projects/puppet/wiki/Big_Picture
[17]. http://www.craigdunn.org/2010/08/part-3-installing-puppet-dashboard-on-centos-puppet-2-6-1/