Puppet Re-provisioning Guide

[AmarnathPai]

This document aims to create a complete checklist for puppet re-provisioning

NOTE: The SA head unit on which the puppet master runs is referred to as "SA Server" and the host which is to be added is referred to as "Appliance"

Prerequisite

• Netwitness version should be 10.4 or above
• The date and time should be in sync on the SA server and the appliance (SA server should never be behind the appliance)
• The appliance should have a host entry for puppetmaster.local in /etc/hosts file resolving to the SA server IP address
• The file /etc/puppet/csr_attributes.yaml on the appliance should have the correct IP address of the appliance

Re-provisioning:

On the Appliance:

•     Stop the puppet, mcollective and rabbitmq service

# service puppet stop

# service mcollective stop

# service rabbitmq-server stop

• Record the node_id using cat /var/lib/puppet/node_id
•     Clear all existing certificates

# rm -rf /var/lib/puppet/ssl/*

• Clear the mcollective certificates

NOTE: Do not delete the directory, only delete the .pem files inside the directory

# cd /etc/mcollective/ssl/

# rm *.pem

# cd /etc/mcollective/ssl/client

# rm *.pem

• Clear the rabbitmq truststore certificate

# cd /etc/rabbitmq/ssl

# rm truststore.pem

On the SA Server:

•  On the UI, select the appliance and click on "Remove and Re-purpose"
•  Open /var/lib/puppet/ssl/ca/inventory.txt and delete the line containing the  node_id of the appliance
• This should clear all traces of the appliance from the SA server

The following steps can be used to verify  that the certificate has been cleared:

# puppet cert list -all | grep <node_id>   (Replace the node_id with the node_id of the appliance. This should not give any output if the node is cleared)

# mongo puppet

> db.nodes.find() (Confirm that the node_id is not available in the mongo db)

• If the node_id is still present, the following command can be used to clear it

# puppet cert clean <node_id>

# /etc/puppet/scripts/delNode.py <node_id>

Once the above steps are performed, go back to the appliance and start the puppet service

# service puppet start

• Monitor the /var/log/messages for entries related to puppet-agent.
• Wait for few seconds and then click on "Discover" in the UI and the appliance should show up.

Few additional checks:

• Once the puppet starts on the appliance, /var/lib/puppet/ssl/ca/requests/ on the SA server should contain a certificate request with the node_id of the appliance
• The puppet master would then sign the certificate and invoke the scripts /etc/puppet/scripts/addNode.py and /etc/puppet/scripts/addService.py
• This can be checked with the following command on the SA Server

# ps -ef | grep puppet

• Once the initial puppet run is complete on the appliance, it should automatically start the rabbitmq service