Chef: Local Cookbook Development

 

Generators
generator1generator2generator3

  1. Execute the command as follows:
    chef generate cookbook test_cookbook
    cd test_cookbook
    Take a look at some of the default files like README.md, spec-helper.rb, default.rb file
  2. We will now create a generator that we will use to create cookbook with our customization:
    mkdir generator
    chef generate generator generator/lcd_origin
    cd generator/lcd_origin/templates/default
    vi README.md.erb insert text of your choice. Save and close.
  3. Edit kitchen.yml.erb change the contents to match the following, as we will be using docker driver and centos 7.2 image:
    ---
    driver:
      name: docker
      privileged: true
      use_sudo: false
    
    provisioner:
      name: chef_zero
      # You may wish to disable always updating cookbooks in CI or other testing environments.
      # For example:
      #   always_update_cookbooks: <%%= !ENV['CI'] %>
      always_update_cookbooks: true
    
    verifier:
      name: inspec
    
    platforms:
      - name: centos-7.2
        driver_config:
          run_command: /usr/lib/systemd/systemd
    
    suites:
      - name: default
        run_list:
          - recipe[<%= cookbook_name %>::default]
        verifier:
          inspec_tests:
            - test/smoke/default
        attributes:
    <span 				data-mce-type="bookmark" 				id="mce_SELREST_start" 				data-mce-style="overflow:hidden;line-height:0" 				style="overflow:hidden;line-height:0" 			></span>
    

    Make sure you are not adding any tabs as they are invalid in ymls.

  4. Edit ../../files/default/spec_helper.rb Add the following lines:
    RSpec.configure do |config|
      config.platform = 'centos'
      config.version = '7.2.1511'
    end<span 				data-mce-type="bookmark" 				id="mce_SELREST_start" 				data-mce-style="overflow:hidden;line-height:0" 				style="overflow:hidden;line-height:0" 			></span>
    
  5. So to tell chef to use our generator when generating cookbooks:
    goto the lcd_origin directory
    mkdir ~/.chef
    vi ~/.chef/config.rb Add the contents as below:

    cookbook_path ['~/chef/cookbooks']
    local_mode true
    if File.basename($PROGRAM_NAME).eql?('chef') && ARGV[0].eql?('generate')
      chefdk.generator.license = "all_rights"
      chefdk.generator.copyright_holder = "Student Name"
      chefdk.generator_cookbook = "/root/cookbooks/lcd_origin"
      chefdk.generator.email = "you@example.com"
      chefdk.generator_cookbook = "~/generator/lcd_origin"
    end
    <span 				data-mce-type="bookmark" 				id="mce_SELREST_start" 				data-mce-style="overflow:hidden;line-height:0" 				style="overflow:hidden;line-height:0" 			></span>
    

    Create directory chef/cookbooks
    mkdir -p ~/chef/cookbooks
    And then generate a cookbook
    cd ~/chef
    chef generate cookbook cookbooks/lcd_web
    If you check the README vi cookbooks/lcd_web/README.md. You will see the text we added previously.
    We can generate attributes by following command:
    chef generate attribute cookbooks/lcd_web default
    We can genrate recipe by:
    chef generate recipe cookbooks/lcd_web users

  6. These attributes can be checked here in lcd_origin/attributes/default.rb
    The recipe can be checked with lcd_origin/recipes/users.rb

Test Driven Development

tdd1.png
tdd2tdd3tdd4tdd5

ChefSpec

chefspec1chefspec2chefspec3chefspec4chefspec5

    1. You can check the chefspec default file  spec/unit/recipes/default_spec.rb
    2. Now let’s run the default_spec
      chef exec rspec
      If you get any warning about platform and platform_version then change the following in your code:
      runner = ChefSpec::ServerRunner.new(platform: ‘centos’, version: ‘7.2.1511’)
      For some of you the platform and the platform_version might be already added. It might be ubuntu and 16.04 change it to centos and 7.2.1511 resp.
      Then execute the rspec again. This will give an output similar to below:
      Finished in 0.48374 seconds (files took 1.83 seconds to load)
      2 examples, 0 failures
    3. Now change the default_spec.rb to match the following to check if httpd package is installed or not:
      #
      # Cookbook:: lcd_web
      # Spec:: default
      #
      # Copyright:: 2017, Student Name, All Rights Reserved.
      
      require 'spec_helper'
      
      describe 'lcd_web::default' do
        context 'CentOS' do
          let(:chef_run) do
            # for a complete list of available platforms and versions see:
            # https://github.com/customink/fauxhai/blob/master/PLATFORMS.md
            runner = ChefSpec::ServerRunner.new(platform: 'centos', version: '7.2.1511')
            runner.converge(described_recipe)
          end
      
          it 'converges successfully' do
            expect { chef_run }.to_not raise_error
          end
      
          it 'installs httpd' do
            expect(chef_run).to install_package('httpd')
          end
        end
      end
      

      Then run chef exec rspec You will get an error as below:

      .F.

      Failures:

      1) lcd_web::default CentOS installs httpd
      Failure/Error: expect(chef_run).to install_package(‘httpd’)

      expected “package[httpd]” with action :install to be in Chef run. Other p ackage resources:

      # ./spec/unit/recipes/default_spec.rb:23:in `block (3 levels) in ‘

      Finished in 0.63225 seconds (files took 1.68 seconds to load)
      3 examples, 1 failure

      Failed examples:

      rspec ./spec/unit/recipes/default_spec.rb:22 # lcd_web::default CentOS installs httpd

      It failed because there was no httpd package present in the system because our cookbook doesn’t do so. Let’s add httpd installation to our default recipe.

      package ‘httpd’ do
      end
      As the default action is install we do not need to specify anything.
      Now let’s execute the rspec: chef exec rspec
      Now the output will be as follows:

      Finished in 0.73318 seconds (files took 1.67 seconds to load)
      3 examples, 0 failures

    4. Add the following below installs httpd task:
          it 'enables the httpd service' do
            expect(chef_run).to enable_service('httpd')
          end
      
          it 'starts the service' do
            expect(chef_run).to start_service('httpd')
          end
      
      Add the following in default.rb recipe:
      service 'httpd' do
        action [:start, :enable]
      end
      <span 				data-mce-type="bookmark" 				id="mce_SELREST_start" 				data-mce-style="overflow:hidden;line-height:0" 				style="overflow:hidden;line-height:0" 			></span>
      

      Then run: chef exec rspec

    5. Now what if I add something in the default recipe that might break my cookbook:
      add the line in default.rb
      include_recipe ‘something’
      Then run rspec you will get error as below:
      Failure/Error: runner.converge(described_recipe)Chef::Exceptions::CookbookNotFound:
      Cookbook something not found. If you’re loading something from another cookbook, make sure you configure the dependency in your metadata
      # /tmp/chefspec20170918-4764-5svx8rfile_cache_path/cookbooks/lcd_web/recipes/default.rb:7:in `from_file’
      # ./spec/unit/recipes/default_spec.rb:15:in `block (3 levels) in ‘
      # ./spec/unit/recipes/default_spec.rb:31:in `block (3 levels) in ‘Finished in 0.81827 seconds (files took 1.71 seconds to load)
      5 examples, 4 failuresFailed examples:rspec ./spec/unit/recipes/default_spec.rb:18 # lcd_web::default CentOS converges successfully
      rspec ./spec/unit/recipes/default_spec.rb:22 # lcd_web::default CentOS installs httpd
      rspec ./spec/unit/recipes/default_spec.rb:26 # lcd_web::default CentOS enables the httpd service
      rspec ./spec/unit/recipes/default_spec.rb:30 # lcd_web::default CentOS starts the service

So you can see the test will fail if the cookbook dependency failes as well. You can remove the include_recipe line and re execute the rspec.

Test Kitchen Configuration

testkit1testkit2testkit3testkit4testkit5testkit6testkit7testkit8testkit9testkit10testkit11

  1. Edit .kitchen.yml to match the following:
    ---
    driver:
      name: docker
      privileged: true
      use_sudo: false
    
    provisioner:
      name: chef_zero
      # You may wish to disable always updating cookbooks in CI or other testing environments.
      # For example:
      #   always_update_cookbooks: <%= !ENV['CI'] %>
      always_update_cookbooks: true
    
    verifier:
      name: inspec
    
    platforms:
      - name: centos-7.2
      - name: centos-6.8
    
    suites:
      - name: dev
        driver:
          run_command: /usr/sbin/init
        run_list:
          - recipe[la_java::default]
        attributes: { 'java': { 'jdk_version': '7' } }
        driver_config:
          forward:
            - 8080:80
        excludes: centos-6.8
      - name: prod
        driver:
          run_command: /sbin/init
        run_list:
          - recipe[la_java::default]
        attributes: { 'java': { 'jdk_version': '6' } }
        driver_config:
          forward:
            - 8081:80
        includes: centos-6.8
    <span 				data-mce-type="bookmark" 				id="mce_SELREST_start" 				data-mce-style="overflow:hidden;line-height:0" 				style="overflow:hidden;line-height:0" 			></span>
    

    Execute the command kitchen list
    If you get error as follows:
    >>>>>> ——Exception——-
    >>>>>> Class: Kitchen::ClientError
    >>>>>> Message: Could not load the ‘docker’ driver from the load path. Please en sure that your driver is installed as a gem or included in your Gemfile if using Bundler.
    >>>>>> ———————-
    >>>>>> Please see .kitchen/logs/kitchen.log for more details
    >>>>>> Also try running `kitchen diagnose –all` for configuration

    Then install the chef gem kitchen-docker with:  chef gem install kitchen-docker
    Then the kitchen list output will be as follows:
    [root@arati7111 lcd_web]# kitchen list
    Instance Driver Provisioner Verifier Transport Last Action Last Error
    dev-centos-72 Docker ChefZero Inspec Ssh
    prod-centos-68 Docker ChefZero Inspec Ssh
    Then run: kitchen converge
    Then kitchen verify
    Thus java 7 will be installed in centos 7 and java 6 on centos 6

  2. If you comment the includes and excludes statements from .kitchen.yml then we will have 4 test cases.
  3. You can download the java cookbook with
    knife cookbook site download
    and add it to the run_list

Using Test Kitchen

kitchen3kitchen4kitchen5

Execute the following commands

  1. mkdir test_kitchen
  2. cd test_kitchen
  3. kitchen init
    If you get the following error while executing kitchen init:Successfully installed kitchen-vagrant-1.2.1
    Fetching: mixlib-install-3.6.0.gem (100%)
    Successfully installed mixlib-install-3.6.0
    Fetching: ffi-1.9.18.gem (100%)
    Building native extensions. This could take a while…
    ERROR: Error installing kitchen-vagrant:
    ERROR: Failed to build gem native extension./usr/bin/ruby extconf.rb
    mkmf.rb can’t find header files for ruby at /usr/share/include/ruby.h

Gem files will remain installed in /home/user/.gem/ruby/gems/ffi-1.9.18 for inspection.
Results logged to /home/user/.gem/ruby/gems/ffi-1.9.18/ext/ffi_c/gem_make.out

Then execute the kitchen init command with sudo rights

  • If even then the issue is not resolved execute the following command:
    chef gem list kitchen-vagrant
    Then you will have output as follows:  kitchen-vagrant (1.2.1)
    chef gem uninstall kitchen-vagrant -v 1.2.1
    Then rerun kitchen init with sudo rights
  • chef generate cookbook my_cookbook
    You can see that the test/integration/default directory
    In the cookbook you can see the test/smoke/default directory
  • Execute kitchen list
    If you get an error as follows:
    >>>>>> ——Exception——-
    >>>>>> Class: Kitchen::UserError
    >>>>>> Message: Vagrant 1.1.0 or higher is not installed. Please download a package from http://www.vagrantup.com/downloads.html.
    >>>>>> ———————-
    That means you will have to install vagrant. you can use the following command:
    rpm -ivh https://releases.hashicorp.com/vagrant/2.0.0/vagrant_2.0.0_x86_64.rpmThen again execute kitchen list  you will get an output as below:
    Instance Driver Provisioner Verifier Transport Last Action Last Error
    default-ubuntu-1404 Vagrant ChefSolo Busser Ssh
    default-centos-72 Vagrant ChefSolo Busser Ssh
    kitchen7kitchen8kitchen9
  • Execute kitchen create
    You will get an error as follows: The provider ‘virtualbox’ that was requested to back the machine ‘default’
    You will need to install virtualbox first: Follow this tutorial for the same:
    https://apassionatechie.wordpress.com/2017/10/09/install-virtualbox-on-centos-7/kitchen10.png
    If you are not able to install vbox then just change the driver name from vagrant to docker in .kitchen.yml

Then execute kitchen converge
kitchen11.png
Then execute kitchen setup
Then you can execute kitchen destroy to delete the created instances
kitchen13kitchen14

InSpec
inspec1inspec2inspec3inspec4inspec5inspec6inspec7inspec8

Create a cookbook with chef generate cookbook lcd_web. Edit the following file test/smoke/default/default_test.rb. Add the following lines
[ ‘net-tools’, ‘httpd’ ].each do |pkg|
describe package(pkg) do
it { should be_installed }
end
end

Then execute kitchen verify it will fail the test as below:

System Package
∅ net-tools should be installed
expected that `System Package net-tools` is installed
System Package
∅ httpd should be installed
expected that `System Package httpd` is installed

Now lets add the chef code to install these packages
Add the following lines in recipes/default.rb
[ ‘net-tools’, ‘httpd’].each do |pkg|
package pkg do
action :install
end
end

Then execute kitchen converge
Then run kitchen verify
The output should be somthing as below:
System Package
✔ net-tools should be installed
System Package
✔ httpd should be installed

Static Code Analysis

static1static2static3static4static5static6static7static8static9
foodcritic -l
FC001: Use strings in preference to symbols to access node attributes
FC002: Avoid string interpolation where not required
FC004: Use a service resource to start and stop services
FC005: Avoid repetition of resource declarations
FC006: Mode should be quoted or fully specified when setting file permissions
FC007: Ensure recipe dependencies are reflected in cookbook metadata
FC008: Generated cookbook metadata needs updating
FC009: Resource attribute not recognised
FC010: Invalid search syntax
FC011: Missing README in markdown format
FC012: Use Markdown for README rather than RDoc
FC013: Use file_cache_path rather than hard-coding tmp paths
FC014: Consider extracting long ruby_block to library
FC015: Consider converting definition to a Custom Resource
FC016: LWRP does not declare a default action
FC017: LWRP does not notify when updated
FC018: LWRP uses deprecated notification syntax

Now cd to lcd_web cookbook
execute: foodcritic .
output: Checking 2 files
x.
FC008: Generated cookbook metadata needs updating: ./metadata.rb:3
FC064: Ensure issues_url is set in metadata: ./metadata.rb:1
FC065: Ensure source_url is set in metadata: ./metadata.rb:1
FC067: Ensure at least one platform supported in metadata: ./metadata.rb:1
FC078: Ensure cookbook shared under an OSI-approved open source license: ./metadata.rb:1

Lets check our metadata.rb file
For error FC067 add the line in metadata.rb: supports ‘centos’
The other errors are not of our concern so we will exclude them with:
foodcritic . -t ~FC008 -t ~FC064 -t ~FC065 -t ~FC078
There should be no errors.

Now we want to persist these exclusion so we will create .foodcritic file:
vi .foodcritic and add the following lines:

~FC008

~FC064

~FC065

~FC078
Save and close. Then execute foodcritic .

Lets have a look at rubocop. To install rubocop execute gem install rubocop

You can have a look at it’s help with command rubocop –help

If you run rubocop in lcd_we cookbook you might get output as follows:

init_test/lcd_web/recipes/default.rb:7:2: C: Space inside square brackets detected.
[ ‘net-tools’, ‘httpd’].each do |pkg|
^
bin/mixlib-install:11:11: C: Prefer single-quoted strings when you don’t need string interpolation or special symbols.
version = “>= 0”
^^^^^^
bin/mixlib-install:15:32: C: Prefer single-quoted strings when you don’t need string interpolation or special symbols.
str = str.dup.force_encoding(“BINARY”) if str.respond_to? :force_encoding
^^^^^^^^
bin/mixlib-install:17:15: C: Avoid the use of Perl-style backrefs.
version = $1
^^

32 files inspected, 53 offenses detected

You might have errors like “line too long” etc. By default the line length specified in 80, Now let’s see if we can override that setting.
We will be needing todo file for that let’s generate it by the command:
rubocop –auto-gen-config
Then edit the file .rubocop.yml and add the line: inherit_from: .rubocop_todo.yml
Then execute rubocop you will see that all of our offenses have been masked.
Let’s have a look at the .rubocop_todo.yml file.
Now in the code you will see code blocks like follows:

# Offense count: 1
# Cop supports --auto-correct.
# Configuration parameters: EnforcedStyle, SupportedStyles.
# SupportedStyles: auto_detection, squiggly, active_support, powerpack, unindent
Layout/IndentHeredoc:
  Exclude:
    - 'generator/lcd_origin/recipes/recipe.rb'

This denotes that there i an indentation issue with recipe.rb file.
rubocop has the feature to auto-correct small issues, you will need to comment code block one by one and then execute rubocop –auto-correct
Output will be something like: generator/lcd_origin/metadata.rb:5:1: C: [Corrected] 1 trailing blank lines detected.

So in this way we can correct some issues with rubocop. You can repeat the same for rest of the issues.
For the other issues which cannot be correct with rubo-cop you can add rules for the same in .rubocop.yml

We will change the LineLength rule. Add the following in .rubocop.yml file:
LineLength:
Max: 200
Then comment the LineLength block in .rubocop_todo.yml file.

Then execute rubocop You will see there are no offenses.

Troubleshooting
ts1ts2ts3ts4ts5ts6

Edit file recipes/default.rb. Comment everything and add the following:

lazy_message = "Hello World"

file 'lazy_message' do
  path '/tmp/lazy.txt'
  content "#{lazy_message}"
end

execute 'yum-makecache' do
  command 'yum makecache'
  notifies :create, 'file[message]', :immediately
  action :nothing
end

package 'bind-utils' do
  action :install
  notifies :run, 'execute[yum-makecache]', :before
end

file 'message' do
  path '/tmp/message.txt'
  content lazy { "#{lazy_message}" }
end

lazy_message = "Goodbye World"

In this code firstly value of lazy_meesage will be set to Hello World. Then file /tmp/lazy.txt will get generated then makecache block will ge executed which in turn will call the file “message” block. But the file content will be wriiten at last due to lazy tag. Then package bind-utils will be installed but before that yum makecache will be called.
Then execute kitchen converge
kitchen login
lazy.txt will contain “Hello World” and message.txt will contain “Goodbye world”

 

Advertisements

Ansible issues

  1. is not a valid attribute for a Play
    When ever you get the above error firstly crosscheck that the ansible attribute you have mentioned is correct. If it is correct then the issue probably is that you have created tasks as follows:

    ---
       - vars_prompt:
            - name: "var1"
              prompt: "Please pass variable"
              private: no
    
       - fail: msg="var1 is not passed or blank"
         when: var1 is undefined or ( var1 is defined and storeid == "" )

    when it should be as follows:

    ---
       vars_prompt:
          - name: "var1"
            prompt: "Please pass variable"
            private: no
    
       tasks:
         - fail: msg="var1 is not passed or blank"
           when: var1 is undefined or ( var1 is defined and storeid == "" )

    the example referenced is just a task. It is not a valid playbook because it is missings a hosts declaration and the module call is not under a tasks section.

  2. ERROR! conflicting action statements: fail, command
    I get this error if I have a task as follows:
    – name: deploy
    win_get_url:
    url: ‘http://server_ip/builds/build.zip’
    dest: ‘D:\build.zip’
    win_unzip:
    src: D:\build.zip
    dest: D:\You cannot have multiple actions listed inside a single task like this. Instead you need to do this:

        - name: deploy get url
          win_get_url:
             url: 'http://server_ip/builds/build.zip'
             dest: 'D:\build.zip'
        - name: deploy unzip
          win_unzip:
             src: D:\build.zip
             dest: D:\
  3. Ansible task to check API status
    Here I am checking ES cluster health:

        - name: Get ES cluster health
          uri:
            url: http://{{inventory_hostname}}:9200/_cluster/health
            return_content: yes
          register: cluster_status
    
        - set_fact:
            es_cluster_health: "{{ cluster_status.content | from_json }}"
    
        - fail: msg="ES cluster not healthy"
          when: "es_cluster_health.status != 'yellow'"
    
    

    You can compare the status with any string you want. Here I am comparing it with string “yellow”

Install and configure chef server on centos 7

  1. Go to this link: https://downloads.chef.io/chef-server
  2. In the on-premise section select your OS and download your package.
  3. Here I will be using centos 7. When the rpm is downloaded you can install it via the command:
    sudo rpm -Uvh
  4. Execute the following command: chef-server-ctl reconfigure
    Because the Chef server is composed of many different services that work together to create a functioning system, this step may take a few minutes to complete.
  5. Run the following command to create an administrator:
    $ chef-server-ctl user-create USER_NAME FIRST_NAME LAST_NAME EMAIL 'PASSWORD' --filename FILE_NAME
    

    An RSA private key is generated automatically. This is the user’s private key and should be saved to a safe location. The --filename option will save the RSA private key to the specified absolute path.

    For example:

    $ chef-server-ctl user-create stevedanno Steve Danno steved@chef.io 'abc123' --filename /path/to/stevedanno.pem

    If you get the following error:

    Specified config file /etc/opscode/pivotal.rb does not exist

    Then check if you have RAM available. Free some memory and rerun the

    chef-server-ctl reconfigure

  6. If you get following error: /users resource does not exist

    Then check the chef logs with following command:

    chef-server-ctl tail

    You might see an error similar to the following:
    ==> /var/log/opscode/nginx/error.log <==
    2017/09/08 11:38:28 [emerg] 29689#0: bind() to 0.0.0.0:80 failed (98: Address already in use)
    2017/09/08 11:38:28 [emerg] 29689#0: bind() to 0.0.0.0:443 failed (98: Address already in use).

    Then execute the following command:
    netstat -ntlp | grep 80
    Check which service is using the port 80 and 443. Majorly it is the httpd service. Stop the httpd service. Then the chef nginx service will start automatically without any issues.

    Then execute the create-user command it will get executed without any error.

  7. Run the following command to create an organization:
    $ chef-server-ctl org-create short_name 'full_organization_name' --association_user user_name --filename ORGANIZATION-validator.pem
    

    The name must begin with a lower-case letter or digit, may only contain lower-case letters, digits, hyphens, and underscores, and must be between 1 and 255 characters. For example: 4thcoffee.

    The full name must begin with a non-white space character and must be between 1 and 1023 characters. For example: 'Fourth Coffee, Inc.'.

    The --association_user option will associate the user_name with the admins security group on the Chef server.

    An RSA private key is generated automatically. This is the chef-validator key and should be saved to a safe location. The --filename option will save the RSA private key to the specified absolute path.

    For example:

    $ chef-server-ctl org-create 4thcoffee 'Fourth Coffee, Inc.' --association_user stevedanno --filename /path/to/4thcoffee-validator.pem
    
  8. Enable additional features of the Chef server! The packages may be downloaded directly as part of the installation process or they may be first downloaded to a local directory, and then installed.Use DownloadsThe install subcommand downloads packages from https://packages.chef.io/ by default. For systems that are not behind a firewall (and have connectivity to https://packages.chef.io/), these packages can be installed as described below.
    Feature Command
    Chef Manage

    Use Chef management console to manage data bags, attributes, run-lists, roles, environments, and cookbooks from a web user interface.

    On the Chef server, run:

    $ chef-server-ctl install chef-manage
    

    then:

    $ chef-server-ctl reconfigure
    

    and then:

    $ chef-manage-ctl reconfigure
    

    Note

    Starting with the Chef management console 2.3.0, the Chef MLSA must be accepted when reconfiguring the product. If the Chef MLSA has not already been accepted, the reconfigure process will prompt for a yes to accept it. Or run chef-manage-ctl reconfigure --accept-license to automatically accept the license.

  9. Open the chef-manage UI in the browser with https:///login
    Login with the credentials of the user you just created in the above step.
    On the nodes panel if you see the following error:  Error An error occurred, please try again
    Then look at the chef logs with the command chef-server-ctl tail
    The error will be majorly with the nginx. The error that I was facing was as follows:
    FAILED SocketConnector@127.0.0.1:8983: java.net.BindException: Address already in use (Bind failed). The issue was that solr service was already running on my server before installing chef. I had to stop the solr service, the issue was resolved. I was able to see the nodes panel without any error.2017-09-11 11_03_01-Mail
  10. Chef Push Jobs

    Use Chef push jobs to run jobs—an action or a command to be executed—against nodes independently of a chef-client run.

    On the Chef server, run:

    $ chef-server-ctl install opscode-push-jobs-server
    

    then:

    $ chef-server-ctl reconfigure
    

    and then:

    $ opscode-push-jobs-server-ctl reconfigure
  11. Reporting

    Use Reporting to keep track of what happens during every chef-client runs across all of the infrastructure being managed by Chef. Run Reporting with Chef management console to view reports from a web user interface.

    On the Chef server, run:

    $ chef-server-ctl install opscode-reporting
    

    then:

    $ chef-server-ctl reconfigure
    

    and then:

    $ opscode-reporting-ctl reconfigure
  12. If you want to login into postgres database of chef and see the data present in there execute the following command:
    chef-server-ctl psql
    You will get an output similar to:
    [ERROR] You must supply a service name. Valid names include: bifrost, bookshelf, oc-id, oc_erchef, oc_id, opscode-erchef, opscode_chef, push-jobs, reporting
    This is the list of databases you can login and see the data from.
    The main database where the chef data is present is opscode-erchef (previously opscode_chef). You can login to the database with the following command:
    chef-server-ctl psql opscode-erchef
    To see the tables in the database execute the following psql command:
    \dt
    This will give you a list as below:
    Schema | Name | Type | Owner
    ——–+——————————————–+——-+—————
    public | checksums | table | opscode-pgsql
    public | clients | table | opscode-pgsql
    public | containers | table | opscode-pgsql
    public | cookbook_artifact_version_checksums | table | opscode-pgsql
    public | cookbook_artifact_versions | table | opscode-pgsql
    public | cookbook_artifacts | table | opscode-pgsql
    public | cookbook_version_checksums | table | opscode-pgsql
    public | cookbook_versions | table | opscode-pgsql
    public | cookbooks | table | opscode-pgsql
    public | data_bag_items | table | opscode-pgsql
    public | data_bags | table | opscode-pgsql
    public | environments | table | opscode-pgsql
    public | groups | table | opscode-pgsql
    public | keys | table | opscode-pgsql
    public | node_policy | table | opscode-pgsql
    public | nodes | table | opscode-pgsql
    public | opc_customers | table | opscode-pgsql
    public | opc_users | table | opscode-pgsql
    public | org_migration_state | table | opscode-pgsql
    public | org_user_associations | table | opscode-pgsql
    public | org_user_invites | table | opscode-pgsql
    public | orgs | table | opscode-pgsql
    public | policies | table | opscode-pgsql
    public | policy_groups | table | opscode-pgsql
    public | policy_revisions | table | opscode-pgsql
    public | policy_revisions_policy_groups_association | table | opscode-pgsql
    public | roles | table | opscode-pgsql
    public | sandboxed_checksums | table | opscode-pgsql
    public | users | table | opscode-pgsqlYou can see the data in every table with the follwoing command:
    select * from <table-name>;
  13. If you want to see all the settings and configurations chef is using you can see the file:
    vi /etc/opscode/chef-server-running.json
  14. Find chef version:
    head -n1 /opt/opscode/version-manifest.txt
  15. Go to the chef dashboard -> Administration-> Organizations-> Starter Kit – > download starter kit.
    unzip chef-starter.zip
    Then unzip the starter kit in workstation. You can also use your chef server for this purpose. A chef-repo directory will be created.
    cd chef-repo/.chef
    cat knife.rb
    Check the configurations in knife.rb file.
  16. Install chefdk
    rpm -ivh https://packages.chef.io/files/stable/chefdk/2.2.1/el/7/chefdk-2.2.1-1.el7.x86_64.rpm
  17. Execute the following command
    knife ssl fetch 
    If you get an permission denied error then execute the command with sudo privileges.
    After execuing the above command you will get an output similar to the above:
    WARNING: Certificates from vrushabh.novalocal will be fetched and placed in your trusted_cert
    directory (/root/chef-repo/.chef/trusted_certs)……
  18. Knife is used to create cookbooks and to upload the cookbooks to chef server, upload roles, runlist etc.
    Knife allows us to communicate with our chef server.
  19. Lets bootstrap our node to chef server. For this you will need another machine which is accessible from the machine you have unzipped the chef-repo directory. Create a user in the node machine with sudo privileges, which can be ‘ssh’ed from our workstation and chef server.
    Execute the command:
    knife bootstrap 192.168.1.240 -N chef-node –ssh-user user1 –sudo
    Where ‘chef-node’ is the node name i am giving to the node. This can be any name. If -N option is not provided the default nodename will be the hostname of the node.
    ‘user1’ is the user I created on node.
    You might get an error: Your private key could not be loaded.
    cd chef-repo then execute the bootstrap command.
    You will see an output similar to the following:
    ……
    192.168.1.240 Converging 0 resources
    192.168.1.240
    192.168.1.240 Running handlers:
    192.168.1.240 Running handlers complete
    192.168.1.240 Chef Client finished, 0/0 resources updated in 16 seconds
  20. Then goto the chef dashboard and on the nodes panel page you can see the newly added node. In the attributes panel you can see the attributes belonging to the node like CPU,RAM etc.
  21. Create a project in github(ex: chef-fluency-badge). Then cd chef-repo and execute the following command:
    git init
    git add ./
    git config –global user.email “”
    git config –global user.name “”
    git commit -am “Adding chef-repo”
    git remote add origin https://github.com//chef-fluency-badge.git
    git push -u origin master
  22. Create a cookbook with the followig command:
    knife cookbook create learn for chef < 12
    chef generate cookbook for chef => 12
    Add this learn cookbook to git repo with the following commands:
    git add ./
    git commit -m “adding learn cookbook”
    git push -u origin master
  23. Lets create apache cookbook:
    cd chef-repo
    chef generate cookbook cookbooks/apache
    cd cookbooks/apache
    Edit the metadata.rb file and enter your details. Save and close.
    cd recipes
    Edit default.rb and add the following lines:
    ##

    package 'apache2' do
            package_name 'httpd'
            action :install
    end
    
    service 'apache2' do
            service_name 'httpd'
            action [:start, :enable]
    end
    
    
  24. Check the ruby syntax by running the command:
    ruby -c default.rb
    foodcritic default.rb 

    foodcritic should not give any error message.
    Now we want to change the default apache webpage.
    In the same recipes directory create websites.rb with following content:
    ##

    file "default www" do
            path '/var/www/html/index.html'
            content 'Hello world'
    end
    &lt;span 				data-mce-type="bookmark" 				id="mce_SELREST_start" 				data-mce-style="overflow:hidden;line-height:0" 				style="overflow:hidden;line-height:0" 			&gt;&amp;#65279;&lt;/span&gt;

    Check it with foodcritic. You can also comeback to cookbooks directory and execute:
    foodcritic apache

  25. Upload the cookbook to the chef server with the following command:
    knife cookbook upload apache
    You can also add this code to git repo.
  26. Then goto the chef-manage UI, in the policies section you will see the apache cookbook. Policy is basically configuration management for a node.
  27. In your workstation machine execute the knife node list command to get a list of nodes.
  28. Then we need to add our cookbook to run_list. Execute the following command for it:
    knife node run_list add chef-node ‘recipe[apache]’

    Then execute the following command to retreive the list of run_list:
    knife node show chef-node
    The above command gives a summary of the node like recipes, OS version etc. To get a detailed list execute the following command:
    knife node show -l chef-node

  29. Now we will dry-run our cookbook on chef-client:
    chef-client –why-run
    OR
    chef-client -W
  30. After the why-run is successful then execute the cookbook with the following command:
    chef-client

    You can check the httpd service status, it should be running. Then go to the browser and hit the <IP> of the client. It will not show our “Hello World” page but the default apache “Testing 123” page.
    This was because our websites recipe did not execute. Edit the default.rb and add the following line:
    include_recipe ‘apache::websites’

  31. Then upload the apache cookbook to chef-server:
    knife cookbook upload apache
  32. Then run chef-client your websites recipe will get executed.
  33. Suppose if we want to run the websites recipe manually everytime.
    Edit the websites.rb file and remove the include_recipe line we just added above. Then again upload the cookbook to the chef-server.
  34. Then we need to add the websites recipe to the run_list otherwise after running chef-client the websites recipe will not get executed.
    You can add the recipe with the following command:
    knife node run_list add chef-node ‘recipe[apache::websites]’ -a recipe[apache]
    or you can also execute knife node run_list add ‘apache’
  35. Now you can run chef-client it will execute the websites recipe.
  36. The client configuration can be found at /etc/chef/client.rb on the client machine.

Git: Your local changes to the following files would be overwritten by merge

If all local changes can be discarded you can simply run git checkout . to do so. After that you can run git pull or similar commands to pull. To ignore certain files (that are not in the commits already) you can add filenames/directories to .gitignore.

unconfigured table sc hema_keyspaces cassandra

The issue you are having here, is that you are using docs to explore Cassandra that are for version 2.0 and 2.1. My guess is that you are probably using Cassandra 3.0. In that case, you will need to query the system_schema keyspace for the “keyspaces” table:

cassandra@cqlsh:system_schema> SELECT * FROM system_schema.keyspaces;