# irrigate-chef-ci
Chef CI/CD repository

This resource helps to implement CI/CD process to Chef development.

**Overview**

Files in scripts/ directory were created for automatic generation of .kitchen.yml file.
They are mostly useful in CI tools such as Jenkins.

cookbook - intended to add access to the node attributes inside our tests. 
And to check if kitchen user exists and have proper permissions, in case we override 
it in our recipes.

kitchen/Dockerfile - for building base image for kitchen-docker

***Pipeline Library***

src/com/orchard - Groovy source files for shared libraries. Directory should look like standard Java source directory structure. This directory is added to the classpath when executing Pipelines.

vars - directory hosts scripts that define global variables accessible from Pipeline. The basename of each *.groovy file should be a Groovy (~ Java) identifier, conventionally camelCase. The matching *.txt, if present, can contain documentation.

resources - files and scripts used by Pipeline (future replacement for the "scripts" directory). A resources directory allows the libraryResource step to be used from an external library to load associated non-Groovy files.

Read more - https://jenkins.io/doc/book/pipeline/shared-libraries/

**How it works**

***Pipeline Jenkinsfile***

Pipeline defined in a Jenkinsfile and checked in to source control.
The default Jenkinsfile:

```
@Library('jenkins-pipeline-lib') _

styleAndIntegration {
  node_label = 'chef-ci'
  int_tests = true # To enable integration tests
}

cookbookUpload {
  node_label = 'chef-ci'
}
```

It has two steps - one for style check and integration tests, and the second - to upload updated cookbook to all chef servers.

***styleAndIntegration overview***  

Defined in vars/styleAndIntegration.groovy

Stage 1: Jsonlint, cookstyle, foodcritic  
Stage 2: run scripts/ci.sh for integration tests


***Integration Tests flow***

1. ci.sh script searches for files, which were changed in current PR. 
It searches only for changes in recipes and test files (files in test/smoke directory). 
_Recipes and tests should have the same names!_

2. According to found recipes jenkins forms .kitchen.yml file with separate suite for all roles, 
which uses changed recipes. It also changes relative paths used in test files (e.g. template/something.erb -> cookbooks/cookbook-name/template/something.erb). 
So if you changed default recipe of irrigate-orchard_base cookbook, which is used in almost all our roles, 
and we have 50 roles - 50 suites will be created. For each suite kitchen-docker creates separate container.  
NOTE! We have recipes which shouldn't be used in test environments (e.g. which try to mount specific producction only available recource...).
ci.sh script will remove all recipes from `ignore_recipes` file from roles. So to avoid specific recipe to be provisioned during testing,
just add it to scripts/ignore_recipes.

3. During the execution encrypted data bags will be decrypted by decryptor.rb file.
If data bag item key is on of the keys in .secure_keys file, it's value will be replaced with `secure_data`

  
  All suites contains special role for testing purposes. 
  This role stores all node attributes to  /tmp/kitchen/node_attributes.json file, so you can use this file in your tests.
  
```
attribute_file = '/tmp/kitchen/node_attributes.json'
if file(attribute_file).exist?
  node = json(attribute_file).params

  some tests which uses node['default']['attribute name']
end
```     

***cookbookUpload overview***

Defined in vars/cookbookUpload.groovy  
Imports library from src/com/orchard/CookbookUploader.groovy  

Stage 1: call the uploadCookbook method of CookbookUploader class  

***uploadCookbook flow***

1. It verifies ENV to find all required binaries
2. It parses the metadata to find the version and name of the cookbook
3. It extracts a list of chef servers from "knife block list" command
4. It connects to each chef server and decide what action should be performed (upload, update or skip)
5. If action is upload or update it uses "berks install" and "berks upload" commands to upload new version of the cookbook


**How to use locally**  

When you develop integration tests, and want to check if they work, you have to create .kitchen.yml file 
in cookbook root directory. You can use kitchen/.kitchen.yml_example as template.

Because we use chef_zero for provisioner, don't forget to add paths for all cookbook you use in run_list, 
including this cookbook, but without cookbook you are currently testing.
Usually it's relative path, because all our cookbooks are private.
For example:
Berksfile content
    
    source "https://supermarket.chef.io"
    
    metadata
    
    cookbook 'irrigate-chef-ci', path: '../irrigate-chef-ci/cookbook'
    
Please note that chef cookbook for irrigate-chef-ci locates in `cookbook` directory, not in the repo root
 

**Notes**

If you create new data bag item or new role, be sure that this data appears in irrigate-chef-repo before the testing.

If you change “popular" cookbook (e.g. irrigate-php), testing on the Jenkins server could take a while.