The Orchard Cucumber Tests === QuickStart ----- ### OSX If you have a choice, it is strongly suggested that you use OSX for cucumber tests. Change directory to the cucumber_tests repo. #### Ruby 0. Fork and clone this repo 0. Install RVM `curl -ssL https://get.rvm.io | bash -s stable --ruby` 0. Use RVM to install and select Ruby 2.5.3: `rvm install ruby-2.5.3` 0. `source ~/.rvm/scripts/rvm` 0. `rvm use 2.5.3` 0. Install the bundler gem: `gem install bundler` #### Native Dependencies and Gems 0. Install imagemagick using brew: `brew install imagemagick@6` 0. `echo 'export PATH="/usr/local/opt/imagemagick@6/bin:$PATH"' >> ~/.bash_profile` 0. `source ~/.bash_profile` 0. `unset HOMEBREW_BUILD_FROM_SOURCE` 0. Install PostgreSQL: `brew install postgresql` 0. Install MySQL: `brew install mysql` 0. Install mysql2 gem `gem install mysql2` 0. Configure `mysql2` gem to use system libraries: `bundle config --local build.mysql2 "--with-ldflags=-L/usr/local/opt/openssl/lib —with-cppflags=-I/usr/local/opt/openssl/include"` 0. Run `export PKG_CONFIG_PATH=/usr/local/Cellar/imagemagick/$IMAGEMAGICK_VERSION/lib/pkgconfig` 0. `gem install rmagick` 0. `brew link --force imagemagick@6` 0. Install gems using bundler: `bundle install` If attempting to `bundle install` fails around installing mysql2, try: `brew unlink mysql` `brew cleanup` `bundle install` #### Verification 0. Run `rake cucumber:init_dotenv` to create your `.env` 0. Ensure you have access to `s3://nickelback/cucumber-configs` 0. Add your production AWS creds to your .env file 0. Run `rake cucumber:sync_env` to download configs from S3 0. Run `CUCUMBER_PROFILE=grid cucumber features/legacy/oa/oa_edit_sales_goals.feature` to verify that Cucumber is functioning correctly 0. Run `cucumber features/legacy/oa/oa_edit_sales_goals.feature` to verify that Selenium is functioning correctly ### Ubuntu Since ruby-2.5.* is past it's EOL and this automation suite is deprecated but still requires support here are some info on how to run it on Ubuntu 22.04 Ubuntu 22.04 comes with openssl3 but this suite is dependant on openssl1. Don't install ruby via rvm, use rbenv instead. You won't need to downgrade openssl version of Ubuntu 22.04, because the rbenv already uses openssl 1.1.1 for your ruby version. #### Installation 0. Install [rbenv](https://github.com/rbenv/): `sudo apt install rbenv` 0. Install and build with [ruby-build](https://github.com/rbenv/ruby-build): 0. First, download a tarball from https://github.com/rbenv/ruby-build/releases/latest. 0. `tar -xzf ruby-build-*.tar.gz` `PREFIX=/usr/local ./ruby-build-*/install.sh` `ruby-build --definitions` `ruby-build 2.5.3 ~/local/ruby-2.5.3` 0. Cone this repo 0. cd to it 0. `bundle install` 0. ... fight your way through bundle install errors 0. Run `rake cucumber:init_dotenv` to create your `.env` 0. Ensure you have access to `s3://nickelback/cucumber-configs` 0. Add your production AWS creds to your .env file 0. Run `rake cucumber:sync_env` to download configs from S3 0. Run `CUCUMBER_PROFILE=grid cucumber features/legacy/oa/oa_edit_sales_goals.feature` to verify that Cucumber is functioning correctly ### Windows > **_NOTE:_** I don't think it's even possible to run these on WIN-based systems at this point, this is heavily outdated info. But if one wants/needs to try his luck here are some findings from previous expedition long ago Again, it is strongly suggested that you use OSX or Linux instead. Any Windows instructions here are not officially supported by anyone in The Orchard and these instructions are likely to get out of date. Furthermore, The Ruby community doesn't typically support it, there are no actively supported version managers for it, and all of the in-house developers use OSX. If you find these are incomplete or incorrect, then please submit a PR. Courtesy of Vladimir Zhyvoktov: #### Ruby 0. Fork and clone the repo 0. Install `ruby-2.5.3` using the executable 0. [Install DevKit](https://github.com/oneclick/rubyinstaller/wiki/Development-Kit) for `ruby-2.5.3`. 0. Install the bundler gem: `gem install bundler` #### Native Dependencies and Gems 0. Delete `Gemfile.lock` from the tests repository 0. Install gems using bundler: `bundle install`. 0. `nokogiri` may fail when attempting to run tests. If it does, try the following: 0. Run `gem uninstall nokogiri`. 0. Add `gem "nokogiri", ">= 1.6.7.rc"` to your Gemfile. 0. Run `bundle install`. 0. Run `bundle update nokogiri`. [Source](http://stackoverflow.com/questions/28999906/require-cannot-load-such-file-nokogiri-nokogiri-loaderror-when-running) 0. Add the following gems to the gemfile: ``` gem "win32-process" gem "windows-pr" ``` Developers -------------- You can run cucumber scenarios against your local environment for quick acceptance test feedback by using the `developer` profile: `DOMAIN_PROFILE=developer cucumber features/standalone/project_manager.feature` `DOMAIN_PROFILE=developer cucumber features/standalone/physical_product_builder.feature` Configuration --- Configuration is implemented using profiles, located in the `configs/cucumber_profiles` and `configs/domain_profiles` directories. To select a profile, export the name as the relevant ENV variable (`CUCUMBER_PROFILE` and `DOMAIN_PROFILE` respectively). Non-default profiles are merged with default.yml, so don't worry about keeping them complete. all profiles starting with custom (including those nested in a custom folder) are in `.gitignore`, so make use of that for profiles you don't want to put under source control. ### Cucumber Profiles Cucumber profiles store settings such as the browser (Chrome vs. Firefox) and platform (local vs. grid). To select a profile, export the relative path as the ENV variable `CUCUMBER_PROFILE` (ex. `CUCUMBER_PROFILE=grid`, or `CUCUMBER_PROFILE=custom/docker`). See `cucumber_profiles/default.yml` for descriptions of the individual configurations. Profile | Description ------------------- | ----------- `cache_warm` | Run with greatly increased timeouts. Used by the cache warm CI build. `default` | Used unless `CUCUMBER_PROFILE` is set. All other profiles inherit from this. This is a good profile for developing locally. `docker_selenium` | Run from a Selenium node in a local docker container. `docker_compose` | Run from a Selenium node within a Docker Compose network (Jenkins only). `grid` | Run from a Selenium node on the grid. Useful for confirming that tests work on the grid. For example, `cucumber` will run the entire test suite locally because `default` is the profile. However, `CUCUMBER_PROFILE=grid cucumber` will run the entire test suite on the grid. ### Domain Profiles Domain profiles store hostnames for environments against which the cucumber suite is to be tested. To select a profile, export the relative path as the ENV variable `DOMAIN_PROFILE` (ex. `DOMAIN_PROFILE=standalone` or `DOMAIN_PROFILE=production`). See `domain_profiles/default.yml` for descriptions of the individual configurations. Profile | Description ------------------- | ----------- `default` | Run against the QA environment. Used unless `DOMAIN_PROFILE` is set. All other profiles inherit from this. `developer` | Run against local development environments. `linux_docker` | Run against an orchard environment in a local docker container using a linux kernel. `production` | Run against the production environment. Only use this if you know what you're doing. `standalone` | Run against standalone React frontends. `docker_compose` | Run from a Selenium node within a Docker Compose network (Jenkins only). `virtualbox_docker` | Run against an orchard environment in a local docker container using a virtualbox VM. For example, `cucumber features/standalone` will run the standalone suite against the QA environment because `default` is the profile. However, `DOMAIN_PROFILE=standalone cucumber features/standalone` will run against the standalone environments. Currently, we support running tests for the Royalties application against a local environment created via Docker Compose. See [here](docker_compose/royalties/README.md) for details. Run the cukes --- To run ALL legacy Cucumber tests: ``` cucumber features/legacy ``` To run all completed legacy Cucumber tests: ``` cucumber features/legacy --tags 'not @wip' ``` You can pass in environment variables like CUCUMBER_PROFILE from the command line: ``` cucumber CUCUMBER_PROFILE=grid features/legacy ``` To run a specific Cucumber feature: ``` cucumber features/legacy/create_volume.feature ``` You could even do a single, specific scenario: ``` cucumber features/legacy/create_volume.feature:7 ``` to run the scenario beginning at line 7 on the feature file. Similarly, you could run the scenarios beginning at lines 7 and 16, back-to-back: ``` cucumber features/legacy/create_volume.feature:7:16 ``` Tags: Tag | Description -------- | ----------- @slow | Tests with runtime > 2 minutes but < 10 minutes @glacial | Tests with runtime > 10 minutes @ignore | Tests that aren't run by the main Jenkins builds @wip | Pending or failing tests that need work from a developer or QA Note that all standalone features are also tagged with any frontends and microservices they may interact with. ## Folder structure We use a somewhat unconventional directory structure for our feature files. Features are separated into different directories depending on their purpose. ### Legacy Legacy tests run against Workstation and OA, which are very slow. They run once every two hours with the `cucumber-tests-master-runthrough` Jenkins build. ### Standalone For frontend services such as mobile-analytics and frontend-distribution, tests can be run against standalone QA environments. These tests execute much more quickly than those on qaorch, and are useful for running after deployments to QA to gate deployments to production. To develop on them, you must use the `standalone` domain profile. See [here](https://docs.google.com/document/d/1vB0hkOrDt_VKiiHr8uqFHz6nDN3nGamDa3B4qa-pNR0/) for details on how to integrate new standalone tests into deployment workflows, and always remember to update the [Microservices, ETLs, and Frontends](https://docs.google.com/spreadsheets/d/1c-txc_SMJyVT5Z4NN8aQpWz6jbFDdTEAr2-w9_YfEVo/) spreadsheet with any new changes. ### Scripts Scripts unrelated to tests are located in the `features/scripts` directory. They are useful for automating manual testing processes and returning the QA environment to a testable state TODO: Consider switching to plain watir scripts instead of cucumber. ## [Best Practices](https://docs.google.com/document/d/1-M8R_SUNxUHbiF2fcR-gbKiYHharZMOKuVSbogjCvAk/edit#heading=h.bjwhqhwj5oo6) ## STYLE 1. Our ruby style guide can be found [here](https://github.com/theorchard/docs/blob/master/styleguides/ruby.md) 2. Our gherkin style guide can be found [here](https://github.com/theorchard/docs/blob/master/styleguides/gherkin.md) ## Known Issues and Other Gotchas 1. SublimeLinter-rubocop does not play nice with RVM: https://github.com/attenzione/SublimeLinter-scss-lint/issues/5. The workaround that worked for me was to do: `rvm @global do gem install rubocop -v 0.27.1` (and if debugger complains it needs another rubocop version then install that one instead) ## I'm New Here I Have Questions Please check out our [onboarding documentation.](https://docs.google.com/document/d/1LXwP8F-ZIiuzlWv63nnde2IOwJSWCk5UoysQPowIxj0/edit#heading=h.9o7zblp3f40) It should answer everything from questions about the music business to our philosophy in regards to automated testing at the Orchard. Don't forget to check out QA team Google Drive [folder](https://drive.google.com/drive/folders/0ADTYk5b2IkMdUk9PVA) - a lot of useful information, faq's, howto's and knowledge it contains.