# mobile-test-automation
## End-to-end test framework (Behave + Appium) for Sony Music's mobile apps (Orchard, AWAL, SME), running on iOS and Android.
Tests run either against a local simulator/emulator with Appium and Python, or against a real device on BrowserStack (locally or via Docker/CI).

*Requires macOS with Xcode installed for local iOS development.*

Skip to `Running on Browserstack` if running tests on local emulators isn't required

### Table of contents

* [Repo structure](#repo-structure)
* [Setup shared dependencies](#setup-shared-dependencies)
* [Environment variables (`.env`)](#environment-variables-env)
* [Prepare the environment for iOS](#prepare-the-environment-for-ios)
* [Prepare the environment for Android](#prepare-the-environment-for-android)
* [Run the tests locally](#run-the-tests-locally)
* [Running on Browserstack](#running-on-browserstack)
* [Test tagging conventions](#test-tagging-conventions)
* [Linting](#linting)
* [CI/CD](#cicd)

### Repo structure

* `features/` - Gherkin `.feature` files (one set per OS, e.g. `android_*.feature` / `ios_*.feature`) and `features/steps` - the Behave step definitions
* `page_objects/` - Page Object Model classes, split into `android_pom/`, `ios_pom/` and shared `mixins/`
* `helpers/` - driver setup, BrowserStack integration, Datadog/JUnit reporting, secrets, test data, gestures, etc.
* `scripts/` - shell scripts used by the `Makefile` (emulator startup, binary download, Docker orchestration, BrowserStack Local)
* `run.py` - uploads the app binary to BrowserStack and drives the `browserstack-sdk behave` run used by the `local_browserstack_*` make targets
* `apps/` - local folder for app binaries (`.app`/`.ipa`/`.apk`), not checked into git

### Setup shared dependencies:

* Install Appium and required drivers:

  `npm install -g appium@next`

  `appium driver install xcuitest`

  `appium driver install uiautomator2`

* Install python dependencies:
  `virtualenv venv`
  `source ./venv/bin/activate`
  `pip install -r requirements.txt`

* Copy shadow `.env` file, add username and password
  `cp .env.shadow .env`

  or download `.env` from S3:
    `make sync_env`

* Create folder for applications binaries for local testing:
  `mkdir apps`

### Environment variables (`.env`)

| Variable | Description |
| --- | --- |
| `BROWSERSTACK_USERNAME` / `BROWSERSTACK_ACCESS_KEY` | Credentials for a BrowserStack user with access to our account |
| `BRAND` | App brand under test: `orchard`, `awal` or `sme` |
| `OS` | `ios` or `android` |
| `EXTENSION` | Binary extension to download: `.apk` for android, `.ipa` for ios |
| `PREFIX` | Prefix used when looking up the binary in S3 (defaults to `e2e`) |
| `SKIP_BINARY_DOWNLOAD` | Set to `true` to skip downloading the app binary from S3 |
| `TAGS` | Optional Behave tag expression to filter which scenarios run |

### Prepare the environment for iOS:

* Install the latest version of Xcode and xcode tools:
  `xcode-select --install`

* Install libraries:
  `brew install carthage`
  `brew install libimobiledevice`
* `brew install ffmpeg`

* Download desired [`.app`] file from aws s3 bucket (https://s3.console.aws.amazon.com/s3/buckets/orcd-mobile-binaries?region=us-east-1&tab=objects)
  and save it in the `apps` folder at the root

* The local simulator caps in `ios_caps.json` specify a `deviceName`/`platformVersion` - create/boot a simulator matching those values, or edit `ios_caps.json` to match a simulator you have available (prefer the latest iOS version your Xcode install supports)

### Prepare the environment for Android:

* Download and install Android Studio from the website:
`https://developer.android.com/studio`

* Launch Android Studio, open preferences and open `SDK Tools` under the `Android SDK` tab. Ensure the following tools are installed:
`Androiod SDK Build-Tools`
`Android SDK Command-line Tools`
`Android Emulator`
`Android SDK Platform-Tools`
`Intel x86 Emulator Accelerator`

* Use the Android Virtual Device manager to add a new device, using the latest available Android version. Name the device to match the `deviceName` in `android_caps.json` (or edit `android_caps.json` to match the device/version you created)

* Set up the JAVA_HOME env variable pointing to the home dir of the latest installed JDK. E.g. `/Library/Java/JavaVirtualMachines/[version].jdk/Contents/Home`

* Set up the ANDROID_HOME env variable pointing to `path/to/Android/sdk`

* Add the following to path:
`export PATH=$PATH:$ANDROID_HOME/tools/bin`
`export PATH=$PATH/:$ANDROID_HOME/platform-tools`
`export PATH=$PATH/:$ANDROID_HOME/build-tools/[version]`
`export PATH=$PATH/:$ANDROID_HOME/emulator`

* Download desired [`.apk`] file from aws s3 bucket (https://s3.console.aws.amazon.com/s3/buckets/orcd-mobile-binaries?region=us-east-1&tab=objects) and save it in the `apps` folder at the root

#### Run the tests locally:

`make local_ios_tests -e -k`
`make local_android_tests -e -k`

These run the `ios_basic_flow.feature` / `android_basic_flow.feature` suites (excluding scenarios tagged `@wip`) and write cucumber JSON / JUnit reports to `cucumber_results` and `junit`.

### Running on Browserstack:

In order to run on browserstack you will need a user with access to our account. The dockerfile provides all required dependencies
to run local tests against browserstack. Before running the make command, ensure your `.env` file has been filled out (including `BRAND` and `OS`), and you
have generated aws tokens with `awsume -o`.

Then run the tests using:

`make browserstack_docker`

This downloads the app binary from S3, cleans previous reports, and runs the suite for the OS/brand configured in `.env` inside the docker container defined in the `Dockerfile`'s `deploy` target.

Below are the setup steps for running browserstack on your local machine without docker, 
none of the above setup is required to run on browserstack.

* Install python dependencies:
  `virtualenv venv`
  `source ./venv/bin/activate`
  `pip install -r requirements.txt`

* Install node dependencies (needed for Datadog JUnit upload):
  `yarn install`

* Download the local browserstack binary from https://www.browserstack.com/docs/local-testing/releases-and-downloads <br />
Then run using the below command, subbing in your browserstack key:
  `./BrowserStackLocal --key [your_browserstack_key] --local-identifier ORCHARD_DEV --enable-logging-for-api`

* `cp .env.shadow .env` <br />
Fill out the username and password of the mobile test user in the env file. Ensure you have 
added your `BROWSERSTACK_USERNAME` and `BROWSERSTACK_ACCESS_KEY`, as well as `BRAND` and `OS`.

 Run tests using the `make local_browserstack_android` / `make local_browserstack_ios` commands, which run the full feature set for that OS via `run.py` (uploads the binary to BrowserStack, then drives `browserstack-sdk behave`).

### Test tagging conventions

* `@wip` - excluded from all `make *_tests` runs (`--tags=~@wip`)
* `@skip` - scenario is always skipped
* `@skip_if_sme` - scenario is skipped when the SME app binary is under test
* `@employee` - runs the scenario as an employee test user instead of a client
* `@orchard-app` / `@awal-app` - restricts a feature to the named brand(s); skipped when `BRAND` doesn't match
* `@language` - on failure, resets the app back to its default language before the next scenario
* `@smoke` - smoke-test subset (combined with `@android`/`@ios`)

### Linting

* `make lint` runs `flake8` against `page_objects`, `features` and `helpers`
* `make docker_lint` runs the same lint inside the `lint` target of the `Dockerfile`/`docker-compose.yml`
* Pre-commit hooks (`.pre-commit-config.yaml`) run trailing-whitespace/EOF/YAML checks plus `make lint` on Python files - install with `pre-commit install`

### CI/CD

The `Jenkinsfile` defines the pipeline: it lints via `make docker_lint`, and on `master` builds and pushes the `deploy` Docker image to ECR (`mobile-test-automation`). Pipeline failures notify the `#e2e-test-results` Slack channel. Test runs upload JUnit results to Datadog (`helpers/datadog_helper.py`) when running under Jenkins (`JOB_NAME` is set).