# Python styleguide

The python styleguide follows two styleguides: [pep8](https://www.python.org/dev/peps/pep-0008/) and [Google Python Styleguide](https://google.github.io/styleguide/pyguide.html). Unless mentioned otherwise, those two styleguides applies to any python code that is written at The Orchard.

**Table of Content**

1. [Readability](python.md#readability)
   * [Lines](python.md#lines)
   * [Naming conventions](python.md#naming-conventions)
   * [Indentation](python.md#indentation)
   * [Imports](python.md#imports)
   * [Default Argument Values](python.md#default-argument-values)
   * [Statements](python.md#statements)
   * [List Comprehension](python.md#list-comprehension)
   * [Parenthesis](python.md#parenthesis)
   * [Quotes](python.md#quotes)
   * [Documentation](python.md#documentation)
2. [Other guidelines](python.md#other-guidelines)
   * [Nested Statements](python.md#nested-statements)
   * [Strings](python.md#strings)
   * [Flake8: Our python linter](python.md#flake8-our-python-linter)
   * [Project Organisation](python.md#project-organisation)
   * [Testing tools](python.md#testing-tools)

## Readability

### Lines

#### Line length

We use the pep8 standard for line length. This means line length is a maximum of 79 characters. Please do not modify python linting tools to use any other line length maximum.

#### Line endings

We do not use semicolons to indicate the end of a line.

Do:

```python
my_age = 30
```

Don't:

```python
my_age = 30;
```

### Naming conventions

Our naming conventions are lowercase with underscore for everything except classes, exceptions and we use uppercase with underscore for all globals and constants.

Do:

```python
CONSTANT_VALUE = 'something'

class BeautifulHouse():
    pass

my_house = BeautifulHouse()
```

Don't:

```python
constantValue = 'something'
class beautiful_house():
    pass

myHouse = BeautifulHouse()
```

#### Avoid unnecessary abbreviations or ambiguous names.

Use only absolutely common abbreviations and try to avoid ambiguous names.

Do:

```python
SNOWFLAKE_DB_NAME = 'something'
DESTINATION_TABLE_NAME = 'foo_bar'
SECONDARY_IDX_COLUMN = 'date'
OS_NAME_REGEXP = '$Linux\s^'

def copy_into(source_url, table_name, import_file_format=None):
    pass
```

Don't:

```python
SF_DB = 'something'
DEST = 'foo_bar'
PATTERN1 = '$Linux\s^'

def copy_into(data, tbl, fmt=None):
    pass
```

#### Use of Leading Underscores in Method Names and Instance Variables

As per PEP-0008:

* Use one leading underscore for non-public methods and instance variables.
* Avoid using two leading underscores except for special situations \(avoiding name conflicts with attributes in classes designed to be subclassed\).

### Indentation

Expanding on the [Google Python Styleguide](https://google.github.io/styleguide/pyguide.html?showone=Indentation#Indentation) and [PEP8](https://www.python.org/dev/peps/pep-0008/#indentation) Indentation sections, when dealing with implied line continuation and functions, always use the '4-space hanging indent; nothing on first line' strategy, not the 'aligning wrapped elements with the opening delimiter' strategy. The 4-space hanging indent strategy allows one to more easily rename variables and methods without readjusting spacing, and makes for a more consistent coding style.

Do:

```python
foo = long_function_name(
    var_one, var_two, var_three, var_four)

def a_long_function_name(
        arg1, arg2, arg3, arg4):
    """Comments here."""
    pass

def long_function_with_long_arguments(
        long_arg1, long_arg2, long_arg3,
        long_arg4, long_arg5, long_arg6):
    """Comments here."""
    pass
```

Don't:

```python
foo = long_function_name(var_one, var_two,
                         var_three, var_four)

def another_long_function_name(
                    arg1, arg2, arg3, arg4):
    pass

def long_function_with_long_arguments(
        long_arg1,
        long_arg2,
        long_arg3,
        long_arg4,
        long_arg5,
        long_arg6):
    pass
```

### Imports

The header of your files should contain:

* One import per line.
* Standard Python libraries go first.
* Third-party imports go next \(i.e. anything you had to install\).
* Local application/library imports go last \(i.e. anything in your code base\).
* Separate each group of imports with an empty line.

All imports should be sorted alphabetically \(easier to read.\) Avoid \(unless absolutely necessary\) to import classes. Only import modules.

Do:

```python
import os
import urllib.parse

import apple
from google import mail
from google import search

from project import feature
from project import service
```

Don't:

```python
import os, sys
```

Don't:

```python
import os
from project import feature
```

Don't:

```python
from project import feature
import os
```

Don't:

```python
from project import service
from project import feature
```

### Default argument values

Similar to the [Google Styleguide](https://google-styleguide.googlecode.com/svn/trunk/pyguide.html?showone=Default_Argument_Values#Default_Argument_Values), you can set default argument values as long as those are immutable.

Do:

```python
def foo(a, b=None):
    if not b:
        b = []

# or

def foo(a, b=None):
    b = b or []
```

Don't:

```python
def foo(a, b=[]):  # lists are mutable.
def foo(a, b=time.time()):  # The time the module was loaded???
def foo(a, b=FLAGS.my_thing):  # sys.argv has not yet been parsed...
def foo(a: str): # no need for annotations
```

### Statements

Only one statement per line, no exception.

Do:

```python
if foo:
    bar(foo)

try:
    something()
except Exception:
    pass
```

Don't:

```python
if foo: bar(foo)

try: something()
except Exception: pass
```

### List comprehension

List comprehensions and generator expressions provide a concise and efficient way to create lists and iterators without resorting to the use of map\(\), filter\(\), or lambda. You can use them for simple cases.

Do:

```python
result = []
for x in range(10):
    for y in range(5):
        if x * y > 10:
            result.append((x, y))

for x in xrange(5):
  for y in xrange(5):
      if x != y:
          for z in xrange(5):
              if y != z:
                  yield (x, y, z)

return ((x, complicated_transform(x))
        for x in long_generator_function(parameter)
        if x is not None)

squares = [x * x for x in range(10)]

eat(jelly_bean for jelly_bean in jelly_beans
    if jelly_bean.color == 'black')
```

Don't \(it looks smaller but more difficult to follow / understand\):

```python
result = [(x, y) for x in range(10) for y in range(5) if x * y > 10]

return ((x, y, z)
    for x in xrange(5)
    for y in xrange(5)
    if x != y
    for z in xrange(5)
    if y != z)
```

### Parenthesis

Python doesn't require parenthesis to work, you can omit them. Always make sure to use them whenever absolutely necessary. For example:

```python
if (not len(house.windows) < MAX_HOUSE_WINDOW_COUNT and
        not len(house.floors) > MIN_HOUSE_FLOOR_COUNT):
    pass
```

### Quotes

It depends on the context. If you're trying to write regular strings, you should use single quotes \(unless you have to escape\). Double quotes should be used for documentation.

```python
def hello(name):
    """Some comment."""
    # note the '
    return "C’est la vie de {name}".format(name=name)

# single quotes
hello('Christian')
```

### Documentation

For python documentation \(and comments in general\), we follow the Sphinx contrib [Napoleon](https://pypi.python.org/pypi/sphinxcontrib-napoleon/), which is a standard at Google. After each doc block comment, add one empty line for readability.

#### Modules

Add a block at the beginning of the file containing a title and a long description with what the module is about.

```python
"""Analytics handlers.

The analytics handlers provide endpoints to facilitate metric grouping and
fetching. Some of those endpoints are currently used as replacements of VAPI,
which requires an extra layer of information (vapi envelope).
"""

from flask import abort
from flask import request

# more code below
```

#### Functions

Do:

```python
def func(arg1, arg2):
    """Summary line.

    Extended description of function.

    Args:
        arg1 (int): Description of arg1.
        arg2 (str): Description of arg2.

    Returns:
        bool: Description of return value.

    Raises:
        UnluckyError: Luck was not on your side.
    """
    if not random.choice([True, False]):
        raise UnluckyError()
    return True

def function():
    """Single summary line example."""
    pass
```

Don't:

```text
:param path: The path of the file to wrap
:type path: str
:param field_storage: The :class:`FileStorage` instance to wrap
:type field_storage: FileStorage
:param temporary: Whether or not to delete the file when the File
   instance is destructed
:type temporary: bool
:returns: A buffered writable file descriptor
:rtype: BufferedFileStorage
```

**Test Functions**

For test functions you do not need a full pydoc, because most of the parameters are fixtures. Just a description is adequate.

#### Generators

Yields for generators can be documented as you expect.

```python
def count_to_ten(i):
    """Create a generator to count to 10.

    Args:
        i (int): seed value

    Yields:
        int: counting value.

    Raises:
        StopIteration: the number went above ten.
    """
    while True:
        if i > 10:
            raise StopIteration()
        yield i
        i += 1
```

#### Namedtuple

The generated namedtuple subclass is a call instead of a declaration, so you cannot inject docstrings as usual.

After creating the subclass, manipulate the `__doc__` attribute directly. The class level docstring already has some useful data, so you might want to append to it.

Please note that editing docstrings on namedtuples is only allowed for python 3.5 and onward.

```python
import sys


if sys.version_info >= (3, 5):
    Book = namedtuple('Book', ['id', 'title', 'authors'])
    Book.__doc__ += ': Hardcover book in active collection.'  # this is appended
    Book.id.__doc__ = '(int): 13-digit ISBN.'
    Book.title.__doc__ = '(str): Title of first printing.'
    Book.authors.__doc__ = '(list): List of authors sorted by last name.'

print(Book.__doc__)
# 'Book(id, title, authors): Hardcover book in active collection.'
```

## Other guidelines

### Nested Statements

Nested structures \(conditionals, statements, loops\) make the code harder to understand than flatter structures with multiple exits predicated with guard clauses. Keeping your code as flat as possible makes it easier to review, and easier to keep bug free.

Variable definitions within nested structure are also problematic as they may be referenced outside. As a rule of thumb, always define a default for each variable used within a nested/looped structure.

Do:

```python
# Loops
for user_id in range(20):
    if user_id % 2:
        continue

    user = get_user_by_id(user_id)
    if user.username == 'fred':
        user.sleep()
    elif user.username == 'ioda':
        user.sing()


# Conditionals
def get_user_username_by_id(user_id):
    """Get a user's username by its id.

    Args:
        user_id (int): the user's id.

    Returns:
        str: the user's username.
    """
    if not user_id:
        return ''

    user = db.find({'_id': user_id})
    if user:
        return user.username
    return ''


def get_followers(account):
    """Get follower count.

    Args:
        account (dict): account information.

    Returns:
        int: the number of followers.
    """
    if account.get('followers'):
        return account.get('followers')

    if account.get('superstar'):
        return account.get('follower_count')
    return 0


# Variable definition
def get_user_last_name(user):
    """Get a user's last name.

    Args:
        user (dict): the user information.

    Returns:
        str: the user's last name.
    """
    last_name = ''
    if not user:
        return last_name

    last_name = user.get('last_name', '')
    if not last_name:
        last_name = user.get('family_name', '')
    return last_name
```

Don't:

```python
# Loops
for user_id in range(20):
    if user_id % 2:
        user = get_user_by_id(user_id)
        if user.username == 'fred':
            user.sleep()
        elif user.username == 'ioda':
            user.sing()

# Conditionals
def get_user_username_by_id(user_id):
    """Get a user by its id."""
    if user_id:
        user = db.find({'id': user_id})
        if user:
            return user.username


def get_followers(account):
    """Get follower count.

    Args:
        account (dict): account information.

    Returns:
        int: the number of followers.
    """
    if account.get('followers'):
        return account.get('followers')
    else:
        if account.get('superstar'):
            return account.get('follower_count')
        return 0


# Variable definition
def get_user_last_name(user):
    """Get a user's last name.

    Args:
        user (dict): the user information.

    Returns:
        str: the user's last name.
    """
    if user:
        if not user.get('last_name'):
            last_name = default_last_name
        else:
            if user.get('family_name'):
                last_name = user.get('family_name')
    # Because this variable is defined only in a nested statement, if the
    # user is not set (None), last_name is not defined and will throw an error.
    return last_name
```

### Strings

Strings in python are by nature not mutable, so any operation done to the string ends up creating a new one \(even if it's just to lower case it\). Two operations frequently done on strings are formatting and concatenation.

Formatting is an explicit operation which creates a string from a generic reusable template that contains placeholders. For instance: `{first_name} is happy.`

Concatenation is the juxtaposition of two or more strings together. In which case, the `+` is often misused \(any mathematical operation requires parts to be executed independently, looping several times on the same words\).

Do:

```python
# Formatting
GRASS_URL = 'https://grassurl/{path}/'
GRASS_URL.format(path='path')

# an alternative but valid solution

import io

output = io.StringIO()
output.write('val1')
output.write('val2')
output.write('val3')
result = output.getvalue()
output.close()


# Concatenation
result = ''.join(['stringA', 'stringB', 'stringC'])
```

Don't:

```python
# Formatting
GRASS_URL = 'https://grassurl/'
GRASS_URL + 'path'  # this is string concatenation, not formatting.

# Concatenation
result = 'stringA' + 'stringB' + 'stringC'
```

### Flake8: Our python linter

Most of the common errors can be caught by using `flake8` \(works with all versions of python.\) You can install it using `pip install flake8`, and running it is as easy as `flake8 folder/`. All python projects should use this linter to check the styles on pull requests.

Additionally, the following flake8 plugins should be installed to help automate checking other parts of our styleguide.

* flake8-docstrings - Check [documentation](python.md#documentation) standards \(and because no one wants to look for periods at the end of sentences\).
* flake8-import-order - Alphabetize and seperate those imports correctly. [See below](python.md#check-import-styling-via-flake8-import-order) for config instructions needed to use this plugin. 
* flake8-quotes - Ensure we use [single quotes](python.md#quotes). 

#### Check Import Styling via flake8-import-order

For those who want flake8 to also check their [import styling](python.md#imports), one can use the [flake8-import-order package](https://pypi.python.org/pypi/flake8-import-order) by:

1. Installing the package via `pip install flake8-import-order`.  
2. Adding a `.flake8` file to the top of their project formatted as:  

```text
[flake8]
application-import-names=<LOCAL_MODULE_NAME_1>,<LOCAL_MODULE_NAME_2>
import-order-style=google
```

\(where the application-import-names option is a comma separated list of names that should be considered local to your application\)

### File structure

Example a file structure:

```python
"""Title of the file.

Description of the file. It may contains some code samples in the case of a
generic util.
"""

from google import exceptions
from google import search


def perform_google_search(query):
    """Description of the method.

    Args:
        query (str): The query to send to Google.

    Returns:
        dict: The response object that was sent back from Google.

    Raise:
        Exception: if no response is found.
    """
    response = search(query)
    if response:
        return response.data
    raise exceptions.NoResponse()
```

### Project organisation

If you need to create a new project, this is the organization we usually go with. By separating the project and the tests it allows one to easily execute tests & produce code coverage reports. The file `requirements.txt` contains all the dependencies of the project \(including the ones for the tests\) and `setup.py` contains information about the project.

```text
├── project_name
│   └── file.py
├── requirements.txt
├── setup.py
└── tests
    └── some_test.py
```

### Testing tool

Because of its ability to be extended, we use [py.test](http://pytest.org/latest/) as our main testing tool \(please avoid using unittest.TestCase\). It provides runners, fixtures, mocking and have several extensions \(for coverage etc.\)