# S3 helper
<p>
	There are 4 main functions here:
	<br>1. look() takes in a directory path (in s3) and lists all files/keys in the directory.
	<br>it also takes wildcards for a regex-like search.
	<br>2. disk_2_s3() loads files from disk to s3
	<br>3. read_[csv/json]() reads a s3 object into memory as a Pandas dataframe
	The syntax is identital to Pandas <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html">read_csv()</a> and <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_json.html">read_json()</a>, with a few features stripped out.
	<br>4. to_[csv/json]() writes a dataframe from memory to an s3 location.
</p>
## Installation
Configure s3 as you would for boto3.
<a href="http://boto3.readthedocs.io/en/latest/guide/configuration.html">read here</a>
Environment Variables or AWS Config File seems to work best.

## Usage
To install requirements.
```pip install -r requirements.txt```

In bash/terminal- traverse to the directory of the module and copy the PATH using: `pwd|pbcopy`

To use in iPython environments like Jupyter
you can include these lines in each script:
```
import sys
sys.path.append("PATH")
import s3

df = s3.read_csv("s3://bucket_name/key/file_name.tsv.gz", 
                 sep='\t', compression='gzip')
```

For continued use, the PATH should be added to the iPython startup script

```
cd ~/.ipython/profile_default/startup
vim first.py
sys.path.append("PATH")
```

For regular scripting, add the module to the to Python path:

```
export PYTHONPATH="${PYTHONPATH}:PATH"
```

To see how this module works, check out <a href="https://github.com/theorchard/datalytics/blob/master/Modules/s3/tutorial.ipynb">tutorial.ipynb</a> for a walkthrough of use-cases.
## Contributing
1. Fork it!
2. Create your feature branch: `git checkout -b my-new-feature`
3. Commit your changes: `git commit -am 'Add some feature'`
4. Push to the branch: `git push origin my-new-feature`
5. Submit a pull request :D

## Credits
Written by Leon Yin

## License
MIT