# ![image](img/infamous.png)  

## Introduction
To access the GitHub repo visit [CEA-Infamous](https://github.com/SME-BUS/cea-infamous).

What you'll find inside:

* **sql-snipit**: Find here shared sql queries by the Kommunity
* **djagitit package**: a shared python package ([djagitit Reference Guide](djagitit/welcome.md))
* **projects-cea**: Find up and running shared projects maintained by some great people from the Kommunity
* **tools**: a bunch of automated scripts to help with your development workflow






## Getting Started
???+ warning "Pre-requisites 1/2"
    To access the repo and leverage its infamous content you'll need:

    * to have a **SME GitHub account**.
    > If you don't, you need to reach to **Tom Koshy** for his team to set you up.
    * to be granted access on the repo.
    > Reach to **Julien** or **Matti** once you get your SME GitHub account 

    **You're good to go if:**   
    
    + You only want to see the **sql-snippit queries**    
    + You just want have a look at the **content of the repo online**.
    + You only want to use the **djagitit package** but **don't want to use the repo as your codebase** (skip to [the djagitit package](#the-djagitit-package) section)
???+ warning "Pre-requisites 2/2"
    **This section is aimed at people who want to clone the repo on their machine to work on python projects**

    * **Git** installed on your machine
    * **Python (>=3.10)** installed on your machine
    * Get access to the **api-server** if you want to be able to deploy and schedule scripts in production  
    > If you don't, you need to reach to **Tom Koshy** for his team to set you up.


In order to work with the cea-infamous repo, there's a simple 2-steps process :

+ Clone the repo
+ Configure your ```.env``` file

Then you'll be ready to work (we recommand to follow our [development workflow]() recommandations) and be able to use the [tools scripts]() and the [djagitit package]()


### clone the repo
???+ note
    **If you use an IDE** (such as [VSCode](https://code.visualstudio.com/)), it might includes functionnalities to do the following directly in the IDE.   
    We'll document here how to do it from the command line.
    
!!! success "Follow the steps"
    
    + Open a terminal   
    + Change the current directory to the location where you want to clone the directory   
    ```   
    cd ./THE_DIRECTORY_I_WANT   
    ```   
    + Run the clone command   
    ```
    git clone https://github.com/SME-BUS/cea-infamous.git
    ```   
    + Congrats!   
    You now have a ```cea-infamous``` directory under ```THE_DIRECTORY_I_WANT```

That's it ! You now need to configure your .env file.  


### configure your .env file
The **.env** file is a configuration file where you will store:   

+ paths to common directories to use accross your projects   
( a directory to store virtual environments, a log directory, a directory to store data inputs and outputs, ...)   
+ credentials to access databases   
+ keys, secrets and credentials to access APIs   
+ ...

The purpose of this configuration file is to store all your sensible information in one place.   
All you scripts would access the information from this file, which allow you not to store any critical information in your scripts, notebooks...    
It will also allow us to easily share code. The same script will use the appropriate user credentials depending on the ```.env``` file that exists on the machine it runs on.
???+ warning "Confidentiality"
    It's important to remember that this file is **stricly personnal** and will **not be commited in the repo nor stored on GitHub**

**It's mandatory to fill some informations within the file in order to work with the cea-infamous repo:**       

+ **The ```djagigit``` package will look for a ```.env``` file to get your information to log to databases, apis, tableau...**   
+ **The ```tools``` scripts will look for a ```.env``` file to get paths and credentials when needed.**

!!! success "Follow this steps"
    + copy the ```sample.env``` file present in the repo as ```.env```
    + open the ```.env``` file with a text editor and fill in the appropriate values (see below for the list of variables)   
    You only need to fill the variables you intend to use and can leave the others blank.    
    You can add your own variables at the end of the file if you need some for your projects.

???+ info "Location of the .env file"
    You can store your ```.env``` file in the location of your choice between:   
    
    + the root of the actual repo (this is safe because the ```.env``` will not be commited in the repo)
    + in any parent directory of the repo (will allow using the same ```.env``` file across several repos if needed):   
    
        + up until your home directory (because the ```tools``` will stop searching above that point):    
            ```/Users/comt001/``` *on macOS*        
            ```C:\Users\comt001``` *on Windows*   
            ```/home/francesys``` *on a Linux system*
        
        + if your repository is not under your home repository, please consider storing the ```.env``` file not too far above the repo to prevent long time searchs

!!! tip "Variables in the .env file"
    ??? abstract "local paths"
        | Variable | Description |
        |----------|-------------|
        |INFAMOUSDIR| The path to your repository |
        |ENVDIR| The directory to store your python virtual environments |
        |DATADIR| (Optionnal) The directory to store your data files |
        |LOGDIR| (Optionnal) The directory to store your logs files |
        |APIDIR| (Optionnal) The directory to store files downloaded from the api server |

    ??? abstract "api-server"

        | Variable | Description |
        |----------|-------------|
        |pem_path | The path to your .pem file |
        |api_user | Your api-server user |
        |api_suuser | Your api-server super user |
        |bastion_server | the url to the bastion-server |
        |api_server | the url to the api-server |

    ??? abstract "gitHub"

        | Variable | Description |
        |----------|-------------|
        | ghToken | Your personnal access token (generated on github.com) |

    ??? abstract "reportingDB"

        | Variable | Description |
        |----------|-------------|
        | hostReportingDB | the url to the reportingDB |
        | portReportingDB | the port to use for the connection |
        | dbnameReportingDB | name of the database |
        | usernameReportingDB | your username |
        | passwordReportingDB | your password |

    ??? abstract "snowflake"

        | Variable | Description |
        |----------|-------------|
        | privateKeySnowflake | the path to you private key (.p8) file |
        | passPhraseSnowflake | the passPhrase for your private key |
        | usernameSnowflake | your username (email) |
        | defaultWarehouseSnowflake | the name of default warehouse when not explicitly specified|

    ??? abstract "s3"

        | Variable | Description |
        |----------|-------------|
        | awsAccessKeyId | Your amazon Access Key |
        | awsSecretAccessKey | Your Amazon Secret Key |
        | awsS3Bucket | The name of your s3 bucket |

    ??? abstract "tableau"

        | Variable | Description |
        |----------|-------------|
        | tableauServer | url of the tableau server |
        | tableauUsername | your username |
        | tableauPassword | your password |

    ??? abstract "spotify API"

        | Variable | Description |
        |----------|-------------|
        | spotifyClientId | Your spotify App ID |
        | spotifyClientSecret | Your spotify App Secret |
        | spotifyTokenDirPath | (Optionnal) the path to store you token if you use scope-limited endpoints |
        | spotifyUsername | (Optionnal) the default username to use if you use scope-limited endpoints |

    ??? abstract "apple music API"

        | Variable | Description |
        |----------|-------------|
        | appleSecretFile | the name of your secret .p8 file |
        | appleSecret | Your apple secret (the full string)
        | appleKeyId | your apple KeyID |
        | appleTeamId | your apple TeamID |

    ??? abstract "google API"

        | Variable | Description |
        |----------|-------------|
        | googleApiKeyPath | The path the .json file containing you api keys |

    ??? abstract "smtp"

        | Variable | Description |
        |----------|-------------|
        | smtpHost | the smtp server url |
        | smtpPort | the port to use to send mail |
        | smtpKey | your smtp key |
        | smtpSecret | your smtp secret |

    ??? abstract "luminate API"

        | Variable | Description |
        |----------|-------------|
        | luminateLogin | the sony api-user login |
        | luminatePassword | the sony api-user password |
        | luminateApiKey | the sony api key |

## Development workflow

This section is dedicated to manage your development workflow with git if you intend to use the repo to be the home for you codebase.

??? tip "Recording of the GitHub Session (March 2024)"
    [Access the video](https://sonymusicentertainment-my.sharepoint.com/:v:/g/personal/julien_comte_sonymusic_com/EezgOd2pnVxHnazCs2lDqOEBpO_jpk5ZHyntrw6ghn9BtA?nav=eyJyZWZlcnJhbEluZm8iOnsicmVmZXJyYWxBcHAiOiJPbmVEcml2ZUZvckJ1c2luZXNzIiwicmVmZXJyYWxBcHBQbGF0Zm9ybSI6IldlYiIsInJlZmVycmFsTW9kZSI6InZpZXciLCJyZWZlcnJhbFZpZXciOiJNeUZpbGVzTGlua0NvcHkifX0&e=TBONKk)

+ The **```main```** branch of the repo is the **latest official community version** of projects and ressources (djagitit package, sql-snippits queries and tools)    
This branch is protected so that no one can push commits directly to it (see [Contribute](#contribute) section below).
+ For every team that wants to use the repo for their own projects, we'll set up a **```team-main```** branch (for instance ```bnx-main``` for Benelux) that will be your **team's latest official version**   
*It's up to you but we strongly suggest enforcing pull-requests to push commits on this main branch, the same way we do for the common one.*

We will apply **branch protection** :  
For every branch that contains the name ```team-```, only members of that particular team will be able to push on it.   
*That means that every user that has access to the repo will be able to see everyone's code but only members of a team can modify the code on their team's branch*


From there :

???+ note
    **If you use an IDE** (such as [VSCode](https://code.visualstudio.com/)), it might includes functionnalities to do the following directly in the IDE.   
    We'll document here how to do it from the command line.

If you want to work on a project/feature:
    
+ create a branch

???+ example  
    === "local project/feature"
        ```
        # Be sure to be on the team-main branch (for instance for France)
        git checkout fr-main

        # Synchronize to get the latest modifications on the remote team-main branch gitHub if any
        git fetch
        git pull

        # create your new branch from the team-main branch 
        git checkout -b fr-my-new-branch

        # push your new branch on gitHub
        git push --set-upstream origin fr-my-new-branch
        ```
    === "community project/feature"
        ```
        # Be sure to be on the main branch
        git checkout main

        # Synchronize to get the latest modifications on the remote main branch gitHub if any
        git fetch
        git pull

        # create your new branch from the main branch 
        # we suggest to prefix your branch name with your name or github user 
        git checkout -b juc0/my-new-branch

        # push your new branch on gitHub
        git push --set-upstream origin juc0/my-new-branch
        ```

+ From there you can do you work



+ Commit your files 
???+ example  
        # Let's say you're working on a project called shook-ones in the project-fr directory of the repo.
        # Go to the project folder
        cd project-fr/shooked-ones

        # Check the modifications
        git status

        # For each modification or group of modifications that you want to commit, stage the files
        # We encourage you to do the most atomic commits possible
        # It's better to have many commits with a few modifications in each, rather than a few commits with many modifications
        git add your-file
        git add other-file

        # Commit your modifications
        git commit -m "my commit message that could be a quick summary of the modifications"

+ Push your modifications to GitHub
???+ example  
        #This will push all new commits on your branch to GitHub
        git push

+ When you're ready, merge it to the main branch
    
    Go to the repo on GitHub and create a Pull-Request from your branch (ie fr-my-new-branch) to your main branch (ie fr-main for a local project or main for a community project).
    
    You can assign someone to review the Pull-Request if needed and use the Pull-Request page on GitHUb to discuss or ask for some modifications.
    Once you're ok, Validate the merge by clicking the Merge button and follow by deleting the branch.

+ Delete the local branch and resync your main branch
???+ example  
        #Switch to the main branch
        git checkout fr-main

        #Get the latests modifications
        git fetch
        git pull

        #Delete your merged branch
        git branch -D fr-my-new-branch

## Contribute

This section will be filled soon