Baseline model variants

In [2]:
import pandas as pd
import numpy as np
%matplotlib inline
%pylab inline
pylab.rcParams['figure.figsize'] = (12, 7)

import matplotlib.pyplot as plt

from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import make_pipeline
from sklearn.grid_search import GridSearchCV
from sklearn.metrics import r2_score
from sklearn.metrics import mean_squared_error

Populating the interactive namespace from numpy and matplotlib

/home/sergiusz/anaconda3/envs/whitelist/lib/python3.6/site-packages/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
  "This module will be removed in 0.20.", DeprecationWarning)
/home/sergiusz/anaconda3/envs/whitelist/lib/python3.6/site-packages/sklearn/grid_search.py:42: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.
  DeprecationWarning)
In [3]:
df = pd.read_feather('../data/basemodel_cumsum.feather')

model = LinearRegression(fit_intercept = True)
x = df[['popularity']]
y = df[['log_all_streams_cumsum']]

def PolynomialRegression(degree=2, **kwargs):
    return make_pipeline(PolynomialFeatures(degree), LinearRegression(**kwargs))

param_grid = {'polynomialfeatures__degree': np.arange(3), 'linearregression__normalize': [True, False]}

grid = GridSearchCV(PolynomialRegression(), param_grid, cv = 5, error_score=mean_squared_error)

Baseline model (2018-05-21 22:12)

In [4]:
def fitAndEvaluate(x,y,grid):
    grid.fit(x,y)
    model = grid.best_estimator_
    print(f'Best model: {model}')
    y_pred = model.predict(x)

    plt.subplot(1,2,1)
    plt.scatter(y_pred, y_pred - y)
    plt.hlines(y = 0, xmin = min(y_pred), xmax = max(y_pred))
    plt.title('Residual Errors')
    
    plt.subplot(1,2,2)
    plt.scatter(x,y)
    lim = plt.axis()
    plt.scatter(x,y_pred, c = 'red')
    plt.title('Prediction versus actual')

    rmse = np.sqrt(np.sum((y_pred - y)**2))[0]
    print(f'RMSE: {rmse}')
    print(f'Rsq: {r2_score(y,y_pred)}')
    

fitAndEvaluate(x,y,grid)

Best model: Pipeline(memory=None,
     steps=[('polynomialfeatures', PolynomialFeatures(degree=2, include_bias=True, interaction_only=False)), ('linearregression', LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=True))])
RMSE: 5.691645826227121
Rsq: 0.5953026658448666

Baseline with outlier removed (2018-05-22 22:15)

  • Evaluate outlier at 0 popularity. It seems to skew the results. It looks like linear would be a good fit here actually
In [5]:
df.sort_values('popularity').head()
Out[5]:
track_id popularity all_streams_cumsum log_all_streams_cumsum
6 0A488iaPeDAUP5q7Jm3paF 0 791514 13.581703
41 1ekNZULmcBHp3WRNKft7ou 8 68719 11.137781
11 0DPXH5sPDVwjsqfgvmi1yt 26 139665 11.847002
37 1cinSNWNJeGVxMgUSNCHRT 27 137729 11.833043
1 00mc2RHScEYMEFlc7FRGaK 28 93919 11.450188

Popularity 0 outlier

There is an outlier that has massive impact on the curve with 0 popularity, ,this could be some corner case but also some data error. Look into this explicitly.

  • 0A488iaPeDAUP5q7Jm3paF

UPDATE this was caused by an erratic jump to 0 and back

In [6]:
df = pd.read_feather('../data/basemodel_cumsum_outlier_removed.feather')

x = df[['popularity']]
y = df[['log_all_streams_cumsum']]
In [7]:
fitAndEvaluate(x,y,grid)

Best model: Pipeline(memory=None,
     steps=[('polynomialfeatures', PolynomialFeatures(degree=1, include_bias=True, interaction_only=False)), ('linearregression', LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=True))])
RMSE: 5.600131028898199
Rsq: 0.608212130063924

Baseline with outlier removed and using days after release (19th July)

In [8]:
df = pd.read_feather('../data/basemodel_cumsum_outlier_removed_days_after_release.feather')

x = df[['popularity']]
y = df[['log_all_streams_cumsum']]
In [9]:
fitAndEvaluate(x,y,grid)

Best model: Pipeline(memory=None,
     steps=[('polynomialfeatures', PolynomialFeatures(degree=2, include_bias=True, interaction_only=False)), ('linearregression', LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False))])
RMSE: 5.79525484445616
Rsq: 0.5770137357378432