{ "cells": [ { "cell_type": "markdown", "id": "9ea2588b", "metadata": {}, "source": [ "### Sagemaker Pipeline examples" ] }, { "cell_type": "markdown", "id": "dbac9189", "metadata": {}, "source": [ "Most of this example is based on the following websites:\n", "* https://sagemaker-examples.readthedocs.io/en/latest/sagemaker-pipelines/index.html\n", "\n", "
\n", "This example uses sample data for predicing housing prices." ] }, { "cell_type": "markdown", "id": "ba48749d", "metadata": {}, "source": [ "Setup environment" ] }, { "cell_type": "code", "execution_count": 11, "id": "d809bd38", "metadata": {}, "outputs": [], "source": [ "import boto3\n", "import sagemaker\n", "\n", "bucket = \"dev-cucumbers\"\n", "sagemaker_session = sagemaker.session.Session(default_bucket = bucket)\n", "\n", "region = sagemaker_session.boto_region_name\n", "role = sagemaker.get_execution_role()\n", "\n", "# we want to use our own bucket not the one assigned to sagemaker by default\n", "default_bucket = sagemaker_session.default_bucket()\n", "\n", "model_package_group_name = f\"HousingModelPackageGroupName\"" ] }, { "cell_type": "markdown", "id": "c3143bd3", "metadata": {}, "source": [ "Implement caching- this basically helps to avoid rerunning some steps of pipeline repeatedly:\n", "* https://docs.aws.amazon.com/sagemaker/latest/dg/pipelines-caching.html" ] }, { "cell_type": "code", "execution_count": 12, "id": "2ec31b14", "metadata": {}, "outputs": [], "source": [ "from sagemaker.workflow.steps import CacheConfig" ] }, { "cell_type": "markdown", "id": "a63d20aa", "metadata": {}, "source": [ "### Create a Pipeline" ] }, { "cell_type": "markdown", "id": "c51bc764", "metadata": {}, "source": [ "Define parameters for your pipeline
\n", "Parameters enable custom pipeline executions and schedules without having to modify the Pipeline definition.\n", "
\n", "* processing_instance_count – The instance count of the processing job.\n", "* input_data – The Amazon S3 location of the input data.\n", "* batch_data – The Amazon S3 location of the input data for batch transformation.\n", "* model_approval_status – The approval status to register the trained model with for CI/CD. For more information, see Automate MLOps with SageMaker Projects." ] }, { "cell_type": "code", "execution_count": 13, "id": "1fb01e2c", "metadata": {}, "outputs": [], "source": [ "input_data_uri = 's3://dev-cucumbers/sagemaker-pipeline-input/housing.csv'\n", "batch_data_uri = 's3://dev-cucumbers/sagemaker-pipeline-input/batch_input/'\n", "\n", "from sagemaker.workflow.parameters import (\n", " ParameterInteger,\n", " ParameterString,\n", " ParameterFloat\n", ")\n", "\n", "processing_instance_count = ParameterInteger(\n", " name=\"ProcessingInstanceCount\",\n", " default_value=1\n", ")\n", "model_approval_status = ParameterString(\n", " name=\"ModelApprovalStatus\",\n", " default_value=\"PendingManualApproval\"\n", ")\n", "input_data = ParameterString(\n", " name=\"InputData\",\n", " default_value=input_data_uri,\n", ")\n", "batch_data = ParameterString(\n", " name=\"BatchData\",\n", " default_value=batch_data_uri,\n", ")\n", "mse_threshold = ParameterFloat(name=\"MseThreshold\", default_value=3000000000.0)\n", "\n", "# Inference step parameters\n", "endpoint_instance_type = ParameterString(name=\"EndpointInstanceType\", default_value=\"ml.m5.large\")" ] }, { "cell_type": "markdown", "id": "d22c8f71", "metadata": {}, "source": [ "#### Define a Processing Step for Feature Engineering\n", "Processing job steps - A simplified, managed experience on SageMaker to run data processing workloads, such as feature engineering, data validation, model evaluation, and model interpretation.
\n", "PrcessingStep takes advantage of SKLearnProcessor that required Python file." ] }, { "cell_type": "code", "execution_count": 14, "id": "87932220", "metadata": {}, "outputs": [], "source": [ "!mkdir -p housing" ] }, { "cell_type": "markdown", "id": "f5cd8fc7", "metadata": {}, "source": [ "The preprocessing script uses scikit-learn to do the following:\n", "* Fill in missing ocean_proximity category data and encode it so that it is suitable for training.\n", "* Scale and normalize all numerical fields, aside from ocean_proximity and rings median_house_value data.\n", "* Split the data into training, validation, and test datasets." ] }, { "cell_type": "code", "execution_count": 15, "id": "d0ad6d34", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Overwriting housing/preprocessing.py\n" ] } ], "source": [ "%%writefile housing/preprocessing.py\n", "import argparse\n", "import os\n", "import requests\n", "import tempfile\n", "import numpy as np\n", "import pandas as pd\n", "\n", "\n", "from sklearn.compose import ColumnTransformer\n", "from sklearn.impute import SimpleImputer\n", "from sklearn.pipeline import Pipeline\n", "from sklearn.preprocessing import StandardScaler, OneHotEncoder\n", "\n", "\n", "if __name__ == \"__main__\":\n", " base_dir = \"/opt/ml/processing\"\n", "\n", " df = pd.read_csv(\n", " f\"{base_dir}/input/housing.csv\"\n", " )\n", " numeric_features = list(df.columns)\n", " for f in [\"ocean_proximity\", \"median_house_value\"]:\n", " numeric_features.remove(f)\n", " \n", " numeric_transformer = Pipeline(\n", " steps=[\n", " (\"imputer\", SimpleImputer(strategy=\"median\")),\n", " (\"scaler\", StandardScaler())\n", " ]\n", " )\n", "\n", " categorical_features = [\"ocean_proximity\"]\n", " categorical_transformer = Pipeline(\n", " steps=[\n", " (\"imputer\", SimpleImputer(strategy=\"constant\", fill_value=\"missing\")),\n", " (\"onehot\", OneHotEncoder(handle_unknown=\"ignore\"))\n", " ]\n", " )\n", "\n", " preprocess = ColumnTransformer(\n", " transformers=[\n", " (\"num\", numeric_transformer, numeric_features),\n", " (\"cat\", categorical_transformer, categorical_features)\n", " ]\n", " )\n", " \n", " y = df.pop(\"median_house_value\")\n", " X_pre = preprocess.fit_transform(df)\n", " y_pre = y.to_numpy().reshape(len(y), 1)\n", " \n", " X = np.concatenate((y_pre, X_pre), axis=1)\n", " \n", " np.random.shuffle(X)\n", " train, validation, test = np.split(X, [int(.7*len(X)), int(.85*len(X))])\n", "\n", " \n", " pd.DataFrame(train).to_csv(f\"{base_dir}/train/train.csv\", header=False, index=False)\n", " pd.DataFrame(validation).to_csv(f\"{base_dir}/validation/validation.csv\", header=False, index=False)\n", " pd.DataFrame(test).to_csv(f\"{base_dir}/test/test.csv\", header=False, index=False)" ] }, { "cell_type": "markdown", "id": "bb3b36db", "metadata": {}, "source": [ " Create an instance of an SKLearnProcessor to pass in to the processing step. " ] }, { "cell_type": "code", "execution_count": 16, "id": "22dd4365", "metadata": {}, "outputs": [], "source": [ "from sagemaker.sklearn.processing import SKLearnProcessor\n", "\n", "\n", "framework_version = \"0.23-1\"\n", "\n", "sklearn_processor = SKLearnProcessor(\n", " framework_version=framework_version,\n", " instance_type=\"ml.m5.xlarge\",\n", " instance_count=processing_instance_count,\n", " base_job_name=\"sklearn-housing-process\",\n", " role=role,\n", " sagemaker_session=sagemaker.session.Session(default_bucket = bucket)\n", ")" ] }, { "cell_type": "code", "execution_count": 17, "id": "1d599361", "metadata": {}, "outputs": [], "source": [ "from sagemaker.processing import ProcessingInput, ProcessingOutput\n", "from sagemaker.workflow.steps import ProcessingStep\n", "\n", "\n", "step_process = ProcessingStep(\n", " name=\"HousingProcess\",\n", " processor=sklearn_processor,\n", " inputs=[\n", " ProcessingInput(source=input_data, destination=\"/opt/ml/processing/input\"), \n", " ],\n", " outputs=[\n", " ProcessingOutput(output_name=\"train\", source=\"/opt/ml/processing/train\"),\n", " ProcessingOutput(output_name=\"validation\", source=\"/opt/ml/processing/validation\"),\n", " ProcessingOutput(output_name=\"test\", source=\"/opt/ml/processing/test\")\n", " ],\n", " code=\"housing/preprocessing.py\",\n", " cache_config=CacheConfig(enable_caching=True, expire_after='PT1H')\n", ")" ] }, { "cell_type": "markdown", "id": "3c70b39e", "metadata": {}, "source": [ "#### Define a Training step\n", "Training job steps - An iterative process that teaches a model to make predictions by presenting examples from a training dataset." ] }, { "cell_type": "code", "execution_count": 18, "id": "528b3420", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "s3://dev-cucumbers/sagemaker-pipeline-steps/HousingTrain\n" ] } ], "source": [ "model_path = f\"s3://{default_bucket}/sagemaker-pipeline-steps/HousingTrain\"\n", "print(model_path)" ] }, { "cell_type": "markdown", "id": "842a67f9", "metadata": {}, "source": [ "Configure an estimator for the XGBoost algorithm and the input dataset" ] }, { "cell_type": "code", "execution_count": 19, "id": "441b8838", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "NOTEBOOK_METADATA_FILE detected but failed to get valid domain and user from it.\n" ] } ], "source": [ "from sagemaker.estimator import Estimator\n", "\n", "\n", "image_uri = sagemaker.image_uris.retrieve(\n", " framework=\"xgboost\",\n", " region=region,\n", " version=\"1.0-1\",\n", " py_version=\"py3\",\n", " instance_type=\"ml.m5.xlarge\"\n", ")\n", "xgb_train = Estimator(\n", " image_uri=image_uri,\n", " instance_type=\"ml.m5.xlarge\",\n", " instance_count=1,\n", " output_path=model_path,\n", " role=role,\n", " disable_profiler=True\n", ")\n", "xgb_train.set_hyperparameters(\n", " objective=\"reg:linear\",\n", " num_round=50,\n", " max_depth=5,\n", " eta=0.2,\n", " gamma=4,\n", " min_child_weight=6,\n", " subsample=0.7,\n", " silent=0\n", ")" ] }, { "cell_type": "markdown", "id": "fe082066", "metadata": {}, "source": [ "Create a TrainingStep using the estimator instance and properties of the ProcessingStep. In particular, pass in the S3Uri of the \"train\" and \"validation\" output channel to the TrainingStep. " ] }, { "cell_type": "code", "execution_count": 20, "id": "cd1b713a", "metadata": {}, "outputs": [], "source": [ "from sagemaker.inputs import TrainingInput\n", "from sagemaker.workflow.steps import TrainingStep\n", "\n", "\n", "step_train = TrainingStep(\n", " name=\"HousingTrain\",\n", " estimator=xgb_train,\n", " inputs={\n", " \"train\": TrainingInput(\n", " s3_data=step_process.properties.ProcessingOutputConfig.Outputs[\n", " \"train\"\n", " ].S3Output.S3Uri,\n", " content_type=\"text/csv\"\n", " ),\n", " \"validation\": TrainingInput(\n", " s3_data=step_process.properties.ProcessingOutputConfig.Outputs[\n", " \"validation\"\n", " ].S3Output.S3Uri,\n", " content_type=\"text/csv\"\n", " )\n", " },\n", " cache_config=CacheConfig(enable_caching=True, expire_after='PT1H')\n", ")" ] }, { "cell_type": "markdown", "id": "53cd812a", "metadata": {}, "source": [ "#### Define a Processing Step for Model Evaluation\n", "\n", "This section shows how to create a processing step to evaluate the accuracy of the model. The result of this model evaluation is used in the condition step to determine which execute path to take.\n" ] }, { "cell_type": "markdown", "id": "fbe1e035", "metadata": {}, "source": [ "The evaluation script uses xgboost to do the following:\n", "* Load the model.\n", "* Read the test data.\n", "* Issue predictions against the test data.\n", "* Build a classification report, including accuracy and ROC curve.\n", "* Save the evaluation report to the evaluation directory." ] }, { "cell_type": "code", "execution_count": 21, "id": "562413b6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Overwriting housing/evaluation.py\n" ] } ], "source": [ "%%writefile housing/evaluation.py\n", "import json\n", "import pathlib\n", "import pickle\n", "import tarfile\n", "import joblib\n", "import numpy as np\n", "import pandas as pd\n", "import xgboost\n", "\n", "\n", "from sklearn.metrics import mean_squared_error\n", "\n", "\n", "if __name__ == \"__main__\":\n", " model_path = f\"/opt/ml/processing/model/model.tar.gz\"\n", " with tarfile.open(model_path) as tar:\n", " tar.extractall(path=\".\")\n", " \n", " model = pickle.load(open(\"xgboost-model\", \"rb\"))\n", "\n", " test_path = \"/opt/ml/processing/test/test.csv\"\n", " df = pd.read_csv(test_path, header=None)\n", " \n", " y_test = df.iloc[:, 0].to_numpy()\n", " df.drop(df.columns[0], axis=1, inplace=True)\n", " \n", " X_test = xgboost.DMatrix(df.values)\n", " \n", " predictions = model.predict(X_test)\n", "\n", " mse = mean_squared_error(y_test, predictions)\n", " std = np.std(y_test - predictions)\n", " report_dict = {\n", " \"regression_metrics\": {\n", " \"mse\": {\n", " \"value\": mse,\n", " \"standard_deviation\": std\n", " },\n", " },\n", " }\n", "\n", " output_dir = \"/opt/ml/processing/evaluation\"\n", " pathlib.Path(output_dir).mkdir(parents=True, exist_ok=True)\n", " \n", " evaluation_path = f\"{output_dir}/evaluation.json\"\n", " with open(evaluation_path, \"w\") as f:\n", " f.write(json.dumps(report_dict))" ] }, { "cell_type": "markdown", "id": "ca037856", "metadata": {}, "source": [ "Create an instance of a ScriptProcessor that is used to create a ProcessingStep." ] }, { "cell_type": "code", "execution_count": 22, "id": "7ca6ba43", "metadata": {}, "outputs": [], "source": [ "from sagemaker.processing import ScriptProcessor\n", "\n", "\n", "script_eval = ScriptProcessor(\n", " image_uri=image_uri,\n", " command=[\"python3\"],\n", " instance_type=\"ml.m5.xlarge\",\n", " instance_count=1,\n", " base_job_name=\"script-housing-eval\",\n", " role=role,\n", " sagemaker_session=sagemaker.session.Session(default_bucket = bucket)\n", ")" ] }, { "cell_type": "markdown", "id": "040fcf5a", "metadata": {}, "source": [ "Create a ProcessingStep using the processor instance, the input and output channels, and the evaluation.py script. In particular, pass in the S3ModelArtifacts property from the step_train training step, as well as the S3Uri of the \"test\" output channel of the step_process processing step. This is very similar to a processor instance's run method in the SageMaker Python SDK. " ] }, { "cell_type": "code", "execution_count": 23, "id": "a88f7ebb", "metadata": {}, "outputs": [], "source": [ "from sagemaker.workflow.properties import PropertyFile\n", "\n", "evaluation_report = PropertyFile(\n", " name=\"EvaluationReport\",\n", " output_name=\"evaluation\",\n", " path=\"evaluation.json\"\n", ")\n", "step_eval = ProcessingStep(\n", " name=\"HousingEval\",\n", " processor=script_eval,\n", " inputs=[\n", " ProcessingInput(\n", " source=step_train.properties.ModelArtifacts.S3ModelArtifacts,\n", " destination=\"/opt/ml/processing/model\"\n", " ),\n", " ProcessingInput(\n", " source=step_process.properties.ProcessingOutputConfig.Outputs[\n", " \"test\"\n", " ].S3Output.S3Uri,\n", " destination=\"/opt/ml/processing/test\"\n", " )\n", " ],\n", " outputs=[\n", " ProcessingOutput(output_name=\"evaluation\", source=\"/opt/ml/processing/evaluation\"),\n", " ],\n", " code=\"housing/evaluation.py\",\n", " property_files=[evaluation_report],\n", ")" ] }, { "cell_type": "markdown", "id": "ffbaa032", "metadata": {}, "source": [ "#### Define a CreateModelStep for Batch Transformation
\n", "AWS recommends using Model Step to create models as of v2.90.0 of the SageMaker Python SDK. CreateModelStep will continue to work in previous versions of the SageMaker Python SDK, but is no longer actively supported." ] }, { "cell_type": "markdown", "id": "0c3858c1", "metadata": {}, "source": [ "Create a SageMaker model. Pass in the S3ModelArtifacts property from the step_train training step." ] }, { "cell_type": "code", "execution_count": 24, "id": "e001abb9", "metadata": {}, "outputs": [], "source": [ "from sagemaker.model import Model\n", "\n", "\n", "model = Model(\n", " image_uri=image_uri,\n", " model_data=step_train.properties.ModelArtifacts.S3ModelArtifacts,\n", " sagemaker_session=sagemaker_session,\n", " role=role,\n", ")" ] }, { "cell_type": "markdown", "id": "bfb53b12", "metadata": {}, "source": [ "Define the model input for your SageMaker model." ] }, { "cell_type": "code", "execution_count": 25, "id": "5fee6e35", "metadata": {}, "outputs": [], "source": [ "from sagemaker.inputs import CreateModelInput\n", "\n", "\n", "inputs = CreateModelInput(\n", " instance_type=\"ml.m5.large\",\n", " accelerator_type=\"ml.eia1.medium\",\n", ")" ] }, { "cell_type": "markdown", "id": "8db9464d", "metadata": {}, "source": [ "Create model steps - A step that creates a model for use in transform steps or later publication as an endpoint." ] }, { "cell_type": "code", "execution_count": 26, "id": "7512a87b", "metadata": {}, "outputs": [], "source": [ "from sagemaker.workflow.steps import CreateModelStep\n", "\n", "\n", "step_create_model = CreateModelStep(\n", " name=\"HousingCreateModel\",\n", " model=model,\n", " inputs=inputs,\n", ")" ] }, { "cell_type": "markdown", "id": "f5991db8", "metadata": {}, "source": [ "Transform job steps - A batch transform to preprocess datasets to remove noise or bias that interferes with training or inference from a dataset, get inferences from large datasets, and run inference when a persistent endpoint is not needed." ] }, { "cell_type": "markdown", "id": "a8d50437", "metadata": {}, "source": [ "This section shows how to create a TransformStep to perform batch transformation on a dataset after the model is trained. This step is passed into the condition step and only executes if the condition step evaluates to true." ] }, { "cell_type": "code", "execution_count": 27, "id": "f96625e2", "metadata": {}, "outputs": [], "source": [ "from sagemaker.transformer import Transformer\n", "from sagemaker.inputs import TransformInput\n", "from sagemaker.workflow.steps import TransformStep\n", "\n", "transformer = Transformer(\n", " model_name=step_create_model.properties.ModelName,\n", " instance_type=\"ml.m5.xlarge\",\n", " accept='text/csv',\n", " strategy='SingleRecord',\n", " assemble_with ='Line',\n", " instance_count=1,\n", " output_path=f\"s3://{default_bucket}/sagemaker-pipeline-steps/\"\n", ")" ] }, { "cell_type": "code", "execution_count": 28, "id": "14f7ab77", "metadata": {}, "outputs": [], "source": [ "from sagemaker.inputs import TransformInput\n", "from sagemaker.workflow.steps import TransformStep\n", "\n", "transform_inputs = TransformInput(\n", " data=batch_data,\n", " data_type='S3Prefix',\n", " content_type='text/csv',\n", " split_type='Line',\n", " join_source='Input',\n", " )\n", "\n", "step_transform = TransformStep(\n", " name=\"HousingTransform\",\n", " transformer=transformer,\n", " inputs=transform_inputs\n", ")" ] }, { "cell_type": "markdown", "id": "65b50b96", "metadata": {}, "source": [ "#### Define a RegisterModel Step to Create a Model Package.
\n", "Register model steps - A step that creates a model package resource in the Model Registry that can be used to create deployable models in Amazon SageMaker.
\n", "AWS recommends using Model Step to register models as of v2.90.0 of the SageMaker Python SDK. RegisterModel will continue to work in previous versions of the SageMaker Python SDK, but is no longer actively supported." ] }, { "cell_type": "markdown", "id": "c90ba558", "metadata": {}, "source": [ "This step is passed into the condition step and only executes if the condition step evaluates to true." ] }, { "cell_type": "code", "execution_count": 32, "id": "326c5917", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Popping out 'ProcessingJobName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n" ] } ], "source": [ "from sagemaker.model_metrics import MetricsSource, ModelMetrics \n", "from sagemaker.workflow.step_collections import RegisterModel\n", "\n", "\n", "model_metrics = ModelMetrics(\n", " model_statistics=MetricsSource(\n", " s3_uri=\"{}/evaluation.json\".format(\n", " step_eval.arguments[\"ProcessingOutputConfig\"][\"Outputs\"][0][\"S3Output\"][\"S3Uri\"]\n", " ),\n", " content_type=\"application/json\"\n", " )\n", ")\n", "\n", "step_register = RegisterModel(\n", " name=\"HousingRegisterModel\",\n", " estimator=xgb_train,\n", " model_data=step_train.properties.ModelArtifacts.S3ModelArtifacts,\n", " content_types=[\"text/csv\"],\n", " response_types=[\"text/csv\"],\n", " inference_instances=[\"ml.t2.medium\", \"ml.m5.xlarge\"],\n", " transform_instances=[\"ml.m5.xlarge\"],\n", " model_package_group_name=model_package_group_name,\n", " approval_status=model_approval_status,\n", " model_metrics=model_metrics\n", ")" ] }, { "cell_type": "markdown", "id": "c98350df", "metadata": {}, "source": [ "#### Define a Fail Step to Terminate the Pipeline Execution and Mark it as Failed\n", "Fail steps - A step that stops a pipeline execution and marks the pipeline execution as failed." ] }, { "cell_type": "code", "execution_count": 33, "id": "ec374ff3", "metadata": {}, "outputs": [], "source": [ "from sagemaker.workflow.fail_step import FailStep\n", "from sagemaker.workflow.functions import Join\n", "\n", "step_fail = FailStep(\n", " name=\"HousingMSEFail\",\n", " error_message=Join(on=\" \", values=[\"Execution failed due to MSE >\", mse_threshold]),\n", ")" ] }, { "cell_type": "markdown", "id": "990dd6df", "metadata": {}, "source": [ "#### Deploy model to SageMaker Endpoint using Lambda Step" ] }, { "cell_type": "code", "execution_count": 34, "id": "6b1e8640", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Overwriting housing/deploy_model_lambda.py\n" ] } ], "source": [ "%%writefile housing/deploy_model_lambda.py\n", "\n", "\n", "\"\"\"\n", "This Lambda function deploys the model to SageMaker Endpoint.\n", "If Endpoint exists, then Endpoint will be updated with new Endpoint Config.\n", "\"\"\"\n", "\n", "import json\n", "import boto3\n", "import time\n", "\n", "\n", "sm_client = boto3.client(\"sagemaker\")\n", "\n", "\n", "def lambda_handler(event, context):\n", " print(f\"Received Event: {event}\")\n", "\n", " current_time = time.strftime(\"%m-%d-%H-%M-%S\", time.localtime())\n", " endpoint_instance_type = event[\"endpoint_instance_type\"]\n", " model_name = event[\"model_name\"]\n", " endpoint_config_name = event[\"endpoint_config_name\"]\n", " endpoint_name = event[\"endpoint_name\"]\n", "\n", " # Create Endpoint Configuration\n", " create_endpoint_config_response = sm_client.create_endpoint_config(\n", " EndpointConfigName=endpoint_config_name,\n", " ProductionVariants=[\n", " {\n", " \"InstanceType\": endpoint_instance_type,\n", " \"InitialVariantWeight\": 1,\n", " \"InitialInstanceCount\": 1,\n", " \"ModelName\": model_name,\n", " \"VariantName\": \"AllTraffic\",\n", " }\n", " ],\n", " )\n", " print(f\"create_endpoint_config_response: {create_endpoint_config_response}\")\n", "\n", " # Check if an endpoint exists. If no - Create new endpoint, if yes - Update existing endpoint\n", " list_endpoints_response = sm_client.list_endpoints(\n", " SortBy=\"CreationTime\",\n", " SortOrder=\"Descending\",\n", " NameContains=endpoint_name,\n", " )\n", " print(f\"list_endpoints_response: {list_endpoints_response}\")\n", "\n", " if len(list_endpoints_response[\"Endpoints\"]) > 0:\n", " print(\"Updating Endpoint with new Endpoint Configuration\")\n", " update_endpoint_response = sm_client.update_endpoint(\n", " EndpointName=endpoint_name, EndpointConfigName=endpoint_config_name\n", " )\n", " print(f\"update_endpoint_response: {update_endpoint_response}\")\n", " else:\n", " print(\"Creating Endpoint\")\n", " create_endpoint_response = sm_client.create_endpoint(\n", " EndpointName=endpoint_name, EndpointConfigName=endpoint_config_name\n", " )\n", " print(f\"create_endpoint_response: {create_endpoint_response}\")\n", "\n", " return {\"statusCode\": 200, \"body\": json.dumps(\"Endpoint Created Successfully\")}" ] }, { "cell_type": "code", "execution_count": 35, "id": "fc24269c", "metadata": {}, "outputs": [], "source": [ "from sagemaker.workflow.lambda_step import LambdaStep\n", "from sagemaker.lambda_helper import Lambda\n", "\n", "import json\n", "import boto3\n", "import time\n", "\n", "current_time = time.strftime(\"%m-%d-%H-%M-%S\", time.localtime())\n", "\n", "endpoint_config_name = \"pipeline-housing-endpoint-\" + current_time\n", "endpoint_name = \"pipeline-housing-endpoint-\" + current_time\n", "\n", "deploy_model_lambda_function_name = \"sagemaker-deploy-model-lambda-\" + current_time\n", "\n", "deploy_model_lambda_function = Lambda(\n", " function_name=deploy_model_lambda_function_name,\n", " execution_role_arn = role,\n", " script=\"housing/deploy_model_lambda.py\",\n", " handler=\"deploy_model_lambda.lambda_handler\",\n", " s3_bucket = \"petsvakala-temporary\"\n", ")\n", "\n", "step_lower_mse_deploy_model_lambda = LambdaStep(\n", " name=\"Deploy-Housing-Model-To-Endpoint\",\n", " lambda_func=deploy_model_lambda_function,\n", " inputs={\n", " \"model_name\": step_create_model.properties.ModelName,\n", " \"endpoint_config_name\": endpoint_config_name,\n", " \"endpoint_name\": endpoint_name,\n", " \"endpoint_instance_type\": endpoint_instance_type,\n", " },\n", ")\n", "\n", "step_lower_mse_deploy_model_lambda.add_depends_on([step_create_model])" ] }, { "cell_type": "markdown", "id": "2d552995", "metadata": {}, "source": [ "#### Define a Condition Step to Verify Model Accuracy\n", "Conditional execution steps - A step that provides conditional execution of branches in a pipeline." ] }, { "cell_type": "markdown", "id": "426d9668", "metadata": {}, "source": [ "A ConditionStep allows SageMaker Pipelines to support conditional execution in your pipeline DAG based on the condition of step properties. In this case, you only want to register a model package if the accuracy of that model, as determined by the model evaluation step, exceeds the required value. If the accuracy exceeds the required value, the pipeline also creates a SageMaker Model and runs batch transformation on a dataset. This section shows how to define the Condition step." ] }, { "cell_type": "code", "execution_count": 36, "id": "3a0c2007", "metadata": {}, "outputs": [], "source": [ "from sagemaker.workflow.conditions import ConditionLessThanOrEqualTo\n", "from sagemaker.workflow.condition_step import ConditionStep\n", "from sagemaker.workflow.functions import JsonGet\n", "\n", "\n", "cond_lte = ConditionLessThanOrEqualTo(\n", " left=JsonGet(\n", " step_name=step_eval.name,\n", " property_file=evaluation_report,\n", " json_path=\"regression_metrics.mse.value\"\n", " ),\n", " right=mse_threshold\n", ")" ] }, { "cell_type": "code", "execution_count": 37, "id": "753a18ce", "metadata": {}, "outputs": [], "source": [ "step_cond = ConditionStep(\n", " name=\"HousingMSECond\",\n", " conditions=[cond_lte],\n", " if_steps=[step_register, step_create_model, step_transform, step_lower_mse_deploy_model_lambda],\n", " else_steps=[step_fail], \n", ")" ] }, { "cell_type": "markdown", "id": "bfae0979", "metadata": {}, "source": [ "Now that you’ve created all of the steps, combine them into a pipeline." ] }, { "cell_type": "markdown", "id": "e159762e", "metadata": {}, "source": [ "NB! A step can only appear once in either the pipeline's step list or the if/else step lists of the condition step. It cannot appear in both. " ] }, { "cell_type": "markdown", "id": "50e908c3", "metadata": {}, "source": [ "#### Create pipeline definition" ] }, { "cell_type": "code", "execution_count": 38, "id": "3332992c", "metadata": {}, "outputs": [], "source": [ "from sagemaker.workflow.pipeline import Pipeline\n", "\n", "\n", "pipeline_name = f\"HousingPipeline\"\n", "pipeline = Pipeline(\n", " name=pipeline_name,\n", " parameters=[\n", " processing_instance_count,\n", " model_approval_status,\n", " input_data,\n", " batch_data,\n", " mse_threshold,\n", " endpoint_instance_type\n", " ],\n", " steps=[step_process, step_train, step_eval, step_cond],\n", ")" ] }, { "cell_type": "markdown", "id": "5715bfd1", "metadata": {}, "source": [ "(Optional) Examine the JSON pipeline definition to ensure that it's well-formed." ] }, { "cell_type": "code", "execution_count": 39, "id": "30694d61", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Popping out 'ProcessingJobName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "Popping out 'TrainingJobName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "Popping out 'ProcessingJobName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "No finished training job found associated with this estimator. Please make sure this estimator is only used for building workflow config\n", "Popping out 'CertifyForMarketplace' from the pipeline definition since it will be overridden in pipeline execution time.\n", "Popping out 'ModelPackageName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "Popping out 'ModelName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "Popping out 'TransformJobName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n" ] }, { "data": { "text/plain": [ "{'Version': '2020-12-01',\n", " 'Metadata': {},\n", " 'Parameters': [{'Name': 'ProcessingInstanceCount',\n", " 'Type': 'Integer',\n", " 'DefaultValue': 1},\n", " {'Name': 'ModelApprovalStatus',\n", " 'Type': 'String',\n", " 'DefaultValue': 'PendingManualApproval'},\n", " {'Name': 'InputData',\n", " 'Type': 'String',\n", " 'DefaultValue': 's3://dev-cucumbers/sagemaker-pipeline-input/housing.csv'},\n", " {'Name': 'BatchData',\n", " 'Type': 'String',\n", " 'DefaultValue': 's3://dev-cucumbers/sagemaker-pipeline-input/batch_input/'},\n", " {'Name': 'MseThreshold', 'Type': 'Float', 'DefaultValue': 3000000000.0},\n", " {'Name': 'EndpointInstanceType',\n", " 'Type': 'String',\n", " 'DefaultValue': 'ml.m5.large'}],\n", " 'PipelineExperimentConfig': {'ExperimentName': {'Get': 'Execution.PipelineName'},\n", " 'TrialName': {'Get': 'Execution.PipelineExecutionId'}},\n", " 'Steps': [{'Name': 'HousingProcess',\n", " 'Type': 'Processing',\n", " 'Arguments': {'ProcessingResources': {'ClusterConfig': {'InstanceType': 'ml.m5.xlarge',\n", " 'InstanceCount': {'Get': 'Parameters.ProcessingInstanceCount'},\n", " 'VolumeSizeInGB': 30}},\n", " 'AppSpecification': {'ImageUri': '683313688378.dkr.ecr.us-east-1.amazonaws.com/sagemaker-scikit-learn:0.23-1-cpu-py3',\n", " 'ContainerEntrypoint': ['python3',\n", " '/opt/ml/processing/input/code/preprocessing.py']},\n", " 'RoleArn': 'arn:aws:iam::103233932089:role/dev-audience-notebook-test-role',\n", " 'ProcessingInputs': [{'InputName': 'input-1',\n", " 'AppManaged': False,\n", " 'S3Input': {'S3Uri': {'Get': 'Parameters.InputData'},\n", " 'LocalPath': '/opt/ml/processing/input',\n", " 'S3DataType': 'S3Prefix',\n", " 'S3InputMode': 'File',\n", " 'S3DataDistributionType': 'FullyReplicated',\n", " 'S3CompressionType': 'None'}},\n", " {'InputName': 'code',\n", " 'AppManaged': False,\n", " 'S3Input': {'S3Uri': 's3://dev-cucumbers/HousingProcess-673c91f3a25bd2ea169d7f1f4713fa43/input/code/preprocessing.py',\n", " 'LocalPath': '/opt/ml/processing/input/code',\n", " 'S3DataType': 'S3Prefix',\n", " 'S3InputMode': 'File',\n", " 'S3DataDistributionType': 'FullyReplicated',\n", " 'S3CompressionType': 'None'}}],\n", " 'ProcessingOutputConfig': {'Outputs': [{'OutputName': 'train',\n", " 'AppManaged': False,\n", " 'S3Output': {'S3Uri': {'Std:Join': {'On': '/',\n", " 'Values': ['s3:/',\n", " 'dev-cucumbers',\n", " 'HousingPipeline',\n", " {'Get': 'Execution.PipelineExecutionId'},\n", " 'HousingProcess',\n", " 'output',\n", " 'train']}},\n", " 'LocalPath': '/opt/ml/processing/train',\n", " 'S3UploadMode': 'EndOfJob'}},\n", " {'OutputName': 'validation',\n", " 'AppManaged': False,\n", " 'S3Output': {'S3Uri': {'Std:Join': {'On': '/',\n", " 'Values': ['s3:/',\n", " 'dev-cucumbers',\n", " 'HousingPipeline',\n", " {'Get': 'Execution.PipelineExecutionId'},\n", " 'HousingProcess',\n", " 'output',\n", " 'validation']}},\n", " 'LocalPath': '/opt/ml/processing/validation',\n", " 'S3UploadMode': 'EndOfJob'}},\n", " {'OutputName': 'test',\n", " 'AppManaged': False,\n", " 'S3Output': {'S3Uri': {'Std:Join': {'On': '/',\n", " 'Values': ['s3:/',\n", " 'dev-cucumbers',\n", " 'HousingPipeline',\n", " {'Get': 'Execution.PipelineExecutionId'},\n", " 'HousingProcess',\n", " 'output',\n", " 'test']}},\n", " 'LocalPath': '/opt/ml/processing/test',\n", " 'S3UploadMode': 'EndOfJob'}}]}},\n", " 'CacheConfig': {'Enabled': True, 'ExpireAfter': 'PT1H'}},\n", " {'Name': 'HousingTrain',\n", " 'Type': 'Training',\n", " 'Arguments': {'AlgorithmSpecification': {'TrainingInputMode': 'File',\n", " 'TrainingImage': '683313688378.dkr.ecr.us-east-1.amazonaws.com/sagemaker-xgboost:1.0-1-cpu-py3'},\n", " 'OutputDataConfig': {'S3OutputPath': 's3://dev-cucumbers/sagemaker-pipeline-steps/HousingTrain'},\n", " 'StoppingCondition': {'MaxRuntimeInSeconds': 86400},\n", " 'ResourceConfig': {'VolumeSizeInGB': 30,\n", " 'InstanceCount': 1,\n", " 'InstanceType': 'ml.m5.xlarge'},\n", " 'RoleArn': 'arn:aws:iam::103233932089:role/dev-audience-notebook-test-role',\n", " 'InputDataConfig': [{'DataSource': {'S3DataSource': {'S3DataType': 'S3Prefix',\n", " 'S3Uri': {'Get': \"Steps.HousingProcess.ProcessingOutputConfig.Outputs['train'].S3Output.S3Uri\"},\n", " 'S3DataDistributionType': 'FullyReplicated'}},\n", " 'ContentType': 'text/csv',\n", " 'ChannelName': 'train'},\n", " {'DataSource': {'S3DataSource': {'S3DataType': 'S3Prefix',\n", " 'S3Uri': {'Get': \"Steps.HousingProcess.ProcessingOutputConfig.Outputs['validation'].S3Output.S3Uri\"},\n", " 'S3DataDistributionType': 'FullyReplicated'}},\n", " 'ContentType': 'text/csv',\n", " 'ChannelName': 'validation'}],\n", " 'HyperParameters': {'objective': 'reg:linear',\n", " 'num_round': '50',\n", " 'max_depth': '5',\n", " 'eta': '0.2',\n", " 'gamma': '4',\n", " 'min_child_weight': '6',\n", " 'subsample': '0.7',\n", " 'silent': '0'},\n", " 'DebugHookConfig': {'S3OutputPath': 's3://dev-cucumbers/sagemaker-pipeline-steps/HousingTrain',\n", " 'CollectionConfigurations': []},\n", " 'ProfilerConfig': {'DisableProfiler': True}},\n", " 'CacheConfig': {'Enabled': True, 'ExpireAfter': 'PT1H'}},\n", " {'Name': 'HousingEval',\n", " 'Type': 'Processing',\n", " 'Arguments': {'ProcessingResources': {'ClusterConfig': {'InstanceType': 'ml.m5.xlarge',\n", " 'InstanceCount': 1,\n", " 'VolumeSizeInGB': 30}},\n", " 'AppSpecification': {'ImageUri': '683313688378.dkr.ecr.us-east-1.amazonaws.com/sagemaker-xgboost:1.0-1-cpu-py3',\n", " 'ContainerEntrypoint': ['python3',\n", " '/opt/ml/processing/input/code/evaluation.py']},\n", " 'RoleArn': 'arn:aws:iam::103233932089:role/dev-audience-notebook-test-role',\n", " 'ProcessingInputs': [{'InputName': 'input-1',\n", " 'AppManaged': False,\n", " 'S3Input': {'S3Uri': {'Get': 'Steps.HousingTrain.ModelArtifacts.S3ModelArtifacts'},\n", " 'LocalPath': '/opt/ml/processing/model',\n", " 'S3DataType': 'S3Prefix',\n", " 'S3InputMode': 'File',\n", " 'S3DataDistributionType': 'FullyReplicated',\n", " 'S3CompressionType': 'None'}},\n", " {'InputName': 'input-2',\n", " 'AppManaged': False,\n", " 'S3Input': {'S3Uri': {'Get': \"Steps.HousingProcess.ProcessingOutputConfig.Outputs['test'].S3Output.S3Uri\"},\n", " 'LocalPath': '/opt/ml/processing/test',\n", " 'S3DataType': 'S3Prefix',\n", " 'S3InputMode': 'File',\n", " 'S3DataDistributionType': 'FullyReplicated',\n", " 'S3CompressionType': 'None'}},\n", " {'InputName': 'code',\n", " 'AppManaged': False,\n", " 'S3Input': {'S3Uri': 's3://dev-cucumbers/HousingEval-19b131b63656129c86bfecbee4904927/input/code/evaluation.py',\n", " 'LocalPath': '/opt/ml/processing/input/code',\n", " 'S3DataType': 'S3Prefix',\n", " 'S3InputMode': 'File',\n", " 'S3DataDistributionType': 'FullyReplicated',\n", " 'S3CompressionType': 'None'}}],\n", " 'ProcessingOutputConfig': {'Outputs': [{'OutputName': 'evaluation',\n", " 'AppManaged': False,\n", " 'S3Output': {'S3Uri': 's3://dev-cucumbers/HousingEval-19b131b63656129c86bfecbee4904927/output/evaluation',\n", " 'LocalPath': '/opt/ml/processing/evaluation',\n", " 'S3UploadMode': 'EndOfJob'}}]}},\n", " 'PropertyFiles': [{'PropertyFileName': 'EvaluationReport',\n", " 'OutputName': 'evaluation',\n", " 'FilePath': 'evaluation.json'}]},\n", " {'Name': 'HousingMSECond',\n", " 'Type': 'Condition',\n", " 'Arguments': {'Conditions': [{'Type': 'LessThanOrEqualTo',\n", " 'LeftValue': {'Std:JsonGet': {'PropertyFile': {'Get': 'Steps.HousingEval.PropertyFiles.EvaluationReport'},\n", " 'Path': 'regression_metrics.mse.value'}},\n", " 'RightValue': {'Get': 'Parameters.MseThreshold'}}],\n", " 'IfSteps': [{'Name': 'HousingRegisterModel-RegisterModel',\n", " 'Type': 'RegisterModel',\n", " 'Arguments': {'ModelPackageGroupName': 'HousingModelPackageGroupName',\n", " 'ModelMetrics': {'ModelQuality': {'Statistics': {'ContentType': 'application/json',\n", " 'S3Uri': 's3://dev-cucumbers/HousingEval-19b131b63656129c86bfecbee4904927/output/evaluation/evaluation.json'}},\n", " 'Bias': {},\n", " 'Explainability': {}},\n", " 'InferenceSpecification': {'Containers': [{'Image': '683313688378.dkr.ecr.us-east-1.amazonaws.com/sagemaker-xgboost:1.0-1-cpu-py3',\n", " 'ModelDataUrl': {'Get': 'Steps.HousingTrain.ModelArtifacts.S3ModelArtifacts'}}],\n", " 'SupportedContentTypes': ['text/csv'],\n", " 'SupportedResponseMIMETypes': ['text/csv'],\n", " 'SupportedRealtimeInferenceInstanceTypes': ['ml.t2.medium',\n", " 'ml.m5.xlarge'],\n", " 'SupportedTransformInstanceTypes': ['ml.m5.xlarge']},\n", " 'ModelApprovalStatus': {'Get': 'Parameters.ModelApprovalStatus'}}},\n", " {'Name': 'HousingCreateModel',\n", " 'Type': 'Model',\n", " 'Arguments': {'ExecutionRoleArn': 'arn:aws:iam::103233932089:role/dev-audience-notebook-test-role',\n", " 'PrimaryContainer': {'Image': '683313688378.dkr.ecr.us-east-1.amazonaws.com/sagemaker-xgboost:1.0-1-cpu-py3',\n", " 'Environment': {},\n", " 'ModelDataUrl': {'Get': 'Steps.HousingTrain.ModelArtifacts.S3ModelArtifacts'}}}},\n", " {'Name': 'HousingTransform',\n", " 'Type': 'Transform',\n", " 'Arguments': {'ModelName': {'Get': 'Steps.HousingCreateModel.ModelName'},\n", " 'TransformInput': {'DataSource': {'S3DataSource': {'S3DataType': 'S3Prefix',\n", " 'S3Uri': {'Get': 'Parameters.BatchData'}}},\n", " 'ContentType': 'text/csv',\n", " 'SplitType': 'Line'},\n", " 'TransformOutput': {'S3OutputPath': 's3://dev-cucumbers/sagemaker-pipeline-steps/',\n", " 'AssembleWith': 'Line',\n", " 'Accept': 'text/csv'},\n", " 'TransformResources': {'InstanceCount': 1,\n", " 'InstanceType': 'ml.m5.xlarge'},\n", " 'BatchStrategy': 'SingleRecord',\n", " 'DataProcessing': {'JoinSource': 'Input'}}},\n", " {'Name': 'Deploy-Housing-Model-To-Endpoint',\n", " 'Type': 'Lambda',\n", " 'Arguments': {'model_name': {'Get': 'Steps.HousingCreateModel.ModelName'},\n", " 'endpoint_config_name': 'pipeline-housing-endpoint-09-22-10-17-41',\n", " 'endpoint_name': 'pipeline-housing-endpoint-09-22-10-17-41',\n", " 'endpoint_instance_type': {'Get': 'Parameters.EndpointInstanceType'}},\n", " 'DependsOn': ['HousingCreateModel'],\n", " 'FunctionArn': 'arn:aws:lambda:us-east-1:103233932089:function:sagemaker-deploy-model-lambda-09-22-10-17-41',\n", " 'OutputParameters': []}],\n", " 'ElseSteps': [{'Name': 'HousingMSEFail',\n", " 'Type': 'Fail',\n", " 'Arguments': {'ErrorMessage': {'Std:Join': {'On': ' ',\n", " 'Values': ['Execution failed due to MSE >',\n", " {'Get': 'Parameters.MseThreshold'}]}}}}]}}]}" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import json\n", "\n", "json.loads(pipeline.definition())" ] }, { "cell_type": "markdown", "id": "c4285549", "metadata": {}, "source": [ "Submit the pipeline definition to the SageMaker Pipelines service to create a pipeline if it doesn't exist, or update the pipeline if it does. The role passed in is used by SageMaker Pipelines to create all of the jobs defined in the steps. " ] }, { "cell_type": "code", "execution_count": 40, "id": "547b7eca", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Popping out 'ProcessingJobName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "Popping out 'TrainingJobName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "Popping out 'ProcessingJobName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "No finished training job found associated with this estimator. Please make sure this estimator is only used for building workflow config\n", "Popping out 'CertifyForMarketplace' from the pipeline definition since it will be overridden in pipeline execution time.\n", "Popping out 'ModelPackageName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "Popping out 'ModelName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "Popping out 'TransformJobName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "Popping out 'ProcessingJobName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "Popping out 'TrainingJobName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "Popping out 'ProcessingJobName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "No finished training job found associated with this estimator. Please make sure this estimator is only used for building workflow config\n", "Popping out 'CertifyForMarketplace' from the pipeline definition since it will be overridden in pipeline execution time.\n", "Popping out 'ModelPackageName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "Popping out 'ModelName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n", "Popping out 'TransformJobName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n" ] }, { "data": { "text/plain": [ "{'PipelineArn': 'arn:aws:sagemaker:us-east-1:103233932089:pipeline/HousingPipeline',\n", " 'ResponseMetadata': {'RequestId': '0adb9604-067d-4dc1-9748-f8b4bd3cca57',\n", " 'HTTPStatusCode': 200,\n", " 'HTTPHeaders': {'x-amzn-requestid': '0adb9604-067d-4dc1-9748-f8b4bd3cca57',\n", " 'content-type': 'application/x-amz-json-1.1',\n", " 'content-length': '83',\n", " 'date': 'Fri, 22 Sep 2023 10:18:13 GMT'},\n", " 'RetryAttempts': 0}}" ] }, "execution_count": 40, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pipeline.upsert(role_arn=role)" ] }, { "cell_type": "markdown", "id": "fc23e1e6", "metadata": {}, "source": [ "Start pipeline" ] }, { "cell_type": "code", "execution_count": 41, "id": "566106c0", "metadata": {}, "outputs": [], "source": [ "execution = pipeline.start()" ] }, { "cell_type": "code", "execution_count": 42, "id": "468e8401", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'PipelineArn': 'arn:aws:sagemaker:us-east-1:103233932089:pipeline/HousingPipeline',\n", " 'PipelineExecutionArn': 'arn:aws:sagemaker:us-east-1:103233932089:pipeline/HousingPipeline/execution/0lfl15rntlkz',\n", " 'PipelineExecutionDisplayName': 'execution-1695377897846',\n", " 'PipelineExecutionStatus': 'Executing',\n", " 'CreationTime': datetime.datetime(2023, 9, 22, 10, 18, 17, 745000, tzinfo=tzlocal()),\n", " 'LastModifiedTime': datetime.datetime(2023, 9, 22, 10, 18, 17, 745000, tzinfo=tzlocal()),\n", " 'CreatedBy': {},\n", " 'LastModifiedBy': {},\n", " 'ResponseMetadata': {'RequestId': '7feac370-8d05-4529-b5b8-8413ea0c3296',\n", " 'HTTPStatusCode': 200,\n", " 'HTTPHeaders': {'x-amzn-requestid': '7feac370-8d05-4529-b5b8-8413ea0c3296',\n", " 'content-type': 'application/x-amz-json-1.1',\n", " 'content-length': '395',\n", " 'date': 'Fri, 22 Sep 2023 10:18:17 GMT'},\n", " 'RetryAttempts': 0}}" ] }, "execution_count": 42, "metadata": {}, "output_type": "execute_result" } ], "source": [ "execution.describe()" ] }, { "cell_type": "markdown", "id": "2b0a220d", "metadata": {}, "source": [ "We can wait until the pipeline execution ends" ] }, { "cell_type": "code", "execution_count": 43, "id": "07441ba5", "metadata": {}, "outputs": [], "source": [ "execution.wait()" ] }, { "cell_type": "markdown", "id": "e6a2c9c0", "metadata": {}, "source": [ "We can list execution steps to see their statuses or also check why step might have failed." ] }, { "cell_type": "code", "execution_count": 44, "id": "878b6802", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[{'StepName': 'HousingTransform',\n", " 'StartTime': datetime.datetime(2023, 9, 22, 10, 31, 5, 405000, tzinfo=tzlocal()),\n", " 'EndTime': datetime.datetime(2023, 9, 22, 10, 36, 10, 851000, tzinfo=tzlocal()),\n", " 'StepStatus': 'Succeeded',\n", " 'AttemptCount': 0,\n", " 'Metadata': {'TransformJob': {'Arn': 'arn:aws:sagemaker:us-east-1:103233932089:transform-job/pipelines-0lfl15rntlkz-HousingTransform-yVQDlOcTk0'}}},\n", " {'StepName': 'Deploy-Housing-Model-To-Endpoint',\n", " 'StartTime': datetime.datetime(2023, 9, 22, 10, 31, 5, 405000, tzinfo=tzlocal()),\n", " 'EndTime': datetime.datetime(2023, 9, 22, 10, 31, 7, 345000, tzinfo=tzlocal()),\n", " 'StepStatus': 'Succeeded',\n", " 'AttemptCount': 0,\n", " 'Metadata': {'Lambda': {'Arn': 'arn:aws:lambda:us-east-1:103233932089:function:sagemaker-deploy-model-lambda-09-22-10-17-41',\n", " 'OutputParameters': [{'Name': 'body',\n", " 'Value': '\"Endpoint Created Successfully\"'},\n", " {'Name': 'statusCode', 'Value': '200.0'}]}}},\n", " {'StepName': 'HousingCreateModel',\n", " 'StartTime': datetime.datetime(2023, 9, 22, 10, 31, 3, 621000, tzinfo=tzlocal()),\n", " 'EndTime': datetime.datetime(2023, 9, 22, 10, 31, 4, 758000, tzinfo=tzlocal()),\n", " 'StepStatus': 'Succeeded',\n", " 'AttemptCount': 0,\n", " 'Metadata': {'Model': {'Arn': 'arn:aws:sagemaker:us-east-1:103233932089:model/pipelines-0lfl15rntlkz-housingcreatemodel-ihanymuvud'}}},\n", " {'StepName': 'HousingRegisterModel-RegisterModel',\n", " 'StartTime': datetime.datetime(2023, 9, 22, 10, 31, 3, 621000, tzinfo=tzlocal()),\n", " 'EndTime': datetime.datetime(2023, 9, 22, 10, 31, 4, 760000, tzinfo=tzlocal()),\n", " 'StepStatus': 'Succeeded',\n", " 'AttemptCount': 0,\n", " 'Metadata': {'RegisterModel': {'Arn': 'arn:aws:sagemaker:us-east-1:103233932089:model-package/HousingModelPackageGroupName/3'}}},\n", " {'StepName': 'HousingMSECond',\n", " 'StartTime': datetime.datetime(2023, 9, 22, 10, 31, 2, 422000, tzinfo=tzlocal()),\n", " 'EndTime': datetime.datetime(2023, 9, 22, 10, 31, 2, 772000, tzinfo=tzlocal()),\n", " 'StepStatus': 'Succeeded',\n", " 'AttemptCount': 0,\n", " 'Metadata': {'Condition': {'Outcome': 'True'}}},\n", " {'StepName': 'HousingEval',\n", " 'StartTime': datetime.datetime(2023, 9, 22, 10, 26, 19, 139000, tzinfo=tzlocal()),\n", " 'EndTime': datetime.datetime(2023, 9, 22, 10, 31, 1, 740000, tzinfo=tzlocal()),\n", " 'StepStatus': 'Succeeded',\n", " 'AttemptCount': 0,\n", " 'Metadata': {'ProcessingJob': {'Arn': 'arn:aws:sagemaker:us-east-1:103233932089:processing-job/pipelines-0lfl15rntlkz-HousingEval-uchzt0fKs2'}}},\n", " {'StepName': 'HousingTrain',\n", " 'StartTime': datetime.datetime(2023, 9, 22, 10, 23, 34, 212000, tzinfo=tzlocal()),\n", " 'EndTime': datetime.datetime(2023, 9, 22, 10, 26, 16, 727000, tzinfo=tzlocal()),\n", " 'StepStatus': 'Succeeded',\n", " 'AttemptCount': 0,\n", " 'Metadata': {'TrainingJob': {'Arn': 'arn:aws:sagemaker:us-east-1:103233932089:training-job/pipelines-0lfl15rntlkz-HousingTrain-8Wem0rwFV9'}}},\n", " {'StepName': 'HousingProcess',\n", " 'StartTime': datetime.datetime(2023, 9, 22, 10, 18, 19, 332000, tzinfo=tzlocal()),\n", " 'EndTime': datetime.datetime(2023, 9, 22, 10, 23, 33, 134000, tzinfo=tzlocal()),\n", " 'StepStatus': 'Succeeded',\n", " 'AttemptCount': 0,\n", " 'Metadata': {'ProcessingJob': {'Arn': 'arn:aws:sagemaker:us-east-1:103233932089:processing-job/pipelines-0lfl15rntlkz-HousingProcess-87MoAqNPYU'}}}]" ] }, "execution_count": 44, "metadata": {}, "output_type": "execute_result" } ], "source": [ "execution.list_steps()" ] }, { "cell_type": "markdown", "id": "0a97ad61", "metadata": {}, "source": [ "#### Examining the Evaluation\n", "Examine the resulting model evaluation after the pipeline completes. Download the resulting evaluation.json file from S3 and print the report." ] }, { "cell_type": "code", "execution_count": 45, "id": "6cbf722c", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Popping out 'ProcessingJobName' from the pipeline definition by default since it will be overridden at pipeline execution time. Please utilize the PipelineDefinitionConfig to persist this field in the pipeline definition if desired.\n" ] }, { "data": { "text/plain": [ "{'regression_metrics': {'mse': {'value': 2669071078.804031,\n", " 'standard_deviation': 51662.22577583345}}}" ] }, "execution_count": 45, "metadata": {}, "output_type": "execute_result" } ], "source": [ "evaluation_json = sagemaker.s3.S3Downloader.read_file(\"{}/evaluation.json\".format(\n", " step_eval.arguments[\"ProcessingOutputConfig\"][\"Outputs\"][0][\"S3Output\"][\"S3Uri\"]\n", "))\n", "json.loads(evaluation_json)" ] }, { "cell_type": "markdown", "id": "cf6c1274", "metadata": {}, "source": [ "#### Lineage\n", "Review the lineage of the artifacts generated by the pipeline." ] }, { "cell_type": "code", "execution_count": 46, "id": "502f3d97", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'StepName': 'HousingProcess', 'StartTime': datetime.datetime(2023, 9, 22, 10, 18, 19, 332000, tzinfo=tzlocal()), 'EndTime': datetime.datetime(2023, 9, 22, 10, 23, 33, 134000, tzinfo=tzlocal()), 'StepStatus': 'Succeeded', 'AttemptCount': 0, 'Metadata': {'ProcessingJob': {'Arn': 'arn:aws:sagemaker:us-east-1:103233932089:processing-job/pipelines-0lfl15rntlkz-HousingProcess-87MoAqNPYU'}}}\n" ] }, { "ename": "ClientError", "evalue": "An error occurred (AccessDeniedException) when calling the DescribeArtifact operation: User: arn:aws:sts::103233932089:assumed-role/dev-audience-notebook-test-role/SageMaker is not authorized to perform: sagemaker:DescribeArtifact on resource: arn:aws:sagemaker:us-east-1:103233932089:artifact/d9184ef62d3d915e5f0dda2996367f3d because no identity-based policy allows the sagemaker:DescribeArtifact action", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mClientError\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[46], line 8\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m execution_step \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mreversed\u001b[39m(execution\u001b[38;5;241m.\u001b[39mlist_steps()):\n\u001b[1;32m 7\u001b[0m \u001b[38;5;28mprint\u001b[39m(execution_step)\n\u001b[0;32m----> 8\u001b[0m display(\u001b[43mviz\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mshow\u001b[49m\u001b[43m(\u001b[49m\u001b[43mpipeline_execution_step\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mexecution_step\u001b[49m\u001b[43m)\u001b[49m)\n\u001b[1;32m 9\u001b[0m time\u001b[38;5;241m.\u001b[39msleep(\u001b[38;5;241m5\u001b[39m)\n", "File \u001b[0;32m~/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/lineage/visualizer.py:97\u001b[0m, in \u001b[0;36mLineageTableVisualizer.show\u001b[0;34m(self, trial_component_name, training_job_name, processing_job_name, pipeline_execution_step, model_package_arn, endpoint_arn, artifact_arn, context_arn, actions_arn)\u001b[0m\n\u001b[1;32m 94\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m actions_arn:\n\u001b[1;32m 95\u001b[0m start_arn \u001b[38;5;241m=\u001b[39m actions_arn\n\u001b[0;32m---> 97\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_get_associations_dataframe\u001b[49m\u001b[43m(\u001b[49m\u001b[43mstart_arn\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/lineage/visualizer.py:160\u001b[0m, in \u001b[0;36mLineageTableVisualizer._get_associations_dataframe\u001b[0;34m(self, arn)\u001b[0m\n\u001b[1;32m 158\u001b[0m upstream_associations: Iterator[AssociationSummary] \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_get_associations(dest_arn\u001b[38;5;241m=\u001b[39marn)\n\u001b[1;32m 159\u001b[0m downstream_associations: Iterator[AssociationSummary] \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_get_associations(src_arn\u001b[38;5;241m=\u001b[39marn)\n\u001b[0;32m--> 160\u001b[0m inputs: \u001b[38;5;28mlist\u001b[39m \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mlist\u001b[39;49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mmap\u001b[39;49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_convert_input_association_to_df_row\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mupstream_associations\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 161\u001b[0m outputs: \u001b[38;5;28mlist\u001b[39m \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mlist\u001b[39m(\n\u001b[1;32m 162\u001b[0m \u001b[38;5;28mmap\u001b[39m(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_convert_output_association_to_df_row, downstream_associations)\n\u001b[1;32m 163\u001b[0m )\n\u001b[1;32m 164\u001b[0m df: DataFrame \u001b[38;5;241m=\u001b[39m pd\u001b[38;5;241m.\u001b[39mDataFrame(\n\u001b[1;32m 165\u001b[0m inputs \u001b[38;5;241m+\u001b[39m outputs,\n\u001b[1;32m 166\u001b[0m columns\u001b[38;5;241m=\u001b[39m[\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 172\u001b[0m ],\n\u001b[1;32m 173\u001b[0m )\n", "File \u001b[0;32m~/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/lineage/visualizer.py:261\u001b[0m, in \u001b[0;36mLineageTableVisualizer._convert_input_association_to_df_row\u001b[0;34m(self, association)\u001b[0m\n\u001b[1;32m 252\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_convert_input_association_to_df_row\u001b[39m(\u001b[38;5;28mself\u001b[39m, association) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m \u001b[38;5;28mlist\u001b[39m:\n\u001b[1;32m 253\u001b[0m \u001b[38;5;250m \u001b[39m\u001b[38;5;124;03m\"\"\"Convert an input association to a data frame row.\u001b[39;00m\n\u001b[1;32m 254\u001b[0m \n\u001b[1;32m 255\u001b[0m \u001b[38;5;124;03m Args:\u001b[39;00m\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 259\u001b[0m \u001b[38;5;124;03m array: Array of column values for the association data frame.\u001b[39;00m\n\u001b[1;32m 260\u001b[0m \u001b[38;5;124;03m \"\"\"\u001b[39;00m\n\u001b[0;32m--> 261\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_convert_association_to_df_row\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 262\u001b[0m \u001b[43m \u001b[49m\u001b[43massociation\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msource_arn\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 263\u001b[0m \u001b[43m \u001b[49m\u001b[43massociation\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msource_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 264\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mInput\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m 265\u001b[0m \u001b[43m \u001b[49m\u001b[43massociation\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msource_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 266\u001b[0m \u001b[43m \u001b[49m\u001b[43massociation\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43massociation_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 267\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/lineage/visualizer.py:310\u001b[0m, in \u001b[0;36mLineageTableVisualizer._convert_association_to_df_row\u001b[0;34m(self, arn, name, direction, src_dest_type, association_type)\u001b[0m\n\u001b[1;32m 308\u001b[0m arn_name \u001b[38;5;241m=\u001b[39m arn\u001b[38;5;241m.\u001b[39msplit(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m:\u001b[39m\u001b[38;5;124m\"\u001b[39m)[\u001b[38;5;241m5\u001b[39m]\n\u001b[1;32m 309\u001b[0m entity_type \u001b[38;5;241m=\u001b[39m arn_name\u001b[38;5;241m.\u001b[39msplit(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m/\u001b[39m\u001b[38;5;124m\"\u001b[39m)[\u001b[38;5;241m0\u001b[39m]\n\u001b[0;32m--> 310\u001b[0m name \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_get_friendly_name\u001b[49m\u001b[43m(\u001b[49m\u001b[43mname\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43marn\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mentity_type\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 311\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m [name, direction, src_dest_type, association_type, entity_type]\n", "File \u001b[0;32m~/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/lineage/visualizer.py:328\u001b[0m, in \u001b[0;36mLineageTableVisualizer._get_friendly_name\u001b[0;34m(self, name, arn, entity_type)\u001b[0m\n\u001b[1;32m 325\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m name\n\u001b[1;32m 327\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m entity_type \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124martifact\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[0;32m--> 328\u001b[0m artifact \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_session\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msagemaker_client\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mdescribe_artifact\u001b[49m\u001b[43m(\u001b[49m\u001b[43mArtifactArn\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43marn\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 329\u001b[0m uri \u001b[38;5;241m=\u001b[39m artifact[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mSource\u001b[39m\u001b[38;5;124m\"\u001b[39m][\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mSourceUri\u001b[39m\u001b[38;5;124m\"\u001b[39m]\n\u001b[1;32m 331\u001b[0m \u001b[38;5;66;03m# shorten the uri if the length is more than 40,\u001b[39;00m\n\u001b[1;32m 332\u001b[0m \u001b[38;5;66;03m# e.g s3://flintstone-end-to-end-tests-gamma-us-west-2-069083975568/results/\u001b[39;00m\n\u001b[1;32m 333\u001b[0m \u001b[38;5;66;03m# canary-auto-1608761252626/preprocessed-data/tuning_data/train.txt\u001b[39;00m\n\u001b[1;32m 334\u001b[0m \u001b[38;5;66;03m# become s3://.../preprocessed-data/tuning_data/train.txt\u001b[39;00m\n", "File \u001b[0;32m~/anaconda3/envs/python3/lib/python3.10/site-packages/botocore/client.py:535\u001b[0m, in \u001b[0;36mClientCreator._create_api_method.._api_call\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 531\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mTypeError\u001b[39;00m(\n\u001b[1;32m 532\u001b[0m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mpy_operation_name\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m() only accepts keyword arguments.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 533\u001b[0m )\n\u001b[1;32m 534\u001b[0m \u001b[38;5;66;03m# The \"self\" in this scope is referring to the BaseClient.\u001b[39;00m\n\u001b[0;32m--> 535\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_make_api_call\u001b[49m\u001b[43m(\u001b[49m\u001b[43moperation_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/anaconda3/envs/python3/lib/python3.10/site-packages/botocore/client.py:980\u001b[0m, in \u001b[0;36mBaseClient._make_api_call\u001b[0;34m(self, operation_name, api_params)\u001b[0m\n\u001b[1;32m 978\u001b[0m error_code \u001b[38;5;241m=\u001b[39m parsed_response\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mError\u001b[39m\u001b[38;5;124m\"\u001b[39m, {})\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mCode\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m 979\u001b[0m error_class \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mexceptions\u001b[38;5;241m.\u001b[39mfrom_code(error_code)\n\u001b[0;32m--> 980\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m error_class(parsed_response, operation_name)\n\u001b[1;32m 981\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 982\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m parsed_response\n", "\u001b[0;31mClientError\u001b[0m: An error occurred (AccessDeniedException) when calling the DescribeArtifact operation: User: arn:aws:sts::103233932089:assumed-role/dev-audience-notebook-test-role/SageMaker is not authorized to perform: sagemaker:DescribeArtifact on resource: arn:aws:sagemaker:us-east-1:103233932089:artifact/d9184ef62d3d915e5f0dda2996367f3d because no identity-based policy allows the sagemaker:DescribeArtifact action" ] } ], "source": [ "import time\n", "from sagemaker.lineage.visualizer import LineageTableVisualizer\n", "\n", "\n", "viz = LineageTableVisualizer(sagemaker.session.Session())\n", "for execution_step in reversed(execution.list_steps()):\n", " print(execution_step)\n", " display(viz.show(pipeline_execution_step=execution_step))\n", " time.sleep(5)" ] }, { "cell_type": "markdown", "id": "4884168a", "metadata": {}, "source": [ "#### Parametrized Executions\n", "You can run additional executions of the pipeline and specify different pipeline parameters. The parameters argument is a dictionary containing parameter names, and where the values are used to override the defaults values.\n", "\n", "Based on the performance of the model, you might want to kick off another pipeline execution on a compute-optimized instance type and set the model approval status to “Approved” automatically. This means that the model package version generated by the RegisterModel step is automatically ready for deployment through CI/CD pipelines, such as with SageMaker Projects." ] }, { "cell_type": "code", "execution_count": null, "id": "fb300cf2", "metadata": {}, "outputs": [], "source": [ "execution = pipeline.start(\n", " parameters=dict(\n", " ModelApprovalStatus=\"Approved\",\n", " )\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "4901b58a", "metadata": {}, "outputs": [], "source": [ "# execution = pipeline.start(parameters=dict(MseThreshold=3.0))" ] }, { "cell_type": "code", "execution_count": null, "id": "ef0623bb", "metadata": {}, "outputs": [], "source": [ "execution.wait()" ] }, { "cell_type": "code", "execution_count": null, "id": "5b531567", "metadata": {}, "outputs": [], "source": [ "execution.list_steps()" ] }, { "cell_type": "markdown", "id": "0ce608a8", "metadata": {}, "source": [ "Apart from that, you might also want to adjust the MSE threshold to a smaller value and raise the bar for the accuracy of the registered model. In this case you can override the MSE threshold like the following:" ] }, { "cell_type": "markdown", "id": "42092f53", "metadata": {}, "source": [ "List model packages" ] }, { "cell_type": "code", "execution_count": 47, "id": "23495bb4", "metadata": {}, "outputs": [], "source": [ "import time\n", "import os\n", "from sagemaker import get_execution_role, session\n", "import boto3\n", "\n", "region = boto3.Session().region_name\n", "\n", "# role = 'arn:aws:iam::566770890926:role/pipeline_petsvakala'\n", "\n", "sm_client = boto3.client('sagemaker', region_name=region)" ] }, { "cell_type": "code", "execution_count": 48, "id": "3cb52f9c", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'ModelPackageSummaryList': [{'ModelPackageGroupName': 'HousingModelPackageGroupName',\n", " 'ModelPackageVersion': 3,\n", " 'ModelPackageArn': 'arn:aws:sagemaker:us-east-1:103233932089:model-package/HousingModelPackageGroupName/3',\n", " 'CreationTime': datetime.datetime(2023, 9, 22, 10, 31, 4, 633000, tzinfo=tzlocal()),\n", " 'ModelPackageStatus': 'Completed',\n", " 'ModelApprovalStatus': 'PendingManualApproval'}],\n", " 'ResponseMetadata': {'RequestId': 'b4d49228-7648-43c9-9237-a42d1104a7c3',\n", " 'HTTPStatusCode': 200,\n", " 'HTTPHeaders': {'x-amzn-requestid': 'b4d49228-7648-43c9-9237-a42d1104a7c3',\n", " 'content-type': 'application/x-amz-json-1.1',\n", " 'content-length': '327',\n", " 'date': 'Fri, 22 Sep 2023 10:39:29 GMT'},\n", " 'RetryAttempts': 0}}" ] }, "execution_count": 48, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sm_client.list_model_packages(ModelPackageGroupName=model_package_group_name)" ] }, { "cell_type": "markdown", "id": "337cae55", "metadata": {}, "source": [ "#### Make prediction" ] }, { "cell_type": "code", "execution_count": 49, "id": "52354222", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "151201.640625\n" ] } ], "source": [ "runtime= boto3.client('runtime.sagemaker')\n", " \n", "line = \"1.13787052,-1.321157625,0.505394187,-0.3473488,-0.545657683,-0.730705932,-0.634388389,1.62368063,0,0,0,0,1\"\n", "line2 = \"-1.707174567,1.586280972,-1.163225463,-0.483031373,-0.562348921,-0.507293489,-0.592538683,-0.221431932,1,0,0,0,0\"\n", "\n", "response = runtime.invoke_endpoint(EndpointName=endpoint_name,\n", " ContentType='text/csv',\n", " Body=line2)\n", "\n", "response_body = response['Body'].read().decode(\"ascii\")\n", "print(response_body)" ] }, { "cell_type": "markdown", "id": "b4fc0a5e", "metadata": {}, "source": [ "#### Clean up resources" ] }, { "cell_type": "code", "execution_count": 50, "id": "51aaef68", "metadata": {}, "outputs": [ { "ename": "ClientError", "evalue": "An error occurred (AccessDeniedException) when calling the DeleteModelPackageGroup operation: User: arn:aws:sts::103233932089:assumed-role/dev-audience-notebook-test-role/SageMaker is not authorized to perform: sagemaker:DeleteModelPackageGroup on resource: arn:aws:sagemaker:us-east-1:103233932089:model-package-group/HousingModelPackageGroupName because no identity-based policy allows the sagemaker:DeleteModelPackageGroup action", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mClientError\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[50], line 30\u001b[0m\n\u001b[1;32m 27\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;132;01m{}\u001b[39;00m\u001b[38;5;124m \u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;241m.\u001b[39mformat(e))\n\u001b[1;32m 28\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m\n\u001b[0;32m---> 30\u001b[0m \u001b[43mempty_and_delete_model_package\u001b[49m\u001b[43m(\u001b[49m\u001b[43msm_client\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmodel_package_group_name\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 31\u001b[0m delete_sagemaker_pipeline(sm_client, pipeline_name)\n", "Cell \u001b[0;32mIn[50], line 16\u001b[0m, in \u001b[0;36mempty_and_delete_model_package\u001b[0;34m(sagemaker_client, mpg_name)\u001b[0m\n\u001b[1;32m 13\u001b[0m time\u001b[38;5;241m.\u001b[39msleep(\u001b[38;5;241m1\u001b[39m)\n\u001b[1;32m 15\u001b[0m \u001b[38;5;66;03m# Delete model package group\u001b[39;00m\n\u001b[0;32m---> 16\u001b[0m \u001b[43msagemaker_client\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mdelete_model_package_group\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 17\u001b[0m \u001b[43m \u001b[49m\u001b[43mModelPackageGroupName\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmpg_name\u001b[49m\n\u001b[1;32m 18\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/anaconda3/envs/python3/lib/python3.10/site-packages/botocore/client.py:535\u001b[0m, in \u001b[0;36mClientCreator._create_api_method.._api_call\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 531\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mTypeError\u001b[39;00m(\n\u001b[1;32m 532\u001b[0m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mpy_operation_name\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m() only accepts keyword arguments.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 533\u001b[0m )\n\u001b[1;32m 534\u001b[0m \u001b[38;5;66;03m# The \"self\" in this scope is referring to the BaseClient.\u001b[39;00m\n\u001b[0;32m--> 535\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_make_api_call\u001b[49m\u001b[43m(\u001b[49m\u001b[43moperation_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/anaconda3/envs/python3/lib/python3.10/site-packages/botocore/client.py:980\u001b[0m, in \u001b[0;36mBaseClient._make_api_call\u001b[0;34m(self, operation_name, api_params)\u001b[0m\n\u001b[1;32m 978\u001b[0m error_code \u001b[38;5;241m=\u001b[39m parsed_response\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mError\u001b[39m\u001b[38;5;124m\"\u001b[39m, {})\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mCode\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m 979\u001b[0m error_class \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mexceptions\u001b[38;5;241m.\u001b[39mfrom_code(error_code)\n\u001b[0;32m--> 980\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m error_class(parsed_response, operation_name)\n\u001b[1;32m 981\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 982\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m parsed_response\n", "\u001b[0;31mClientError\u001b[0m: An error occurred (AccessDeniedException) when calling the DeleteModelPackageGroup operation: User: arn:aws:sts::103233932089:assumed-role/dev-audience-notebook-test-role/SageMaker is not authorized to perform: sagemaker:DeleteModelPackageGroup on resource: arn:aws:sagemaker:us-east-1:103233932089:model-package-group/HousingModelPackageGroupName because no identity-based policy allows the sagemaker:DeleteModelPackageGroup action" ] } ], "source": [ "def empty_and_delete_model_package(sagemaker_client, mpg_name):\n", " mpg = sagemaker_client.list_model_packages(\n", " ModelPackageGroupName=mpg_name,\n", " )\n", " \n", " # Delete model packages if Group not empty\n", " model_packages = mpg.get('ModelPackageSummaryList')\n", " if model_packages:\n", " for mp in model_packages:\n", " sagemaker_client.delete_model_package(\n", " ModelPackageName=mp['ModelPackageArn']\n", " )\n", " time.sleep(1)\n", "\n", " # Delete model package group\n", " sagemaker_client.delete_model_package_group(\n", " ModelPackageGroupName=mpg_name\n", " )\n", " \n", "def delete_sagemaker_pipeline(sm_client, pipeline_name):\n", " try:\n", " sm_client.delete_pipeline(\n", " PipelineName=pipeline_name,\n", " )\n", " print(\"{} pipeline deleted\".format(pipeline_name))\n", " except Exception as e:\n", " print(\"{} \\n\".format(e))\n", " return\n", " \n", "empty_and_delete_model_package(sm_client, model_package_group_name)\n", "delete_sagemaker_pipeline(sm_client, pipeline_name)" ] }, { "cell_type": "markdown", "id": "7cbaecc1", "metadata": {}, "source": [ "Delete model, endpoint configuration and endpoint" ] }, { "cell_type": "code", "execution_count": 59, "id": "c11d0658", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'pipelines-0lfl15rntlkz-HousingCreateModel-IhanyMuvUd'" ] }, "execution_count": 59, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sm_client.list_models(NameContains=\"pipeline\")['Models'][0]['ModelName']" ] }, { "cell_type": "code", "execution_count": 60, "id": "9ef97cb1", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'ResponseMetadata': {'RequestId': '49ce85a0-132f-474f-960d-2ea75c430069',\n", " 'HTTPStatusCode': 200,\n", " 'HTTPHeaders': {'x-amzn-requestid': '49ce85a0-132f-474f-960d-2ea75c430069',\n", " 'content-type': 'application/x-amz-json-1.1',\n", " 'content-length': '0',\n", " 'date': 'Fri, 22 Sep 2023 10:44:21 GMT'},\n", " 'RetryAttempts': 0}}" ] }, "execution_count": 60, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Delete the Model\n", "del_model_name = step_create_model.properties.ModelName\n", "del_model_name = 'pipelines-0lfl15rntlkz-HousingCreateModel-IhanyMuvUd'\n", "sm_client.delete_model(ModelName=del_model_name)\n", "\n", "# Delete the EndpointConfig\n", "sm_client.delete_endpoint_config(EndpointConfigName=endpoint_config_name)\n", "\n", "# Delete the Endpoint\n", "sm_client.delete_endpoint(EndpointName=endpoint_name)" ] }, { "cell_type": "markdown", "id": "678de45a", "metadata": {}, "source": [ "Delete lambda function" ] }, { "cell_type": "code", "execution_count": 61, "id": "5b60b09c", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'ResponseMetadata': {'RequestId': 'c74c8b6e-a1c8-4a0e-bde2-27b3e25d44ad',\n", " 'HTTPStatusCode': 204,\n", " 'HTTPHeaders': {'date': 'Fri, 22 Sep 2023 10:44:24 GMT',\n", " 'content-type': 'application/json',\n", " 'connection': 'keep-alive',\n", " 'x-amzn-requestid': 'c74c8b6e-a1c8-4a0e-bde2-27b3e25d44ad'},\n", " 'RetryAttempts': 0}}" ] }, "execution_count": 61, "metadata": {}, "output_type": "execute_result" } ], "source": [ "deploy_model_lambda_function.delete()" ] } ], "metadata": { "kernelspec": { "display_name": "conda_python3", "language": "python", "name": "conda_python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12" } }, "nbformat": 4, "nbformat_minor": 5 }