Deploying pretrained MobileNet V2 model over AWS using serverless open source framework

 Deployment  mobilenet v2 classification Aws lambda Imagenet
MobileNet V2 architecture overview
Figure 1. MobileNet V2 as the lightweight image-classification model used for serverless inference.

01 · Checklist

Checklist

  • Ubuntu 18.04/20.04, Miniconda, Docker, and Node.js.
  • npm plus the Serverless Framework.
  • An AWS account on the free tier.
  • Package the handler, put the traced model on S3, and call the live endpoint.

02 · Context

Context

Deploying pretrained MobileNet V2 model on ImageNet, using serverless open source framework to build and run the application on AWS Lambda and fetch the model from an S3 bucket.

Serverless framework manages all the resources in AWS so you can focus on the application. API endpoints, Lambda functions, CloudFormation stacks, and application packages on S3 are created automatically.

Key Idea

Keep the model artifact outside the Lambda package. Lambda runs the inference code, while S3 stores the heavier traced PyTorch model file.

This article belongs to a learning series on training and serving models for constrained environments such as mobile devices, embedded systems, Jetson Nano, Raspberry Pi, and neural compute sticks. This first step focuses on packaging the model and deploying it into AWS.

03 · Concept

Concept

Mental Model

The deployment has three moving parts: model storage in S3, inference execution in Lambda, and public request routing through API Gateway.

04 · Code

Code

MobileNet V2 ImageNet Classifier

Upload a small image and call the AWS Lambda endpoint to classify it with the deployed model.

Choose a PNG, JPEG, or WebP image under 5 MB.

Selected image preview
Operational Note

A cold Lambda container can make the first request slow. If the request times out, wait about 90 seconds and try again with a small image.

Model Artifact

First, load a pretrained MobileNet V2 model, trace it with a dummy input, and save it as a deployable TorchScript artifact.

Pretrained MobileNet V2 model artifact
Figure 2. The pretrained model is converted into a traced artifact before deployment.
### Importing the Libraries
import torch
from torchvision import models

### Instancing a pre-trained model will download it's weights to a cache memory
model = models.mobilenet_v2(pretrained = True)

### Deactivating some layers in the model, so that model output it's inferences as expected.
model.eval()

### Trace the model with some dummy input
traced_model = torch.jit.trace(model,torch.randn(1,3,224,224))

### Saving the model
traced_model.save('mobilenet_v2.pt')
  • Create a serverless Lambda(service) function. It generates boilerplate for template: aws-python3
Serverless Framework setup output
Figure 3. Serverless Framework creates the AWS Python service structure.
### Setting up serverless(sls) with the AWS user credentials
sudo sls config credentials --provider aws --key <AWS-key> --secret <AWS-secret-key>

### Create a service (Lambda function) in current directory with service name as new folder.
#serverless create --template <name-of-the-template> --path <name-of-the-service>
serverless create --template aws-python3 --path mobilenet-v2
Note

After creating the Lambda service, Serverless generates the function folder with files such as handler.py and serverless.yml.

  • Installing a serverless plugin which will automatically bundle dependencies from a requirement.txt file.
sudo serverless plugin install -n serverless-python-requirements

Change the permission of lambda function folder, run below command inside the folder (mobilenet-v2).if used sudo while creating it.

sudo chown -R $USER:$USER .
Warning

A Lambda deployment package cannot exceed AWS package limits. PyTorch can be large, so the packaging strategy matters as much as the inference code.

Requirements file for Lambda packaging
Figure 4. Requirements configuration used for packaging Python dependencies.
  • To enable deploying package through npm command, add a deploy script to package.json file
{
  "name": "mobilenet-v2",
  "description": "",
  "version": "0.1.0",
  "dependencies": {},
  "scipts":{"deploy":"serverless deploy"},
  "devDependencies": {
    "serverless-python-requirements": "^5.1.0"
  }
}
  • Update handler function in handler.py
"""Code to download pre-trained pytorch model.
Download the pre-trained model and convert into trace model
"""
try:
    import unzip_requirements
except ImportError:
    pass
from requests_toolbelt.multipart import decoder
import torch
import torchvision
import torchvision.transforms as transforms
from PIL import Image

from torchvision.models import resnet50
from torch import nn

import boto3
import os
import tarfile  
import io
import base64
import json
print("Import End")

# define environmental variables if there are not existing
S3_BUCKET = os.environ['S3_BUCKET'] if 'S3_BUCKET' in os.environ else 'name-of-the-bucket'
MODEL_PATH = os.environ['MODEL_PATH'] if 'MODEL_PATH' in os.environ else 'mobilenet_v2.pt'
json_data = os.environ['json_data'] if 'json_data' in os.environ else 'imagenet1000_clsidx_to_labels.json'

print("Downloading the model:")

s3 = boto3.resource('s3')

try:
    if os.path.isfile(MODEL_PATH) != True:
        #get object from s3
        obj = s3.Bucket(S3_BUCKET).Object(MODEL_PATH)
        # read it in memory
        print("creating Bytestream")
        bytestream = io.BytesIO(obj.get()['Body'].read())
        print("Loading Model")
        model = torch.jit.load(bytestream)
        print("Model loaded")

except Exception as e:
    print(repr(e))
    raise(e)

def transform_image(image_bytes):

    """Transform the image for pre-trained model.
    Transform the image which includes resizing, centercrop and normalize.
    Args:
        image_bytes: Input image in bytes
    Returns:
        Tensor
    Raises:
        Except: An error occurred accessing the bytes.
    """
    try:
        transformations = transforms.Compose([
            transforms.Resize(255), # Resnet needed 224  image
            transforms.CenterCrop(224),
            transforms.ToTensor(),
            transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])])
        image = Image.open(io.BytesIO(image_bytes))
        return transformations(image).unsqueeze(0) #converted image into batch size of 1
    except Exception as e:
        print(repr(e))
        raise(e)

def imagenet1000_classidx_to_label(class_idx):

    """Convert class index to class labels, Imagenet.
    Converting predicted class index to class labels, Imagenet.
    Args:
        class_idx: str, predicted class index.
    Returns:
        A dict mapping keys to the corresponding table row data
        fetched. Each row is represented as a tuple of strings.
        Returned keys are always bytes.  If a key from the keys argument is
        missing from the dictionary, then that row was not found in the
        table (and require_all_keys must have been False).
    Raises:
        Exception: An error occurred accessing the json file.
    """
    try:
        #content_object = s3.Object(S3_BUCKET,json_data)
        print('Entered into Class prediction level')
        content_object = s3.Bucket(S3_BUCKET).Object(json_data)
        file_content = content_object.get()['Body'].read().decode('utf-8')
        json_content = json.loads(file_content)
        return json_content[class_idx]
    except Exception as e:
        print("Error",repr(e))
        return "Class Not Found"

def get_prediction(image_bytes):

    """Prediction from pre-trained model.
    Inferecing using pre-trained model
    Args:
        image_bytes: Transformed image_bytes
    Returns:
        int, predicted class index from imagenet
    """
    tensor = transform_image(image_bytes=image_bytes)
    return model(tensor).argmax().item()

def classify_image(event, context):

    """Classify image using api.
    Function is called from this template python: handler.py
    Args:
        event: Dictionary containing API inputs 
        context: Dictionary
    Returns:
        dictionary: API response
    Raises:
        Exception: Returning API repsonse 500
    """
    try:
        print("Entering into event")
        content_type_header = event['headers']['content-type']
        #print(event['body']) #printing the actual image in hex format
        body = base64.b64decode(event["body"]) #decoding the base64 image
        print("Body loaded")

        picture = decoder.MultipartDecoder(body, content_type_header).parts[0]
        prediction = get_prediction(image_bytes=picture.content)
        print(prediction)

        prediction_label = imagenet1000_classidx_to_label(str(prediction))

        filename = picture.headers[b'Content-Disposition'].decode().split(';')[1].split('=')[1]
        if len(filename) < 4:
            filename = picture.headers[b'Content-Disposition'].decode().split(';')[2].split('=')[1]

        return { #returning the result for the event
            "statusCode": 200,
            "headers": {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*',
                "Access-Control-Allow-Credentials": True
                },
            "body": json.dumps({'file': filename.replace('"', ''),'predicted': f"{prediction}, {prediction_label}"})
    }
    except Exception as e:
        print(repr(e))
        return {
            "statusCode": 500,
            "headers": {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*',
                "Access-Control-Allow-Credentials": True
            },
            "body": json.dumps({"error": repr(e)})
        }

The above codeblock define as the entry handler function for Lambda. It contains application specific code. Following functions are implemented in this file:

1. S3 bucket and path for model file in S3.

2. Loading of the model and JSON from S3 bucket.

3. Entry handler functions which will be invoke every time AWS Lanbda function is triggered.

4. Processing of incoming image content and using model for image's classification.

  • Update configurations in serverless.yml
service: mobilenet-v2

provider:
  name: aws
  runtime: python3.8
  stage: dev
  region: ap-south-1
  timeout: 60
  environment:
    S3_BUCKET: s3-bucket-name
    MODEL_PATH: mobilenet_v2.pt
  iam:
    role:
      statements:
      - Effect: "Allow"
        Action:
          - s3:getObject
        Resource: arn:aws:s3:::s3-bucket-name/*

custom:
  pythonRequirements:
    dockerizePip: true
    zip: true
    slim: true
    strip: false
    noDeploy:
      - docutils
      - jmespath
      - pip
      - python-dateutil
      - setuptools
      - six
      - tensorboard
    useStaticCache: true
    useDownloadCache: true
    cacheLocation: "./cache"

package:
  patterns:
    - '!package.json'
    - '!package-log.json'
    - '!node_modules/**'
    - '!cache/**'
    - '!test/**'
    - '!__pycache__/**'
    - '!.pytest_cache/**'
    - '!model/**'

functions:
  classify_image:
    handler: handler.classify_image
    memorySize: 3008
    timeout: 60
    events:
      - http:
          path: mobilenetV2-classify
          method: post
          cors: true

plugins:
  - serverless-python-requirements
  • Deploy the package
serverless deploy
Serverless deployment output
Figure 5. Serverless deployment output after creating the AWS resources.
Result

After deployment, Serverless creates the Lambda function, deployment artifacts, CloudFormation resources, and an API Gateway endpoint for inference requests.

  • API gateway configuration for Binary Media Type

We need to allow multipart/form-data content type for the HTTP request and hence visit AWS API gateway console and set "Binary Media Type" as "multipart/form-data"

  • Testing Deploying using API tool

Install Insomnia API tool is used to test the deployment. Invoke API with following configurations:

    HTTP Method: POST

    URL: {generated API link after deployment}

    Content-Type: multipart/form-data

    Select image file for clasification: dog.jpg
Image classification API response in Insomnia
Figure 6. Testing the deployed image-classification endpoint with an API client.

05 · Conclusion

Conclusion

  • Add strict CORS for the production domain instead of a wildcard origin.
  • Add API Gateway throttling, file-size validation, structured error responses, and CloudWatch alarms.
  • Move this demo toward a modern version with container-based Lambda or a lightweight inference service.

06 · Comments

Comments

Questions, corrections, and deployment notes are welcome.