Deploying Python AWS Lambda Functions

When it comes to deploying lambda function into AWS, I have been using mostly Java for the implementation language and runtime. However, using Python has some advantages, especially if the code does not require parallel threads. E.g., the deployment package is usually a lot smaller and waking the lambda function, after it hasn’t been used for a while seems faster.

My preferred build automation system for Java-based projects is gradle and I found the com.github.kaklakariada.aws-sam-deploy plugin to be the most convenient way, to quickly build and deploy a stack. Basically a one-click thing, once the credentials are stored in ~/.aws/credentials and the project has a proper template.yaml file.

Even longer than gradle, IntelliJ Idea is me preferred IDE for all things Java, and while JetBrains offers a plugin to use IntelliJ for python development, I find their dedicated IDE for Python, aka PyCharm easier to use.

What follows are the steps to use PyCharm and JetBrains’ AWS ToolKit to create an environment for convenient deployment into AWS.

  1. Creating an AWS Account
  2. Installing AWS CLI and AWS SAM CLI
  3. Deploying with command tools
  4. Deploying with JetBrains’ AWS Toolkit

1 Creating an AWS Account

Create an AWS Account (aka Root User Account) here:
https://portal.aws.amazon.com/gp/aws/developer/registration/index.html
A credit card number and working phone is needed during the setup process. Select “Basic” for the support plan.

1.1 Creating an IAM Account

Create an IAM Account (aka AWS Identity and Access Management (IAM) user) here:
https://console.aws.amazon.com/iam

1.2 Group

Select ‘Group’ on the left side and then click the “Create new Group” button.
Enter a name (e.g. MyEduGroup) and in the next step add these two policies. As I found out just recently, policies for a specific Lambda function can be referenced in its template.yml file, which I will tell you about at the end of this post. It’s still good to know right now that policies can be attached to a function instead of attaching it to a group or a user.

  • IAMFullAccess
  • PowerUserAccess

Review and click the “Create Group” button.

1.3 User

Select ‘User’ on the left side and then click the “Add user” button, to create a new IAM user.
Create a user name (e.g. MyEduUser) and select “Programmatic access” only.
Next, add the user to the group, mentioned above, (e.g. MyEduGroup)


Review and click the “Create user” button and don’t forget to download the csv file, containing the credentials. Add those credentials into a (hidden) file: ~/.aws/credentials file, which should then look like so:

[edu]
aws_access_key_id = AKI...
aws_secret_access_key = A+zgQz...

Btw, the string inside the square brackets is called the profile name

1.4 Bucket

While in the AWS web interface, click in Services and navigate/find S3. There you will have the chance to create a bucket. Do that. I used ‘edu-bucket’ and clicked through, going with all the defaults.

2. Installing AWS CLI and AWS SAM CLI

Jetbrains’ AWS Toolkit is basically just a UI accessing the AWS Command-line Tools, which therefore must be installed before we can use the Toolkit.

2.1 AWS CLI

How to install the AWS CLI version 2 is explained here. However, with Homebrew installed on your Mac, it can be installed like so:

brew install awscli

and confirming the installation like so:

aws --version
aws-cli/2.0.8 Python/3.8.2 Darwin/19.4.0 botocore/2.0.0dev12

2.2 AWS SAM CLI

How to install the AWS SAM CLI is explained here. Again, with Homebrew installed on your Mac, it can be installed like so:

brew tap aws/tap
brew install aws-sam-cli

and confirming the installation like so:

sam --version
SAM CLI, version 0.47.0

3. Deploying with Command-line Tools

Still using the command line, I create a new folder and excuse ‘sam init’:

mkdir ~/PycharmProjects/AWSLambda
cd  ~/PycharmProjects/AWSLambda
sam init

which starts collecting some information, before proceeding:
I chose (1) for AWS Quick Start Templates
python3.7 for the runtime
entered hello for a project name
and (1) Hello World Example for the quick start application template

All that led to the following output:

-----------------------
Generating application:
-----------------------
Name: hello
Runtime: python3.7
Dependency Manager: pip
Application Template: hello-world
Output Directory: .
Next steps can be found in the README file at ./hello/README.md

The sam init command initializes a serverless application with a SAM template and some boilerplate code. Here is what the folder/file structure looks like.


With the AWS credentials stored in the ~/.aws/credentials file and the previously created s-3 bucket, the following deploy script can be used to create an AWS CloudFormation stack and deploy the lambda function.

#!/bin/zsh

PROFILE="edu"
REGION="us-west-2"
BUCKET="edu-bucket"
STACK="Edu-Stack"

# build

sam build

# deploy

sam deploy\
 --capabilities="CAPABILITY_IAM"\
 --s3-bucket $BUCKET\
 --region $REGION\
 --profile $PROFILE\
 --stack-name $STACK

# delete the stack with this command:
# aws cloudformation delete-stack --stack-name $STACK --region $REGION --profile $PROFILE

Creating and running the deploy script in the project folder will result in something like this:

Building resource 'HelloWorldFunction'
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource
Build Succeeded
Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Deploying with following values
===============================
Stack name                 : Edu-Stack
Region                     : us-west-2
Confirm changeset          : False
Deployment s3 bucket       : edu-bucket
Capabilities               : ["CAPABILITY_IAM"]
Parameter overrides        : {}

CloudFormation outputs from deployed stack
Outputs
Key                 HelloWorldFunctionIamRole
Description         Implicit IAM Role created for Hello World function
Value               arn:aws:iam::178522735890:role/Edu-Stack-HelloWorldFunctionRole-1F9K36EUWBDRL
Key                 HelloWorldApi
Description         API Gateway endpoint URL for Prod stage for Hello World function
Value               https://1lcw260mvj.execute-api.us-west-2.amazonaws.com/Prod/hello/
Key                 HelloWorldFunction
Description         Hello World Lambda Function ARN
Value               arn:aws:lambda:us-west-2:178522735890:function:Edu-Stack-HelloWorldFunction-1FWZM46M02QR3                                                                                                                               
Successfully created/updated stack - Edu-Stack in us-west-2

After finding the endpoint the the output above, a tool like httpie can be used to quickly run a request:

▶ http get https://pk2700q2a5.execute-api.us-west-2.amazonaws.com/Prod/hello/
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 26
Content-Type: application/json
Date: Tue, 21 Apr 2020 20:22:31 GMT
Via: 1.1 02b9f097e6bccd6a8d89ebf056435cb5.cloudfront.net (CloudFront)
X-Amz-Cf-Id: FCvYo61XLpyhOTy-TKoYFM5kQj7dbggdrRCKrlX_PqMVy0kt2ppzzA==
X-Amz-Cf-Pop: LAX50-C2
X-Amzn-Trace-Id: Root=1-5e9f5607-ceb665183f353ae439380d0e;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: LWphKH53vHcFnvw=
x-amzn-RequestId: 42ccbde7-b8d6-4668-9d9f-0e7bb9e939d9
{
    "message": "hello world"
}

Of course the just created stack shows up in the AWS Web GUI as well:

Let’s delete the Stack and do the same with PyCharm and the AWS ToolKit installed.

4. Deploying with JetBrains’ AWS Toolkit

 

After launching PyCharm start with installing the AWS Toolkit plugin. Re-launch PyCharm and create a new project, selecting “AWS Serverless Application” application type. Just like before, I select python 3.7 and the “AWS SAM Hello World” SAM Template, and let PyCharm create my project structure.

 

At this point a dialog with pop up asking me if I want to create a project from existing source? You need to Click [NO]

A now already familiar looking folder/file structure will appear. Open the ‘AWS Explorer’ Tab at the lower left side of the IDE. Click on Configure AWS connection and select your profile and region, after which at least the CloudWatch logs from before (we forgot to clean those up) will show up.

 

Here is the ToolKit’s User Guide. To quickly create the stack and deploy the lambda function, simply right-click the ‘template.yaml’ file and select ‘Deploy Serverless Application’. In the following dialog I provided a stack name, this time Edu-Stack1 and in the ‘CloudFormation’ tab at the very bottom of the IDE, the process of the deployment can be monitored.

After refreshing the AWS Explorer tab, the Edu-Stack1 and the lambda function shows up as well. A right-click on the function followed by “Run” will allow you to invoke the  function.

Let’s open app.py, the python script that contains the actual lambda function, and make a code change. E.g. change the message t0 “Hello Serverless World”. Now, right-click the ‘template.yaml’ file and select ‘Deploy Serverless Application’ again. This time Update Stack is pre-selected and clicking Deploy succeeds a little quicker than before.

A right-click on the function followed by “Run” will show the returned response, of the updated lambda function.


 

 

 

 

Share this post:

Leave a Reply