Blog - Einleitung

Ancud Blog

Herzlich Willkommen zum Ancud Blog. Hier finden Sie eine Vielzahl von interessanten Artikeln zu verschiedenen Themen. Tauchen Sie ein in unsere Welt des Wissens! 

Blogs

Blogs

Data Engineers on AWS: SNS Notification and Lambda Subscribe

Hey, data engineers! We’re back to explore more in the exciting field we’ve been diving into. Today, we’re checking out some AWS services about sending and receiving messages.

I. Introduction :

In today’s world, lots of apps are shifting to real-time. The IT world aims to get info to users ASAP, helping them stay updated for quick decision-making. Think apps for airplane details or live football scores. What’s the basic setup for these kinds of apps?

II. Basic Architecture :

Keep in mind that the basic architecture is split into three main parts: the sender of messages or information, the place where messages are stored (the topic), and the entity that uses or consumes those messages.

To get a clearer picture, let’s take a look at the image below:


 
Fig 1 : Basic architeture

III. AWS services :

Cloud providers have developed services to make this pub/sub feature possible. Today, we’re diving into AWS, and specifically, we’ll discuss AWS Simple Notification Service (SNS).

Let’s start by defining AWS SNS. It’s a fully managed messaging service that allows for the creation and distribution of messages to a distributed group of recipients or endpoints.

AWS SNS offers two notification types: A2A (application to application) and A2P (application to person).
- A2A involves communication between two software applications.
- A2P covers interactions between applications and individuals (persons), including notifications, alerts, and marketing messages.


 
Fig 2 : AWS SNS

Now, let’s delve into some use cases of SNS:

Application Integration :
Fanout Scenario : This occurs when a message in an SNS topic is replicated to multiple endpoints. These endpoints can include Kinesis Data Firehose delivery streams, Amazon SQS queues, HTTP(S) endpoints, and Lambda functions.


 
Fig 3 : Fanout

Application Alerts :
if you define a specific threshold and want to trigger an event when that threshold is reached, you can use AWS SNS to send emails. This proves to be very useful for alerting in applications based on predefined conditions or thresholds you’ve set .

  • User notification : send sms or emails to users , for example order confirmation email in an e-commerce website .

IV. Tutorial :

In our tutorial, we’ll build an e-commerce confirmation message system using an AWS SNS topic. Picture it as a website that sends an email or an SMS to a customer after they’ve made a purchase. To handle this, we’ll subscribe to the SNS topic using an AWS Lambda function. This setup allows for efficient communication and confirmation in an e-commerce scenario.

Requirements :

  • AWS account

 
Fig 4 : Tutorial architecture
  • Step 1 : Create SNS topic

Navigate to the AWS Management Console, and access the SNS service.


 

 
  • Step 2 : Create lambda function for async invocations from SNS topic

Now, it’s time to create an AWS Lambda function that processes the data coming from the SNS topic.


 

It’s a straightforward process — just follow the UI, give the Lambda function a name, and leave the default values as they are. It should be created without any issues.

Step 3 : subscribe the lambda function to the SNS topic


 

 

When you click on the “Add” button, you’ll notice in the configuration section that a resource-based policy has been created. If you’re not familiar with what a resource-based policy is, no worries — let’s break it down together.


 

Imagine you have a treasure chest (AWS resource), and you want to decide who gets to open it and take what’s inside. The resource-based policy is like a set of rules you attach to the treasure chest to say who is allowed to open it and what they are allowed to take.


 
  • Step 4 : develop the lambda function

Open the Lambda function UI and navigate to the “Test” tab. In the template dropdown list, select our created SNS topic. Upon doing so, you’ll see an output similar to this .


 

we go to the code part and we will write javascript code in order to extract the events from SNS .

exports.handler = async (event) => {
    console.log("event:", JSON.stringify(event, undefined, 2));
    
    var message = event.Records[0].Sns.Message;
    console.log('Message received from SNS:', message);
};

Sure, let’s break it down for better understanding :

The first line exports a function named handler. In the context of AWS Lambda, the handler function is the entry point for the Lambda execution. It is triggered when an event occurs that is configured to invoke this Lambda function

exports.handler = async (event) => {
  • This line logs information about the incoming event to the console. The event parameter contains information about the event that triggered the Lambda function. In our case, it's an event from an AWS Simple Notification Service (SNS) topic.
 console.log("event:", JSON.stringify(event, undefined, 2));
  • This line extracts the message payload from the SNS event. SNS events often have a structure with a Records array, and in this case, it's assuming that the first record (index 0) contains an SNS message. It then extracts the message from the Sns.Message field.
var message = event.Records[0].Sns.Message;
  • Finally, this line logs the extracted SNS message to the console. The message variable now holds the content of the SNS message, and it's logged for debugging or monitoring purposes.
console.log('Message received from SNS:', message);

 
  • Step 6: Publish a message from SNS and visualize it.

we will go to the SNS topic in the UI and we will click on the publish message button .


 

we will write our first message :


 

Now, click on the “Publish message” button.


 
  • Step 7: Visualize the message using CloudWatch.

Navigate to the Lambda UI, and in the “Monitor” section, click on “View CloudWatch logs.” You’ll observe the log streams displayed like this:


 

Click on it, and you should see our message displayed like this:


 
  • Step 8: Don’t forget to clean up the resources you’ve created.

V. Conclusion :

In our article today, we dived into some AWS tools for working with data. The crucial point is to really grasp the ideas and understand how the different parts of the system fit together. Doing this helps you feel confident about working in any setup with any tool.

Authorname Chiheb Mhamdi