Amazon Simple Queue Service (SQS) and Amazon Simple Notification Service (SNS) are both messaging services provided by AWS, but they serve different purposes and are designed for distinct use cases.

1. Overview

• Amazon SQS:
  • A fully managed message queuing service designed for decoupling and scaling microservices, distributed systems, and serverless applications.
  • Focuses on reliable message storage and delivery, allowing consumers to process messages at their own pace.
  • Operates as a pull-based system where consumers poll the queue to retrieve messages.
• Amazon SNS:
  • A fully managed pub/sub (publish/subscribe) messaging service for sending notifications and enabling communication between distributed systems.
  • Focuses on broadcasting messages to multiple subscribers in real-time.
  • Operates as a push-based system where messages are delivered to subscribers automatically.

2. Core Concepts

3. Key Features

4. Performance and Scalability

• SQS:
  • Throughput: Nearly unlimited for Standard queues (thousands of messages per second). FIFO queues have a limit of 300 transactions per second (or 3,000 with batching).
  • Scalability: Automatically scales to handle high message volumes.
  • Latency: Polling introduces slight latency (configurable with long/short polling).
• SNS:
  • Throughput: High throughput for Standard topics (tens of thousands of messages per second). FIFO topics have lower throughput (similar to SQS FIFO).
  • Scalability: Scales automatically to handle large numbers of publishers and subscribers.
  • Latency: Near real-time delivery to subscribers.

5. Use Cases

Example Use Cases:

• SQS: A web application sends image processing tasks to an SQS queue, which is polled by a fleet of EC2 instances for processing.
• SNS: An e-commerce platform sends order confirmation notifications to customers via email, SMS, and mobile push, while also triggering a Lambda function to update inventory.

6. Integration

• SQS:
  • Commonly used with EC2, Lambda, or ECS for processing messages.
  • Can be integrated with SNS for fan-out (SNS publishes to multiple SQS queues).
  • Supports AWS SDKs and CLI for polling and message management.
• SNS:
  • Integrates with SQS, Lambda, HTTP/S endpoints, email, SMS, and mobile push.
  • Often used with S3, CloudWatch, or IoT for event-driven architectures.
  • Supports AWS SDKs and CLI for publishing and subscription management.

Common Integration: SNS + SQS for fan-out:

• SNS topic receives a message and fans it out to multiple SQS queues.
• Each SQS queue is processed by a different consumer (e.g., one for logging, one for analytics).

7. Security

8. Pricing

• SQS:
  • Charged based on the number of requests (e.g., SendMessage, ReceiveMessage).
  • $0.40 per million requests (first 1 million requests free per month).
  • Additional costs for FIFO queues (higher per-request cost).
  • Data transfer costs may apply for cross-region or external traffic.
• SNS:
  • Charged based on the number of messages published and deliveries.
  • $0.50 per million messages published, plus delivery costs (e.g., $0.02 per 100 SMS in the US).
  • FIFO topics have higher costs.
  • Free tier: 1 million requests and certain deliveries free per month.

Cost Consideration:

• SQS costs depend on polling frequency (e.g., long polling reduces costs).
• SNS costs depend on the number of subscribers and protocol (e.g., SMS is more expensive than SQS).

9. Monitoring and Logging

• SQS:
  • CloudWatch metrics: Queue length, number of messages sent/received, age of oldest message.
  • CloudTrail for API call logging.
  • No native delivery logging; consumers must implement logging.
• SNS:
  • CloudWatch metrics: Number of messages published, delivery successes/failures.
  • CloudWatch Logs for delivery logging (optional, for specific protocols).
  • CloudTrail for API call logging.

10. Limitations

11. When to Use

• Use SQS:
  • When you need to queue tasks for asynchronous processing.
  • When consumers need to control the pace of message processing.
  • When message persistence is required (e.g., retry failed tasks).
  • Example: Processing user uploads, batch jobs, or workflow tasks.
• Use SNS:
  • When you need to broadcast messages to multiple subscribers in real-time.
  • When you need push-based notifications (e.g., email, SMS, mobile push).
  • When implementing event-driven architectures with fan-out.
  • Example: Sending alerts, triggering multiple downstream systems, or notifying users.
• Use SQS + SNS:
  • For fan-out scenarios where a single event needs to trigger multiple processes.
  • Example: SNS topic fans out to multiple SQS queues, each processed by a different service.

12. Example

Example: SNS + SQS Fan-Out Workflow

Scenario

An e-commerce platform needs to process an order event by:

1. Sending a confirmation email to the customer.
2. Logging the order to an analytics queue.
3. Updating inventory via a processing queue.

Workflow

1. Create an SNS Topic:
   • Topic: OrderEvents.
   • ARN: arn:aws:sns:us-east-1:123456789012:OrderEvents.
2. Create SQS Queues:
   • AnalyticsQueue: For logging order data.
   • InventoryQueue: For updating inventory.
3. Subscribe Endpoints to SNS Topic:
   • Email: customer@example.com (protocol: email).
   • AnalyticsQueue (protocol: SQS).
   • InventoryQueue (protocol: SQS).
4. Publish a Message:
   • Publisher sends an order event to the OrderEvents topic:

     {
       "order_id": "12345",
       "customer_email": "customer@example.com",
       "item": "Laptop",
       "quantity": 1
     }
     

5. Message Delivery:
   • SNS delivers the message to:
     • Email: Customer receives a confirmation email.
     • AnalyticsQueue: Analytics service polls the queue and logs the order.
     • InventoryQueue: Inventory service polls the queue and updates stock.

Implementation (AWS CLI)

# Create SNS topic
aws sns create-topic --name OrderEvents

# Create SQS queues
aws sqs create-queue --queue-name AnalyticsQueue
aws sqs create-queue --queue-name InventoryQueue

# Subscribe SQS queues to SNS topic
aws sns subscribe --topic-arn arn:aws:sns:us-east-1:123456789012:OrderEvents \
  --protocol sqs \
  --notification-endpoint arn:aws:sqs:us-east-1:123456789012:AnalyticsQueue

aws sns subscribe --topic-arn arn:aws:sns:us-east-1:123456789012:OrderEvents \
  --protocol sqs \
  --notification-endpoint arn:aws:sqs:us-east-1:123456789012:InventoryQueue

# Subscribe email to SNS topic
aws sns subscribe --topic-arn arn:aws:sns:us-east-1:123456789012:OrderEvents \
  --protocol email \
  --notification-endpoint customer@example.com

# Publish a message
aws sns publish --topic-arn arn:aws:sns:us-east-1:123456789012:OrderEvents \
  --message '{"order_id":"12345","customer_email":"customer@example.com","item":"Laptop","quantity":1}'

Notes

• Use SNS for broadcasting the event to multiple endpoints.
• Use SQS for reliable, asynchronous processing by downstream services.
• Configure DLQs for both SNS and SQS to handle failures.

13. Summary

• SQS is best for queuing and asynchronous processing, with persistent message storage and pull-based consumption.
• SNS is best for real-time notifications and fan-out, with push-based delivery to multiple subscribers.
• Combine SQS and SNS for powerful event-driven architectures, leveraging SNS for broadcasting and SQS for processing.