Multiple messages from Lambda to SQS using Destination trigger

Question

Can I write multiple messages from Lambda to SQS using destination triggers?

My use case is, I have to read an s3 file and add rows to SQS queue which will be further processed by another lambda function. S3 file is huge so I can't process everything using a single lambda function.

Should be doable. How many rows at once do you plan on submitting? There are limits on how much inflight messages there can be in one sqs. — Marcin
– Marcin, Commented May 21, 2020 at 5:00
Doable using destination triggers? I can batch so that I'm not going over inflight limit. — RockerZ
– RockerZ, Commented May 21, 2020 at 5:19
By destination triggers you mean "S3 notifications"? If yes, then the notification triggers only for the entire file, not for each individual row of data. — Marcin
– Marcin, Commented May 21, 2020 at 5:21
No, from Lambda to SQS using this: aws.amazon.com/about-aws/whats-new/2019/11/… — RockerZ
– RockerZ, Commented May 21, 2020 at 5:25
Yes. Just need to be aware of the fact that SQS does not guarantee order of the messages nor uniqueness. Thus your destination functions may get duplicate copies of some rows, and in different order then originally sent. — Marcin
– Marcin, Commented May 21, 2020 at 5:56

RockerZ · Accepted Answer · 2020-05-24 00:05:17Z

You can't directly write multiple messages from single lambda to SQS using destination functions. I ended up writing the messages from lambda-1 to SQS and using another lambda-2 to do row level processing

Collectives™ on Stack Overflow

Multiple messages from Lambda to SQS using Destination trigger

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related