2

I have a lambda function which is triggered by a FIFO SQS. I only want one instance of the function running. The function invokes a state machine and the state machine takes longer to finish than the lambda function. I want the lambda function to finish only after the step function has completed it's execution.

Improve this question

3 Answers 3

Reset to default
5

AWS Express Step Functions now supports synchronous invocations.

Keep in mind that Express Step Functions are meant for workloads that require higher event rates and have shorter durations. Also, keep in mind that you will be billed based on the number of requests and the duration of the workflow just like lambda.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

3

AWS Step Functions are only invoked asynchronously. A state machine can run for up to 1 year so synchronous invocation is not possible. Depending on your workflow you might find Activities useful.

Activities are an AWS Step Functions feature that enables you to have a task in your state machine where the work is performed by a worker that can be hosted on Amazon Elastic Compute Cloud (Amazon EC2), Amazon Elastic Container Service (Amazon ECS), mobile devices—basically anywhere.

Improve this answer

1 Comment

stackoverflow.com/a/65137575/8553816 as posted at the bottom now step functions support synchronous transactions.
2

Edit - note that the solution below is usually unnecessary now, since AWS has introduced ways to invoke express step functions in a synchronous manner

If you expect your state machine to complete in a reasonable time, then you can get your Lambda (or anything else that starts a state machine execution) to await a response.

This is an accepted pattern, but has quite a few moving parts:

  • Set up an SNS topic; state machine results will be published onto it
  • During your Lambda's initialisation:
    • generate a random "gateway ID"
    • create a SQS queue
    • subscribe to the SQS queue, filtering by gateway ID
  • When your Lambda invokes the state machine:
    • generate a random "correlation ID"
    • include both the gateway and correlation IDs in the inputs to the state machine
  • As the final step of the state machine, publish the result to the SNS topic. Include the gateway and correlation IDs in the response.
  • In the lambda, after submitting the state machine execution:
    • await a message on the SQS queue.
    • give some thought to timeouts on this. You should set a timeout for the response. You can therefore set a time-to-live on the state machine -- if its outcome is going to be useless once this Lambda dies.
    • When it arrives, ensure that its correlation ID matches yours, then use the data in that message as your response.
Improve this answer

2 Comments

This is a good solution. I am curious what would happen to the messages that the Lambda polls from SQS that does not match with the gateway and/or correlation IDs? Would the Lambda put them back without processing? Also, there could be multiple such Lambdas waiting for their own messages, which could be picked up by others. What are your thoughts on this?
@ACloudRoamer By filtering on the gatewayId, each subscriber should only get events it's expecting. If another subscriber also gets it, that doesn't affect the legitimate subscriber. If this goes wrong, it's up to you how to handle it. You could silently drop uncorrelated events, log it, or DLQ it.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.