If you do want your Lambda in a VPC for extra security (or other reasons), there are two options I can think of for connecting to RDS: 1) Making it capable of reaching the public internet; or 2) Making it connect to RDS without leaving the AWS Network. You can even use both depending on your use cases (I'll explain the differences later).
For both solutions, first you need to figure which subnets and security group is linked to your lambda:

Solution 1. Associate Elastic IPs to the Lambda's Network Interfaces
Next, go to EC2 Service, find the Public IPs menu under Network & Security. Allocate one IP for each subnet (in the example above there are two subnets).
Go to Network Interfaces menu, find the network interfaces attached to your lambda (same subnet and security group).

Associate the Public IPs in the actions menu for each one:


Solution 2. Create a VPC Endpoint for RDS
Go to VPC Service in the console, open the Endpoints menu option under Virtual private cloud. Click the Create Endpoint button on top.

Choose the AWS Service you want to connect (e.g. RDS, S3, Secrets Manager), select the VPC your lambda is in:

Select the subnets (based on the AZ) and Security Group your lambda is linked to:

Do this for each AWS Service your Lambda needs access to (e.g. RDS, S3, etc).
Explanation and How to choose one
Your Lambda loses access to public internet when inside a VPC, so you can't fetch data from external services/APIs, which includes most AWS Services because connection is made via public internet.
Solution 1 will make your Lambda capable of reaching public internet again, which is convenient.
Solution 2 will make your Lambda capable of connecting to AWS Services without leaving the AWS Network, which reduces latency and improves security.
If your lambda only needs access to RDS and a few other AWS Services, you can go with solution 2. If your lambda needs to access external APIs (unrelated to AWS), you need solution 1, but even in that case you could mix them and use solution 2 for better security and reduced latency, so you can use both.