10

How can I create an RDS instance with the create-environment or another subcommand of aws elasticbeanstalk? I've tried several combinations of parameters to no avail. Below is an example.

APP_NAME="randall-railsapp" aws s3api create-bucket --bucket "$APP_NAME" APP_VERSION="$(git describe --always)" APP_FILE="deploy-$APP_NAME-$APP_VERSION.zip" git archive -o "$APP_FILE" HEAD aws s3 cp "$APP_FILE" "s3://$APP_NAME/$APP_FILE" aws --region us-east-1 elasticbeanstalk create-application-version \ --auto-create-application \ --application-name "$APP_NAME" \ --version-label "$APP_VERSION" \ --source-bundle S3Bucket="$APP_NAME",S3Key="$APP_FILE" aws --region us-east-1 elasticbeanstalk create-environment \ --application-name "$APP_NAME" \ --version-label "$APP_VERSION" \ --environment-name "$APP_NAME-env" \ --description "randall's rails app environment" \ --solution-stack-name "64bit Amazon Linux 2014.03 v1.0.0 running Ruby 2.1 (Puma)" \ --cname-prefix "$APP_NAME-test" \ --option-settings file://test.json

And the contents of test.json:

[ { "OptionName": "EC2KeyName", "Namespace": "aws:autoscaling:launchconfiguration", "Value": "a-key-is-here" }, { "OptionName": "EnvironmentType", "Namespace": "aws:elasticbeanstalk:environment", "Value": "SingleInstance" }, { "OptionName": "SECRET_KEY_BASE", "Namespace": "aws:elasticbeanstalk:application:environment", "Value": "HAHAHAHAHAHA" }, { "OptionName": "DBPassword", "Namespace": "aws:rds:dbinstance", "Value": "hunter2" }, { "OptionName": "DBUser", "Namespace": "aws:rds:dbinstance", "Value": "random" }, { "OptionName": "DBEngineVersion", "Namespace": "aws:rds:dbinstance", "Value": "9.3" }, { "OptionName": "DBEngine", "Namespace": "aws:rds:dbinstance", "Value": "postgres" } ]

Anyone know why this is failing? Anything I specify with a aws:rds:dbinstance namespace seems to get removed from the configuration.

Improve this question

4 Answers 4

Reset to default
8

Just setting the aws:rds:dbinstance options does not create an RDS database. Currently you can create an RDS instance using one of the following techniques:

  1. Create using AWS Console
  2. Use eb cli
  3. Use Resources section of ebextensions to create an RDS resource

The first two approaches are the most convenient as they do all the heavy lifting for you but for the third one you have to do some extra work. The third approach is what you would want to use if you are not using the console or eb CLI.

You can create an RDS resource for your beanstalk environment using the following ebextension snippet. Create a file called 01-rds.config in the .ebextensions directory of your app source.

Resources: AWSEBRDSDatabase: Type: AWS::RDS::DBInstance Properties: AllocatedStorage: 5 DBInstanceClass: db.t2.micro DBName: myawesomeapp Engine: postgres EngineVersion: 9.3 MasterUsername: myAwesomeUsername MasterUserPassword: myCrazyPassword

This file is in YAML format so indentation is important. You could also use JSON if you like. These are not option settings so you cannot pass it as --option-settings test.json. You just need to bundle this file with your app source.

Read more about what properties you can configure on your RDS database here. On this page you can also find what properties are required and what properties are optional.

Let me know if the above does not work for you or if you have any further questions.

Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

So is the 01-rds.config how the EB command line tools do it?
Also did you try this? When I try it my environments instantly terminate and I can't seem to get any logs from them.
SWEET I figured it out. The ecstatic joy I'm experiencing right now is fantastic. You guys rock. I've made a small edit to your answer correcting the parameter names for 01-rds.config I've figured out this is how the eb command line tools do it.
I'm now having issues getting the environment variables to go through.
What specific issues? You can see how CUSTOM_ENV is set here using a config file containing option settings. stackoverflow.com/a/14491294/161628 Since it is an option setting you can either specify it in .ebextensions or pass it through the command line option --option-settings
|
3

As of December 2017 we use the following ebextensions

$ cat .ebextensions/rds.config Resources: AWSEBRDSDBSecurityGroup: Type: AWS::RDS::DBSecurityGroup Properties: EC2VpcId: Fn::GetOptionSetting: OptionName: "VpcId" GroupDescription: RDS DB VPC Security Group DBSecurityGroupIngress: - EC2SecurityGroupId: Ref: AWSEBSecurityGroup AWSEBRDSDBSubnetGroup: Type: AWS::RDS::DBSubnetGroup Properties: DBSubnetGroupDescription: RDS DB Subnet Group SubnetIds: Fn::Split: - "," - Fn::GetOptionSetting: OptionName: DBSubnets AWSEBRDSDatabase: Type: AWS::RDS::DBInstance DeletionPolicy: Delete Properties: PubliclyAccessible: true MultiAZ: false Engine: mysql EngineVersion: 5.7 BackupRetentionPeriod: 0 DBName: test MasterUsername: toor MasterUserPassword: 123456789 AllocatedStorage: 10 DBInstanceClass: db.t2.micro DBSecurityGroups: - Ref: AWSEBRDSDBSecurityGroup DBSubnetGroupName: Ref: AWSEBRDSDBSubnetGroup Outputs: RDSId: Description: "RDS instance identifier" Value: Ref: "AWSEBRDSDatabase" RDSEndpointAddress: Description: "RDS endpoint address" Value: Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Address"] RDSEndpointPort: Description: "RDS endpoint port" Value: Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Port"] AWSEBRDSDatabaseProperties: Description: Properties associated with the RDS database instance Value: Fn::Join: - "," - - Ref: AWSEBRDSDatabase - Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Address"] - Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Port"]

With such custom options

$ cat .ebextensions/custom-options.config option_settings: "aws:elasticbeanstalk:customoption": DBSubnets: subnet-1234567,subnet-7654321 VpcId: vpc-1234567

The only things - you have to explicitly pass RDS_* env variables to your application.

Improve this answer

Comments

2

The other answers did not work in my environment as of Sept 2015. After much trial and error, the following worked for me:

config template snippet (YAML):

 aws:rds:dbinstance: DBAllocatedStorage: '5' DBDeletionPolicy: Delete DBEngine: postgres DBEngineVersion: 9.3.9 DBInstanceClass: db.t2.micro DBPassword: PASSWORD_HERE DBUser: USERNAME_HERE MultiAZDatabase: false

.ebextensions/rds.config file (JSON):

{ "Parameters": { "AWSEBDBUser": { "NoEcho": "true", "Description": "The name of master user for the client DB Instance.", "Default": "ebroot", "Type": "String", "ConstraintDescription": "must begin with a letter and contain only alphanumeric characters" }, "AWSEBDBPassword": { "NoEcho": "true", "Description": "The master password for the DB instance.", "Type": "String", "ConstraintDescription": "must contain only alphanumeric characters" }, "AWSEBDBName": { "NoEcho": "true", "Description": "The DB Name of the RDS instance", "Default": "ebdb", "Type": "String", "ConstraintDescription": "must contain only alphanumeric characters" } }, "Resources": { "AWSEBAutoScalingGroup": { "Metadata": { "AWS::ElasticBeanstalk::Ext": { "_ParameterTriggers": { "_TriggerConfigDeployment": { "CmpFn::Insert": { "values": [ { "CmpFn::Ref": "Parameter.AWSEBDBUser" }, { "CmpFn::Ref": "Parameter.AWSEBDBPassword" }, { "CmpFn::Ref": "Parameter.AWSEBDBName" } ] } } }, "_ContainerConfigFileContent": { "plugins": { "rds": { "Description": "RDS Environment variables", "env": { "RDS_USERNAME": { "Ref": { "CmpFn::Ref": "Parameter.AWSEBDBUser" } }, "RDS_PASSWORD": { "Ref": { "CmpFn::Ref": "Parameter.AWSEBDBPassword" } }, "RDS_DB_NAME": { "Ref": { "CmpFn::Ref": "Parameter.AWSEBDBName" } }, "RDS_HOSTNAME": { "Fn::GetAtt": [ "AWSEBRDSDatabase", "Endpoint.Address" ] }, "RDS_PORT": { "Fn::GetAtt": [ "AWSEBRDSDatabase", "Endpoint.Port" ] } } } } } } } }, "AWSEBRDSDatabase": { "Type": "AWS::RDS::DBInstance", "DeletionPolicy": "Delete", "Properties": { "DBName": { "Ref": { "CmpFn::Ref": "Parameter.AWSEBDBName" } }, "AllocatedStorage": "5", "DBInstanceClass": "db.t2.micro", "Engine": "postgres", "DBSecurityGroups": [ { "Ref": "AWSEBRDSDBSecurityGroup" } ], "MasterUsername": { "Ref": { "CmpFn::Ref": "Parameter.AWSEBDBUser" } }, "MasterUserPassword": { "Ref": { "CmpFn::Ref": "Parameter.AWSEBDBPassword" } }, "MultiAZ": false } }, "AWSEBRDSDBSecurityGroup": { "Type": "AWS::RDS::DBSecurityGroup", "Properties": { "DBSecurityGroupIngress": { "EC2SecurityGroupName": { "Ref": "AWSEBSecurityGroup" } }, "GroupDescription": "Enable database access to Beanstalk application" } } } }
Improve this answer

1 Comment

Update that as of Jan 2016 the above approach no longer works.
1

I had the same problem, couldn't get it to work via .ebextensions, and I don't like the EB CLI tool.

EB CLI uses an undocumented API feature, and a customized version of the botocore library ('eb_botocore') to make this work. :(

So I went ahead and forked botocore, and merged in the API data file used by eb_botocore: https://github.com/boto/botocore/pull/396

Then I ran 'python setup.py install' on both my modified botocore and aws-cli (both at master), and aws-cli now accepts a --template-specification option on the 'aws elasticbeanstalk create-environment' command. Hooray!

Example usage:

aws elasticbeanstalk create-environment\ ...various options...\ --option-settings file://option-settings.json --template-specification file://rds.us-west-2.json

where rds.us-west-2.json is:

{ "TemplateSnippets": [{ "SnippetName": "RdsExtensionEB", "Order": 10000, "SourceUrl": "https://s3.amazonaws.com/elasticbeanstalk-env-resources-us-west-2/eb_snippets/rds/rds.json" }] }

(it appears you must select a snippet specific to your EB region).

and option-settings.json contains RDS-related settings similar to ones listed in the question (DBEngine, DBInstanceClass, DBAllocatedStorage, DBPassword).

It works great. I hope the AWS CLI team allows us to use this feature in the official tool in the future. I'm guessing it's not a trivial change or they would have done it already, but it's a pretty major omission functionality-wise from the Elastic Beanstalk API and AWS CLI tool, so hopefully they take a crack at it.

Improve this answer

1 Comment

Just an update that this answer isn't up to date as of 9/2015 the CLI is pretty different now.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.