Loading

Connect to deployments in another Elastic Cloud organization

ECH

This section explains how to configure a deployment to connect remotely to clusters belonging to a different Elastic Cloud organization.

Note about terminology

In the case of remote clusters, the Elasticsearch cluster or deployment initiating the connection and requests is often referred to as the local cluster, while the Elasticsearch cluster or deployment receiving the requests is referred to as the remote cluster.

Note

If network security policies are applied to the remote cluster, the remote cluster administrator must configure a private connection policy of type remote cluster, using either the organization ID or the Elasticsearch cluster ID of the local cluster as the filtering criteria. For more information, refer to Remote clusters and network security.

Before you start, consider the security model that you would prefer to use for authenticating remote connections between clusters, and follow the corresponding steps.

API key
For deployments based on Elastic Stack 8.14 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model uses a dedicated service endpoint, on port 9443 by default, and gives administrators fine-grained control over remote access. The API key is created on the remote cluster and defines the permissions available to all cross-cluster requests, while local user roles can further restrict, but not extend, those permissions.
TLS certificate (deprecated in Elastic Stack 9.0.0)
This model uses mutual TLS authentication over the Elasticsearch transport interface for cross-cluster operations. User authentication is performed on the local cluster and a user's role names are passed to the remote cluster for authorization. Because a superuser on the local cluster automatically gains full read access to the remote cluster, this model is only suitable for clusters within the same security domain.

Follow these steps to configure the API key security model for remote clusters. If you run into any issues, refer to Troubleshooting.

  • The local and remote deployments must be on Elastic Stack 8.14 or later.
  • Contrary to the certificate security model, the API key security model does not require that both local and remote clusters trust each other.
  • On the deployment you will use as remote, use the Elasticsearch API or Kibana to create a cross-cluster API key. Configure it with access to the indices you want to use for cross-cluster search or cross-cluster replication.
  • Copy the encoded key (encoded in the response) to a safe location. You will need it in the next step.

The API key created previously will be used by the local deployment to authenticate with the corresponding set of permissions to the remote deployment. For that, you need to add the API key to the local deployment's keystore.

  1. Log in to the Elastic Cloud Console.

  2. On the home page, find your hosted deployment and select Manage to access it directly. Or, select Hosted deployments to go to the Hosted deployments page to view all of your deployments.

    On the Hosted deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list.

  3. From the navigation menu, select Security.

  4. Locate Remote Connections > Trust management > Connections using API keys and select Add API key.

    1. Fill both fields.

      • For the Remote cluster name, enter the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores.
      • For the Cross-cluster API key, paste the encoded cross-cluster API key.

    2. Click Add to save the API key.

  5. Restart the local deployment to reload the new setting. To do that, go to the deployment's main page, locate the Actions menu, and select Restart Elasticsearch.

    Note

    If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys.

If you need to update the remote connection with different permissions later, refer to Change a cross-cluster API key used for a remote connection.

A deployment can be configured to trust all or specific deployments in another Elastic Cloud Hosted organization. To add cross-organization trust:

  1. From the Security page, select Remote Connections > Add trusted environment and select Elastic Cloud. Then click Next.

  2. Select Certificates as authentication mechanism and click Next.

  3. Enter the ID of the deployment’s organization which you want to establish trust with. You can find that ID on the Organization page. It is usually made of 10 digits.

  4. Choose one of following options to configure the level of trust with the other organization:

    • All deployments - This deployment trusts all deployments in the other organization, including new deployments when they are created.
    • Specific deployments - Specify which of the existing deployments you want to trust in the other organization. The full Elasticsearch cluster ID must be entered for each remote cluster. The Elasticsearch Cluster ID can be found in the deployment overview page under Applications.

  5. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s Security page.

  6. Select Create trust to complete the configuration.

  7. Repeat these steps from each of the deployments you want to use for CCS or CCR in both organizations. You will only be able to connect two deployments successfully when both of them trust each other.

Note

The organization ID and cluster IDs must be entered fully and correctly. For security reasons, verification of the IDs is not possible. If cross-organization trust does not appear to be working, double-checking the IDs is a good place to start.

Using the API

You can update a deployment using the appropriate trust settings for the Elasticsearch payload.

In order to trust a deployment with cluster id cf659f7fe6164d9691b284ae36811be1 (NOTE: use the Elasticsearch cluster ID, not the deployment ID) in another organization with Organization ID 1053523734, you need to update the trust settings with an additional account like this:

 { "trust":{ "accounts":[ { "account_id":"ec38dd0aa45f4a69909ca5c81c27138a", "trust_all":true }, { "account_id":"1053523734", "trust_all":false, "trust_allowlist":[ "cf659f7fe6164d9691b284ae36811be1" ] } ] } }

You can now connect remotely to the trusted clusters.

On the local cluster, add the remote cluster using Kibana or the Elasticsearch API.

  1. Go to the Remote Clusters management page in the navigation menu or use the global search field.

  2. Select Add a remote cluster.

  3. In Select connection type, choose the authentication mechanism you prepared earlier (API keys or Certificates), and then click Next.

  4. In Add connection information, fill in the following fields:

    • Remote cluster name: This cluster alias is a unique identifier that represents the connection to the remote cluster and is used to distinguish local and remote indices.

      When using API key authentication, this alias must match the Remote cluster name you configured when adding the API key in the Cloud UI.

    • Remote address: This value can be found on the Security page of the Elastic Cloud Hosted deployment you want to use as a remote. Copy the Proxy address from the Remote cluster parameters section.

      Note

      If you’re using API keys as security model, change the port to 9443.

    • Configure advanced options (optional): Expand this section if you need to customize additional settings.

      • TLS server name: Specify a value if the certificate presented by the remote cluster is signed for a different name than the remote address.

        This value can be found on the Security page of the Elastic Cloud Hosted deployment you want to use as a remote. Copy the Server name from the Remote cluster parameters section.

      • Socket connections: Define the number of connections to open with the remote cluster.

    For a full list of available client connection settings, refer to the remote cluster settings reference.

  5. Click Next.

  6. In Confirm setup, click Add remote cluster (you have already established trust in a previous step).

To configure a deployment as a remote cluster, use the cluster update settings API. Configure the following fields:

  • Remote cluster alias: When using API key authentication, the cluster alias must match the one you configured when adding the API key in the Cloud UI as Remote cluster name.

  • mode: proxy

  • proxy_address: This value can be found on the Security page of the Elastic Cloud Hosted deployment you want to use as a remote. Copy the Proxy address from the Remote cluster parameters section.

    Using the API, this value can be obtained from the Elasticsearch resource info, concatenating the field metadata.endpoint and port 9400 using a semicolon.

    Note

    If you’re using API keys as security model, change the port to 9443.

  • server_name: This value can be found on the Security page of the Elastic Cloud Hosted deployment you want to use as a remote. Copy the Server name from the Remote cluster parameters section.

    Using the API, this can be obtained from the Elasticsearch resource info field metadata.endpoint.

This example shows the API call to add or update a remote cluster. The alias alias-for-my-remote-cluster must match the remote cluster name used when adding the API key to the deployment:

 PUT /_cluster/settings { "persistent": { "cluster": { "remote": { "alias-for-my-remote-cluster": { "mode":"proxy", "proxy_address": "<REMOTE_CLUSTER_ADDRESS>:9443", "server_name": "<REMOTE_CLUSTER_SERVER_NAME>" } } } } }
  1. Remote cluster alias

For a full list of available client connection settings in proxy mode, refer to the remote cluster settings reference.

If you're using the API key based security model, to use a remote cluster for cross-cluster replication or cross-cluster search, you need to create user roles with remote indices privileges on the local cluster. Refer to Configure roles and users.