Remote clusters missing use cases and structural changes

deploy-manage/remote-clusters/_snippets/apikeys-create-key.md

@@ -1,2 +1,2 @@
-* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [{{kib}}](/deploy-manage/api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}.
-* Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next step.
+1. On the remote cluster, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [{{kib}}](/deploy-manage/api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it to include access to the indices you want to use for {{ccs}} or {{ccr}}.
+2. Copy the encoded key (`encoded` in the response) to a safe location. It is required for the local cluster configuration.

deploy-manage/remote-clusters/_snippets/apikeys-local-config-intro.md

@@ -1 +1 @@
-The API key created previously will be used by the local deployment to authenticate with the corresponding set of permissions to the remote deployment. For that, you need to add the API key to the local deployment's keystore.
+The API key created previously is needed by the local {{local_type_generic}} to authenticate with the corresponding set of permissions to the remote {{remote_type_generic}}. To enable this, add the API key to the local {{local_type_generic}}'s keystore.

deploy-manage/remote-clusters/_snippets/apikeys-prerequisites-limitations.md

@@ -1,2 +1,2 @@
 * The local and remote deployments must be on {{stack}} 8.14 or later.
-* Contrary to the certificate security model, the API key security model does not require that both local and remote clusters trust each other.
+* Unlike the certificate-based security model, the API key model does not require mutual trust between clusters; only the local cluster is required to trust the remote cluster’s certificate.

deploy-manage/remote-clusters/_snippets/configure-roles-and-users.md

@@ -1,3 +1,3 @@
 % this will need improvement in a future PR, as the text below is only valid for API key based security model
 
-If you're using the API key based security model, to use a remote cluster for {{ccr}} or {{ccs}}, you need to create user roles with [remote indices privileges](/deploy-manage/users-roles/cluster-or-deployment-auth/role-structure.md#roles-remote-indices-priv) on the local cluster. Refer to [Configure roles and users](/deploy-manage/remote-clusters/remote-clusters-api-key.md#remote-clusters-privileges-api-key).
+If you're using the API key–based security model for {{ccr}} or {{ccs}}, you can define user roles with [remote indices privileges](/deploy-manage/users-roles/cluster-or-deployment-auth/role-structure.md#roles-remote-indices-priv) on the local cluster to further restrict the permissions granted by the API key. For more details, refer to [Configure roles and users](/deploy-manage/remote-clusters/remote-clusters-api-key.md#remote-clusters-privileges-api-key).

deploy-manage/remote-clusters/_snippets/eck_apikey_secret.md

@@ -0,0 +1,15 @@
+The following command creates a secret containing the encoded API key obtained earlier:
+
+```sh
+cat <<EOF | kubectl apply -f -
+apiVersion: v1
+kind: Secret
+metadata:
+  name: remote-api-keys
+type: Opaque
+stringData:
+  cluster.remote.<remote-cluster-name>.credentials: <encoded value> <1>
+EOF
+```
+1. For the `<remote-cluster-name>`, enter the alias of your choice. This alias is used when connecting to the remote cluster. It must be lowercase and only contain letters, numbers, dashes, and underscores.
+

deploy-manage/remote-clusters/_snippets/eck_expose_transport.md

@@ -1,10 +1,16 @@
+<!--
+This snippet is in use in the following locations:
+- eck-remote-clusters-from-external.md
+- ec-enable-ccs-for-eck.md
+- ece-enable-ccs-for-eck.md
+-->
 Expose the transport service (defaults to port `9300`) of your ECK cluster to allow external {{es}} clusters to connect:
 
 ```yaml
 apiVersion: elasticsearch.k8s.elastic.co/v1
 kind: Elasticsearch
 metadata:
-  name: <cluster-name>
+  name: <remote-cluster-name>
 spec:
   transport:
     service:

deploy-manage/remote-clusters/_snippets/eck_rcs_connect_intro.md

@@ -1,4 +1,12 @@
-On the local deployment, add the remote ECK cluster using {{kib}} or the {{es}} API with the following connection settings:
+<!--
+This snippet is in use in the following locations:
+- eck-remote-clusters-from-external.md
+- ec-enable-ccs-for-eck.md
+- ece-enable-ccs-for-eck.md
+
+It requires local_type_generic substitution to be defined
+-->
+On the local {{local_type_generic}}, add the remote ECK cluster using {{kib}} or the {{es}} API with the following connection settings:
 
 * **Remote address**: Use the FQDN or IP address of the LoadBalancer service, or similar resource, you created to expose the remote cluster server interface (for API key-based authentication) or the transport interface (for TLS certificate-based authentication).
 

deploy-manage/remote-clusters/_snippets/eck_rcs_enable.md

@@ -1,4 +1,4 @@
-By default, the remote cluster server interface is deactivated on ECK-managed clusters. To use the API key–based security model for cross-cluster connections, you must first enable it on the remote {{es}} cluster:
+By default, the remote cluster server interface is deactivated on ECK-managed clusters. To use the API key–based security model for cross-cluster connections, you must first enable it on the remote {{es}} cluster by setting `spec.remoteClusterServer.enabled: true`:
 
 ```yaml subs=true
 apiVersion: elasticsearch.k8s.elastic.co/v1

deploy-manage/remote-clusters/_snippets/eck_rcs_intro.md

@@ -0,0 +1 @@
+The [remote clusters module](/deploy-manage/remote-clusters.md) in {{es}} enables you to establish uni-directional connections to a remote cluster. This functionality is used in cross-cluster replication (CCR) and cross-cluster search (CCS).

deploy-manage/remote-clusters/_snippets/rcs-elasticsearch-api-snippet-self.md

@@ -4,10 +4,11 @@ This snippet is in use in the following locations:
 - ec-remote-cluster-self-managed.md
 - ece-enable-ccs-for-eck.md
 - ec-enable-ccs-for-eck.md
+- eck-remote-clusters-from-external.md
 -->
 To add a remote cluster, use the [cluster update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings). Configure the following fields:
 
-* `Remote cluster alias`: When using API key authentication, the cluster alias must match the one you configured when adding the API key in the Cloud UI as **Remote cluster name**.
+* `Remote cluster alias`: When using API key authentication, the cluster alias must match the one you configured when adding the API key.
 * `mode`: `proxy`
 * `proxy_address`: Enter the endpoint of the remote cluster, including the hostname, FQDN, or IP address, and the port. Both IPv4 and IPv6 addresses are supported.
 

deploy-manage/remote-clusters/_snippets/rcs-kibana-api-snippet-self.md

@@ -4,6 +4,7 @@ This snippet is in use in the following locations:
 - ec-remote-cluster-self-managed.md
 - ece-enable-ccs-for-eck.md
 - ec-enable-ccs-for-eck.md
+- eck-remote-clusters-from-external.md
 -->
 1. Go to the **Remote Clusters** management page in the navigation menu or use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md).
 2. Select **Add a remote cluster**.
@@ -13,7 +14,7 @@ This snippet is in use in the following locations:
 
     * **Remote cluster name**: This *cluster alias* is a unique identifier that represents the connection to the remote cluster and is used to distinguish local and remote indices.
 
-      When using API key authentication, this alias must match the **Remote cluster name** you configured when adding the API key in the Cloud UI.
+      When using API key authentication, this alias must match the **Remote cluster name** you configured when adding the API key.
     * **Remote address**: Enter the endpoint of the remote cluster, including the hostname, FQDN, or IP address, and the port.
 
       Make sure you use the correct port for your authentication method:

deploy-manage/remote-clusters/_snippets/self_rcs_enable.md

@@ -0,0 +1,60 @@
+1. Enable the remote cluster server on every node of the remote cluster. In [`elasticsearch.yml`](/deploy-manage/stack-settings.md):
+
+    1. Set [`remote_cluster_server.enabled`](elasticsearch://reference/elasticsearch/configuration-reference/networking-settings.md#remote-cluster-network-settings) to `true`.
+    2. Configure the bind and publish address for remote cluster server traffic, for example using [`remote_cluster.host`](elasticsearch://reference/elasticsearch/configuration-reference/networking-settings.md#remote-cluster-network-settings). Without configuring the address, remote cluster traffic can be bound to the local interface, and remote clusters running on other machines can't connect.
+    3. Optionally, configure the remote server port using [`remote_cluster.port`](elasticsearch://reference/elasticsearch/configuration-reference/networking-settings.md#remote_cluster.port) (defaults to `9443`).
+
+2. Next, generate a certificate authority (CA) and a server certificate/key pair. On one of the nodes of the remote cluster, from the directory where {{es}} has been installed:
+
+    1. Create a CA, if you don't have a CA already:
+
+        ```sh
+        ./bin/elasticsearch-certutil ca --pem --out=cross-cluster-ca.zip --pass CA_PASSWORD
+        ```
+
+        Replace `CA_PASSWORD` with the password you want to use for the CA. You can remove the `--pass` option and its argument if you are not deploying to a production environment.
+
+    2. Unzip the generated `cross-cluster-ca.zip` file. This compressed file contains the following content:
+
+        ```txt
+        /ca
+        |_ ca.crt
+        |_ ca.key
+        ```
+
+    3. Generate a certificate and private key pair for the nodes in the remote cluster:
+
+        ```sh
+        ./bin/elasticsearch-certutil cert --out=cross-cluster.p12 --pass=CERT_PASSWORD --ca-cert=ca/ca.crt --ca-key=ca/ca.key --ca-pass=CA_PASSWORD --dns=<CLUSTER_FQDN> --ip=192.0.2.1
+        ```
+
+        * Replace `CA_PASSWORD` with the CA password from the previous step.
+        * Replace `CERT_PASSWORD` with the password you want to use for the generated private key.
+        * Use the `--dns` option to specify the relevant DNS name for the certificate. You can specify it multiple times for multiple DNS.
+        * Use the `--ip` option to specify the relevant IP address for the certificate. You can specify it multiple times for multiple IP addresses.
+
+    4. If the remote cluster has multiple nodes, you can either:
+
+        * create a single wildcard certificate for all nodes;
+        * or, create separate certificates for each node either manually or in batch with the [silent mode](elasticsearch://reference/elasticsearch/command-line-tools/certutil.md#certutil-silent).
+
+3. On every node of the remote cluster:
+
+    1. Copy the `cross-cluster.p12` file from the earlier step to the `config` directory. If you didn't create a wildcard certificate, make sure you copy the correct node-specific p12 file.
+    2. Add following configuration to [`elasticsearch.yml`](/deploy-manage/stack-settings.md):
+
+        ```yaml
+        xpack.security.remote_cluster_server.ssl.enabled: true
+        xpack.security.remote_cluster_server.ssl.keystore.path: cross-cluster.p12
+        ```
+
+    3. Add the SSL keystore password to the {{es}} keystore:
+
+        ```sh
+        ./bin/elasticsearch-keystore add xpack.security.remote_cluster_server.ssl.keystore.secure_password
+        ```
+
+        When prompted, enter the `CERT_PASSWORD` from the earlier step.
+
+4. Restart the remote cluster.
+

deploy-manage/remote-clusters/_snippets/self_rcs_local_config.md

@@ -0,0 +1,30 @@
+<!--
+This snippet is in use in the following locations:
+- remote-clusters-api-key.md
+- eck-remote-clusters-from-external.md
+-->
+1. On every node of the local cluster:
+
+    1. Copy the `ca.crt` file generated on the remote cluster earlier into the `config` directory, renaming the file `remote-cluster-ca.crt`.
+    2. Add following configuration to [`elasticsearch.yml`](/deploy-manage/stack-settings.md):
+
+        ```yaml
+        xpack.security.remote_cluster_client.ssl.enabled: true
+        xpack.security.remote_cluster_client.ssl.certificate_authorities: [ "remote-cluster-ca.crt" ]
+        ```
+
+        ::::{tip}
+        If the remote cluster uses a publicly trusted certificate, don't include the `certificate_authorities` setting. This example assumes the remote is using the private certificates [created earlier](#remote-clusters-security-api-key-remote-action), which require the CA to be added.
+        ::::
+
+    3. Add the cross-cluster API key, created on the remote cluster earlier, to the keystore:
+
+        ```sh
+        ./bin/elasticsearch-keystore add cluster.remote.ALIAS.credentials
+        ```
+
+        Replace `ALIAS` with the same name that you intend to use to create the remote cluster entry later. When prompted, enter the encoded cross-cluster API key created on the remote cluster earlier.
+
+2. Restart the local cluster to load changes to the keystore and settings.
+
+    If you are configuring only the cross-cluster API key, you can use the [Nodes reload secure settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) instead of restarting the cluster. Configuring the `remote_cluster_client` settings in `elasticsearch.yml` still requires a restart.

deploy-manage/remote-clusters/ec-enable-ccs-for-eck.md

@@ -9,6 +9,8 @@ applies_to:
 products:
   - id: cloud-hosted
 sub:
+  local_type_generic: deployment
+  remote_type_generic: cluster
   remote_type: Self-managed
 ---
 

deploy-manage/remote-clusters/ec-remote-cluster-ece.md

@@ -9,6 +9,8 @@ applies_to:
 products:
   - id: cloud-hosted
 sub:
+  local_type_generic: deployment
+  remote_type_generic: deployment
   remote_type: Elastic Cloud Enterprise
 ---
 

deploy-manage/remote-clusters/ec-remote-cluster-other-ess.md

@@ -8,6 +8,8 @@ applies_to:
 products:
   - id: cloud-hosted
 sub:
+  local_type_generic: deployment
+  remote_type_generic: deployment
   remote_type: Elastic Cloud Hosted
 ---
 

deploy-manage/remote-clusters/ec-remote-cluster-same-ess.md

@@ -8,6 +8,8 @@ applies_to:
 products:
   - id: cloud-hosted
 sub:
+  local_type_generic: deployment
+  remote_type_generic: deployment
   remote_type: Elastic Cloud Hosted
 ---
 

deploy-manage/remote-clusters/ec-remote-cluster-self-managed.md

@@ -9,6 +9,8 @@ applies_to:
 products:
   - id: cloud-hosted
 sub:
+  local_type_generic: deployment
+  remote_type_generic: cluster
   remote_type: Self-managed
 ---
 

deploy-manage/remote-clusters/ece-enable-ccs-for-eck.md

@@ -9,6 +9,8 @@ applies_to:
 products:
   - id: cloud-enterprise
 sub:
+  local_type_generic: deployment
+  remote_type_generic: cluster
   remote_type: Self-managed
 ---
 

deploy-manage/remote-clusters/ece-remote-cluster-ece-ess.md

@@ -9,6 +9,8 @@ applies_to:
 products:
   - id: cloud-enterprise
 sub:
+  local_type_generic: deployment
+  remote_type_generic: deployment
   remote_type: Elastic Cloud Hosted
 ---
 

deploy-manage/remote-clusters/ece-remote-cluster-other-ece.md

@@ -8,6 +8,8 @@ applies_to:
 products:
   - id: cloud-enterprise
 sub:
+  local_type_generic: deployment
+  remote_type_generic: deployment
   remote_type: Elastic Cloud Enterprise
 ---
 

deploy-manage/remote-clusters/ece-remote-cluster-same-ece.md

@@ -8,6 +8,8 @@ applies_to:
 products:
   - id: cloud-enterprise
 sub:
+  local_type_generic: deployment
+  remote_type_generic: deployment
   remote_type: Elastic Cloud Enterprise
 ---
 

deploy-manage/remote-clusters/ece-remote-cluster-self-managed.md

@@ -9,6 +9,8 @@ applies_to:
 products:
   - id: cloud-enterprise
 sub:
+  local_type_generic: deployment
+  remote_type_generic: cluster
   remote_type: Self-managed
 ---