Skip to content

Commit

Permalink
Merge pull request #2561 from kuzzleio/beta
Browse files Browse the repository at this point in the history 
Update the documentation
  • Loading branch information
@rolljee
rolljee authored Oct 31, 2024
2 parents 441bbc8 + f487c89 commit 0b09e92
Show file tree
Hide file tree
Showing 9 changed files with 816 additions and 221 deletions.
2 changes: 1 addition & 1 deletion doc/2/api/controllers/security/create-first-admin/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Creates a Kuzzle administrator account, only if none exist.
### HTTP

```http
URL: http://kuzzle:7512/<kuid>/_createFirstAdmin[?reset]
URL: http://kuzzle:7512/_createFirstAdmin/<kuid>[?reset]
URL(2): http://kuzzle:7512/_createFirstAdmin[?reset]
Method: POST
Body:
Expand Down
107 changes: 35 additions & 72 deletions doc/2/guides/elasticsearch/migrate-project-from-v7-to-v8/index.md
View file
Original file line number Diff line number Diff line change
@@ -1,94 +1,57 @@
---
code: false
type: page
order: 50
title: Migrate to Elasticsearch 8 | Elasticsearch | Guide | Core
order: 200
title: Elasticsearch 8 | Elasticsearch | Guide | Core
meta:
- name: description
content: Migrate your Kuzzle project from Elasticsearch 7 to Elasticsearch 8
content: Configure Kuzzle to use Elasticsearch 8
- name: keywords
content: Kuzzle, Documentation, kuzzle write pluggins, General purpose backend, iot, backend, opensource, API Controllers
---

# Migrate a project from Elasticsearch 7 to Elasticsearch 8
# Elasticsearch 8

<SinceBadge version="2.32.0"/>
<SinceBadge version="2.30.0"/>

Kuzzle relies on Elasticsearch as a [NoSQL document store](/core/2/guides/main-concepts/data-storage).

The support of Elasticsearch 8 has been introduced in Kuzzle 2.30.0.
## Migrating production data from Elasticsearch 7 to 8

To avoid any breaking changes around the support of Elasticsearch 8, we kept Kuzzle working seemlessly with Elasticsearch 7 and Elasticsearch 8.
Is this section, we will see how to migrate a production environnement data from Elasticsearch 7.x to Elasticsearch 8.x

The use of Elasticsearch 8 is an **opt-in** option, so no modification is needed on your behalf if you want to keep using Elasticsearch 7.
### Prerequisites

The default major version of Elasticsearch will be 7 until the next major version of Kuzzle that would ne Kuzzle v3.
Before starting the migration process, ensure the following:
* __Backup your data__: Always backup your indices and cluster settings before starting the migration. Use the Snapshot and Restore feature for this.
* __Version Check__: Make sure your Elasticsearch 7.x is at the latest minor version. Elasticsearch supports migrating from the last minor version of the previous major version.

## How to setup your project to use Elasticsearch 8
### Check Deprecation API

### Setup Kuzzle to use Elasticsearch 8
* Elasticsearch Deprecation API can be used to check for any features or settings in your current cluster that are deprecated or removed in the 8.x version. Address these issues before proceeding.
* Test in a **Non-Production Environment**
Conduct a dry run in a development environment to spot potential issues and estimate the duration the migration process might take.

#### Upgrade the npm package
First you need to upgrade you Kuzzle package to version `>= 2.30.0-es8` in the `package.json` file. Then run `npm install` to upgrade the packages for you application.
### Migration Methods

### Configure Kuzzle
A new configuration key `majorVersion` has been introduced ine the `storageEngine` section to allow the selection of the Eleasticsearch version you want to support for your project.
Theire are 2 strategies to upgrade Elasticsearch in a production environment:
1. Re-indexing
* Step 1: Create a new cluster running Elasticsearch 8.x.
* Step 2: Take a snapshot of your data in the current 7.x cluster.
* Step 3: Restore the snapshot into the new 8.x cluster.
1. Rolling Upgrade
* Step 1: Disable Shard Allocation.
* Step 2: Stop and upgrade a single Elasticsearch node.
* Step 3: Enable Shard Allocation and allow the node to join the cluster and the cluster to re-balance.
* Step 4: Repeat for each node in the cluster.

When not specified, it will be considered to be version 7, specify 8 if you want to switch Kuzzle to support Elasticasearch 8.
After you have migrated your data:
1. Post Upgrade Checks
* Run the health and stats APIs to ensure the health of your newly upgraded cluster.
* Update your clients and integrations to the latest version that's compatible with Elasticsearch 8.x, if not done already.
* Monitor your cluster using the Monitoring API or third-party monitoring services.
1. Troubleshoot
* If you encounter any issues during the migration process, take advantage of the Elasticsearch documentation, forums, and issue trackers for troubleshooting information and support.

This has to be add to you kuzzlerc file, or provided via an environnement variable (see RC doc for details on kuzzlerc configation options)
> Note: Migration steps can vary depending on your setup and needs. Always refer to the official Elasticsearch documentation for the most accurate information, you can find it [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html).
```json
{
"services": {
"storageEngine": {
"majorVersion": 8
}
}
}
```

:::warning
You can not set the `majorVersion` key to 8 if you are using a version of Kuzzle that does not support it. (older versions of Kuzzle won't complain about this value)
:::

:::info
Kuzzle cannot connect to both Elasticsearch 7 and Elasticsearch 8 at the same time.
:::

Once the version is set to 8, Kuzzle will use the Elasticsearch 8 API to communicate with the database.

### Launch Elasticsearch 8 (dev environnement)

Next you will have to change the docker-compose.yml file so that it pulls Elasticsearch 8 image with the recommanded configuration to work with Kuzzle

You can replace the original `elasticsearch` section with the following exemple

```yaml
elasticsearch:
image: elasticsearch:8.11.3
container_name: kuzzle_elasticsearch
environment:
- xpack.security.enabled=false
- action.destructive_requires_name=false
- cluster.name=kuzzle
- node.name=alyx
- discovery.type=single-node
- ingest.geoip.downloader.enabled=false
- indices.id_field_data.enabled=true
ports:
- '9200:9200'
healthcheck:
test: ['CMD', 'curl', '-f', 'http://localhost:9200']
interval: 2s
timeout: 2s
retries: 10
ulimits:
nofile: 65536
```
### Data migration
In the context of running the project in a development envinronnement, you can run your usual initialisation scripts as usual, or use Kourou to dump data from the project still running on Elasticsearch 7 and import them when you are done with setuping the project to run with Elasticsearch 8.
In the context of an hosted environment such as pre-prodcution or production environnement, we recommand following this guide.
Disclaimer: The above steps provide a general migration guide. Migrations can be complex and it's advised to always test these steps in a **non-production environment** before applying them to production.
68 changes: 27 additions & 41 deletions doc/2/guides/elasticsearch/v8/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,17 @@
code: false
type: page
order: 100
title: Elasticsearch 8 | Elasticsearch | Guide | Core
title: Migrate to Elasticsearch 8 | Elasticsearch | Guide | Core
meta:
- name: description
content: Configure Kuzzle to use Elasticsearch 8
content: Migrate your Kuzzle project from Elasticsearch 7 to Elasticsearch 8
- name: keywords
content: Kuzzle, Documentation, kuzzle write pluggins, General purpose backend, iot, backend, opensource, API Controllers
---

# Elasticsearch 8
# Elasticsearch 8 support

<SinceBadge version="2.32.0"/>
<SinceBadge version="2.30.0"/>

Kuzzle uses Elasticsearch as a [NoSQL document store](/core/2/guides/main-concepts/data-storage).

Expand Down Expand Up @@ -117,43 +117,29 @@ services:
Or you can run `kourou app:scaffold sandbox` to create a new Kuzzle project with a `docker-compose.yml` file that uses Elasticsearch 8.

## Migrating production data from Elasticsearch 7 to 8
## Inside a kuberenetes cluster

Is this section, we will see how to migration en production environnement data from Elasticsearch 7.x to Elasticsearch 8.x
While running Kuzzle inside a Kubernetes cluster, you can use the following configuration to run Kuzzle with Elasticsearch 8:

### Prerequisites

Before starting the migration process, ensure the following:
* __Backup your data__: Always backup your indices and cluster settings before starting the migration. Use the Snapshot and Restore feature for this.
* __Version Check__: Make sure your Elasticsearch 7.x is at the latest minor version. Elasticsearch supports migrating from the last minor version of the previous major version.

### Check Deprecation API

* Elasticsearch Deprecation API can be used to check for any features or settings in your current cluster that are deprecated or removed in the 8.x version. Address these issues before proceeding.
* Test in a **Non-Production Environment**
Conduct a dry run in a development environment to spot potential issues and estimate the duration the migration process might take.

### Migration Methods

Theire are 2 strategies to upgrade Elasticsearch in a production environment:
1. Re-indexing
* Step 1: Create a new cluster running Elasticsearch 8.x.
* Step 2: Take a snapshot of your data in the current 7.x cluster.
* Step 3: Restore the snapshot into the new 8.x cluster.
1. Rolling Upgrade
* Step 1: Disable Shard Allocation.
* Step 2: Stop and upgrade a single Elasticsearch node.
* Step 3: Enable Shard Allocation and allow the node to join the cluster and the cluster to re-balance.
* Step 4: Repeat for each node in the cluster.

After you have migrated your data:
1. Post Upgrade Checks
* Run the health and stats APIs to ensure the health of your newly upgraded cluster.
* Update your clients and integrations to the latest version that's compatible with Elasticsearch 8.x, if not done already.
* Monitor your cluster using the Monitoring API or third-party monitoring services.
1. Troubleshoot
* If you encounter any issues during the migration process, take advantage of the Elasticsearch documentation, forums, and issue trackers for troubleshooting information and support.

> Note: Migration steps can vary depending on your setup and needs. Always refer to the official Elasticsearch documentation for the most accurate information, you can find it [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html).
```yaml
# SSL/TLS Configuration
- name: kuzzle_services__storageEngine__client__tls__ca
valueFrom:
secretKeyRef:
name: elasticsearch-master-certs
key: ca.crt
- name: kuzzle_services__storageEngine__client__tls__rejectUnauthorized
value: "false"
- name: kuzzle_services__storageEngine__client__tls__cert
valueFrom:
secretKeyRef:
name: elasticsearch-master-certs
key: tls.crt
- name: kuzzle_services__storageEngine__client__tls__key
valueFrom:
secretKeyRef:
name: elasticsearch-master-certs
key: tls.key
```

Disclaimer: The above steps provide a general migration guide. Migrations can be complex and it's advised to always test these steps in a **non-production environment** before applying them to production.
This will allow Kuzzle to connect to Elasticsearch 8 using a secure connection.
102 changes: 70 additions & 32 deletions doc/2/guides/getting-started/authenticate-users/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,40 +49,88 @@ We saw that in the [Set Up Permission](/core/2/guides/getting-started/set-up-per
```

::: warning
New authentication strategies are made available by [authentication plugins](/core/2/guides/write-plugins/integrate-authentication-strategy).
New authentication strategies are made available by [authentication plugins](/core/2/guides/write-plugins/integrate-authentication-strategy).

By default, only the [local strategy](/core/2/guides/main-concepts/authentication#local-strategy) is available.
By default, only the [local strategy](/core/2/guides/main-concepts/authentication#local-strategy) is available.

We also provide an authentication plugin for the [OAuth strategy](/core/2/guides/main-concepts/authentication#oauth-strategy) but it's not available by default and need to be [added to your application](/core/2/guides/develop-on-kuzzle/external-plugins).
:::

### Getting an authentication token

Kuzzle uses **authentication tokens** to identify user sessions.
Kuzzle uses **authentication tokens** to identify user sessions.

First we need to get one with the [auth:login](/core/2/api/controllers/auth/login) action. This action takes the `strategy` used as a mean to authenticate, and any additional information needed by that strategy.

In our example we will use the [local](/core/2/guides/main-concepts/authentication#local-strategy) strategy so we have to provide a `username` and a `password`:

We previously created a user with the username `melis` and the password `password`. If you don't have this user, you can create it with the following command:

```bash
kourou security:createUser '{
content: {
profileIds: ["default"]
},
credentials: {
local: {
username: "melis",
password: "password"
}
}
}'
```

Then we can log in with the following command:

#### Using API

```bash
curl -XPOST 'http://localhost:7512/_login/local' \
-H 'Content-Type: application/json' \
-d '{
"username": "melis",
"password": "password"
}'

## Response
{
"action": "login",
"controller": "auth",
"error": null,
"headers": {},
"node": "knode-glamorous-flaubert-1113",
"requestId": "688feaf7-d720-4d23-9ba6-cc43487f0108",
"result": {
"_id": "kuid-tricky-comedian-10492",
"expiresAt": 1729776561843,
"jwt": "kauth-eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJfaWQiOiJrdWlkLXRyaWNreS1jb21lZGlhbi0xMDQ5MiIsImlhdCI6MTcyOTc3Mjk2MSwiZXhwIjoxNzI5Nzc2NTYxfQ.m_c8h3aLxqOa45afgFgowRnQ5f4uSPG3QVKDW1taYak",
"ttl": 3600000
},
"status": 200,
"volatile": null
}
```

#### Using CLI

```bash
kourou auth:login -a strategy=local --body '{
username: "melis",
password: "password"
}'
```

```bash
[ℹ] Unknown command "auth:login", fallback to API method
## Response
[ℹ] Unknown command "auth:login", fallback to API action

🚀 Kourou - Executes an API query.
[ℹ] Connecting to http://localhost:7512 ...
{
"_id": "62843356-d826-42fb-adf1-e930e90b6006",
"expiresAt": 1602600225701,
"jwt": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJfaWQiOiI2Mjg0MzM1Ni1kODI2LTQyZmItYWRmMS1lOTMwZTkwYjYwMDYiLCJpYXQiOjE2MDI1OTY2MjUsImV4cCI6MTYwMjYwMDIyNX0.0HZF_AhyTzPCRxdaMbT6hlwLflYG4emmLlTD6YV_Nmo",

[ℹ] Connecting to ws://localhost:7512 ...
{
"_id": "kuid-tricky-comedian-10492",
"expiresAt": 1729776399610,
"jwt": "kauth-eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJfaWQiOiJrdWlkLXRyaWNreS1jb21lZGlhbi0xMDQ5MiIsImlhdCI6MTcyOTc3Mjc5OSwiZXhwIjoxNzI5Nzc2Mzk5fQ.9rBVc4h6hV3Rsb0Z6kvLKhlZNxI-9O7xeWJnC6LfCEQ",
"ttl": 3600000
}
}
[✔] Successfully executed "auth:login"
```

Expand All @@ -102,23 +150,19 @@ Now that we have a token, we must pass it to API requests, either in the **HTTP
When using Kourou with `--username` and `--password` flags, the [auth:login](/core/2/api/controllers/auth/login) action is called and the received token is automatically used along with subsequent requests.
:::


:::: tabs
::: tab Kourou
#### Using API

```bash
kourou auth:getCurrentUser -a jwt=<token>
curl -H "Authorization: Bearer <token>" http://localhost:7512/_me
```

:::
::: tab HTTP
#### Using CLI

``` bash
curl -H "Authorization: Bearer <token>" http://localhost:7512/_me
```bash
kourou auth:getCurrentUser -a jwt=<token>
```

:::
::: tab WebSocket
#### Using Websocket

```bash
npx wscat -c ws://localhost:7512 --execute '{
Expand All @@ -128,27 +172,21 @@ npx wscat -c ws://localhost:7512 --execute '{
}'
```

:::

::: tab Javascript
#### Using javascript SDK

```bash
kourou sdk:execute '
sdk.jwt = "<token>";
console.log(await sdk.auth.getCurrentUser());
'
```

::: info
Kourou is able to [execute](https://github.com/kuzzleio/kourou/blob/master/README.md#kourou-sdkexecute-code) Javascript code snippets.
A `sdk` variable is exposed and refers to an instance of the [Javascript SDK](/sdk/js/7), connected to Kuzzle and authenticated if credentials are provided.
:::

::::
:::

<GuidesLinks
:prev="{ text: 'Set up Permissions', url: '/guides/getting-started/set-up-permissions' }"
:next="{ text: 'Subscribe to Realtime Notifications', url: '/guides/getting-started/subscribe-realtime-notifications/' }"
/>

Loading

0 comments on commit 0b09e92

Please sign in to comment.