Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't access dashboard - Token Invalid, Authentik #2338

Open
Pshemas opened this issue Jul 28, 2024 · 9 comments
Open

Can't access dashboard - Token Invalid, Authentik #2338

Pshemas opened this issue Jul 28, 2024 · 9 comments

Comments

@Pshemas
Copy link

Pshemas commented Jul 28, 2024

I've been looking at similar reports - and I couldn't figure out which one would be best for this one, in the end decided on new one, hopefully all the appropriate ones will be merged.

So I had a working self hosted instance of Netbird with Authentik as a IdP provider. After a while it stopped working with Token Invalid error message... which "magically" fixed itself. But now it stopped working again and I can't access the dashboard (the service itself works, the agents can connect, but I can't do any management atm).

Here's what I see in the logs:

2024-07-28T12:35:58Z DEBG management/server/idp/authentik.go:134: requesting new jwt token for authentik idp manager
2024-07-28T12:35:58Z ERRO [context: HTTP, requestID: a7655341-9864-4855-b005-3fa72ca9b82a] management/server/http/middleware/auth_middleware.go:89: Error when validating JWT claims: unable to get authentik token, statusCode 400
2024-07-28T12:35:58Z ERRO [context: HTTP, requestID: a7655341-9864-4855-b005-3fa72ca9b82a] management/server/http/util/util.go:81: got a handler error: token invalid
2024-07-28T12:35:58Z ERRO [requestID: a7655341-9864-4855-b005-3fa72ca9b82a, context: HTTP] management/server/telemetry/http_api_metrics.go:191: HTTP response a7655341-9864-4855-b005-3fa72ca9b82a: GET /api/users status 401
2024-07-28T12:35:58Z DEBG [context: HTTP, requestID: a7655341-9864-4855-b005-3fa72ca9b82a] management/server/telemetry/http_api_metrics.go:211: request GET /api/users took 305 ms and finished with status 401
2024-07-28T12:35:59Z DEBG [context: SYSTEM] management/server/jwtclaims/jwtValidator.go:112: keys refreshed, new UTC expiration time: 2024-07-28 12:35:59.293866388 +0000 UTC
2024-07-28T12:35:59Z DEBG [context: HTTP, requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418] management/server/account.go:1667: overriding JWT Domain and DomainCategory claims since single account mode is enabled
2024-07-28T12:35:59Z DEBG [context: HTTP, requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418] management/server/account.go:1816: Acquired global lock in 8.327µs for user 7
2024-07-28T12:35:59Z DEBG [context: HTTP, requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418] management/server/sql_store.go:169: took 8 ms to persist an account to the store
2024-07-28T12:35:59Z DEBG [context: HTTP, requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418] management/server/account.go:1301: looking up user 7 of account cpk3ikv7g7ts73c049h0 in cache
2024-07-28T12:35:59Z DEBG management/server/account.go:1239: account cpk3ikv7g7ts73c049h0 not found in cache, reloading
2024-07-28T12:35:59Z DEBG management/server/idp/authentik.go:134: requesting new jwt token for authentik idp manager
2024-07-28T12:35:59Z ERRO [context: HTTP, requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418] management/server/http/middleware/auth_middleware.go:89: Error when validating JWT claims: unable to get authentik token, statusCode 400
2024-07-28T12:35:59Z ERRO [requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418, context: HTTP] management/server/http/util/util.go:81: got a handler error: token invalid
2024-07-28T12:35:59Z ERRO [context: HTTP, requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418] management/server/telemetry/http_api_metrics.go:191: HTTP response 859af32c-cfd2-4633-a3b1-2c2bba6b0418: GET /api/users status 401
2024-07-28T12:35:59Z DEBG [context: HTTP, requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418] management/server/telemetry/http_api_metrics.go:211: request GET /api/users took 321 ms and finished with status 401

Here's sanitized management.json:

{
    "Stuns": [{
        "Proto": "udp",
        "URI": "stun:mydomain.com:3478",
        "Username": "",
        "Password": ""
    }],
    "TURNConfig": {
        "TimeBasedCredentials": false,
        "CredentialsTTL": "12h0m0s",
        "Secret": "secret",
        "Turns": [{
            "Proto": "udp",
            "URI": "turn:mydomain.com:3478",
            "Username": "self",
            "Password": "someturnpassword"
        }]
    },
    "Signal": {
        "Proto": "http",
        "URI": "mydomain.com:10000",
        "Username": "",
        "Password": ""
    },
    "Datadir": "/var/lib/netbird/",
    "DataStoreEncryptionKey": "somekey",
    "HttpConfig": {
        "LetsEncryptDomain": "",
        "CertFile": "/etc/letsencrypt/live/mydomain.com/fullchain.pem",
        "CertKey": "/etc/letsencrypt/live/mydomain.com/privkey.pem",
        "AuthAudience": "OauthProiderClientID",
        "AuthIssuer": "https://authentik.mydomain.com/application/o/netbird/",
        "AuthUserIDClaim": "",
        "AuthKeysLocation": "https://authentik.mydomain.com/application/o/netbird/jwks/",
        "OIDCConfigEndpoint": "https://authentik.mydomain.com/application/o/netbird/.well-known/openid-configuration",
        "IdpSignKeyRefreshEnabled": true
    },
    "IdpManagerConfig": {
        "ManagerType": "authentik",
        "ClientConfig": {
            "Issuer": "https://authentik.mydomain.com/application/o/netbird",
            "TokenEndpoint": "https://authentik.mydomain.com/application/o/token/",
            "ClientID": "OauthProiderClientID",
            "ClientSecret": "",
            "GrantType": "client_credentials"
        },
        "ExtraConfig": {
            "Password": "ServiceAccountToken",
            "Username": "Netbird"
        },
        "Auth0ClientCredentials": null,
        "AzureClientCredentials": null,
        "KeycloakClientCredentials": null,
        "ZitadelClientCredentials": null
    },
    "DeviceAuthorizationFlow": {
        "Provider": "hosted",
        "ProviderConfig": {
            "ClientID": "OauthProiderClientID",
            "ClientSecret": "",
            "Domain": "authentik.mydomain.com",
            "Audience": "OauthProiderClientID",
            "TokenEndpoint": "https://authentik.mydomain.com/application/o/token/",
            "DeviceAuthEndpoint": "https://authentik.mydomain.com/application/o/device/",
            "AuthorizationEndpoint": "",
            "Scope": "openid",
            "UseIDToken": false,
            "RedirectURLs": null
        }
    },
    "PKCEAuthorizationFlow": {
        "ProviderConfig": {
            "ClientID": "OauthProiderClientID",
            "ClientSecret": "",
            "Domain": "",
            "Audience": "OauthProiderClientID",
            "TokenEndpoint": "https://authentik.mydomain.com/application/o/token/",
            "DeviceAuthEndpoint": "",
            "AuthorizationEndpoint": "https://authentik.mydomain.com/application/o/authorize/",
            "Scope": "openid profile email offline_access api",
            "UseIDToken": false,
            "RedirectURLs": [
                "http://localhost:53000"
            ]
        }
    },
    "StoreConfig": {
        "Engine": "sqlite"
    },
    "ReverseProxy": {
        "TrustedHTTPProxies": [],
        "TrustedHTTPProxiesCount": 0,
        "TrustedPeers": [
            "0.0.0.0/0"
        ]
    }
}

Here's sanitized openid config:

{
    "issuer": "https://authentik.mydomain.com/application/o/netbird/",
    "authorization_endpoint": "https://authentik.mydomain.com/application/o/authorize/",
    "token_endpoint": "https://authentik.mydomain.com/application/o/token/",
    "userinfo_endpoint": "https://authentik.mydomain.com/application/o/userinfo/",
    "end_session_endpoint": "https://authentik.mydomain.com/application/o/netbird/end-session/",
    "introspection_endpoint": "https://authentik.mydomain.com/application/o/introspect/",
    "revocation_endpoint": "https://authentik.mydomain.com/application/o/revoke/",
    "device_authorization_endpoint": "https://authentik.mydomain.com/application/o/device/",
    "response_types_supported": [
        "code",
        "id_token",
        "id_token token",
        "code token",
        "code id_token",
        "code id_token token"
    ],
    "response_modes_supported": [
        "query",
        "fragment",
        "form_post"
    ],
    "jwks_uri": "https://authentik.mydomain.com/application/o/netbird/jwks/",
    "grant_types_supported": [
        "authorization_code",
        "refresh_token",
        "implicit",
        "client_credentials",
        "password",
        "urn:ietf:params:oauth:grant-type:device_code"
    ],
    "id_token_signing_alg_values_supported": [
        "RS256"
    ],
    "subject_types_supported": [
        "public"
    ],
    "token_endpoint_auth_methods_supported": [
        "client_secret_post",
        "client_secret_basic"
    ],
    "acr_values_supported": [
        "goauthentik.io/providers/oauth2/default"
    ],
    "scopes_supported": [
        "email",
        "profile",
        "openid"
    ],
    "request_parameter_supported": false,
    "claims_supported": [
        "sub",
        "iss",
        "aud",
        "exp",
        "iat",
        "auth_time",
        "acr",
        "amr",
        "nonce",
        "email",
        "email_verified",
        "name",
        "given_name",
        "preferred_username",
        "nickname",
        "groups"
    ],
    "claims_parameter_supported": false,
    "code_challenge_methods_supported": [
        "plain",
        "S256"
    ]
}

Netbird is running inside Docker container, while Authentik in Podman one, on a separate server (with Caddy reverse proxy and Cloudflare).

I'm using Authentik for several other apps and I don't have any issues there (but there's one difference - for other apps I don't use service account setup).

On the side of Authentik I don't see any problems. Here's raw event info:

{
    "user": {
        "pk": 7,
        "email": "myemail",
        "username": "myusername"
    },
    "action": "authorize_application",
    "app": "authentik.providers.oauth2.views.authorize",
    "context": {
        "flow": "someflow",
        "scopes": "offline_access openid email profile",
        "http_request": {
            "args": {
                "scope": "openid profile email offline_access api",
                "state": "7Cwo6bqD1f",
                "audience": "OauthProviderClientID",
                "client_id": "OauthProviderClientID",
                "redirect_uri": "https://mydomain.com/#callback",
                "response_type": "code",
                "code_challenge": "somechallenge",
                "code_challenge_method": "S256"
            },
            "path": "/api/v3/flows/executor/default-provider-authorization-explicit-consent/",
            "method": "GET",
            "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0"
        },
        "authorized_application": {
            "pk": "somepk",
            "app": "authentik_core",
            "name": "Netbird",
            "model_name": "application"
        }
    },
    "client_ip": "someip",
    "expires": "2025-07-28T12:51:42.272Z",
    "brand": {
        "pk": "somepk",
        "app": "authentik_brands",
        "name": "Default brand",
        "model_name": "brand"
    }
}

In credentials / tokens for a user that wishes to access Netbird I see:

obraz

obraz

obraz

Here are provider settings:

obraz

obraz

obraz

Any suggestions howto resolve the issue and get into the management panel are greatly appreciated. At this point I'm just blindly clicking various options as the suggestions in other topics are all over the place - it seems that I'm not the only one who has issues in pinpointing the cause / fix.

If there's some more info needed plz let me know - I'll be happy to provide it.

@Pshemas
Copy link
Author

Pshemas commented Jul 29, 2024

for the time being I've created a new provider and service account to get into the dashboard, but I fully expect the problem to reappear when token expires.

@ne0YT
Copy link

ne0YT commented Oct 7, 2024

same issue:
Request failed with status code 401

Error: Token invalid

@ne0YT
Copy link

ne0YT commented Oct 8, 2024

@Pshemas did you ever figure this out?

@wbarnard81
Copy link

wbarnard81 commented Nov 22, 2024

I updated to Authentik 2024.10.4 this morning and now I am getting this error. All my netbird users can't work. :-(

2024-11-22T14:04:04Z ERRO [context: HTTP, requestID: 0b96d0df-5391-4083-b654-554e28c5cf10] management/server/http/util/util.go:81: got a handler error: token invalid
2024-11-22T14:04:04Z ERRO [context: HTTP, requestID: 0b96d0df-5391-4083-b654-554e28c5cf10] management/server/telemetry/http_api_metrics.go:168: HTTP response 0b96d0df-5391-4083-b654-554e28c5cf10: GET /api/users status 401

@mvivaldi
Copy link

I updated to Authentik 2024.10.4 this morning and now I am getting this error. All my netbird users can't work. :-(

2024-11-22T14:04:04Z ERRO [context: HTTP, requestID: 0b96d0df-5391-4083-b654-554e28c5cf10] management/server/http/util/util.go:81: got a handler error: token invalid
2024-11-22T14:04:04Z ERRO [context: HTTP, requestID: 0b96d0df-5391-4083-b654-554e28c5cf10] management/server/telemetry/http_api_metrics.go:168: HTTP response 0b96d0df-5391-4083-b654-554e28c5cf10: GET /api/users status 401

Yeah, same proble here

@Pshemas
Copy link
Author

Pshemas commented Nov 24, 2024

@ne0YT sadly not. I created new provider and added it to NB. This made it work... until today. With Authentik 2024.10.4 I first couldn't launch at all - got redirection URI error. Initially the tip to change middle option for URI to regex didn't help, but later on it did... But then it brought me back to "Token Invalid" error.
Super tired of this. At this point I'm starting to test other IdP . If other options wouldn't break that often or at the very least there will be a working workarounds / pointers what's wrong I'll switch (even though I like the Authentik approach to lots of user / admin facing things).

@Pshemas
Copy link
Author

Pshemas commented Nov 24, 2024

Here are the latest logs from management:

2024-11-24T14:50:53Z ERRO [context: HTTP, requestID: d0451fc8-98d0-4418-b834-8db0cb66f495] management/server/http/middleware/auth_middleware.go:89: Error when validating JWT claims: 403 Forbidden
2024-11-24T14:50:53Z ERRO [requestID: d0451fc8-98d0-4418-b834-8db0cb66f495, context: HTTP] management/server/http/util/util.go:81: got a handler error: token invalid
2024-11-24T14:50:53Z ERRO [context: HTTP, requestID: d0451fc8-98d0-4418-b834-8db0cb66f495] management/server/telemetry/http_api_metrics.go:168: HTTP response d0451fc8-98d0-4418-b834-8db0cb66f495: GET /api/users status 401
2024-11-24T14:50:53Z DEBG [context: HTTP, requestID: d0451fc8-98d0-4418-b834-8db0cb66f495] management/server/telemetry/http_api_metrics.go:181: request GET /api/users took 521 ms and finished with status 401

@Spiritreader
Copy link

Spiritreader commented Nov 24, 2024

Please see
#2941 (comment)
for a fix / workaround if you get 403 forbidden and the service account login is verified to be working.

@wbarnard81
Copy link

Thank you @Spiritreader That worked for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants