"match state" error only in Chrome (non-incognito)

[cjstone]

mod_auth_openidc-2.4.11.1
Ubuntu 20.04
OAuth Provider: GitLab
Apache conf:

 <Location />
    AuthType openid-connect
    Require valid-user
  </Location>

OIDC conf:

OIDCStateInputHeaders user-agent
OIDCRedirectURI          https://foo.blah.com/redirect_uri
OIDCCryptoPassphrase     blahblah
OIDCProviderMetadataURL  https://blah.blah.com/.well-known/openid-configuration
OIDCClientID             blahblah
OIDCClientSecret         blahblah

This configuration works fine with these browsers: Safari, Firefox & Palemoon
This also works works with Chrome in incognito mode.

However using Chrome in normal mode produces an error consistently:

[Wed Apr 27 18:47:44.963893 2022] [auth_openidc:error] [pid 11058:tid 140199912781568] [client XXX.XXX.XXX.XXX:57169] oidc_restore_proto_state: calculated state from cookie does not match state parameter passed back in URL: "tZdG9hqx62c_LONwXhev8p4ul8M" != "qpIroZ91rbjEJLNVdGIwEFzfn4U", referer: https://blah.blah.com/
[Wed Apr 27 18:47:44.964033 2022] [auth_openidc:error] [pid 11058:tid 140199912781568] [client XXX.XXX.XXX.XXX:57169] oidc_authorization_response_match_state: unable to restore state, referer: https://blah.blah.com/
[Wed Apr 27 18:47:44.964068 2022] [auth_openidc:error] [pid 11058:tid 140199912781568] [client XXX.XXX.XXX.XXX:57169] oidc_handle_authorization_response: invalid authorization response state and no default SSO URL is set, sending an error..., referer: https://blah.blah.com/

At first I thought it might have to do with the fact that GitLab has two backend servers, so I configured stick sessions on the fronts (haproxy). I confirmed that the sessions were indeed sticky, yet the problem persists.

Also, from reading #683, I tried using:

OIDCStateInputHeaders user-agent

But found no change in behavior.

Can someone provide clues on how to proceed with troubleshooting?

Thanks,

--C

[zandbelt]

The problem is somewhere with/in the browser, since the backend works with other browsers. Perhaps you can check which User-Agent header the Chrome browser sends in both modes. Also, the full debug log would help more.

[cjstone]

Thanks you @zandbelt . The UA looks the same for both modes:

"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36"

And the debug logs:

[Wed Apr 27 19:26:49.532232 2022] [ssl:debug] [pid 11241:tid 140288806815488] ssl_engine_kernel.c(415): [client xxx.xxx.xxx.xxx:57868] AH02034: Initial (No.1) HTTPS request received for child 79 (server blah-docs.blah.com:443), referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.532443 2022] [authz_core:debug] [pid 11241:tid 140288806815488] mod_authz_core.c(817): [client xxx.xxx.xxx.xxx:57868] AH01626: authorization result of Require valid-user : denied (no authenticated user yet), referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.532503 2022] [authz_core:debug] [pid 11241:tid 140288806815488] mod_authz_core.c(817): [client xxx.xxx.xxx.xxx:57868] AH01626: authorization result of <RequireAny>: denied (no authenticated user yet), referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.532572 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/mod_auth_openidc.c(3958): [client xxx.xxx.xxx.xxx:57868] oidc_check_user_id: incoming request: "/redirect_uri?code=16b5664c6885c5467dd2f837606d887054adc691d0065fa2ee9767d0e3b95d60&state=O5shhcaOuM_uVwP4jdowcMKAJvY", ap_is_initial_req(r)=1, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.532632 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/util.c(2479): [client xxx.xxx.xxx.xxx:57868] oidc_util_hdr_in_get: Cookie=mod_auth_openidc_state_O5shhcaOuM_uVwP4jdowcMKAJvY=ybymw_6Vgxyw5rmV.cqriSWYzPTnjUIIMuFIj4c6sShbn_Pq1y7_R9jQ9QbykQKbU6ZKBR30AM8Ulrl78FcO1OphYNRGIAZp4YZX0xve2WbklyD7eyOlSFK_ao6fvZGzYhcdfZlKApbCghuyLwJrrnhV8Ui4IrAglVRoF5rVeCvfsItgFWRJ9Vpt0MAohfD7FxkbNVMMEi7rzYKHeSQaS5KmHV9Dz6ZvRvhi7HP2_nx1OVrh7x5itcGkk217KezrdRASVTxyMiu6JrNiN6a48DNGb770DpXoT6zpXTsNkfqaLWliDgULtdvswCwxNA7DlUnP2u4siIoik_2C3_ARugB3_oaRMn2DlCkMkRSy9SviwylQXqpItQ3TC-lqEpwV5gsr63-_Jv55QvM9xQt8.bLjP9Zjr1864Sfct3JebZg, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.532695 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/util.c(1231): [client xxx.xxx.xxx.xxx:57868] oidc_util_get_cookie: returning "mod_auth_openidc_session" = <null>, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.532746 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/util.c(1394): [client xxx.xxx.xxx.xxx:57868] oidc_util_request_matches_url: comparing "/redirect_uri"=="/redirect_uri", referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.532804 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/mod_auth_openidc.c(2080): [client xxx.xxx.xxx.xxx:57868] oidc_handle_redirect_authorization_response: enter, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.532879 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/util.c(1718): [client xxx.xxx.xxx.xxx:57868] oidc_util_read_form_encoded_params: read: code=16b5664c6885c5467dd2f837606d887054adc691d0065fa2ee9767d0e3b95d60, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.532942 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/util.c(1718): [client xxx.xxx.xxx.xxx:57868] oidc_util_read_form_encoded_params: read: state=O5shhcaOuM_uVwP4jdowcMKAJvY, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.532992 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/util.c(1722): [client xxx.xxx.xxx.xxx:57868] oidc_util_read_form_encoded_params: parsed: 103 bytes into 2 elements, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533058 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/mod_auth_openidc.c(1896): [client xxx.xxx.xxx.xxx:57868] oidc_handle_authorization_response: enter, response_mode=query, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533106 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/mod_auth_openidc.c(1525): [client xxx.xxx.xxx.xxx:57868] oidc_authorization_response_match_state: enter (state=O5shhcaOuM_uVwP4jdowcMKAJvY), referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533153 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/mod_auth_openidc.c(616): [client xxx.xxx.xxx.xxx:57868] oidc_restore_proto_state: enter, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533207 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/util.c(2479): [client xxx.xxx.xxx.xxx:57868] oidc_util_hdr_in_get: Cookie=mod_auth_openidc_state_O5shhcaOuM_uVwP4jdowcMKAJvY=ybymw_6Vgxyw5rmV.cqriSWYzPTnjUIIMuFIj4c6sShbn_Pq1y7_R9jQ9QbykQKbU6ZKBR30AM8Ulrl78FcO1OphYNRGIAZp4YZX0xve2WbklyD7eyOlSFK_ao6fvZGzYhcdfZlKApbCghuyLwJrrnhV8Ui4IrAglVRoF5rVeCvfsItgFWRJ9Vpt0MAohfD7FxkbNVMMEi7rzYKHeSQaS5KmHV9Dz6ZvRvhi7HP2_nx1OVrh7x5itcGkk217KezrdRASVTxyMiu6JrNiN6a48DNGb770DpXoT6zpXTsNkfqaLWliDgULtdvswCwxNA7DlUnP2u4siIoik_2C3_ARugB3_oaRMn2DlCkMkRSy9SviwylQXqpItQ3TC-lqEpwV5gsr63-_Jv55QvM9xQt8.bLjP9Zjr1864Sfct3JebZg, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533309 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/util.c(2479): [client xxx.xxx.xxx.xxx:57868] oidc_util_hdr_in_get: Cookie=mod_auth_openidc_state_O5shhcaOuM_uVwP4jdowcMKAJvY=ybymw_6Vgxyw5rmV.cqriSWYzPTnjUIIMuFIj4c6sShbn_Pq1y7_R9jQ9QbykQKbU6ZKBR30AM8Ulrl78FcO1OphYNRGIAZp4YZX0xve2WbklyD7eyOlSFK_ao6fvZGzYhcdfZlKApbCghuyLwJrrnhV8Ui4IrAglVRoF5rVeCvfsItgFWRJ9Vpt0MAohfD7FxkbNVMMEi7rzYKHeSQaS5KmHV9Dz6ZvRvhi7HP2_nx1OVrh7x5itcGkk217KezrdRASVTxyMiu6JrNiN6a48DNGb770DpXoT6zpXTsNkfqaLWliDgULtdvswCwxNA7DlUnP2u4siIoik_2C3_ARugB3_oaRMn2DlCkMkRSy9SviwylQXqpItQ3TC-lqEpwV5gsr63-_Jv55QvM9xQt8.bLjP9Zjr1864Sfct3JebZg, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533366 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/util.c(1231): [client xxx.xxx.xxx.xxx:57868] oidc_util_get_cookie: returning "mod_auth_openidc_state_O5shhcaOuM_uVwP4jdowcMKAJvY" = "ybymw_6Vgxyw5rmV.cqriSWYzPTnjUIIMuFIj4c6sShbn_Pq1y7_R9jQ9QbykQKbU6ZKBR30AM8Ulrl78FcO1OphYNRGIAZp4YZX0xve2WbklyD7eyOlSFK_ao6fvZGzYhcdfZlKApbCghuyLwJrrnhV8Ui4IrAglVRoF5rVeCvfsItgFWRJ9Vpt0MAohfD7FxkbNVMMEi7rzYKHeSQaS5KmHV9Dz6ZvRvhi7HP2_nx1OVrh7x5itcGkk217KezrdRASVTxyMiu6JrNiN6a48DNGb770DpXoT6zpXTsNkfqaLWliDgULtdvswCwxNA7DlUnP2u4siIoik_2C3_ARugB3_oaRMn2DlCkMkRSy9SviwylQXqpItQ3TC-lqEpwV5gsr63-_Jv55QvM9xQt8.bLjP9Zjr1864Sfct3JebZg", referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533424 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/util.c(1121): [client xxx.xxx.xxx.xxx:57868] oidc_util_set_cookie_append_value: no cookie append environment variable OIDC_SET_COOKIE_APPEND found, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533473 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/util.c(2557): [client xxx.xxx.xxx.xxx:57868] oidc_util_hdr_err_out_add: Set-Cookie: mod_auth_openidc_state_O5shhcaOuM_uVwP4jdowcMKAJvY=; Path=/; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Secure; HttpOnly; SameSite=None, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533523 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/util.c(199): [client xxx.xxx.xxx.xxx:57868] oidc_util_jwt_verify: enter: JWT header={"alg": "dir", "enc": "A256GCM"}, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533582 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/util.c(2294): [client xxx.xxx.xxx.xxx:57868] oidc_util_create_symmetric_key: key_len=32, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533690 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/mod_auth_openidc.c(209): [client xxx.xxx.xxx.xxx:57868] oidc_get_browser_state_hash: enter, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533755 2022] [auth_openidc:debug] [pid 11241:tid 140288806815488] src/util.c(2479): [client xxx.xxx.xxx.xxx:57868] oidc_util_hdr_in_get: User-Agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533806 2022] [auth_openidc:error] [pid 11241:tid 140288806815488] [client xxx.xxx.xxx.xxx:57868] oidc_restore_proto_state: calculated state from cookie does not match state parameter passed back in URL: "O5shhcaOuM_uVwP4jdowcMKAJvY" != "crEhpY-Xrhay5XCRnYea6SqHDmA", referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533855 2022] [auth_openidc:error] [pid 11241:tid 140288806815488] [client xxx.xxx.xxx.xxx:57868] oidc_authorization_response_match_state: unable to restore state, referer: https://resources.blah.com/
[Wed Apr 27 19:26:49.533907 2022] [auth_openidc:error] [pid 11241:tid 140288806815488] [client xxx.xxx.xxx.xxx:57868] oidc_handle_authorization_response: invalid authorization response state and no default SSO URL is set, sending an error..., referer: https://resources.blah.com/

Thanks again,

--C

[zandbelt]

the logs before this would tell you what user-agent value was used when constructing the cookie value, so you can compare that with the one oidc_get_browser_state_hash when validating

[cjstone]

Ah, OK... oidc_get_browser_state_hash is different; it has an in-house chrome extension name appended. Is there any way to take this into account in the configuration?

oidc_util_hdr_in_get: User-Agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36 CustomExtension/0.1.1

Thanks again,

--C

[zandbelt]

no, you need to make sure that Chrome behaves like in incognito mode or set OIDCStateInputHeaders to none

[cjstone]

I set OIDCStateInputHeaders to none and we're golden.

Thanks!

--Chris