Clever Geek Handbook

在Kubernetes上以HA模式启动Keycloak



TL; DR：将描述Keycloak，开源访问控制系统，内部设备分析，设置详细信息。



介绍和主要思想



在本文中，我们将看到在Kubernetes上部署Keycloak集群时要牢记的基本思想。



Keycloak — . , — , ( , , . ).



Keycloak — , Java Wildfly. , framework , SSO (single sign-on).



.



Keycloak



Keycloak :



  • , ,
  • Datagrid cache, , , . Infinispan, . Infinispan — - .


Keycloak :



  • — , standalone.xml
  • ( ) — , . standalone-ha.xml, .
  • — , . . domain.xml
  • — , Keycloak , . Keycloak .


, , , Kubernetes. Kubernetes ( Keycloak), .



, Keycloak, , Kubernetes.



Keycloak



Keycloak :



  • ip multicast


, . , - — . .



, Keycloak (HA) , , Wildfly .



Wildfly , , — . , . :



  • mod_cluster: Apache HTTP, TCP multicast . .



  • infinispan: , JGroups . HotRod Infinispan .



  • jgroups: JGroups. , , , , .







ingress Kubernetes :



Keycloak , , HTTP , ip- . ingress HTTP X-Forwarded-For X-Forwarded-Proto, HOST. ingress-nginx (> 0.22.0)



proxy-address-forwarding PROXY_ADDRESS_FORWARDING true Keycloak , proxy.



sticky sessions ingress. Keycloak Infinispan , . , , , .



cookie AUTH_SESSION_ID. Keycloak , cookie sticky session.

Keycloak , , AUTH_SESSION_ID, , . JAVA_OPTS jboss.node.name jboss.tx.node.id — . — 23 jboss, StatefulSet, Deployment.



— , . , . — Wildfly , /opt/jboss/startup-scripts :



embed-server --server-config=standalone-ha.xml --std-out=echo
batch

echo * Setting CACHE_OWNERS to "${env.CACHE_OWNERS}" in all cache-containers

/subsystem=infinispan/cache-container=keycloak/distributed-cache=sessions:write-attribute(name=owners, value=${env.CACHE_OWNERS:1})
/subsystem=infinispan/cache-container=keycloak/distributed-cache=authenticationSessions:write-attribute(name=owners, value=${env.CACHE_OWNERS:1})
/subsystem=infinispan/cache-container=keycloak/distributed-cache=actionTokens:write-attribute(name=owners, value=${env.CACHE_OWNERS:1})
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineSessions:write-attribute(name=owners, value=${env.CACHE_OWNERS:1})
/subsystem=infinispan/cache-container=keycloak/distributed-cache=clientSessions:write-attribute(name=owners, value=${env.CACHE_OWNERS:1})
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineClientSessions:write-attribute(name=owners, value=${env.CACHE_OWNERS:1})
/subsystem=infinispan/cache-container=keycloak/distributed-cache=loginFailures:write-attribute(name=owners, value=${env.CACHE_OWNERS:1})

run-batch
stop-embedded-server


CACHE_OWNERS .



ip multicast



Weavenet CNI, multicast — Keycloak , .



ip multicast Kubernetes, JGroups .



— KUBE_DNS, headless service Keycloak, JGroups , .



— KUBE_PING, API ( serviceAccount list get, serviceAccount).



JGroups JGROUPS_DISCOVERY_PROTOCOL JGROUPS_DISCOVERY_PROPERTIES. KUBE_PING namespace labels.



️ multicast Keycloak Kubernetes ( namespace production, — staging) — Keycloak . multicast jboss.default.multicast.address jboss.modcluster.multicast.address JAVA_OPTS.








Keycloak Infinispan , Keycloack, Keycloak. Keycloak .



Keycloak Java Data Grid ( Infinispan) . Infinispan HotRod.



Infinispan remoteStore, , ( , . ) . infinispan JDG , , JDG1 site1 JDG2 site2.



, JDG Keycloak , HotRod. Keycloak site2 Infinispan, Keycloak site2.



Infinispan. remote-store Infinispan ( standalone-ha.xml), replicated-cache Infinispan .





Keycloak:



  • . , , . realm, , . , — Keycloak. — , . . work , .



  • . , offline , . — , , .





Infinispan



— Keycloak, , authenticationSessions, . Keycloak, . sticky sessions, , Active-Active .



. , , , , - . , forget password actionTokens — , . .



, . , . Keycloak , , . Keycloak realms, users authorization .



work, . - , . , , Keycloak , . .



. sessions, clientSessions, offlineSessions offlineClientSessions, , . , HTTP , sticky sessions .



. loginFailures , . — . . , , — .



Infinispan :



<replicated-cache-configuration name="keycloak-sessions" mode="ASYNC" start="EAGER" batching="false">
</replicated-cache-configuration>

<replicated-cache name="work" configuration="keycloak-sessions" />
<replicated-cache name="sessions" configuration="keycloak-sessions" />
<replicated-cache name="offlineSessions" configuration="keycloak-sessions" />
<replicated-cache name="actionTokens" configuration="keycloak-sessions" />
<replicated-cache name="loginFailures" configuration="keycloak-sessions" />
<replicated-cache name="clientSessions" configuration="keycloak-sessions" />
<replicated-cache name="offlineClientSessions" configuration="keycloak-sessions" />


Infinispan Keycloak

remoteStore Keycloak . , , CACHE_OWNERS, /opt/jboss/startup-scripts:



embed-server --server-config=standalone-ha.xml --std-out=echo
batch

echo *** Update infinispan subsystem ***
/subsystem=infinispan/cache-container=keycloak:write-attribute(name=module, value=org.keycloak.keycloak-model-infinispan)

echo ** Add remote socket binding to infinispan server **
/socket-binding-group=standard-sockets/remote-destination-outbound-socket-binding=remote-cache:add(host=${remote.cache.host:localhost}, port=${remote.cache.port:11222})

echo ** Update replicated-cache work element **
/subsystem=infinispan/cache-container=keycloak/replicated-cache=work/store=remote:add( \
    passivation=false, \
    fetch-state=false, \
    purge=false, \
    preload=false, \
    shared=true, \
    remote-servers=["remote-cache"], \
    cache=work, \
    properties={ \
        rawValues=true, \
        marshaller=org.keycloak.cluster.infinispan.KeycloakHotRodMarshallerFactory, \
        protocolVersion=${keycloak.connectionsInfinispan.hotrodProtocolVersion} \
    } \
)

/subsystem=infinispan/cache-container=keycloak/replicated-cache=work:write-attribute(name=statistics-enabled,value=true)

echo ** Update distributed-cache sessions element **
/subsystem=infinispan/cache-container=keycloak/distributed-cache=sessions/store=remote:add( \
    passivation=false, \
    fetch-state=false, \
    purge=false, \
    preload=false, \
    shared=true, \
    remote-servers=["remote-cache"], \
    cache=sessions, \
    properties={ \
        rawValues=true, \
        marshaller=org.keycloak.cluster.infinispan.KeycloakHotRodMarshallerFactory, \
        protocolVersion=${keycloak.connectionsInfinispan.hotrodProtocolVersion} \
    } \
)
/subsystem=infinispan/cache-container=keycloak/distributed-cache=sessions:write-attribute(name=statistics-enabled,value=true)

echo ** Update distributed-cache offlineSessions element **
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineSessions/store=remote:add( \
    passivation=false, \
    fetch-state=false, \
    purge=false, \
    preload=false, \
    shared=true, \
    remote-servers=["remote-cache"], \
    cache=offlineSessions, \
    properties={ \
        rawValues=true, \
        marshaller=org.keycloak.cluster.infinispan.KeycloakHotRodMarshallerFactory, \
        protocolVersion=${keycloak.connectionsInfinispan.hotrodProtocolVersion} \
    } \
)
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineSessions:write-attribute(name=statistics-enabled,value=true)

echo ** Update distributed-cache clientSessions element **
/subsystem=infinispan/cache-container=keycloak/distributed-cache=clientSessions/store=remote:add( \
    passivation=false, \
    fetch-state=false, \
    purge=false, \
    preload=false, \
    shared=true, \
    remote-servers=["remote-cache"], \
    cache=clientSessions, \
    properties={ \
        rawValues=true, \
        marshaller=org.keycloak.cluster.infinispan.KeycloakHotRodMarshallerFactory, \
        protocolVersion=${keycloak.connectionsInfinispan.hotrodProtocolVersion} \
    } \
)
/subsystem=infinispan/cache-container=keycloak/distributed-cache=clientSessions:write-attribute(name=statistics-enabled,value=true)

echo ** Update distributed-cache offlineClientSessions element **
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineClientSessions/store=remote:add( \
    passivation=false, \
    fetch-state=false, \
    purge=false, \
    preload=false, \
    shared=true, \
    remote-servers=["remote-cache"], \
    cache=offlineClientSessions, \
    properties={ \
        rawValues=true, \
        marshaller=org.keycloak.cluster.infinispan.KeycloakHotRodMarshallerFactory, \
        protocolVersion=${keycloak.connectionsInfinispan.hotrodProtocolVersion} \
    } \
)
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineClientSessions:write-attribute(name=statistics-enabled,value=true)

echo ** Update distributed-cache loginFailures element **
/subsystem=infinispan/cache-container=keycloak/distributed-cache=loginFailures/store=remote:add( \
    passivation=false, \
    fetch-state=false, \
    purge=false, \
    preload=false, \
    shared=true, \
    remote-servers=["remote-cache"], \
    cache=loginFailures, \
    properties={ \
        rawValues=true, \
        marshaller=org.keycloak.cluster.infinispan.KeycloakHotRodMarshallerFactory, \
        protocolVersion=${keycloak.connectionsInfinispan.hotrodProtocolVersion} \
    } \
)
/subsystem=infinispan/cache-container=keycloak/distributed-cache=loginFailures:write-attribute(name=statistics-enabled,value=true)

echo ** Update distributed-cache actionTokens element **
/subsystem=infinispan/cache-container=keycloak/distributed-cache=actionTokens/store=remote:add( \
    passivation=false, \
    fetch-state=false, \
    purge=false, \
    preload=false, \
    shared=true, \
    cache=actionTokens, \
    remote-servers=["remote-cache"], \
    properties={ \
        rawValues=true, \
        marshaller=org.keycloak.cluster.infinispan.KeycloakHotRodMarshallerFactory, \
        protocolVersion=${keycloak.connectionsInfinispan.hotrodProtocolVersion} \
    } \
)
/subsystem=infinispan/cache-container=keycloak/distributed-cache=actionTokens:write-attribute(name=statistics-enabled,value=true)

echo ** Update distributed-cache authenticationSessions element **
/subsystem=infinispan/cache-container=keycloak/distributed-cache=authenticationSessions:write-attribute(name=statistics-enabled,value=true)

echo *** Update undertow subsystem ***
/subsystem=undertow/server=default-server/http-listener=default:write-attribute(name=proxy-address-forwarding,value=true)

run-batch
stop-embedded-server


JAVA_OPTS Keycloak HotRod: remote.cache.host, remote.cache.port jboss.site.name.







— , (Kubernetes, DevOps, Docker, Ansible, Ceph, SRE)



More articles:


All Articles