Kubernetes日志记录：EFK与PLG

. . , — , , .

. : EFK (Elasticsearch) PLG (Loki) .

EFK

, ELK EFK. : Elasticsearch ( ), Logstash FluentD ( ) Kibana .

:

Elasticsearch — . , . JSON, . , , Apache Lucene.

FluentD — , . JSON . , , , .

Kibana — Elasticsearch , , , , .

Elasticsearch

Elasticsearch . . , .

:

• aster node — , , ;
• data node — ;
• ingest node — ;
• coordinating node — , , ;
• alerting node — ;
• machine learning node — .

, .

, , :

, helm chart:

$ helm install efk-stack stable/elastic-stack --set logstash.enabled=false --set fluentd.enabled=true --set fluentd-elastics

PLG

, , Grafana Loki. , . , , Grafana, . , Prometheus, Loki, . Loki , , .

Promtail — Loki. Grafana — Loki.

Loki , Prometheus, Kubernetes.

Loki

Loki , , .

, , . . . , .

:

— ( ):

:

Promtail — , ( ), API Kubernetes , . Loki. , Prometheus.

Distributor — -, . , . , , . . n (3, ).

Ingester — -. . , . , Chunks ( ). , .

Index — , DynamoDB, Cassandra, Google BigTable .

Chunks — , , , S3.

Querier — , . , . .

.

Kubernetes helm. , ( ! . )

.

$ helm repo add loki https://grafana.github.io/loki/charts
$ helm repo update
$ helm upgrade --install loki loki/loki-stack --set grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false

, Prometheus Etcd Loki Etcd.

, .

Elasticsearch Query DSL Lucene query language, . . .

— LogQL, Loki, PromQL (Prometheus query language). . , , Elastic language.

Loki — , .

, Loki , , . Loki .

— OPEX, . Elasticsearch : , , , . Loki HTTP X-Scope-OrgID.

Loki - , , . (), , Elasticsearch.

EFK , Kibana , . .

Loki Kubernetes - . Grafana .

, Loki .

— . , GKE Stackdriver, . .

:

• https://github.com/grafana/loki/blob/master/docs/overview/comparisons.md
• https://www.elastic.co/blog/found-elasticsearch-from-the-bottom-up
• https://www.elastic.co/blog/found-elasticsearch-in-production/

— , (Kubernetes, DevOps, Docker, Ansible, Ceph, SRE, Agile)