Install Log Agent on Kubernetes

Before installing the agent on the operating systems we support below, you need to download the certificate according to the instructions at Initialize Certificate . Information on setting up the agent is in the readme file, and the instruction scripts are also in the downloaded certificate file. Use this information with the instructions below to complete Agent for Log setup.

Setting

You can install the agent using Kubectl.

The goal when deploying in a k8s environment is usually to push the logs of all other pods. Therefore, we should deploy the agent as a daemon-set .

Kubectl

• After initializing the Certificate, you will save the user authentication information to the k8s master node (or any machine that has kubectl and has execute rights on kubernetes).

• Move to the agent you want to install, k8s / kubectl folder. Run command:

Copy kubectl apply -f namespace.yml kubectl apply -f configmap.yml kubectl apply -f secret.yml kubectl apply -f daemonset.yml

In order for the gent log to be able to read the logs of other pods, you need to repair your computer and disable selinux. On the k8s home page , it is also specified to disable selinux with centos. On k8s nodes, run the command:

Copy setenforce 0

and go to the file /etc/sysconfig/selinuxto edit SELINUX=enforcingand become SELINUX=disabled.

The configuration files below have been prepared by us in the script when downloading the certificate. The description below helps readers imagine what it would be like if we created a manual.

Configuration

You need to edit the {parts in brackets} in the files below to suit your environment:

Filebeat

Filenamespace.yml. Copy apiVersion: v1 kind: Namespace metadata: name: agent-vmonitor-platform File configmap.yml. For example, the configuration below will push all pod logs in the web-app namespace to the system {$BOOTSTRAP_SERVERS}, {$TOPIC}Read the info.md ​​file in the downloaded certificate folder. Copy apiVersion: v1 kind: ConfigMap metadata: name: filebeat namespace: agent-vmonitor-platform data: filebeat.yml: >- filebeat.inputs: - type: log paths: - /var/log/pods/*/*/*.log output.kafka: hosts: {$BOOTSTRAP_SERVERS} topic: {$TOPIC} partition.round_robin: reachable_only: false required_acks: 1 compression: gzip max_message_bytes: 1000000 ssl.certificate_authorities: - /usr/share/filebeat/VNG.trust.pem ssl.certificate: /usr/share/filebeat/user.cer.pem ssl.key: /usr/share/filebeat/user.key.pem ssl.verification_mode: "none" logging.level: info logging.to_files: true logging.files: path: /var/log/filebeat name: filebeat keepfiles: 7 permissions: 0644 Filesecret.yml Copy apiVersion: v1 kind: Secret metadata: namespace: agent-vmonitor-platform name: filebeat data: VNG.trust.pem: {$vng.trust.pem} user.cer.pem: {$user.cer.pem} user.key.pem: {$user.key.pem} type: Opaque $vng.trust.pem, $user.cer.pem, $user.key.pem The content is md5 hash of the corresponding files in the certificate directory or create a secret with --from-file cert Filedaemonset.yml Copy apiVersion: apps/v1 kind: DaemonSet metadata: name: filebeat namespace: agent-vmonitor-platform labels: app: filebeat spec: selector: matchLabels: app: filebeat template: metadata: name: filebeat labels: app: filebeat spec: containers: - name: filebeat image: docker.elastic.co/beats/filebeat:8.7.0 imagePullPolicy: IfNotPresent volumeMounts: - name: config mountPath: /usr/share/filebeat/filebeat.yml subPath: filebeat.yml - name: certificate mountPath: /usr/share/filebeat/VNG.trust.pem subPath: VNG.trust.pem - name: certificate mountPath: /usr/share/filebeat/user.cer.pem subPath: user.cer.pem - name: certificate mountPath: /usr/share/filebeat/user.key.pem subPath: user.key.pem - name: varlog mountPath: /var/log/ readOnly: true - name: varlibdockercontainers mountPath: /var/lib/docker/containers readOnly: true resources: limits: cpu: '1' memory: 2Gi volumes: - name: varlog hostPath: path: /var/log/ - name: varlibdockercontainers hostPath: path: /var/lib/docker/containers - name: config configMap: name: filebeat items: - key: filebeat.yml path: filebeat.yml - name: certificate secret: secretName: filebeat items: - key: VNG.trust.pem path: VNG.trust.pem - key: user.cer.pem path: user.cer.pem - key: user.key.pem path: user.key.pem securityContext: runAsUser: 0 restartPolicy: Always tolerations: - key: vmonitor-log operator: Equal value: 'true' effect: NoSchedule

Copy apiVersion: v1 kind: Namespace metadata: name: agent-vmonitor-platform

Copy apiVersion: v1 kind: ConfigMap metadata: name: filebeat namespace: agent-vmonitor-platform data: filebeat.yml: >- filebeat.inputs: - type: log paths: - /var/log/pods/*/*/*.log output.kafka: hosts: {$BOOTSTRAP_SERVERS} topic: {$TOPIC} partition.round_robin: reachable_only: false required_acks: 1 compression: gzip max_message_bytes: 1000000 ssl.certificate_authorities: - /usr/share/filebeat/VNG.trust.pem ssl.certificate: /usr/share/filebeat/user.cer.pem ssl.key: /usr/share/filebeat/user.key.pem ssl.verification_mode: "none" logging.level: info logging.to_files: true logging.files: path: /var/log/filebeat name: filebeat keepfiles: 7 permissions: 0644

Copy apiVersion: v1 kind: Secret metadata: namespace: agent-vmonitor-platform name: filebeat data: VNG.trust.pem: {$vng.trust.pem} user.cer.pem: {$user.cer.pem} user.key.pem: {$user.key.pem} type: Opaque

Copy apiVersion: apps/v1 kind: DaemonSet metadata: name: filebeat namespace: agent-vmonitor-platform labels: app: filebeat spec: selector: matchLabels: app: filebeat template: metadata: name: filebeat labels: app: filebeat spec: containers: - name: filebeat image: docker.elastic.co/beats/filebeat:8.7.0 imagePullPolicy: IfNotPresent volumeMounts: - name: config mountPath: /usr/share/filebeat/filebeat.yml subPath: filebeat.yml - name: certificate mountPath: /usr/share/filebeat/VNG.trust.pem subPath: VNG.trust.pem - name: certificate mountPath: /usr/share/filebeat/user.cer.pem subPath: user.cer.pem - name: certificate mountPath: /usr/share/filebeat/user.key.pem subPath: user.key.pem - name: varlog mountPath: /var/log/ readOnly: true - name: varlibdockercontainers mountPath: /var/lib/docker/containers readOnly: true resources: limits: cpu: '1' memory: 2Gi volumes: - name: varlog hostPath: path: /var/log/ - name: varlibdockercontainers hostPath: path: /var/lib/docker/containers - name: config configMap: name: filebeat items: - key: filebeat.yml path: filebeat.yml - name: certificate secret: secretName: filebeat items: - key: VNG.trust.pem path: VNG.trust.pem - key: user.cer.pem path: user.cer.pem - key: user.key.pem path: user.key.pem securityContext: runAsUser: 0 restartPolicy: Always tolerations: - key: vmonitor-log operator: Equal value: 'true' effect: NoSchedule