Skip to content

Commit

Permalink
en: update view logs and collect logs (pingcap#459)
Browse files Browse the repository at this point in the history
* en: update view logs and collect logs

* add logs-collection

* Apply suggestions from code review

Co-authored-by: Keke Yi <[email protected]>

* modify aliases

Co-authored-by: Keke Yi <[email protected]>
  • Loading branch information
ran-huang and yikeke authored Jun 28, 2020
1 parent 27aeb37 commit 5360c22
Show file tree
Hide file tree
Showing 6 changed files with 82 additions and 89 deletions.
3 changes: 2 additions & 1 deletion en/TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
- [Restore Data from S3-Compatible Storage Using BR](restore-from-aws-s3-using-br.md)
- [Restart a TiDB Cluster](restart-a-tidb-cluster.md)
- [Maintain a Kubernetes Node](maintain-a-kubernetes-node.md)
- [Collect TiDB Logs](collect-tidb-logs.md)
- [View TiDB Logs](view-logs.md)
- [Enable Automatic Failover](use-auto-failover.md)
- [Destroy a TiDB Cluster](destroy-a-tidb-cluster.md)
+ Disaster Recovery
Expand All @@ -74,3 +74,4 @@
- [Configure tidb-drainer Chart](configure-tidb-binlog-drainer.md)
- [Configure tidb-cluster Chart](tidb-cluster-chart-config.md)
- [Configure tidb-backup Chart](configure-backup.md)
- [Log Collection](logs-collection.md)
87 changes: 0 additions & 87 deletions en/collect-tidb-logs.md

This file was deleted.

35 changes: 35 additions & 0 deletions en/logs-collection.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
title: TiDB Log Collection in Kubernetes
summary: Learn the methods of collecting logs of TiDB and its related components.
category: reference
aliases: ['/docs/tidb-in-kubernetes/dev/collect-tidb-logs/']
---

# TiDB Log Collection in Kubernetes

The system and application logs can be useful for troubleshooting issues and automating operations. This article briefly introduces the methods of collecting logs of TiDB and its related components.

## Collect logs of TiDB and Kubernetes components

The TiDB components deployed by TiDB Operator output the logs in the `stdout` and `stderr` of the container by default. For Kubernetes, these logs are stored in the host's `/var/log/containers` directory, and the file name contains information such as the Pod name and the container name. For this reason, you can collect the logs of the application in the container directly on the host.

If you already have a system for collecting logs in your existing infrastructure, you only need to add the `/var/log/containers/*.log` file on the host in which Kubernetes is located in the collection scope by common methods; if there is no available log collection system, or you want to deploy a separate system for collecting relevant logs, you are free to use any system or solution that you are familiar with.

[ElasticSearch](https://kubernetes.io/docs/tasks/debug-application-cluster/logging-elasticsearch-kibana/) and [Stackdriver](https://kubernetes.io/docs/tasks/debug-application-cluster/logging-stackdriver/) are provided as two log collection methods for reference in the Kubernetes official documentation.

Common open source tools that can be used to collect Kubernetes logs are:

- [Fluentd](https://www.fluentd.org/)
- [Fluent-bit](https://fluentbit.io/)
- [Filebeat](https://www.elastic.co/products/beats/filebeat)
- [Logstash](https://www.elastic.co/products/logstash)

Collected Logs can usually be aggregated and stored on a specific server or in a dedicated storage and analysis system such as ElasticSearch.

Some cloud service providers or specialized performance monitoring service providers also have their own free or charged log collection options that you can choose from.

## Collect system logs

System logs can be collected on Kubernetes hosts in the usual way. If you already have a system for collecting logs in your existing infrastructure, you only need to add the relevant servers and log files in the collection scope by conventional means; if there is no available log collection system, or you want to deploy a separate set of systems for collecting relevant logs, you are free to use any system or solution that you are familiar with.

All of the common log collection tools mentioned above support collecting system logs. Some cloud service providers or specialized performance monitoring service providers also have their own free or charged log collection options that you can choose from.
2 changes: 1 addition & 1 deletion en/tidb-operator-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ After the deployment is complete, see the following documents to use, operate, a
+ [Restore a TiDB Cluster](restore-from-aws-s3-using-br.md)
+ [Automatic Failover](use-auto-failover.md)
+ [Monitor a TiDB Cluster in Kubernetes](monitor-a-tidb-cluster.md)
+ [Collect TiDB Logs in Kubernetes](collect-tidb-logs.md)
+ [View TiDB Logs in Kubernetes](view-logs.md)
+ [Maintain Kubernetes Nodes that Hold the TiDB Cluster](maintain-a-kubernetes-node.md)

When a problem occurs and the cluster needs diagnosis, you can:
Expand Down
43 changes: 43 additions & 0 deletions en/view-logs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
title: View TiDB Logs in Kubernetes
summary: Learn how to view TiDB slow logs and application logs in Kubernetes.
category: how-to
---

# View TiDB Logs in Kubernetes

This document introduces the methods to view logs of TiDB components and TiDB slow log.

## View logs of TiDB components

The TiDB components deployed by TiDB Operator output the logs in the `stdout` and `stderr` of the container by default. You can view the log of a single Pod by running the following command:

{{< copyable "shell-regular" >}}

```shell
kubectl logs -n ${namespace} ${pod_name}
```

If the Pod has multiple containers, you can also view the logs of a container in this Pod:

{{< copyable "shell-regular" >}}

```shell
kubectl logs -n ${namespace} ${pod_name} -c ${container_name}
```

For more methods to view Pod logs, run `kubectl logs --help`.

## View slow query logs of TiDB components

For TiDB 3.0 or later versions, TiDB separates slow query logs from application logs. You can view slow query logs from the sidecar container named `slowlog`:

{{< copyable "shell-regular" >}}

```shell
kubectl logs -n ${namespace} ${pod_name} -c slowlog
```

> **Note:**
>
> The format of TiDB slow query logs is the same as that of MySQL slow query logs. However, due to the characteristics of TiDB itself, some of the specific fields might be different. For this reason, the tool for parsing MySQL slow query logs may not be fully compatible with TiDB slow query logs.
1 change: 1 addition & 0 deletions zh/logs-collection.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
title: 日志收集
summary: 介绍收集 TiDB 及相关组件日志的方法。
category: reference
aliases: ['/docs-cn/tidb-in-kubernetes/dev/collect-tidb-logs/']
---

# 日志收集
Expand Down

0 comments on commit 5360c22

Please sign in to comment.