Skip to content

Latest commit

 

History

History
57 lines (33 loc) · 4.04 KB

tiup-troubleshooting-guide.md

File metadata and controls

57 lines (33 loc) · 4.04 KB
Raw
title summary aliases
TiUP Troubleshooting Guide
Introduce the troubleshooting methods and solutions if you encounter issues when using TiUP.
/docs/dev/tiup/tiup-troubleshooting-guide/

TiUP Troubleshooting Guide

This document introduces some common issues when you use TiUP and the troubleshooting methods. If this document does not include the issues you bump into, file a new issue in the Github TiUP repository.

Troubleshoot TiUP commands

Can't see the latest component list using tiup list

TiUP does not update the latest component list from the mirror server every time. You can forcibly refresh the component list by running tiup list.

Can't see the latest version information of a component using tiup list <component>

Same as the previous issue, the component version information is only obtained from the mirror server when there is no local cache. You can refresh the component list by running tiup list <component>.

Component downloading process is interrupted

Unstable network might result in an interrupted component downloading process. You can try to download the component again. If you cannot download it after trying multiple times, it might be caused by the CDN server and you can report the issue here.

A checksum error occurs during component downloading process

Because the CDN server has a short cache time, the new checksum file might not match the component package. Try to download again after 5 minutes. If the new checksum file still does not match the component package, report the issue here.

Troubleshoot TiUP cluster component

unable to authenticate, attempted methods [none publickey] is prompted during deployment

During deployment, component packages are uploaded to the remote host and the initialization is performed. This process requires connecting to the remote host. This error is caused by the failure to find the SSH private key to connect to the remote host.

To solve this issue, confirm whether you have specified the private key by running tiup cluster deploy -i identity_file:

  • If the -i flag is not specified, it might be that TiUP does not automatically find the private key path. It is recommended to explicitly specify the private key path using -i.
  • If the -i flag is specified, it might be that TiUP cannot log in to the remote host using the specified private key. You can verify it by manually executing the ssh -i identity_file user@remote command.
  • If a password is used to log in to the remote host, make sure that you have specified the -p flag and entered the correct login password.

The process of upgrading the cluster using the TiUP cluster component is interrupted

To avoid misuse cases, the TiUP cluster component does not support the upgrade of specified nodes, so after the upgrade fails, you need to perform the upgrade operations again, including idempotent operations during the upgrade process.

The upgrade process can be divided into the following steps:

  1. Back up the old version of components on all nodes
  2. Distribute new components to remote
  3. Perform a rolling restart to all components

If the upgrade is interrupted during a rolling restart, instead of repeating the tiup cluster upgrade operation, you can use tiup cluster restart -N <node1> -N <node2> to restart the nodes that have not completed the restart.

If the number of un-restarted nodes of the same component is relatively large, you can also restart a certain type of component by running tiup cluster restart -R <component>.

During the upgrade, you find that node_exporter-9100.service/blackbox_exporter-9115.service does not exist

If you previously migrated your cluster from TiDB Ansible and the exporter was not deployed in TiDB Ansible, this situation might happen. To solve it, you can manually copy the missing files from other nodes to the new node for the time being. The TiUP team will complete the missing components during the migration process.