[close #651] fix HA issue for pd client

[iosmanthus]

Signed-off-by: iosmanthus myosmanthustree@gmail.com

What problem does this PR solve?

Issue Number: close #651

Problem Description:

See the issue.

What is changed and how does it work?

1. Force getMembers and checkHealth produces a return value and handle it.
2. Handle TsoBatchUsedUp error from tikv server.

Check List for Tests

This PR has been tested by at least one of the following methods:

• Unit test
• Integration test

Related changes

• Need to cherry-pick the release branch

[codecov]

Codecov Report

Patch coverage has no change and project coverage change: +0.18 🎉

Comparison is base (bcd11f3) 37.93% compared to head (599152a) 38.11%.

Additional details and impacted files

@@             Coverage Diff              @@
##             master     #652      +/-   ##
============================================
+ Coverage     37.93%   38.11%   +0.18%     
  Complexity     1610     1610              
============================================
  Files           278      278              
  Lines         17482    17482              
  Branches       1986     1986              
============================================
+ Hits           6631     6664      +33     
+ Misses        10194    10160      -34     
- Partials        657      658       +1     

Impacted Files	Coverage Δ
src/main/java/org/tikv/common/pd/PDError.java	0.00% <0.00%> (-30.31%)	⬇️
...java/org/tikv/common/operation/PDErrorHandler.java	40.00% <0.00%> (-20.00%)	⬇️
src/main/java/org/tikv/common/PDClient.java	56.88% <0.00%> (-0.23%)	⬇️
src/main/java/io/grpc/internal/ClientCallImpl.java	57.14% <0.00%> (+0.48%)	⬆️
...ty/handler/codec/http2/Http2ConnectionHandler.java	47.43% <0.00%> (+0.97%)	⬆️
src/main/java/io/grpc/stub/ClientCalls.java	48.51% <0.00%> (+2.31%)	⬆️
src/main/java/org/tikv/common/region/TiStore.java	61.53% <0.00%> (+5.12%)	⬆️
...rc/main/java/io/grpc/netty/NettyClientHandler.java	59.69% <0.00%> (+5.60%)	⬆️
...va/org/tikv/common/region/StoreHealthyChecker.java	73.41% <0.00%> (+12.65%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[github-actions]

This PR is stale because it has been open 30 days with no activity. Remove the status/stale label or comment or this PR will be closed in 7 days.

[github-actions]

This PR was closed because it has been stalled for 7 days with no activity.