Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tools/simulator: Make simulator work with large scale cluster #8269

Merged
merged 14 commits into from
Jul 26, 2024
Merged

tools/simulator: Make simulator work with large scale cluster #8269

merged 14 commits into from
Jul 26, 2024

Conversation

HuSharp
Copy link
Member

@HuSharp HuSharp commented Jun 7, 2024

What problem does this PR solve?

separate regions and heartbeat to make simulator work with large scale cluster

Issue Number: Close #5683, Ref #8135

What is changed and how does it work?

support 300 stores with 1000k regions
tiup playground nightly --kv.binpath ./pd-simulator --kv=1 --db=0 --kv.config=../conf/simconfig.toml --pd 1

image

image

after adding a node
image

image

Check List

Tests

  • Manual test (add detailed scripts or steps below)

Release note

None.

@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jun 7, 2024

[REVIEW NOTIFICATION]

This pull request has not been approved.

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jun 7, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: yes Indicates the PR's author has signed the dco. labels Jun 7, 2024
@ti-chi-bot ti-chi-bot bot requested review from nolouch and rleungx June 7, 2024 00:16
@ti-chi-bot ti-chi-bot bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 7, 2024
@codecov Codecov
Copy link

codecov bot commented Jun 7, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.17%. Comparing base (bdbe73e) to head (3aecdb5).
Report is 2 commits behind head on master.

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #8269      +/-   ##
==========================================
- Coverage   77.32%   77.17%   -0.16%     
==========================================
  Files         470      470              
  Lines       61687    61684       -3     
==========================================
- Hits        47701    47602      -99     
- Misses      10405    10487      +82     
- Partials     3581     3595      +14
Flag Coverage Δ
unittests 77.17% <ø> (-0.16%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

rleungx
rleungx reviewed Jun 7, 2024
View reviewed changes
conf/simconfig.toml Outdated
## the tick interval when starting PD inside (default: "100ms")
sim-tick-interval = "100ms"
total-store = 300
total-region = 10000000
case-name = "stable"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do other cases work? like scale in/out

Copy link
Member Author

@HuSharp HuSharp Jun 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I have tested in balance-leader as well
image

image

The stable case is convenient for add/remove node

@HuSharp HuSharp mentioned this pull request Jun 18, 2024
Open
@HuSharp
merge master
64f0fad 
Signed-off-by: husharp <[email protected]>
@HuSharp HuSharp marked this pull request as ready for review June 21, 2024 02:16
@ti-chi-bot ti-chi-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 21, 2024
@HuSharp
remove redundant code
e39ec5e 
Signed-off-by: husharp <[email protected]>
@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Jul 4, 2024
rleungx
rleungx reviewed Jul 4, 2024
View reviewed changes
tools/pd-simulator/simulator/cases/stable_env.go
)

// newStableEnv provides a stable environment for test.
func newStableEnv(config *sc.SimConfig) *Case {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible that we just create a cluster with a fixed region and store rather than cases?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may not be an easy thing for the current simulator code structure

@lhy1024
Copy link
Contributor

lhy1024 commented Jul 4, 2024 •
edited
Loading

run simulator with this pr while regions are merging.

[2024/07/04 20:36:30.203 +08:00] [INFO] [pd_service_discovery.go:1018] ["[pd] switch leader"] [new-leader=http://192.168.8.88:2379] [old-leader=]
[2024/07/04 20:36:30.203 +08:00] [INFO] [pd_service_discovery.go:498] ["[pd] init cluster id"] [cluster-id=7387758571852092432]
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write
fatal error: concurrent map iteration and map write

goroutine 242707 [running]:
github.com/tikv/pd/pkg/core.(*RegionsInfo).TraverseRegions(...)
        /home/lhy1024/pd/pkg/core/region.go:2216
github.com/tikv/pd/tools/pd-simulator/simulator.(*RaftEngine).TraverseRegions(...)
        /home/lhy1024/pd/tools/pd-simulator/simulator/raft.go:256
github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).regionHeartBeat(0x184c9a0?, 0xc072746b40?)
        /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:193 +0x10d
created by github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat in goroutine 2287
        /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x1db

@lhy1024
Copy link
Contributor

lhy1024 commented Jul 4, 2024 •
edited
Loading

run with "-race"

lots of data race 
==================
WARNING: DATA RACE
Write at 0x00c00014f5b8 by main goroutine:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:175 +0x54
  main.simStart()
      /home/lhy1024/pd/tools/pd-simulator/main.go:173 +0x8bd
  main.run()
      /home/lhy1024/pd/tools/pd-simulator/main.go:95 +0x2c6
  main.main()
      /home/lhy1024/pd/tools/pd-simulator/main.go:89 +0xb4d

Previous read at 0x00c00014f5b8 by goroutine 2298:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).Tick.func2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:180 +0x44

Goroutine 2298 (finished) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:179 +0x13c
  main.simStart()
      /home/lhy1024/pd/tools/pd-simulator/main.go:173 +0x8bd
  main.run()
      /home/lhy1024/pd/tools/pd-simulator/main.go:95 +0x2c6
  main.main()
      /home/lhy1024/pd/tools/pd-simulator/main.go:89 +0xb4d
==================
==================
WARNING: DATA RACE
Write at 0x00c0001fde30 by goroutine 2381:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).compaction()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:184 +0x1c6
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepCompaction()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:157 +0x149
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:131 +0x112
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Previous read at 0x00c0001fde30 by main goroutine:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).Check()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:262 +0x309
  main.simStart()
      /home/lhy1024/pd/tools/pd-simulator/main.go:174 +0x8ca
  main.run()
      /home/lhy1024/pd/tools/pd-simulator/main.go:95 +0x2c6
  main.main()
      /home/lhy1024/pd/tools/pd-simulator/main.go:89 +0xb4d

Goroutine 2381 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Write at 0x00c016bdc310 by goroutine 2367:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).compaction()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:183 +0x127
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepCompaction()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:157 +0x149
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:131 +0x112
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Previous read at 0x00c016bdc310 by main goroutine:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).Check()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:262 +0x309
  main.simStart()
      /home/lhy1024/pd/tools/pd-simulator/main.go:174 +0x8ca
  main.run()
      /home/lhy1024/pd/tools/pd-simulator/main.go:95 +0x2c6
  main.main()
      /home/lhy1024/pd/tools/pd-simulator/main.go:89 +0xb4d

Goroutine 2367 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Write at 0x00c016bdc460 by goroutine 2367:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).compaction()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:185 +0x204
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepCompaction()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:157 +0x149
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:131 +0x112
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Previous read at 0x00c016bdc460 by main goroutine:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).Check()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:262 +0x309
  main.simStart()
      /home/lhy1024/pd/tools/pd-simulator/main.go:174 +0x8ca
  main.run()
      /home/lhy1024/pd/tools/pd-simulator/main.go:95 +0x2c6
  main.main()
      /home/lhy1024/pd/tools/pd-simulator/main.go:89 +0xb4d

Goroutine 2367 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Write at 0x00c00014f5b8 by main goroutine:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:175 +0x54
  main.simStart()
      /home/lhy1024/pd/tools/pd-simulator/main.go:173 +0x8bd
  main.run()
      /home/lhy1024/pd/tools/pd-simulator/main.go:95 +0x2c6
  main.main()
      /home/lhy1024/pd/tools/pd-simulator/main.go:89 +0xb4d

Previous read at 0x00c00014f5b8 by goroutine 2841:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).Tick.func3()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:183 +0x44

Goroutine 2841 (finished) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:182 +0x1a5
  main.simStart()
      /home/lhy1024/pd/tools/pd-simulator/main.go:173 +0x8bd
  main.run()
      /home/lhy1024/pd/tools/pd-simulator/main.go:95 +0x2c6
  main.main()
      /home/lhy1024/pd/tools/pd-simulator/main.go:89 +0xb4d
==================
==================
WARNING: DATA RACE
Write at 0x00c016bd7d88 by goroutine 4406:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).storeHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:170 +0x26a
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x44

Previous read at 0x00c016bd7d88 by main goroutine:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).Check()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:262 +0x309
  main.simStart()
      /home/lhy1024/pd/tools/pd-simulator/main.go:174 +0x8ca
  main.run()
      /home/lhy1024/pd/tools/pd-simulator/main.go:95 +0x2c6
  main.main()
      /home/lhy1024/pd/tools/pd-simulator/main.go:89 +0xb4d

Goroutine 4406 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x2bb
  main.simStart.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/main.go:165 +0x4f
==================
==================
WARNING: DATA RACE
Write at 0x000005ce7260 by goroutine 18269:
  github.com/tikv/pd/tools/pd-simulator/simulator.getLeaderURL()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:385 +0x2d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).requestWithRetry()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:361 +0x130
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).StoreHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:417 +0xaf
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).storeHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:171 +0x31d
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x44

Previous read at 0x000005ce7260 by goroutine 528:
  github.com/tikv/pd/tools/pd-simulator/simulator.requestHeader()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:71 +0x549
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:215 +0x773
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Goroutine 18269 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x2bb
  main.simStart.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/main.go:165 +0x4f

Goroutine 528 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42
==================
==================
WARNING: DATA RACE
Write at 0x000005ce70af by goroutine 2295:
  github.com/tikv/pd/tools/pd-simulator/simulator.ChooseToHaltPDSchedule()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:546 +0x30
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat.func1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:242 +0x1e
  sync.(*Once).doSlow()
      /usr/lib/go/src/sync/once.go:74 +0xf0
  sync.(*Once).Do()
      /usr/lib/go/src/sync/once.go:65 +0x44
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:240 +0x2a8
  main.simStart.gowrap3()
      /home/lhy1024/pd/tools/pd-simulator/main.go:166 +0x4f

Previous read at 0x000005ce70af by main goroutine:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).Check()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:255 +0x3d
  main.simStart()
      /home/lhy1024/pd/tools/pd-simulator/main.go:174 +0x8ca
  main.run()
      /home/lhy1024/pd/tools/pd-simulator/main.go:95 +0x2c6
  main.main()
      /home/lhy1024/pd/tools/pd-simulator/main.go:89 +0xb4d

Goroutine 2295 (running) created at:
  main.simStart()
      /home/lhy1024/pd/tools/pd-simulator/main.go:166 +0x744
  main.run()
      /home/lhy1024/pd/tools/pd-simulator/main.go:95 +0x2c6
  main.main()
      /home/lhy1024/pd/tools/pd-simulator/main.go:89 +0xb4d
==================
==================
WARNING: DATA RACE
Write at 0x00c00de2e980 by goroutine 22082:
  github.com/tikv/pd/pkg/core.(*RegionsInfo).setRegionLocked()
      /home/lhy1024/pd/pkg/core/region.go:1229 +0x364
  github.com/tikv/pd/pkg/core.(*RegionsInfo).SetRegion()
      /home/lhy1024/pd/pkg/core/region.go:1195 +0xea
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RaftEngine).SetRegion()
      /home/lhy1024/pd/tools/pd-simulator/simulator/raft.go:263 +0xf0
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:215 +0x2d2
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Previous read at 0x00c00de2e980 by goroutine 20870:
  github.com/tikv/pd/pkg/core.(*RegionsInfo).TraverseRegions()
      /home/lhy1024/pd/pkg/core/region.go:2217 +0x278
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RaftEngine).TraverseRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/raft.go:256 +0x180
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).regionHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:193 +0xd7
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x44

Goroutine 22082 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f

Goroutine 20870 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x2c9
  main.simStart.gowrap3()
      /home/lhy1024/pd/tools/pd-simulator/main.go:166 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c06f728c50 by goroutine 20851:
  github.com/tikv/pd/pkg/core.(*RegionInfo).GetLeader()
      /home/lhy1024/pd/pkg/core/region.go:676 +0x5b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).regionHeartBeat.func1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:194 +0x1d
  github.com/tikv/pd/pkg/core.(*RegionsInfo).TraverseRegions()
      /home/lhy1024/pd/pkg/core/region.go:2217 +0x28f
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RaftEngine).TraverseRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/raft.go:256 +0x180
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).regionHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:193 +0xd7
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x44

Previous write at 0x00c06f728c50 by goroutine 22082:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*transferLeader).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:276 +0x51a
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 20851 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x2c9
  main.simStart.gowrap3()
      /home/lhy1024/pd/tools/pd-simulator/main.go:166 +0x4f

Goroutine 22082 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05a5cf350 by goroutine 20879:
  github.com/tikv/pd/pkg/core.(*RegionInfo).GetLeader()
      /home/lhy1024/pd/pkg/core/region.go:676 +0x5b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).regionHeartBeat.func1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:194 +0x1d
  github.com/tikv/pd/pkg/core.(*RegionsInfo).TraverseRegions()
      /home/lhy1024/pd/pkg/core/region.go:2217 +0x28f
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RaftEngine).TraverseRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/raft.go:256 +0x180
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).regionHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:193 +0xd7
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x44

Previous write at 0x00c05a5cf350 by goroutine 24977:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:432 +0xe85
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 20879 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x2c9
  main.simStart.gowrap3()
      /home/lhy1024/pd/tools/pd-simulator/main.go:166 +0x4f

Goroutine 24977 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c029e9e448 by goroutine 20879:
  github.com/pingcap/kvproto/pkg/metapb.(*Peer).GetStoreId()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:786 +0xa7
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).regionHeartBeat.func1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:194 +0x1d
  github.com/tikv/pd/pkg/core.(*RegionsInfo).TraverseRegions()
      /home/lhy1024/pd/pkg/core/region.go:2217 +0x28f
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RaftEngine).TraverseRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/raft.go:256 +0x180
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).regionHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:193 +0xd7
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x44

Previous write at 0x00c029e9e448 by goroutine 24977:
  github.com/tikv/pd/pkg/core.init.func2()
      /home/lhy1024/pd/pkg/core/factory.go:27 +0x33
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:38 +0x15c
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:294 +0x489
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:432 +0xe85
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 20879 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x2c9
  main.simStart.gowrap3()
      /home/lhy1024/pd/tools/pd-simulator/main.go:166 +0x4f

Goroutine 24977 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Write at 0x000005ce7260 by goroutine 26776:
  github.com/tikv/pd/tools/pd-simulator/simulator.getLeaderURL()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:385 +0x2d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).requestWithRetry()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:361 +0x130
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).StoreHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:417 +0xaf
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).storeHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:171 +0x31d
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x44

Previous read at 0x000005ce7260 by goroutine 26773:
  github.com/tikv/pd/tools/pd-simulator/simulator.getLeaderURL()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:386 +0x2ec
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).requestWithRetry()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:361 +0x130
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).StoreHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:417 +0xaf
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).storeHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:171 +0x31d
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x44

Goroutine 26776 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x2bb
  main.simStart.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/main.go:165 +0x4f

Goroutine 26773 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x2bb
  main.simStart.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/main.go:165 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c029166250 by goroutine 31300:
  github.com/tikv/pd/pkg/core.(*RegionInfo).GetLeader()
      /home/lhy1024/pd/pkg/core/region.go:676 +0x5b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).regionHeartBeat.func1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:194 +0x1d
  github.com/tikv/pd/pkg/core.(*RegionsInfo).TraverseRegions()
      /home/lhy1024/pd/pkg/core/region.go:2217 +0x28f
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RaftEngine).TraverseRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/raft.go:256 +0x180
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).regionHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:193 +0xd7
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x44

Previous write at 0x00c029166250 by goroutine 31190:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 31300 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x2c9
  main.simStart.gowrap3()
      /home/lhy1024/pd/tools/pd-simulator/main.go:166 +0x4f

Goroutine 31190 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c07ba58108 by goroutine 31300:
  github.com/pingcap/kvproto/pkg/metapb.(*Peer).GetStoreId()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:786 +0xa7
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).regionHeartBeat.func1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:194 +0x1d
  github.com/tikv/pd/pkg/core.(*RegionsInfo).TraverseRegions()
      /home/lhy1024/pd/pkg/core/region.go:2217 +0x28f
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RaftEngine).TraverseRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/raft.go:256 +0x180
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).regionHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:193 +0xd7
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x44

Previous write at 0x00c07ba58108 by goroutine 31190:
  github.com/tikv/pd/pkg/core.init.func2()
      /home/lhy1024/pd/pkg/core/factory.go:27 +0x33
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:38 +0x15c
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:294 +0x489
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 31300 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x2c9
  main.simStart.gowrap3()
      /home/lhy1024/pd/tools/pd-simulator/main.go:166 +0x4f

Goroutine 31190 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Write at 0x000005ce7260 by goroutine 33803:
  github.com/tikv/pd/tools/pd-simulator/simulator.getLeaderURL()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:385 +0x2d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).requestWithRetry()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:361 +0x130
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).StoreHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:417 +0xaf
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).storeHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:171 +0x31d
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x44

Previous read at 0x000005ce7260 by goroutine 33898:
  github.com/tikv/pd/tools/pd-simulator/simulator.requestHeader()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:71 +0x18b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).StoreHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:288 +0x211
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).StoreHeartbeat.func1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:418 +0x74
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).requestWithRetry()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:353 +0x46
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).StoreHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:417 +0xaf
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).storeHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:171 +0x31d
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x44

Goroutine 33803 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x2bb
  main.simStart.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/main.go:165 +0x4f

Goroutine 33898 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x2bb
  main.simStart.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/main.go:165 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c06299ea50 by goroutine 31242:
  github.com/tikv/pd/pkg/core.(*RegionInfo).GetLeader()
      /home/lhy1024/pd/pkg/core/region.go:676 +0x5b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).regionHeartBeat.func1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:194 +0x1d
  github.com/tikv/pd/pkg/core.(*RegionsInfo).TraverseRegions()
      /home/lhy1024/pd/pkg/core/region.go:2217 +0x28f
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RaftEngine).TraverseRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/raft.go:256 +0x180
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).regionHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:193 +0xd7
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x44

Previous write at 0x00c06299ea50 by goroutine 31120:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:294 +0x644
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 31242 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).RegionsHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:237 +0x2c9
  main.simStart.gowrap3()
      /home/lhy1024/pd/tools/pd-simulator/main.go:166 +0x4f

Goroutine 31120 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d158 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:282 +0x64
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d158 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d170 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:286 +0x244
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d170 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee66180 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:287 +0x304
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee66180 by goroutine 31154:
  runtime.slicecopy()
      /usr/lib/go/src/runtime/slice.go:325 +0x0
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick.WithPendingPeers.func3()
      /home/lhy1024/pd/pkg/core/region_option.go:50 +0x91
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:312 +0xbcf
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d100 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x3f8
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d100 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee605a0 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1687 +0x4d
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee605a0 by goroutine 31154:
  github.com/tikv/pd/pkg/core.init.func1()
      /home/lhy1024/pd/pkg/core/factory.go:25 +0x33
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:38 +0x15c
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee605a8 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1690 +0xae
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee605a8 by goroutine 31154:
  github.com/tikv/pd/pkg/core.init.func1()
      /home/lhy1024/pd/pkg/core/factory.go:25 +0x33
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:38 +0x15c
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee605c0 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1694 +0x119
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee605c0 by goroutine 31154:
  github.com/tikv/pd/pkg/core.init.func1()
      /home/lhy1024/pd/pkg/core/factory.go:25 +0x33
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:38 +0x15c
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee605d8 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1698 +0x187
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee605d8 by goroutine 31154:
  github.com/tikv/pd/pkg/core.init.func1()
      /home/lhy1024/pd/pkg/core/factory.go:25 +0x33
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:38 +0x15c
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee01530 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*RegionEpoch).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1584 +0x1d5
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1699 +0x1a5
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee01530 by goroutine 31154:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Unmarshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:3268 +0x1984
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:39 +0x185
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee01538 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*RegionEpoch).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1587 +0x234
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1699 +0x1a5
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee01538 by goroutine 31154:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Unmarshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:3268 +0x1984
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:39 +0x185
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee01540 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*RegionEpoch).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1590 +0x2a4
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1699 +0x1a5
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee01540 by goroutine 31154:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Unmarshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:3268 +0x1984
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:39 +0x185
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee605e0 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1702 +0x326
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee605e0 by goroutine 31154:
  github.com/tikv/pd/pkg/core.init.func1()
      /home/lhy1024/pd/pkg/core/factory.go:25 +0x33
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:38 +0x15c
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c07a67b340 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Peer).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1730 +0x37
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1704 +0x658
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c07a67b340 by goroutine 31154:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Unmarshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:3303 +0x140f
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:39 +0x185
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c07a67b348 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Peer).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1733 +0x95
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1704 +0x658
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c07a67b348 by goroutine 31154:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Unmarshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:3303 +0x140f
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:39 +0x185
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c07a67b350 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Peer).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1736 +0x104
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1704 +0x658
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c07a67b350 by goroutine 31154:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Unmarshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:3303 +0x140f
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:39 +0x185
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c07a67b358 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Peer).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1742 +0x19b
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1704 +0x658
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c07a67b358 by goroutine 31154:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Unmarshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:3303 +0x140f
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:39 +0x185
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee70030 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1703 +0x645
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee70030 by goroutine 31154:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Unmarshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:3303 +0x14d6
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:39 +0x185
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee70038 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1703 +0x645
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee70038 by goroutine 31154:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick.WithAddPeer.func1()
      /home/lhy1024/pd/pkg/core/region_option.go:352 +0xb5
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:312 +0xbcf
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee605f8 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1708 +0x397
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee605f8 by goroutine 31154:
  github.com/tikv/pd/pkg/core.init.func1()
      /home/lhy1024/pd/pkg/core/factory.go:25 +0x33
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:38 +0x15c
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee60600 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1712 +0x524
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee60600 by goroutine 31154:
  github.com/tikv/pd/pkg/core.init.func1()
      /home/lhy1024/pd/pkg/core/factory.go:25 +0x33
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:38 +0x15c
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee60608 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1715 +0x554
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee60608 by goroutine 31154:
  github.com/tikv/pd/pkg/core.init.func1()
      /home/lhy1024/pd/pkg/core/factory.go:25 +0x33
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:38 +0x15c
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee60610 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1718 +0x5e4
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1319 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee60610 by goroutine 31154:
  github.com/tikv/pd/pkg/core.init.func1()
      /home/lhy1024/pd/pkg/core/factory.go:25 +0x33
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:38 +0x15c
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c01838ed20 by goroutine 1988:
  runtime.slicecopy()
      /usr/lib/go/src/runtime/slice.go:325 +0x0
  github.com/pingcap/kvproto/pkg/metapb.(*Region).MarshalToSizedBuffer()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1397 +0x97b
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1321 +0x64
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c01838ed20 by goroutine 31154:
  runtime.slicecopy()
      /usr/lib/go/src/runtime/slice.go:325 +0x0
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Unmarshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:3233 +0x1e1a
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:39 +0x185
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c01838ece0 by goroutine 1988:
  runtime.slicecopy()
      /usr/lib/go/src/runtime/slice.go:325 +0x0
  github.com/pingcap/kvproto/pkg/metapb.(*Region).MarshalToSizedBuffer()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1404 +0xbbb
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1321 +0x64
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c01838ece0 by goroutine 31154:
  runtime.slicecopy()
      /usr/lib/go/src/runtime/slice.go:325 +0x0
  github.com/pingcap/kvproto/pkg/metapb.(*Region).Unmarshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:3199 +0x2224
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:39 +0x185
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:293 +0x432
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c07a67b440 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Peer).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1730 +0x37
  github.com/pingcap/kvproto/pkg/metapb.(*Peer).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1418 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:294 +0x489
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c07a67b440 by goroutine 31154:
  github.com/tikv/pd/pkg/core.init.func2()
      /home/lhy1024/pd/pkg/core/factory.go:27 +0x33
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:38 +0x15c
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:294 +0x489
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c07a67b450 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Peer).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1736 +0x104
  github.com/pingcap/kvproto/pkg/metapb.(*Peer).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1418 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:294 +0x489
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c07a67b450 by goroutine 31154:
  github.com/tikv/pd/pkg/core.init.func2()
      /home/lhy1024/pd/pkg/core/factory.go:27 +0x33
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:38 +0x15c
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:294 +0x489
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c07a67b458 by goroutine 1988:
  github.com/pingcap/kvproto/pkg/metapb.(*Peer).Size()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1742 +0x19b
  github.com/pingcap/kvproto/pkg/metapb.(*Peer).Marshal()
      /home/lhy1024/.go/pkg/mod/github.com/pingcap/[email protected]/pkg/metapb/metapb.pb.go:1418 +0x2e
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:33 +0x131
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:294 +0x489
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c07a67b458 by goroutine 31154:
  github.com/tikv/pd/pkg/core.init.func2()
      /home/lhy1024/pd/pkg/core/factory.go:27 +0x33
  github.com/tikv/pd/pkg/utils/typeutil.DeepClone[go.shape.*uint8]()
      /home/lhy1024/pd/pkg/utils/typeutil/clone.go:38 +0x15c
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:294 +0x489
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d1d0 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:305 +0x4a8
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d1d0 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d1e0 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:308 +0x505
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d1e0 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d188 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:292 +0x5c7
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d188 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d190 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:297 +0x747
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d190 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d198 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:298 +0x7a4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d198 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d1a0 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:299 +0x7fc
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d1a0 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d1a8 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:300 +0x854
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d1a8 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d1b0 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:301 +0x8ac
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d1b0 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d1b8 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:302 +0x904
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d1b8 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d1c0 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:303 +0x95c
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d1c0 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d1c8 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:304 +0x9b4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d1c8 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d1d8 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:306 +0xa5a
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d1d8 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c05ee4d1f0 by goroutine 1988:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:307 +0xae4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).reportRegionHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:213 +0x4e4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x8f

Previous write at 0x00c05ee4d1f0 by goroutine 31154:
  github.com/tikv/pd/pkg/core.(*RegionInfo).Clone()
      /home/lhy1024/pd/pkg/core/region.go:291 +0x567
  github.com/tikv/pd/tools/pd-simulator/simulator.(*addPeer).tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:416 +0x711
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Task).Step()
      /home/lhy1024/pd/tools/pd-simulator/simulator/task.go:211 +0x21b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).stepTask()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:145 +0x1d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).Tick()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:132 +0x153
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x44

Goroutine 1988 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).HeartbeatStreamLoop()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:156 +0x378
  github.com/tikv/pd/tools/pd-simulator/simulator.NewRetryClient.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:346 +0x42

Goroutine 31154 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StepRegions()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:196 +0x1fd
  main.simStart.gowrap4()
      /home/lhy1024/pd/tools/pd-simulator/main.go:167 +0x4f
==================
==================
WARNING: DATA RACE
Write at 0x000005ce7260 by goroutine 40120:
  github.com/tikv/pd/tools/pd-simulator/simulator.getLeaderURL()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:385 +0x2d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).requestWithRetry()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:361 +0x130
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).StoreHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:417 +0xaf
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).storeHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:171 +0x31d
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x44

Previous read at 0x000005ce7260 by goroutine 40075:
  github.com/tikv/pd/tools/pd-simulator/simulator.requestHeader()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:71 +0x18b
  github.com/tikv/pd/tools/pd-simulator/simulator.(*client).StoreHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:288 +0x211
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).StoreHeartbeat.func1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:418 +0x74
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).requestWithRetry()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:370 +0x498
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).StoreHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:417 +0xaf
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).storeHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:171 +0x31d
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x44

Goroutine 40120 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x2bb
  main.simStart.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/main.go:165 +0x4f

Goroutine 40075 (finished) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x2bb
  main.simStart.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/main.go:165 +0x4f
==================
==================
WARNING: DATA RACE
Write at 0x000005ce7260 by goroutine 41138:
  github.com/tikv/pd/tools/pd-simulator/simulator.getLeaderURL()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:385 +0x2d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).requestWithRetry()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:361 +0x130
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).StoreHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:417 +0xaf
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).storeHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:171 +0x31d
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x44

Previous write at 0x000005ce7260 by goroutine 41142:
  github.com/tikv/pd/tools/pd-simulator/simulator.getLeaderURL()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:385 +0x2d4
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).requestWithRetry()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:361 +0x130
  github.com/tikv/pd/tools/pd-simulator/simulator.(*RetryClient).StoreHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/client.go:417 +0xaf
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Node).storeHeartBeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/node.go:171 +0x31d
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat.gowrap1()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x44

Goroutine 41138 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x2bb
  main.simStart.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/main.go:165 +0x4f

Goroutine 41142 (running) created at:
  github.com/tikv/pd/tools/pd-simulator/simulator.(*Driver).StoresHeartbeat()
      /home/lhy1024/pd/tools/pd-simulator/simulator/drive.go:215 +0x2bb
  main.simStart.gowrap2()
      /home/lhy1024/pd/tools/pd-simulator/main.go:165 +0x4f
==================

@HuSharp
Copy link
Member Author

HuSharp commented Jul 8, 2024

run with "-race"

lots of data race

fixed in #8376, PTAL, thx!

@ti-chi-bot ti-chi-bot bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 9, 2024
@HuSharp
merge master
bf58b48 
Signed-off-by: husharp <[email protected]>
@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jul 9, 2024
@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jul 26, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lhy1024, nolouch

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 26, 2024
@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jul 26, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-07-04 07:21:19.877192118 +0000 UTC m=+1481806.362680949: ☑️ agreed by lhy1024.
  • 2024-07-26 06:14:46.491786524 +0000 UTC m=+1198508.482727979: ☑️ agreed by nolouch.

@ti-chi-bot ti-chi-bot bot merged commit 2baee83 into tikv:master Jul 26, 2024
21 checks passed
@HuSharp HuSharp deleted the seperate_heartbeat branch July 26, 2024 06:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rleungx rleungx rleungx left review comments

@lhy1024 lhy1024 lhy1024 approved these changes

@nolouch nolouch nolouch approved these changes

Assignees
No one assigned
Labels
approved dco-signoff: yes Indicates the PR's author has signed the dco. lgtm release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Make simulator work with large scale cluster
4 participants
@HuSharp @lhy1024 @nolouch @rleungx