Source file src/runtime/metrics/doc.go
1 // Copyright 2020 The Go Authors. All rights reserved. 2 // Use of this source code is governed by a BSD-style 3 // license that can be found in the LICENSE file. 4 5 // Note: run 'go generate' (which will run 'go test -generate') to update the "Supported metrics" list. 6 //go:generate go test -run=Docs -generate 7 8 /* 9 Package metrics provides a stable interface to access implementation-defined 10 metrics exported by the Go runtime. This package is similar to existing functions 11 like [runtime.ReadMemStats] and [runtime/debug.ReadGCStats], but significantly more general. 12 13 The set of metrics defined by this package may evolve as the runtime itself 14 evolves, and also enables variation across Go implementations, whose relevant 15 metric sets may not intersect. 16 17 # Interface 18 19 Metrics are designated by a string key, rather than, for example, a field name in 20 a struct. The full list of supported metrics is always available in the slice of 21 Descriptions returned by [All]. Each [Description] also includes useful information 22 about the metric. 23 24 Thus, users of this API are encouraged to sample supported metrics defined by the 25 slice returned by All to remain compatible across Go versions. Of course, situations 26 arise where reading specific metrics is critical. For these cases, users are 27 encouraged to use build tags, and although metrics may be deprecated and removed, 28 users should consider this to be an exceptional and rare event, coinciding with a 29 very large change in a particular Go implementation. 30 31 Each metric key also has a "kind" (see [ValueKind]) that describes the format of the 32 metric's value. 33 In the interest of not breaking users of this package, the "kind" for a given metric 34 is guaranteed not to change. If it must change, then a new metric will be introduced 35 with a new key and a new "kind." 36 37 # Metric key format 38 39 As mentioned earlier, metric keys are strings. Their format is simple and well-defined, 40 designed to be both human and machine readable. It is split into two components, 41 separated by a colon: a rooted path and a unit. The choice to include the unit in 42 the key is motivated by compatibility: if a metric's unit changes, its semantics likely 43 did also, and a new key should be introduced. 44 45 For more details on the precise definition of the metric key's path and unit formats, see 46 the documentation of the Name field of the Description struct. 47 48 # A note about floats 49 50 This package supports metrics whose values have a floating-point representation. In 51 order to improve ease-of-use, this package promises to never produce the following 52 classes of floating-point values: NaN, infinity. 53 54 # Supported metrics 55 56 Below is the full list of supported metrics, ordered lexicographically. 57 58 /cgo/go-to-c-calls:calls 59 Count of calls made from Go to C by the current process. 60 61 /cpu/classes/gc/mark/assist:cpu-seconds 62 Estimated total CPU time goroutines spent performing GC 63 tasks to assist the GC and prevent it from falling behind the 64 application. This metric is an overestimate, and not directly 65 comparable to system CPU time measurements. Compare only with 66 other /cpu/classes metrics. 67 68 /cpu/classes/gc/mark/dedicated:cpu-seconds 69 Estimated total CPU time spent performing GC tasks on processors 70 (as defined by GOMAXPROCS) dedicated to those tasks. This metric 71 is an overestimate, and not directly comparable to system CPU 72 time measurements. Compare only with other /cpu/classes metrics. 73 74 /cpu/classes/gc/mark/idle:cpu-seconds 75 Estimated total CPU time spent performing GC tasks on spare CPU 76 resources that the Go scheduler could not otherwise find a use 77 for. This should be subtracted from the total GC CPU time to 78 obtain a measure of compulsory GC CPU time. This metric is an 79 overestimate, and not directly comparable to system CPU time 80 measurements. Compare only with other /cpu/classes metrics. 81 82 /cpu/classes/gc/pause:cpu-seconds 83 Estimated total CPU time spent with the application paused by 84 the GC. Even if only one thread is running during the pause, 85 this is computed as GOMAXPROCS times the pause latency because 86 nothing else can be executing. This is the exact sum of samples 87 in /sched/pauses/total/gc:seconds if each sample is multiplied 88 by GOMAXPROCS at the time it is taken. This metric is an 89 overestimate, and not directly comparable to system CPU time 90 measurements. Compare only with other /cpu/classes metrics. 91 92 /cpu/classes/gc/total:cpu-seconds 93 Estimated total CPU time spent performing GC tasks. This metric 94 is an overestimate, and not directly comparable to system CPU 95 time measurements. Compare only with other /cpu/classes metrics. 96 Sum of all metrics in /cpu/classes/gc. 97 98 /cpu/classes/idle:cpu-seconds 99 Estimated total available CPU time not spent executing 100 any Go or Go runtime code. In other words, the part of 101 /cpu/classes/total:cpu-seconds that was unused. This metric is 102 an overestimate, and not directly comparable to system CPU time 103 measurements. Compare only with other /cpu/classes metrics. 104 105 /cpu/classes/scavenge/assist:cpu-seconds 106 Estimated total CPU time spent returning unused memory to the 107 underlying platform in response eagerly to memory pressure. This 108 metric is an overestimate, and not directly comparable to system 109 CPU time measurements. Compare only with other /cpu/classes 110 metrics. 111 112 /cpu/classes/scavenge/background:cpu-seconds 113 Estimated total CPU time spent performing background tasks to 114 return unused memory to the underlying platform. This metric is 115 an overestimate, and not directly comparable to system CPU time 116 measurements. Compare only with other /cpu/classes metrics. 117 118 /cpu/classes/scavenge/total:cpu-seconds 119 Estimated total CPU time spent performing tasks that return 120 unused memory to the underlying platform. This metric is an 121 overestimate, and not directly comparable to system CPU time 122 measurements. Compare only with other /cpu/classes metrics. 123 Sum of all metrics in /cpu/classes/scavenge. 124 125 /cpu/classes/total:cpu-seconds 126 Estimated total available CPU time for user Go code or the Go 127 runtime, as defined by GOMAXPROCS. In other words, GOMAXPROCS 128 integrated over the wall-clock duration this process has been 129 executing for. This metric is an overestimate, and not directly 130 comparable to system CPU time measurements. Compare only with 131 other /cpu/classes metrics. Sum of all metrics in /cpu/classes. 132 133 /cpu/classes/user:cpu-seconds 134 Estimated total CPU time spent running user Go code. This may 135 also include some small amount of time spent in the Go runtime. 136 This metric is an overestimate, and not directly comparable 137 to system CPU time measurements. Compare only with other 138 /cpu/classes metrics. 139 140 /gc/cleanups/executed:cleanups 141 Approximate total count of cleanup functions (created 142 by runtime.AddCleanup) executed by the runtime. Subtract 143 /gc/cleanups/queued:cleanups to approximate cleanup queue 144 length. Useful for detecting slow cleanups holding up the queue. 145 146 /gc/cleanups/queued:cleanups 147 Approximate total count of cleanup functions (created by 148 runtime.AddCleanup) queued by the runtime for execution. 149 Subtract from /gc/cleanups/executed:cleanups to approximate 150 cleanup queue length. Useful for detecting slow cleanups holding 151 up the queue. 152 153 /gc/cycles/automatic:gc-cycles 154 Count of completed GC cycles generated by the Go runtime. 155 156 /gc/cycles/forced:gc-cycles 157 Count of completed GC cycles forced by the application. 158 159 /gc/cycles/total:gc-cycles 160 Count of all completed GC cycles. 161 162 /gc/finalizers/executed:finalizers 163 Total count of finalizer functions (created by 164 runtime.SetFinalizer) executed by the runtime. Subtract 165 /gc/finalizers/queued:finalizers to approximate finalizer queue 166 length. Useful for detecting finalizers overwhelming the queue, 167 either by being too slow, or by there being too many of them. 168 169 /gc/finalizers/queued:finalizers 170 Total count of finalizer functions (created by 171 runtime.SetFinalizer) and queued by the runtime for execution. 172 Subtract from /gc/finalizers/executed:finalizers to approximate 173 finalizer queue length. Useful for detecting slow finalizers 174 holding up the queue. 175 176 /gc/gogc:percent 177 Heap size target percentage configured by the user, otherwise 178 100. This value is set by the GOGC environment variable, and the 179 runtime/debug.SetGCPercent function. 180 181 /gc/gomemlimit:bytes 182 Go runtime memory limit configured by the user, otherwise 183 math.MaxInt64. This value is set by the GOMEMLIMIT environment 184 variable, and the runtime/debug.SetMemoryLimit function. 185 186 /gc/heap/allocs-by-size:bytes 187 Distribution of heap allocations by approximate size. 188 Bucket counts increase monotonically. Note that this does not 189 include tiny objects as defined by /gc/heap/tiny/allocs:objects, 190 only tiny blocks. 191 192 /gc/heap/allocs:bytes 193 Cumulative sum of memory allocated to the heap by the 194 application. 195 196 /gc/heap/allocs:objects 197 Cumulative count of heap allocations triggered by the 198 application. Note that this does not include tiny objects as 199 defined by /gc/heap/tiny/allocs:objects, only tiny blocks. 200 201 /gc/heap/frees-by-size:bytes 202 Distribution of freed heap allocations by approximate size. 203 Bucket counts increase monotonically. Note that this does not 204 include tiny objects as defined by /gc/heap/tiny/allocs:objects, 205 only tiny blocks. 206 207 /gc/heap/frees:bytes 208 Cumulative sum of heap memory freed by the garbage collector. 209 210 /gc/heap/frees:objects 211 Cumulative count of heap allocations whose storage was freed 212 by the garbage collector. Note that this does not include tiny 213 objects as defined by /gc/heap/tiny/allocs:objects, only tiny 214 blocks. 215 216 /gc/heap/goal:bytes 217 Heap size target for the end of the GC cycle. 218 219 /gc/heap/live:bytes 220 Heap memory occupied by live objects that were marked by the 221 previous GC. 222 223 /gc/heap/objects:objects 224 Number of objects, live or unswept, occupying heap memory. 225 226 /gc/heap/tiny/allocs:objects 227 Count of small allocations that are packed together into blocks. 228 These allocations are counted separately from other allocations 229 because each individual allocation is not tracked by the 230 runtime, only their block. Each block is already accounted for 231 in allocs-by-size and frees-by-size. 232 233 /gc/limiter/last-enabled:gc-cycle 234 GC cycle the last time the GC CPU limiter was enabled. 235 This metric is useful for diagnosing the root cause of an 236 out-of-memory error, because the limiter trades memory for CPU 237 time when the GC's CPU time gets too high. This is most likely 238 to occur with use of SetMemoryLimit. The first GC cycle is cycle 239 1, so a value of 0 indicates that it was never enabled. 240 241 /gc/pauses:seconds 242 Deprecated. Prefer the identical /sched/pauses/total/gc:seconds. 243 244 /gc/scan/globals:bytes 245 The total amount of global variable space that is scannable. 246 247 /gc/scan/heap:bytes 248 The total amount of heap space that is scannable. 249 250 /gc/scan/stack:bytes 251 The number of bytes of stack that were scanned last GC cycle. 252 253 /gc/scan/total:bytes 254 The total amount space that is scannable. Sum of all metrics in 255 /gc/scan. 256 257 /gc/stack/starting-size:bytes 258 The stack size of new goroutines. 259 260 /godebug/non-default-behavior/allowmultiplevcs:events 261 The number of non-default behaviors executed by the cmd/go 262 package due to a non-default GODEBUG=allowmultiplevcs=... 263 setting. 264 265 /godebug/non-default-behavior/asynctimerchan:events 266 The number of non-default behaviors executed by the time package 267 due to a non-default GODEBUG=asynctimerchan=... setting. 268 269 /godebug/non-default-behavior/containermaxprocs:events 270 The number of non-default behaviors executed by the runtime 271 package due to a non-default GODEBUG=containermaxprocs=... 272 setting. 273 274 /godebug/non-default-behavior/cryptocustomrand:events 275 The number of non-default behaviors executed by the crypto 276 package due to a non-default GODEBUG=cryptocustomrand=... 277 setting. 278 279 /godebug/non-default-behavior/embedfollowsymlinks:events 280 The number of non-default behaviors executed by the cmd/go 281 package due to a non-default GODEBUG=embedfollowsymlinks=... 282 setting. 283 284 /godebug/non-default-behavior/execerrdot:events 285 The number of non-default behaviors executed by the os/exec 286 package due to a non-default GODEBUG=execerrdot=... setting. 287 288 /godebug/non-default-behavior/gocachehash:events 289 The number of non-default behaviors executed by the cmd/go 290 package due to a non-default GODEBUG=gocachehash=... setting. 291 292 /godebug/non-default-behavior/gocachetest:events 293 The number of non-default behaviors executed by the cmd/go 294 package due to a non-default GODEBUG=gocachetest=... setting. 295 296 /godebug/non-default-behavior/gocacheverify:events 297 The number of non-default behaviors executed by the cmd/go 298 package due to a non-default GODEBUG=gocacheverify=... setting. 299 300 /godebug/non-default-behavior/gotestjsonbuildtext:events 301 The number of non-default behaviors executed by the cmd/go 302 package due to a non-default GODEBUG=gotestjsonbuildtext=... 303 setting. 304 305 /godebug/non-default-behavior/htmlmetacontenturlescape:events 306 The number of non-default behaviors executed by 307 the html/template package due to a non-default 308 GODEBUG=htmlmetacontenturlescape=... setting. 309 310 /godebug/non-default-behavior/http2client:events 311 The number of non-default behaviors executed by the net/http 312 package due to a non-default GODEBUG=http2client=... setting. 313 314 /godebug/non-default-behavior/http2server:events 315 The number of non-default behaviors executed by the net/http 316 package due to a non-default GODEBUG=http2server=... setting. 317 318 /godebug/non-default-behavior/httpcookiemaxnum:events 319 The number of non-default behaviors executed by the net/http 320 package due to a non-default GODEBUG=httpcookiemaxnum=... 321 setting. 322 323 /godebug/non-default-behavior/httplaxcontentlength:events 324 The number of non-default behaviors executed by the net/http 325 package due to a non-default GODEBUG=httplaxcontentlength=... 326 setting. 327 328 /godebug/non-default-behavior/httpmuxgo121:events 329 The number of non-default behaviors executed by the net/http 330 package due to a non-default GODEBUG=httpmuxgo121=... setting. 331 332 /godebug/non-default-behavior/httpservecontentkeepheaders:events 333 The number of non-default behaviors executed 334 by the net/http package due to a non-default 335 GODEBUG=httpservecontentkeepheaders=... setting. 336 337 /godebug/non-default-behavior/installgoroot:events 338 The number of non-default behaviors executed by the go/build 339 package due to a non-default GODEBUG=installgoroot=... setting. 340 341 /godebug/non-default-behavior/multipartmaxheaders:events 342 The number of non-default behaviors executed by 343 the mime/multipart package due to a non-default 344 GODEBUG=multipartmaxheaders=... setting. 345 346 /godebug/non-default-behavior/multipartmaxparts:events 347 The number of non-default behaviors executed by 348 the mime/multipart package due to a non-default 349 GODEBUG=multipartmaxparts=... setting. 350 351 /godebug/non-default-behavior/multipathtcp:events 352 The number of non-default behaviors executed by the net package 353 due to a non-default GODEBUG=multipathtcp=... setting. 354 355 /godebug/non-default-behavior/netedns0:events 356 The number of non-default behaviors executed by the net package 357 due to a non-default GODEBUG=netedns0=... setting. 358 359 /godebug/non-default-behavior/panicnil:events 360 The number of non-default behaviors executed by the runtime 361 package due to a non-default GODEBUG=panicnil=... setting. 362 363 /godebug/non-default-behavior/randautoseed:events 364 The number of non-default behaviors executed by the math/rand 365 package due to a non-default GODEBUG=randautoseed=... setting. 366 367 /godebug/non-default-behavior/randseednop:events 368 The number of non-default behaviors executed by the math/rand 369 package due to a non-default GODEBUG=randseednop=... setting. 370 371 /godebug/non-default-behavior/rsa1024min:events 372 The number of non-default behaviors executed by the crypto/rsa 373 package due to a non-default GODEBUG=rsa1024min=... setting. 374 375 /godebug/non-default-behavior/tarinsecurepath:events 376 The number of non-default behaviors executed by the archive/tar 377 package due to a non-default GODEBUG=tarinsecurepath=... 378 setting. 379 380 /godebug/non-default-behavior/tls10server:events 381 The number of non-default behaviors executed by the crypto/tls 382 package due to a non-default GODEBUG=tls10server=... setting. 383 384 /godebug/non-default-behavior/tls3des:events 385 The number of non-default behaviors executed by the crypto/tls 386 package due to a non-default GODEBUG=tls3des=... setting. 387 388 /godebug/non-default-behavior/tlsmaxrsasize:events 389 The number of non-default behaviors executed by the crypto/tls 390 package due to a non-default GODEBUG=tlsmaxrsasize=... setting. 391 392 /godebug/non-default-behavior/tlsrsakex:events 393 The number of non-default behaviors executed by the crypto/tls 394 package due to a non-default GODEBUG=tlsrsakex=... setting. 395 396 /godebug/non-default-behavior/tlssha1:events 397 The number of non-default behaviors executed by the crypto/tls 398 package due to a non-default GODEBUG=tlssha1=... setting. 399 400 /godebug/non-default-behavior/tlsunsafeekm:events 401 The number of non-default behaviors executed by the crypto/tls 402 package due to a non-default GODEBUG=tlsunsafeekm=... setting. 403 404 /godebug/non-default-behavior/updatemaxprocs:events 405 The number of non-default behaviors executed by the runtime 406 package due to a non-default GODEBUG=updatemaxprocs=... setting. 407 408 /godebug/non-default-behavior/urlmaxqueryparams:events 409 The number of non-default behaviors executed by the net/url 410 package due to a non-default GODEBUG=urlmaxqueryparams=... 411 setting. 412 413 /godebug/non-default-behavior/urlstrictcolons:events 414 The number of non-default behaviors executed by the net/url 415 package due to a non-default GODEBUG=urlstrictcolons=... 416 setting. 417 418 /godebug/non-default-behavior/winreadlinkvolume:events 419 The number of non-default behaviors executed by the os package 420 due to a non-default GODEBUG=winreadlinkvolume=... setting. 421 422 /godebug/non-default-behavior/winsymlink:events 423 The number of non-default behaviors executed by the os package 424 due to a non-default GODEBUG=winsymlink=... setting. 425 426 /godebug/non-default-behavior/x509keypairleaf:events 427 The number of non-default behaviors executed by the crypto/tls 428 package due to a non-default GODEBUG=x509keypairleaf=... 429 setting. 430 431 /godebug/non-default-behavior/x509negativeserial:events 432 The number of non-default behaviors executed by the crypto/x509 433 package due to a non-default GODEBUG=x509negativeserial=... 434 setting. 435 436 /godebug/non-default-behavior/x509rsacrt:events 437 The number of non-default behaviors executed by the crypto/x509 438 package due to a non-default GODEBUG=x509rsacrt=... setting. 439 440 /godebug/non-default-behavior/x509sha256skid:events 441 The number of non-default behaviors executed by the crypto/x509 442 package due to a non-default GODEBUG=x509sha256skid=... setting. 443 444 /godebug/non-default-behavior/x509usefallbackroots:events 445 The number of non-default behaviors executed by the crypto/x509 446 package due to a non-default GODEBUG=x509usefallbackroots=... 447 setting. 448 449 /godebug/non-default-behavior/x509usepolicies:events 450 The number of non-default behaviors executed by the crypto/x509 451 package due to a non-default GODEBUG=x509usepolicies=... 452 setting. 453 454 /godebug/non-default-behavior/zipinsecurepath:events 455 The number of non-default behaviors executed by the archive/zip 456 package due to a non-default GODEBUG=zipinsecurepath=... 457 setting. 458 459 /memory/classes/heap/free:bytes 460 Memory that is completely free and eligible to be returned to 461 the underlying system, but has not been. This metric is the 462 runtime's estimate of free address space that is backed by 463 physical memory. 464 465 /memory/classes/heap/objects:bytes 466 Memory occupied by live objects and dead objects that have not 467 yet been marked free by the garbage collector. 468 469 /memory/classes/heap/released:bytes 470 Memory that is completely free and has been returned to the 471 underlying system. This metric is the runtime's estimate of free 472 address space that is still mapped into the process, but is not 473 backed by physical memory. 474 475 /memory/classes/heap/stacks:bytes 476 Memory allocated from the heap that is reserved for stack space, 477 whether or not it is currently in-use. Currently, this 478 represents all stack memory for goroutines. It also includes all 479 OS thread stacks in non-cgo programs. Note that stacks may be 480 allocated differently in the future, and this may change. 481 482 /memory/classes/heap/unused:bytes 483 Memory that is reserved for heap objects but is not currently 484 used to hold heap objects. 485 486 /memory/classes/metadata/mcache/free:bytes 487 Memory that is reserved for runtime mcache structures, but not 488 in-use. 489 490 /memory/classes/metadata/mcache/inuse:bytes 491 Memory that is occupied by runtime mcache structures that are 492 currently being used. 493 494 /memory/classes/metadata/mspan/free:bytes 495 Memory that is reserved for runtime mspan structures, but not 496 in-use. 497 498 /memory/classes/metadata/mspan/inuse:bytes 499 Memory that is occupied by runtime mspan structures that are 500 currently being used. 501 502 /memory/classes/metadata/other:bytes 503 Memory that is reserved for or used to hold runtime metadata. 504 505 /memory/classes/os-stacks:bytes 506 Stack memory allocated by the underlying operating system. 507 In non-cgo programs this metric is currently zero. This may 508 change in the future.In cgo programs this metric includes 509 OS thread stacks allocated directly from the OS. Currently, 510 this only accounts for one stack in c-shared and c-archive build 511 modes, and other sources of stacks from the OS are not measured. 512 This too may change in the future. 513 514 /memory/classes/other:bytes 515 Memory used by execution trace buffers, structures for debugging 516 the runtime, finalizer and profiler specials, and more. 517 518 /memory/classes/profiling/buckets:bytes 519 Memory that is used by the stack trace hash map used for 520 profiling. 521 522 /memory/classes/total:bytes 523 All memory mapped by the Go runtime into the current process 524 as read-write. Note that this does not include memory mapped 525 by code called via cgo or via the syscall package. Sum of all 526 metrics in /memory/classes. 527 528 /sched/gomaxprocs:threads 529 The current runtime.GOMAXPROCS setting, or the number of 530 operating system threads that can execute user-level Go code 531 simultaneously. 532 533 /sched/goroutines-created:goroutines 534 Count of goroutines created since program start. 535 536 /sched/goroutines/not-in-go:goroutines 537 Approximate count of goroutines running or blocked in 538 a system call or cgo call. Not guaranteed to add up to 539 /sched/goroutines:goroutines with other goroutine metrics. 540 541 /sched/goroutines/runnable:goroutines 542 Approximate count of goroutines ready to execute, 543 but not executing. Not guaranteed to add up to 544 /sched/goroutines:goroutines with other goroutine metrics. 545 546 /sched/goroutines/running:goroutines 547 Approximate count of goroutines executing. Always less than or 548 equal to /sched/gomaxprocs:threads. Not guaranteed to add up to 549 /sched/goroutines:goroutines with other goroutine metrics. 550 551 /sched/goroutines/waiting:goroutines 552 Approximate count of goroutines waiting on a resource 553 (I/O or sync primitives). Not guaranteed to add up to 554 /sched/goroutines:goroutines with other goroutine metrics. 555 556 /sched/goroutines:goroutines 557 Count of live goroutines. 558 559 /sched/latencies:seconds 560 Distribution of the time goroutines have spent in the scheduler 561 in a runnable state before actually running. Bucket counts 562 increase monotonically. 563 564 /sched/pauses/stopping/gc:seconds 565 Distribution of individual GC-related stop-the-world stopping 566 latencies. This is the time it takes from deciding to stop the 567 world until all Ps are stopped. This is a subset of the total 568 GC-related stop-the-world time (/sched/pauses/total/gc:seconds). 569 During this time, some threads may be executing. Bucket counts 570 increase monotonically. 571 572 /sched/pauses/stopping/other:seconds 573 Distribution of individual non-GC-related stop-the-world 574 stopping latencies. This is the time it takes from deciding 575 to stop the world until all Ps are stopped. This is a 576 subset of the total non-GC-related stop-the-world time 577 (/sched/pauses/total/other:seconds). During this time, some 578 threads may be executing. Bucket counts increase monotonically. 579 580 /sched/pauses/total/gc:seconds 581 Distribution of individual GC-related stop-the-world pause 582 latencies. This is the time from deciding to stop the world 583 until the world is started again. Some of this time is spent 584 getting all threads to stop (this is measured directly in 585 /sched/pauses/stopping/gc:seconds), during which some threads 586 may still be running. Bucket counts increase monotonically. 587 588 /sched/pauses/total/other:seconds 589 Distribution of individual non-GC-related stop-the-world 590 pause latencies. This is the time from deciding to stop the 591 world until the world is started again. Some of this time 592 is spent getting all threads to stop (measured directly in 593 /sched/pauses/stopping/other:seconds). Bucket counts increase 594 monotonically. 595 596 /sched/threads/total:threads 597 The current count of live threads that are owned by the Go 598 runtime. 599 600 /sync/mutex/wait/total:seconds 601 Approximate cumulative time goroutines have spent blocked on a 602 sync.Mutex, sync.RWMutex, or runtime-internal lock. This metric 603 is useful for identifying global changes in lock contention. 604 Collect a mutex or block profile using the runtime/pprof package 605 for more detailed contention data. 606 */ 607 package metrics 608