Prometheus

用途說明

可透過Prometheus Server來抓取所提供之指標，指標資訊包含所有代理的服務，且指標格式使用Prometheus之公開格式。

指標可以透過API Manager所提供的admin api :
<API Manager protocol>://<API Manager ip>:<API Manager admin api port>/metrics 來取得指標資訊。

此插件提供的指標為節點級別，需設定Prometheus服務檢測機制來檢測到所有API Manager之服務節點，並透過每個服務節點配置的/metrics api 端點來取得指標數據。

可用指標

• Status codes: 上游服務返回的HTTP狀態代碼。每個服務都會使用此指標。
• Latencies Histograms: API Manager的延遲測量。
  • Request: API Manager和上游服務用於處理請求的總時間。
  • Kong: 路由請求和運行所有已配置插件所需的時間。
  • Upstream: 上游服務回應請求所花費的時間。
• Bandwidth: 流經API Manager的總頻寬（egress/ingress）。每個服務都會使用此指標。
• DB reachability: 值為0或1的儀表類型，表示API Manager節點是否可以訪問DB。
• Connections: 各種Nginx連接指標，例如活躍，讀取，寫入和接受的連接數。
• Target Health: 屬於已設置之上游目標的健康狀態(healthchecks_off、 healthy、 unhealthy、dns_error)。

用法示例

指標輸出範例

透過/metrics api 端點可取得以下指標輸出範例：

curl -i <API Manager protocol>://<API Manager ip>:<API Manager admin api port>/metrics

HTTP/1.1 200 OK

Server: openresty/1.15.8.3

Date: Tue, 7 Jun 2020 16:35:40 GMT

Content-Type: text/plain; charset=UTF-8

Transfer-Encoding: chunked

Connection: keep-alive

Access-Control-Allow-Origin: *

# HELP kong_bandwidth_total Total bandwidth in bytes for all proxied requests in Kong

# TYPE kong_bandwidth_total counter

kong_bandwidth_total{type="egress"} 1277

kong_bandwidth_total{type="ingress"} 254

# HELP kong_bandwidth Total bandwidth in bytes consumed per service in Kong

# TYPE kong_bandwidth counter

kong_bandwidth{type="egress",service="google"} 1277

kong_bandwidth{type="ingress",service="google"} 254

# HELP kong_datastore_reachable Datastore reachable from Kong, 0 is unreachable

# TYPE kong_datastore_reachable gauge

kong_datastore_reachable 1

# HELP kong_http_status_total HTTP status codes aggreggated across all services in Kong

# TYPE kong_http_status_total counter

kong_http_status_total{code="301"} 2

# HELP kong_http_status HTTP status codes per service in Kong

# TYPE kong_http_status counter

kong_http_status{code="301",service="google"} 2

# HELP kong_latency Latency added by Kong, total request time and upstream latency for each service in Kong

# TYPE kong_latency histogram

kong_latency_bucket{type="kong",service="google",le="00001.0"} 1

kong_latency_bucket{type="kong",service="google",le="00002.0"} 1

.

.

.

kong_latency_bucket{type="kong",service="google",le="+Inf"} 2

kong_latency_bucket{type="request",service="google",le="00300.0"} 1

kong_latency_bucket{type="request",service="google",le="00400.0"} 1

.

.

kong_latency_bucket{type="request",service="google",le="+Inf"} 2

kong_latency_bucket{type="upstream",service="google",le="00300.0"} 2

kong_latency_bucket{type="upstream",service="google",le="00400.0"} 2

.

.

kong_latency_bucket{type="upstream",service="google",le="+Inf"} 2

kong_latency_count{type="kong",service="google"} 2

kong_latency_count{type="request",service="google"} 2

kong_latency_count{type="upstream",service="google"} 2

kong_latency_sum{type="kong",service="google"} 2145

kong_latency_sum{type="request",service="google"} 2672

kong_latency_sum{type="upstream",service="google"} 527

# HELP kong_latency_total Latency added by Kong, total request time and upstream latency aggreggated across all services in Kong

# TYPE kong_latency_total histogram

kong_latency_total_bucket{type="kong",le="00001.0"} 1

kong_latency_total_bucket{type="kong",le="00002.0"} 1

.

.

kong_latency_total_bucket{type="kong",le="+Inf"} 2

kong_latency_total_bucket{type="request",le="00300.0"} 1

kong_latency_total_bucket{type="request",le="00400.0"} 1

.

.

kong_latency_total_bucket{type="request",le="+Inf"} 2

kong_latency_total_bucket{type="upstream",le="00300.0"} 2

kong_latency_total_bucket{type="upstream",le="00400.0"} 2

.

.

.

kong_latency_total_bucket{type="upstream",le="+Inf"} 2

kong_latency_total_count{type="kong"} 2

kong_latency_total_count{type="request"} 2

kong_latency_total_count{type="upstream"} 2

kong_latency_total_sum{type="kong"} 2145

kong_latency_total_sum{type="request"} 2672

kong_latency_total_sum{type="upstream"} 527

# HELP kong_nginx_http_current_connections Number of HTTP connections

# TYPE kong_nginx_http_current_connections gauge

kong_nginx_http_current_connections{state="accepted"} 8

kong_nginx_http_current_connections{state="active"} 1

kong_nginx_http_current_connections{state="handled"} 8

kong_nginx_http_current_connections{state="reading"} 0

kong_nginx_http_current_connections{state="total"} 8

kong_nginx_http_current_connections{state="waiting"} 0

kong_nginx_http_current_connections{state="writing"} 1

# HELP kong_nginx_metric_errors_total Number of nginx-lua-prometheus errors

# TYPE kong_nginx_metric_errors_total counter

kong_nginx_metric_errors_total 0

# HELP kong_upstream_target_health Health status of targets of upstream. States = healthchecks_off|healthy|unhealthy|dns_error, value is 1 when state is populated.

kong_upstream_target_health{upstream="<upstream_name>",target="<target>",address="<ip>:<port>",state="healthchecks_off"} 0

kong_upstream_target_health{upstream="<upstream_name>",target="<target>",address="<ip>:<port>",state="healthy"} 1

kong_upstream_target_health{upstream="<upstream_name>",target="<target>",address="<ip>:<port>",state="unhealthy"} 0

kong_upstream_target_health{upstream="<upstream_name>",target="<target>",address="<ip>:<port>",state="dns_error"} 0

在全局啟用插件

1. 從網站左邊Menu中 外掛插件 頁面中，點選右上角的 新增外掛插件 ：

2. 點選後，選擇 分析及監控 頁籤，並啟用Prometheus，填寫內容參考欄位配置說明，設定成功後，任何請求(不分服務、路由、用戶)都會在該節點上紀錄相關指標數據。

在服務端上啟用插件

1. 從網站左邊Menu中 服務 > 服務列表 頁面中，選擇要啟用此插件的服務，假設為google，點選對應的編輯按鈕：

2. 在編輯畫面中，點選上方的 外掛插件 頁籤，再點選頁籤內容上方的 新增外掛插件 按鈕：

點選後，選擇 分析及監控 頁籤，並啟用Prometheus，填寫內容參考欄位配置說明，設定成功後，僅有此服務(範例為google)請求會在該節點上紀錄相關指標數據。

在路由端上啟用插件

可以由兩種方式來選擇路由，並啟用插件：

方式一：路由列表

1. 從網站左邊Menu中 服務 > 路由列表 頁面中，選擇要啟用此插件的路由，假設為google，點選對應的編輯按鈕：

方式二：服務 > 服務列表 > 路由列表

1. 從網站左邊Menu中 服務 > 服務列表 頁面中，選擇要啟用此插件的路由 所屬之服務(假設為google)，點選對應的編輯按鈕。

在編輯畫面中，點選上方的 路由 頁籤，選擇要啟用此插件的路由(假設為google)，點選對應的編輯按鈕：

---

2. 承第1步，點擊上述兩種方式之一的編輯按鈕後，在編輯畫面中，點選上方的 外掛插件 頁籤，再點選頁籤內容上方的 新增外掛插件 按鈕：

點選新增外掛插件 按鈕後，選擇 分析及監控 頁籤，並啟用Prometheus，填寫內容參考欄位配置說明，設定成功後，僅有此路由(範例為google)請求會在該節點上紀錄相關指標數據。

在用戶端上啟用插件

1. 從網站左邊Menu中 訂閱用戶 > 用戶列表 頁面中，選擇要啟用此插件的用戶，假設為portaladmin，點選對應的編輯按鈕：

2. 在編輯畫面中，點選上方的 外掛插件 頁籤，再點選頁籤內容上方的 新增外掛插件 按鈕：

點選後，選擇 分析及監控 頁籤，並啟用Prometheus，填寫內容參考欄位配置說明，設定成功後，僅有此用戶(範例為portaladmin)請求會在該節點上紀錄相關指標數據。

使用Grafana儀表板

Prometheus所蒐集的指標數據可以透過Grafana來以圖形方式顯示，生成儀表板。

驗證

1. 在全域啟用Prometheus插件後，透過API Manager呼叫的任何服務，都會產生對應的指標數據。

2. 確認是否已經部署Prometheus服務，Prometheus有對應的scrape_configs內的job設定，可以設定定時N秒獲取一次 /metrics 端點的指標數據，並指定獲取目標的端點，需定義為API Manager的ip 及其 admin api port，才能取得數據。

3. 可透過Prometheus內建之服務網頁(一般預設為9090 port)來觀看是否有獲取到指標相關數據。 或是可透過呼叫api，如指標輸出範例所示，確認指標數據是否有獲取成功，以下為實際範例：

curl <API Manager protocol>://<API Manager ip>:<API Manager admin api port>/metrics -i

HTTP/1.1 200 OK

Date: Tue, 13 Apr 2021 03:14:14 GMT

Content-Type: text/plain; charset=UTF-8

Transfer-Encoding: chunked

Connection: keep-alive

Access-Control-Allow-Origin: *

X-Kong-Admin-Latency: 10

Server: kong/2.3.3

# HELP kong_bandwidth Total bandwidth in bytes consumed per service in Kong

# TYPE kong_bandwidth counter

kong_bandwidth{type="egress",service="backend",route="backend"} 7552

kong_bandwidth{type="egress",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 13373

kong_bandwidth{type="egress",service="gocv3",route="gocv312"} 32080

kong_bandwidth{type="ingress",service="backend",route="backend"} 850

kong_bandwidth{type="ingress",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 2645

kong_bandwidth{type="ingress",service="gocv3",route="gocv312"} 847

# HELP kong_datastore_reachable Datastore reachable from Kong, 0 is unreachable

# TYPE kong_datastore_reachable gauge

kong_datastore_reachable 1

# HELP kong_http_status HTTP status codes per service in Kong

# TYPE kong_http_status counter

kong_http_status{code="200",service="backend",route="backend"} 10

kong_http_status{code="200",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 2

kong_http_status{code="200",service="gocv3",route="gocv312"} 7

kong_http_status{code="403",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 1

# HELP kong_latency Latency added by Kong, total request time and upstream latency for each service in Kong

# TYPE kong_latency histogram

kong_latency_bucket{type="kong",service="backend",route="backend",le="00002.0"} 5

kong_latency_bucket{type="kong",service="backend",route="backend",le="00005.0"} 7

kong_latency_bucket{type="kong",service="backend",route="backend",le="00007.0"} 7

kong_latency_bucket{type="kong",service="backend",route="backend",le="00010.0"} 7

kong_latency_bucket{type="kong",service="backend",route="backend",le="00015.0"} 7

kong_latency_bucket{type="kong",service="backend",route="backend",le="00020.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00025.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00030.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00040.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00050.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00060.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00070.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00080.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00090.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00100.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00200.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="00300.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="00400.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="00500.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="01000.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="02000.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="05000.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="10000.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="30000.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="60000.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="+Inf"} 10

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00007.0"} 1

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00010.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00015.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00020.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00025.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00030.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00040.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00050.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00060.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00070.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00080.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00090.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00100.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00200.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00300.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00400.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00500.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="01000.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="02000.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="05000.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="10000.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="30000.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="60000.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="+Inf"} 3

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00001.0"} 2

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00002.0"} 3

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00005.0"} 4

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00007.0"} 5

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00010.0"} 6

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00015.0"} 6

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00020.0"} 6

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00025.0"} 6

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00030.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00040.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00050.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00060.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00070.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00080.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00090.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00100.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00200.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00300.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00400.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00500.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="01000.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="02000.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="05000.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="10000.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="30000.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="60000.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="+Inf"} 7

kong_latency_bucket{type="request",service="backend",route="backend",le="00005.0"} 2

kong_latency_bucket{type="request",service="backend",route="backend",le="00007.0"} 3

kong_latency_bucket{type="request",service="backend",route="backend",le="00010.0"} 4

kong_latency_bucket{type="request",service="backend",route="backend",le="00015.0"} 6

kong_latency_bucket{type="request",service="backend",route="backend",le="00020.0"} 6

kong_latency_bucket{type="request",service="backend",route="backend",le="00025.0"} 7

kong_latency_bucket{type="request",service="backend",route="backend",le="00030.0"} 7

kong_latency_bucket{type="request",service="backend",route="backend",le="00040.0"} 7

kong_latency_bucket{type="request",service="backend",route="backend",le="00050.0"} 7

kong_latency_bucket{type="request",service="backend",route="backend",le="00060.0"} 8

kong_latency_bucket{type="request",service="backend",route="backend",le="00070.0"} 8

kong_latency_bucket{type="request",service="backend",route="backend",le="00080.0"} 8

kong_latency_bucket{type="request",service="backend",route="backend",le="00090.0"} 8

kong_latency_bucket{type="request",service="backend",route="backend",le="00100.0"} 8

kong_latency_bucket{type="request",service="backend",route="backend",le="00200.0"} 9

kong_latency_bucket{type="request",service="backend",route="backend",le="00300.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="00400.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="00500.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="01000.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="02000.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="05000.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="10000.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="30000.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="60000.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="+Inf"} 10

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00007.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00010.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00015.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00020.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00025.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00030.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00040.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00050.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00060.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00070.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00080.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00090.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00100.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00200.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00300.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00400.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00500.0"} 2

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="01000.0"} 3

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="02000.0"} 3

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="05000.0"} 3

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="10000.0"} 3

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="30000.0"} 3

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="60000.0"} 3

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="+Inf"} 3

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00040.0"} 2

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00050.0"} 3

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00060.0"} 5

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00070.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00080.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00090.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00100.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00200.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00300.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00400.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00500.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="01000.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="02000.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="05000.0"} 7

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="10000.0"} 7

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="30000.0"} 7

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="60000.0"} 7

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="+Inf"} 7

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00005.0"} 3

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00007.0"} 5

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00010.0"} 7

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00015.0"} 8

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00020.0"} 8

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00025.0"} 8

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00030.0"} 8

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00040.0"} 8

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00050.0"} 9

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00060.0"} 9

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00070.0"} 9

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00080.0"} 9

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00090.0"} 9

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00100.0"} 9

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00200.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00300.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00400.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00500.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="01000.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="02000.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="05000.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="10000.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="30000.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="60000.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="+Inf"} 10

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00500.0"} 2

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="01000.0"} 2

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="02000.0"} 2

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="05000.0"} 2

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="10000.0"} 2

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="30000.0"} 2

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="60000.0"} 2

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="+Inf"} 2

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00040.0"} 3

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00050.0"} 4

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00060.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00070.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00080.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00090.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00100.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00200.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00300.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00400.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00500.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="01000.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="02000.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="05000.0"} 7

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="10000.0"} 7

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="30000.0"} 7

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="60000.0"} 7

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="+Inf"} 7

kong_latency_count{type="kong",service="backend",route="backend"} 10

kong_latency_count{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 3

kong_latency_count{type="kong",service="gocv3",route="gocv312"} 7

kong_latency_count{type="request",service="backend",route="backend"} 10

kong_latency_count{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 3

kong_latency_count{type="request",service="gocv3",route="gocv312"} 7

kong_latency_count{type="upstream",service="backend",route="backend"} 10

kong_latency_count{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 2

kong_latency_count{type="upstream",service="gocv3",route="gocv312"} 7

kong_latency_sum{type="kong",service="backend",route="backend"} 289

kong_latency_sum{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 200

kong_latency_sum{type="kong",service="gocv3",route="gocv312"} 51

kong_latency_sum{type="request",service="backend",route="backend"} 527

kong_latency_sum{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 1138

kong_latency_sum{type="request",service="gocv3",route="gocv312"} 3296

kong_latency_sum{type="upstream",service="backend",route="backend"} 235

kong_latency_sum{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 937

kong_latency_sum{type="upstream",service="gocv3",route="gocv312"} 3244

# HELP kong_nginx_http_current_connections Number of HTTP connections

# TYPE kong_nginx_http_current_connections gauge

kong_nginx_http_current_connections{state="accepted"} 51

kong_nginx_http_current_connections{state="active"} 1

kong_nginx_http_current_connections{state="handled"} 51

kong_nginx_http_current_connections{state="reading"} 0

kong_nginx_http_current_connections{state="total"} 421

kong_nginx_http_current_connections{state="waiting"} 0

kong_nginx_http_current_connections{state="writing"} 1

# HELP kong_nginx_metric_errors_total Number of nginx-lua-prometheus errors

# TYPE kong_nginx_metric_errors_total counter

kong_nginx_metric_errors_total 0