Prometheus

用途說明

可透過 Prometheus Server 來抓取所提供之指標，指標資訊包含所有代理的服務，且指標格式使用 Prometheus 之公開格式。

可以透過 GOC API Gateway 所提供的 Admin API :
<GOC API Gateway protocol>://<GOC API Gateway ip>:<GOC API Gateway admin api port>/metrics 來取得指標資訊，如下方的 INFO 區塊內範例：

• protocol 為 http。
• ip 為 10.112.1.3。
• admin api port 為 31218。(跟管理介面不同 port)

info

上述的 Admin API 不等於目前管理介面的 URL，可參考下圖：

若不清楚如何取得 GOC API Gateway 所提供的 Admin API，可以參考 常見問題。

此插件提供的指標為節點級別，需設定 Prometheus 服務檢測機制來檢測到所有 GOC API Gateway 之服務節點，並透過每個服務節點配置的 /metrics api 端點來取得指標數據。

可用指標

• Status codes： 上游服務返回的 HTTP 狀態代碼。每個服務都會使用此指標。
• Latencies Histograms： GOC API Gateway 的延遲測量。
  • Request： GOC API Gateway 和上游服務用於處理請求的總時間。
  • Kong： 路由請求和運行所有已配置插件所需的時間。
  • Upstream： 上游服務回應請求所花費的時間。
• Bandwidth： 流經 GOC API Gateway 的總頻寬（egress/ingress）。每個服務都會使用此指標。
• DB reachability： 值為 0 或 1 的儀表類型，表示 GOC API Gateway 節點是否可以訪問 DB。
• Connections： 各種 Nginx 連接指標，例如活躍，讀取，寫入和接受的連接數。
• Target Health： 屬於已設置之上游目標的健康狀態 (healthchecks_off、 healthy、 unhealthy、dns_error)。

用法示例

指標輸出範例

呼叫 Admin API 加上 /metrics 可取得以下指標輸出範例：

curl -i <GOC API Gateway protocol>://<GOC API Gateway ip>:<GOC API Gateway admin api port>/metrics

HTTP/1.1 200 OK

Server: openresty/1.15.8.3

Date: Tue, 7 Jun 2020 16:35:40 GMT

Content-Type: text/plain; charset=UTF-8

Transfer-Encoding: chunked

Connection: keep-alive

Access-Control-Allow-Origin: *

# HELP kong_bandwidth_total Total bandwidth in bytes for all proxied requests in Kong

# TYPE kong_bandwidth_total counter

kong_bandwidth_total{type="egress"} 1277

kong_bandwidth_total{type="ingress"} 254

# HELP kong_bandwidth Total bandwidth in bytes consumed per service in Kong

# TYPE kong_bandwidth counter

kong_bandwidth{type="egress",service="google"} 1277

kong_bandwidth{type="ingress",service="google"} 254

# HELP kong_datastore_reachable Datastore reachable from Kong, 0 is unreachable

# TYPE kong_datastore_reachable gauge

kong_datastore_reachable 1

# HELP kong_http_status_total HTTP status codes aggreggated across all services in Kong

# TYPE kong_http_status_total counter

kong_http_status_total{code="301"} 2

# HELP kong_http_status HTTP status codes per service in Kong

# TYPE kong_http_status counter

kong_http_status{code="301",service="google"} 2

# HELP kong_latency Latency added by Kong, total request time and upstream latency for each service in Kong

# TYPE kong_latency histogram

kong_latency_bucket{type="kong",service="google",le="00001.0"} 1

kong_latency_bucket{type="kong",service="google",le="00002.0"} 1

.

.

.

kong_latency_bucket{type="kong",service="google",le="+Inf"} 2

kong_latency_bucket{type="request",service="google",le="00300.0"} 1

kong_latency_bucket{type="request",service="google",le="00400.0"} 1

.

.

kong_latency_bucket{type="request",service="google",le="+Inf"} 2

kong_latency_bucket{type="upstream",service="google",le="00300.0"} 2

kong_latency_bucket{type="upstream",service="google",le="00400.0"} 2

.

.

kong_latency_bucket{type="upstream",service="google",le="+Inf"} 2

kong_latency_count{type="kong",service="google"} 2

kong_latency_count{type="request",service="google"} 2

kong_latency_count{type="upstream",service="google"} 2

kong_latency_sum{type="kong",service="google"} 2145

kong_latency_sum{type="request",service="google"} 2672

kong_latency_sum{type="upstream",service="google"} 527

# HELP kong_latency_total Latency added by Kong, total request time and upstream latency aggreggated across all services in Kong

# TYPE kong_latency_total histogram

kong_latency_total_bucket{type="kong",le="00001.0"} 1

kong_latency_total_bucket{type="kong",le="00002.0"} 1

.

.

kong_latency_total_bucket{type="kong",le="+Inf"} 2

kong_latency_total_bucket{type="request",le="00300.0"} 1

kong_latency_total_bucket{type="request",le="00400.0"} 1

.

.

kong_latency_total_bucket{type="request",le="+Inf"} 2

kong_latency_total_bucket{type="upstream",le="00300.0"} 2

kong_latency_total_bucket{type="upstream",le="00400.0"} 2

.

.

.

kong_latency_total_bucket{type="upstream",le="+Inf"} 2

kong_latency_total_count{type="kong"} 2

kong_latency_total_count{type="request"} 2

kong_latency_total_count{type="upstream"} 2

kong_latency_total_sum{type="kong"} 2145

kong_latency_total_sum{type="request"} 2672

kong_latency_total_sum{type="upstream"} 527

# HELP kong_nginx_http_current_connections Number of HTTP connections

# TYPE kong_nginx_http_current_connections gauge

kong_nginx_http_current_connections{state="accepted"} 8

kong_nginx_http_current_connections{state="active"} 1

kong_nginx_http_current_connections{state="handled"} 8

kong_nginx_http_current_connections{state="reading"} 0

kong_nginx_http_current_connections{state="total"} 8

kong_nginx_http_current_connections{state="waiting"} 0

kong_nginx_http_current_connections{state="writing"} 1

# HELP kong_nginx_metric_errors_total Number of nginx-lua-prometheus errors

# TYPE kong_nginx_metric_errors_total counter

kong_nginx_metric_errors_total 0

# HELP kong_upstream_target_health Health status of targets of upstream. States = healthchecks_off|healthy|unhealthy|dns_error, value is 1 when state is populated.

kong_upstream_target_health{upstream="<upstream_name>",target="<target>",address="<ip>:<port>",state="healthchecks_off"} 0

kong_upstream_target_health{upstream="<upstream_name>",target="<target>",address="<ip>:<port>",state="healthy"} 1

kong_upstream_target_health{upstream="<upstream_name>",target="<target>",address="<ip>:<port>",state="unhealthy"} 0

kong_upstream_target_health{upstream="<upstream_name>",target="<target>",address="<ip>:<port>",state="dns_error"} 0

在全局啟用插件

1. 從網站左邊 Menu 中 外掛插件 頁面中，點選右上角的 新增外掛插件 ：

2. 點選後，選擇 分析及監控 頁籤，並啟用 Prometheus，填寫內容參考欄位配置說明，設定成功後，任何請求(不分服務、路由、用戶)都會在該節點上紀錄相關指標數據。

在服務端上啟用插件

1. 從網站左邊 Menu 中 服務 > 服務列表 頁面中，選擇要啟用此插件的服務，假設為 google，點選對應的編輯按鈕：

2. 在編輯畫面中，點選上方的 外掛插件 頁籤，再點選頁籤內容上方的 新增外掛插件 按鈕：

點選後，選擇 分析及監控 頁籤，並啟用 Prometheus，填寫內容參考欄位配置說明，設定成功後，僅有此服務(範例為google)請求會在該節點上紀錄相關指標數據。

在路由端上啟用插件

可以由兩種方式來選擇路由，並啟用插件：

方式一：路由列表

1. 從網站左邊 Menu 中 服務 > 路由列表 頁面中，選擇要啟用此插件的路由，假設為 google，點選對應的編輯按鈕：

方式二：服務 > 服務列表 > 路由列表

1. 從網站左邊 Menu 中 服務 > 服務列表 頁面中，選擇要啟用此插件的路由 所屬之服務(假設為 google)，點選對應的編輯按鈕。

在編輯畫面中，點選上方的 路由 頁籤，選擇要啟用此插件的路由(假設為 google)，點選對應的編輯按鈕：

---

2. 承第1步，點擊上述兩種方式之一的編輯按鈕後，在編輯畫面中，點選上方的 外掛插件 頁籤，再點選頁籤內容上方的 新增外掛插件 按鈕：

點選新增外掛插件 按鈕後，選擇 分析及監控 頁籤，並啟用 Prometheus，填寫內容參考欄位配置說明，設定成功後，僅有此路由(範例為google)請求會在該節點上紀錄相關指標數據。

在用戶端上啟用插件

1. 從網站左邊 Menu 中 訂閱用戶 > 用戶列表 頁面中，選擇要啟用此插件的用戶，假設為 portaladmin，點選對應的編輯按鈕：

2. 在編輯畫面中，點選上方的 外掛插件 頁籤，再點選頁籤內容上方的 新增外掛插件 按鈕：

點選後，選擇 分析及監控 頁籤，並啟用 Prometheus，填寫內容參考欄位配置說明，設定成功後，僅有此用戶(範例為 portaladmin)請求會在該節點上紀錄相關指標數據。

使用 Grafana 儀表板

Prometheus 所蒐集的指標數據可以透過 Grafana 來以圖形方式顯示，生成儀表板。

驗證

1. 在全域啟用 Prometheus 插件後，透過 GOC API Gateway 呼叫的任何服務，都會產生對應的指標數據。

2. 確認是否已經部署 Prometheus 服務，Prometheus 有對應的 scrape_configs 內的 job 設定，可以設定定時 N 秒獲取一次 /metrics 端點的指標數據，並指定獲取目標的端點。

需定義為 GOC API Gateway 的 ip 及其 admin api port，才能取得數據。

info

若不清楚如何取得 GOC API Gateway 所提供的 Admin API，可以參考 常見問題。

3. 可透過 Prometheus 內建之服務網頁(一般預設為 9090 port)來觀看是否有獲取到指標相關數據。 或是可透過呼叫 api，如指標輸出範例所示，確認指標數據是否有獲取成功，以下為實際範例：

curl <GOC API Gateway protocol>://<GOC API Gateway ip>:<GOC API Gateway admin api port>/metrics -i

HTTP/1.1 200 OK

Date: Tue, 13 Apr 2021 03:14:14 GMT

Content-Type: text/plain; charset=UTF-8

Transfer-Encoding: chunked

Connection: keep-alive

Access-Control-Allow-Origin: *

X-Kong-Admin-Latency: 10

# HELP kong_bandwidth Total bandwidth in bytes consumed per service in Kong

# TYPE kong_bandwidth counter

kong_bandwidth{type="egress",service="backend",route="backend"} 7552

kong_bandwidth{type="egress",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 13373

kong_bandwidth{type="egress",service="gocv3",route="gocv312"} 32080

kong_bandwidth{type="ingress",service="backend",route="backend"} 850

kong_bandwidth{type="ingress",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 2645

kong_bandwidth{type="ingress",service="gocv3",route="gocv312"} 847

# HELP kong_datastore_reachable Datastore reachable from Kong, 0 is unreachable

# TYPE kong_datastore_reachable gauge

kong_datastore_reachable 1

# HELP kong_http_status HTTP status codes per service in Kong

# TYPE kong_http_status counter

kong_http_status{code="200",service="backend",route="backend"} 10

kong_http_status{code="200",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 2

kong_http_status{code="200",service="gocv3",route="gocv312"} 7

kong_http_status{code="403",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 1

# HELP kong_latency Latency added by Kong, total request time and upstream latency for each service in Kong

# TYPE kong_latency histogram

kong_latency_bucket{type="kong",service="backend",route="backend",le="00002.0"} 5

kong_latency_bucket{type="kong",service="backend",route="backend",le="00005.0"} 7

kong_latency_bucket{type="kong",service="backend",route="backend",le="00007.0"} 7

kong_latency_bucket{type="kong",service="backend",route="backend",le="00010.0"} 7

kong_latency_bucket{type="kong",service="backend",route="backend",le="00015.0"} 7

kong_latency_bucket{type="kong",service="backend",route="backend",le="00020.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00025.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00030.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00040.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00050.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00060.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00070.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00080.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00090.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00100.0"} 8

kong_latency_bucket{type="kong",service="backend",route="backend",le="00200.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="00300.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="00400.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="00500.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="01000.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="02000.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="05000.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="10000.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="30000.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="60000.0"} 10

kong_latency_bucket{type="kong",service="backend",route="backend",le="+Inf"} 10

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00007.0"} 1

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00010.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00015.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00020.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00025.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00030.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00040.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00050.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00060.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00070.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00080.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00090.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00100.0"} 2

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00200.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00300.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00400.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00500.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="01000.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="02000.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="05000.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="10000.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="30000.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="60000.0"} 3

kong_latency_bucket{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="+Inf"} 3

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00001.0"} 2

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00002.0"} 3

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00005.0"} 4

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00007.0"} 5

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00010.0"} 6

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00015.0"} 6

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00020.0"} 6

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00025.0"} 6

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00030.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00040.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00050.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00060.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00070.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00080.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00090.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00100.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00200.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00300.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00400.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="00500.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="01000.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="02000.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="05000.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="10000.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="30000.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="60000.0"} 7

kong_latency_bucket{type="kong",service="gocv3",route="gocv312",le="+Inf"} 7

kong_latency_bucket{type="request",service="backend",route="backend",le="00005.0"} 2

kong_latency_bucket{type="request",service="backend",route="backend",le="00007.0"} 3

kong_latency_bucket{type="request",service="backend",route="backend",le="00010.0"} 4

kong_latency_bucket{type="request",service="backend",route="backend",le="00015.0"} 6

kong_latency_bucket{type="request",service="backend",route="backend",le="00020.0"} 6

kong_latency_bucket{type="request",service="backend",route="backend",le="00025.0"} 7

kong_latency_bucket{type="request",service="backend",route="backend",le="00030.0"} 7

kong_latency_bucket{type="request",service="backend",route="backend",le="00040.0"} 7

kong_latency_bucket{type="request",service="backend",route="backend",le="00050.0"} 7

kong_latency_bucket{type="request",service="backend",route="backend",le="00060.0"} 8

kong_latency_bucket{type="request",service="backend",route="backend",le="00070.0"} 8

kong_latency_bucket{type="request",service="backend",route="backend",le="00080.0"} 8

kong_latency_bucket{type="request",service="backend",route="backend",le="00090.0"} 8

kong_latency_bucket{type="request",service="backend",route="backend",le="00100.0"} 8

kong_latency_bucket{type="request",service="backend",route="backend",le="00200.0"} 9

kong_latency_bucket{type="request",service="backend",route="backend",le="00300.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="00400.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="00500.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="01000.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="02000.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="05000.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="10000.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="30000.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="60000.0"} 10

kong_latency_bucket{type="request",service="backend",route="backend",le="+Inf"} 10

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00007.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00010.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00015.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00020.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00025.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00030.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00040.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00050.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00060.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00070.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00080.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00090.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00100.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00200.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00300.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00400.0"} 1

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00500.0"} 2

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="01000.0"} 3

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="02000.0"} 3

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="05000.0"} 3

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="10000.0"} 3

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="30000.0"} 3

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="60000.0"} 3

kong_latency_bucket{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="+Inf"} 3

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00040.0"} 2

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00050.0"} 3

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00060.0"} 5

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00070.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00080.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00090.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00100.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00200.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00300.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00400.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="00500.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="01000.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="02000.0"} 6

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="05000.0"} 7

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="10000.0"} 7

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="30000.0"} 7

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="60000.0"} 7

kong_latency_bucket{type="request",service="gocv3",route="gocv312",le="+Inf"} 7

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00005.0"} 3

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00007.0"} 5

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00010.0"} 7

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00015.0"} 8

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00020.0"} 8

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00025.0"} 8

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00030.0"} 8

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00040.0"} 8

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00050.0"} 9

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00060.0"} 9

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00070.0"} 9

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00080.0"} 9

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00090.0"} 9

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00100.0"} 9

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00200.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00300.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00400.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="00500.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="01000.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="02000.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="05000.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="10000.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="30000.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="60000.0"} 10

kong_latency_bucket{type="upstream",service="backend",route="backend",le="+Inf"} 10

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="00500.0"} 2

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="01000.0"} 2

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="02000.0"} 2

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="05000.0"} 2

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="10000.0"} 2

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="30000.0"} 2

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="60000.0"} 2

kong_latency_bucket{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92",le="+Inf"} 2

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00040.0"} 3

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00050.0"} 4

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00060.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00070.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00080.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00090.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00100.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00200.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00300.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00400.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="00500.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="01000.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="02000.0"} 6

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="05000.0"} 7

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="10000.0"} 7

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="30000.0"} 7

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="60000.0"} 7

kong_latency_bucket{type="upstream",service="gocv3",route="gocv312",le="+Inf"} 7

kong_latency_count{type="kong",service="backend",route="backend"} 10

kong_latency_count{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 3

kong_latency_count{type="kong",service="gocv3",route="gocv312"} 7

kong_latency_count{type="request",service="backend",route="backend"} 10

kong_latency_count{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 3

kong_latency_count{type="request",service="gocv3",route="gocv312"} 7

kong_latency_count{type="upstream",service="backend",route="backend"} 10

kong_latency_count{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 2

kong_latency_count{type="upstream",service="gocv3",route="gocv312"} 7

kong_latency_sum{type="kong",service="backend",route="backend"} 289

kong_latency_sum{type="kong",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 200

kong_latency_sum{type="kong",service="gocv3",route="gocv312"} 51

kong_latency_sum{type="request",service="backend",route="backend"} 527

kong_latency_sum{type="request",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 1138

kong_latency_sum{type="request",service="gocv3",route="gocv312"} 3296

kong_latency_sum{type="upstream",service="backend",route="backend"} 235

kong_latency_sum{type="upstream",service="example-service",route="a46087ef-664e-46e9-96ff-77432f31db92"} 937

kong_latency_sum{type="upstream",service="gocv3",route="gocv312"} 3244

# HELP kong_nginx_http_current_connections Number of HTTP connections

# TYPE kong_nginx_http_current_connections gauge

kong_nginx_http_current_connections{state="accepted"} 51

kong_nginx_http_current_connections{state="active"} 1

kong_nginx_http_current_connections{state="handled"} 51

kong_nginx_http_current_connections{state="reading"} 0

kong_nginx_http_current_connections{state="total"} 421

kong_nginx_http_current_connections{state="waiting"} 0

kong_nginx_http_current_connections{state="writing"} 1

# HELP kong_nginx_metric_errors_total Number of nginx-lua-prometheus errors

# TYPE kong_nginx_metric_errors_total counter

kong_nginx_metric_errors_total 0