Skip to main content
Star us on GitHub Star

File Reporter

The Ziti Controller event system can write metrics to a file for direct viewing or post-processing into a metrics system of your choice.

Ziti Configuration

There are two independent configurations that need to be set up for metric event reporting to work:

  1. Report Interval: The controller and routers need to have a metric reporting interval set. The interval determines how often metrics will be sent to the controller and possibly written out to file
  2. Event Subscription: The subscription is configured in the controller, and determines which of the reported metrics are written.

Metrics Reporting

The reporting interval is defined in the metrics.reportInterval property. While the controller and each router can have a different reporting interval, we suggest that you set them all the same to make lining up metrics across the cluster easier.

Example:

metrics:
reportInterval: 20s

Routers support an additional configuration parameter MessageQueueSize which is the number of unsent metrics messages that can sit in the router metrics queue before metrics gathering is paused.

Example:

metrics:
reportInterval: 20s
messageQueueSize: 20 // Default 10

Metrics Writing

Writing of metrics is broken into two pieces:

  1. subscription: Which metrics will be written
  2. hander: how the metrics will be written

Metrics Subscription

The metrics subscription defines which metrics will be written and how they will be written.

There are two parts to a metrics event reporter The subscription has three components

  1. sourceFilter: Which components to write metrics for. This is a regular expression.
    • ctrl_client: Special marker for the controller
    • Router ID: Get metrics for one and only one router
    • .*: Get metrics from the controller and all routers
  2. metricFilter: Which metrics to report. This is a regular expression
    • .*pool.*: Report only pool metrics
    • .*: Report all metrics
Example:
events:
allControllerMetrics:
subscriptions:
- type: metrics
sourceFilter: ctrl_client
metricFilter: .*
justEdgeRouterPoolMetrics:
subscriptions:
- type: metrics
sourceFilter: .*
metricFilter: .*pool.*

Metrics Handling

The metrics handler defined how metrics are to be written. It is comprised of:

  1. type: The type of handler. Supported types are:
  • file: Metrics will be written to a file
  1. maxsizemb: File rolling - log files will be rolled when they reach this size. Default size is 10mb.
  2. maxbackups File rolling - the number of files to keep. Default is 0 (unlimited).
  3. format: The format of the metric. Supported formats are: json
  4. path: The name of the file to write metrics to
File Rolling

files are rolled when they reach a size of maxsizemb. Files are renamed from name.log to name-iso8601.log

For example, name-2022-06-07T18-50-44.568.log

Example

Write 100mb files, saving 2 of them.

    handler:
type: file
format: json
maxsizemb: 100
maxbackups: 2
path: /tmp/controller-metrics.log

Putting it all together

Controller configuration file:

metrics:
reportInterval: 20s

events:
allControllerMetrics:
subscriptions:
- type: metrics
sourceFilter: ctrl_client
metricFilter: .*
handler:
type: file
format: json
maxsizemb: 50
maxbackups: 2
path: /tmp/controller-metrics.log

justPoolMetrics:
subscriptions:
- type: metrics
sourceFilter: .*
metricFilter: .*pool.*
handler:
type: file
format: json
maxsizemb: 100
maxbackups: 5
path: /tmp/router-pool-metrics.log

Router configuration files:

metrics:
reportInterval: 20s

Metric Examples

intValue

{
"metric_type": "intValue",
"namespace": "metrics",
"source_id": "ctrl_client",
"version": 2,
"timestamp": {
"seconds": 1654625684,
"nanos": 479708609
},
"metric": "pool.listener.mgmt.worker_count",
"metrics": {
"value": 1
},
"source_event_id": "acb85925-0e17-4ca0-90cb-9a2498b33bc8"
}

Histogram

{
"metric_type": "histogram",
"namespace": "metrics",
"source_id": "ctrl_client",
"source_entity_id": "xpw7BEDAk",
"version": 2,
"timestamp": {
"seconds": 1654625684,
"nanos": 479708609
},
"metric": "ctrl.queue_time",
"metrics": {
"count": 57,
"max": 21647,
"mean": 15266.508771929824,
"min": 5753,
"p50": 15670,
"p75": 16558.5,
"p95": 18362.699999999997,
"p99": 21647,
"p999": 21647,
"p9999": 21647,
"std_dev": 2604.8795245927113,
"variance": 6785397.337642349
},
"source_event_id": "acb85925-0e17-4ca0-90cb-9a2498b33bc8"
}

Timer

{
"metric_type": "timer",
"namespace": "metrics",
"source_id": "ctrl_client",
"version": 2,
"timestamp": {
"seconds": 1654625684,
"nanos": 479708609
},
"metric": "api.session.enforcer.run",
"metrics": {
"count": 11,
"m15_rate": 0.2,
"m1_rate": 0.2,
"m5_rate": 0.2,
"max": 6842849,
"mean": 1096126.3636363635,
"mean_rate": 0.20374652060865114,
"min": 254514,
"p50": 335348,
"p75": 1212318,
"p95": 6842849,
"p99": 6842849,
"p999": 6842849,
"p9999": 6842849,
"std_dev": 1858257.4031879376,
"variance": 3453120576502.777
},
"source_event_id": "acb85925-0e17-4ca0-90cb-9a2498b33bc8"
}