etcd Monitoring

Monitor etcd servers using Site24x7 and gain in-depth visibility into critical performance metrics.

Start 30-day free trial Try now, sign up in 30 seconds

Etcd is an open source distributed key value store that provides a reliable way to store data across a cluster of machines. Analyze and optimize your etcd server by configuring our plugin. Use our key indicators to ensure continuous functioning of your etcd server and make informed troubleshooting decisions.

This document details how to configure the etcd plugin and the monitoring metrics for providing in-depth visibility into the performance, availability, and usage stats of etcd servers.

Etcd performance monitoring metrics:

Key metrics we support are:

Successful/failed gets

The metrics "gets_success", "gets_fail" gives the rate of successful or failed get requests that the etcd server recieved.

Successful/failed sets

The metrics "sets_success", "sets_fail" gives the rate of successful or failed set requests that the etcd server recieved.

Successful/failed deletes

The metric "delete_success", "delete_fail" gives the rate of successful or failed delete requests that the etcd server recieved.

Successful/failed updates

The metrics "update_success", "update_fail" gives the rate of successful or failed update requests that the etcd server recieved.

Successful/failed creates

The metrics "create_success", "create_fail" gives the rate of successful or failed create requests that the etcd server recieved.

Successful/Failed compares and swaps

The metric "compare_and_swap_success", "compare_and_swap_fail" gives the rate of successful or failed compare and swap requests that the etcd server recieved.

Successful/failed compares and deletes

The metrics "compare_and_delete_success", "compare_and_delete_fail" gives the rate of successful or failed compare and delete requests that the etcd server recieved.

Expire count

Use the metric "expire_count" and get the total count of expired keys in your etcd setup.

Watchers

The metric "watchers" gives the total count of watchers in your etcd setup.

Append requests recieved

Use the metric "self_recv_appendreq_cnt" and get the total count of append requests that this node has processed.

Requests per second recieved

Use the metric "self_recv_pkg_rate" and get the total number of requests per second this node is recieving. This metric works only if the node is a follower. For a leader node the value is always 0.

Bytes per second recieved

Use the metric "self_recv_bandwidth_rate" and get the total number of bytes per second this node is recieving. This metric works only if the node is a follower. For a leader node the value is always 0.

Requests sent

Use the metric "self_appendreq_cnt" and get the total count of append requests that this node has sent.

Requests per second sent

Use the metric "self_send_pkg_rate" and get the total number of requests per second this node is sending. This metric works only if the node is a leader. For a follower node the value is always 0.

Bytes per second sent

Use the metric "sendbandwidthrate" and get the total number of bytes per second this node is sending. This metric works only if the node is a leader. For a follower node the value is always 0.

How it works?

  • Log-in to your Site24x7 account. Sign up here if you don't have one
  • Download and install the latest version of Site24x7 Linux agent or Site24x7 Windows agent based on what operating system you are currently using
  • Install the etcd plugin
  • The agent will execute the etcd plugin and push the data to the Site24x7 server.

Prerequisites:

  • The plugin requires Python module "psycopg2" to fetch the statistics from the etcd database if your operating system is Linux.

Etcd plugin Linux installation:

  • Go to the Site24x7 Linux agent's plugin directory - /opt/site24x7/monagent/plugins/ and create a directory "etcd_store"
    cd /opt/site24x7/monagent/plugins/
    sudo mkdir etcd_store
  • Download the file ""etcd_store.py" from our GitHub repository and place it under the "etcd_store" directory
    cd etcd_store
    sudo wget https://raw.githubusercontent.com/site24x7/plugins/master/etcd_store/etcd_store.py
  • Similarly, in the Site24x7 Linux agent's plugin directory - /opt/site24x7/monagent/plugins/, create a directory "etcd_self"
    cd /opt/site24x7/monagent/plugins/
    sudo mkdir etcd_self
  • Download the file ""etcd_self.py" from our GitHub repository and place it under the "etcd_self" directory
    cd etcd_self
    sudo wget https://raw.githubusercontent.com/site24x7/plugins/master/etcd_self/etcd_self.py

Etcd plugin Linux installation:

  • Go to the Site24x7 Linux agent's plugin directory - /opt/site24x7/monagent/plugins/ and create a directory "etcd_store"
    cd /opt/site24x7/monagent/plugins/
    sudo mkdir etcd_store
  • Download the file ""etcd_store.py" from our GitHub repository and place it under the "etcd_store" directory
    cd etcd_store
    sudo wget https://raw.githubusercontent.com/site24x7/plugins/master/etcd_store/etcd_store.py
  • Similarly, in the Site24x7 Linux agent's plugin directory - /opt/site24x7/monagent/plugins/, create a directory "etcd_self"
    cd /opt/site24x7/monagent/plugins/
    sudo mkdir etcd_self
  • Download the file ""etcd_self.py" from our GitHub repository and place it under the "etcd_self" directory
    cd etcd_self
    sudo wget https://raw.githubusercontent.com/site24x7/plugins/master/etcd_self/etcd_self.py

Etcd plugin Windows installation:

  • Create a directory with the name "etcd_store" and place them under the Site24x7 Windows agent's plugin directory - C:\Program Files\Site24x7\WinAgent\monitoring\Plugins\
    cd C:\Program Files\Site24x7\WinAgent\monitoring\Plugins\
    mkdir etcd_store
  • Download the file ""etcd_store.ps1" from our GitHub repository and place it under the "etcd_store" directory
    cd etcd_store
    wget https://raw.githubusercontent.com/site24x7/plugins/master/etcd_store/etcd_store.ps1
  • Similarly, go to the Site24x7 Windows agent's plugin directory - C:\Program Files\Site24x7\WinAgent\monitoring\Plugins\ and create a directory "etcd_self"
    cd C:\Program Files\Site24x7\WinAgent\monitoring\Plugins\
    mkdir etcd_self
  • Download the file ""etcd_self.ps1" from our GitHub repository and place it under the "etcd_self" directory
    cd etcd_self
    wget https://raw.githubusercontent.com/site24x7/plugins/master/etcd_self/etcd_self.ps1

Etcd plugin configuration:

    The below example is for a Linux installation

  • The default python path given in the plugin script is #!/usr/bin/python. If you wish to provide an alternate python path, replace the existing one preceded by the shebang character "#!".
  • Configure host and port values for the etcd plugin
    Eg :
    HOST = "localhost"
    ADMINPORT = "4848"
  • Save the changes and restart the agent.
    /etc/init.d/site24x7monagent restart

Monitor additional metrics:

  • To monitor additional metrics, edit the "etcd_self.py" and "etcd_store.py" file for Linux installation and add the new metrics that need monitoring. For Windows installation, edit "etcd_store.ps1" and "etcd_self.ps1".
  • Increment the plugin version value in the file where changes have been made, to view the newly added metrics (For e.g., change the default plugin version from PLUGIN_VERSION = "1" to "PLUGIN_VERSION = "2")

Related plugins:

  • Redis plugin - Monitor performance metrics of your Redis databases
  • MongoDB plugin - Monitor performance metrics of your MongoDB databases
  • Memcached plugin - Analyze performance of your Memcached server
  • Nagios plugin - Execute thousands of Nagios plugins in Site24x7 without the need of running a Nagios server
  • Out-of-the-box plugins - Monitor your entire app stack with our extensive list of integrations
  • Create custom plugins - Create custom Linux and Windows plugins and monitor custom attributes

Troubleshooting tips: