How do we design system monitoring and alerting

Monitoring and alerting play a vital role in effective database management. However, the prevailing “monitor and alert on everything” approach is not suitable when considering environmental sustainability. At XOOTS, we believe in comprehensive and efficient monitoring practices that align with our environmental principles. In this article, we explore key considerations for designing effective monitoring and alerting systems.

Key Monitoring Considerations:

• Focus on Necessity: Monitoring a database consumes CPU, which correlates with carbon emissions. Therefore, it is crucial to monitor only essential metrics and avoid over-monitoring. Each company and database may require different metrics, and it’s important to regularly review, update, or remove monitoring as needs evolve

• Carbon-Efficient Monitoring Queries: Database monitoring queries should be designed to consume minimal CPU resources. This ensures that monitoring activities do not impact application performance

• Avoid Metric Duplication: Prevent redundant monitoring by providing clear ownership of metrics or consolidating monitoring platforms. Multiple teams monitoring the same metrics leads to wasteful practices, potential performance issues, and unnecessary resource consumption

Key Alerting Considerations:

• Selective Alerting: Only set up alerts for critical events that require immediate action. Avoid the “alert on everything” approach, which inundates operational teams with non-actionable alerts. Focusing on relevant alerts ensures important issues are not lost amid a sea of notifications

• Consider Resource Consumption: Sending alerts across the network consumes CPU resources and contributes to carbon emissions. Minimize the number of alerts to reduce environmental impact. While individual alerts may not have a significant impact, scaling this practice across an organization can result in substantial resource usage

• Avoid Alert Duplication: Similar to monitoring, duplicate alerts should be avoided to streamline processes, improve efficiency, and reduce unnecessary resource consumption

• Automate Fixes: Whenever possible, automate fixes for common issues. Implement intelligent systems that can adapt to a changing environment, such as automatic space increases or auto-scaling during peak processing. Automation reduces the need for manual interventions, enhancing both efficiency and sustainability

Performance Monitoring:

Traditional approaches to performance monitoring often focus on individual metrics, such as CPU usage, query times, and IO wait times. However, a more holistic approach is recommended. By gathering key wait and statistic metrics and plotting them over time, organizations can gain deeper insights into performance. This approach enables comparisons between different time periods, facilitating effective troubleshooting and problem resolution. It also allows for the identification of high-consumption CPU statements, inefficient execution plans, and redundant queries, leading to improved overall CPU utilization and reduced environmental impact

Data Retention:

Storing data for the appropriate duration is essential. Businesses operate in cycles, such as peak trading periods or seasonal fluctuations. It is vital to retain relevant metrics across a complete business cycle, typically a year. Keeping at least 400 days of data allows for comparisons and analysis, providing valuable insights during critical trading days or problem investigations.

At XOOTS, we are committed to designing monitoring and alerting systems that prioritize environmental sustainability, resource efficiency, and actionable insights. By emphasizing necessity, carbon-efficient monitoring, selective alerting, and holistic performance monitoring, organizations can optimize resource utilization, minimize environmental impact, and enhance problem resolution. Furthermore, capturing and retaining data for the appropriate duration empowers businesses with valuable insights into long-term trends and strengthens their ability to overcome challenges effectively. With a focus on sustainability and efficiency, XOOTS is dedicated to helping companies achieve their goals while minimizing their environmental footprint.