Issue:
BOBJ services failed to start after a routine system reboot, despite no configuration changes.
Initial Findings:
Process Monitor and BiWhy identified SPLUNK (an auxiliary OS monitoring tool) as a lead CPU consumer. Disabling SPLUNK, however, had no impact.
Root Cause & Resolution:
A quick review of hardware metrics in BiWhy and comparison witha healthy node, revealed unusually high disk queue lengths and utilization on the affected node.
Disk performance tests confirmed the disk was operating up to 40x slower than normal.
The issue was brought to the Cloud/VM team, who replaced the disk. Services resumed normal operation.
Apparently some hardware change event was triggered by cloud infrastructure software, with no notification to the tenant.
Takeaway:
It’s unclear why enterprise-grade tools like SPLUNK failed to detect the root cause.
Whether due to software limitations, complex configuration, or unintuitive interfaces that hinder effective use by administrators, but this is not the first time a simple solution like BiWhy has identified an issue where enterprise tools have not.
A simple, intuitive, and informative UI is often what enterprise software lacks.
Screenshots:
Issue:BOBJ services failed to start after a routine system reboot, despite no configuration changes. Initial Findings:Process Monitor and BiWhy identified SPLUNK (an auxiliary OS monitoring tool) as a lead CPU consumer. Disabling SPLUNK, however, had no impact. Root Cause & Resolution:A quick review of hardware metrics in BiWhy and comparison witha healthy node, revealed unusually high disk queue lengths and utilization on the affected node. Disk performance tests confirmed the disk was operating up to 40x slower than normal.The issue was brought to the Cloud/VM team, who replaced the disk. Services resumed normal operation.Apparently some hardware change event was triggered by cloud infrastructure software, with no notification to the tenant. Takeaway:It’s unclear why enterprise-grade tools like SPLUNK failed to detect the root cause.Whether due to software limitations, complex configuration, or unintuitive interfaces that hinder effective use by administrators, but this is not the first time a simple solution like BiWhy has identified an issue where enterprise tools have not.A simple, intuitive, and informative UI is often what enterprise software lacks. Screenshots: Read More Technology Blog Posts by SAP articles
#SAP
#SAPTechnologyblog
