In this Technology Spa blog, we explore the next evolution of event management and how the discipline is evolving towards a concept we term “event intelligence”.
| ITIL defines Event Management as the process that monitors all events that occur through the IT infrastructure. It allows for normal operation and detects and escalates exception conditions. |
An IDC study of the IT operations management (ITOM) market estimated $9.5 billion in 2018 revenue while the Globe Newswire estimated the Security Information and Event Management market at $2.6 billion in 2018 revenue. “The market is being driven by the need to monitor, manage, and optimize systems, infrastructure, applications, and end-user experience across increasingly complex on-premise, hybrid cloud, public cloud, multicloud, and containerized deployments” – IDC.
The Increasing Scope of Event Management
Beyond the ITIL focus on service availability, IT event management is also vital for governance, risk and compliance (GRC), security incident and event management (SIEM) and many other service management-related areas such as incident and problem management, root cause analysis, effectiveness of controls, data integrity and continuous improvement.
In the SIEM arena alone, IT professionals can be looking at a wide array of events related to data loss prevention, data classification, data exfiltration, bad actors, malware, phishing, spoofing, unauthorized access, compliance, vulnerabilities and threat hunting and much more.
These events can come from many different types of devices such as operating systems, network devices, applications, firewalls, endpoints, security agents, monitoring systems, 3rd party APIs, email, facility equipment such as HVACs and cameras, IOT devices, point of sale systems and many others.
The ways to consume this information is highly varied as well. Raw event information is often made available to IT decision makers via log files, monitoring alerts, SNMP, WMI, APIs, RSS feeds, event hubs, dashboards, discovery tools, SIEM tools, APM tools, ITSM tools, email and even user-reported events.
Today’s Challenges with Event Management
With many organizations adopting a “cloud-first” strategy and finding themselves with services distributed across multiple cloud service providers (see “3 Recommendations for your Multi-Vendor Cloud Strategy”), the ability to manage events across all these providers creates new challenges.
In addition to typical integration and complexity challenges, other challenges often include staff limitations (such as lack of time, prioritization or attention as well as potential human error), data issues (such as data that’s difficult to read, interpret and correlate) and storage limitations (such as volume and flow rates of raw event data, associated costs and data retention requirements).
Today’s technology solutions address these issues to some extent, but often experience:
- High initial expense, with ongoing costs continuing to rise
- Competition with cloud native tools that provide similar capabilities in a limited scope
- Solutions deployed in silos (APMs, SIEMs, syslog servers)
- Risks related to sensitive log data (localization, encryption)
- Limited data transformation and customizations available
- Limited integration into ITSM tools
Overall, these numerous challenges can be grouped into data overload, siloed tools and data, and the cost of proprietary systems as follows:
- Data Overload – The amount of data that IT must deal with in order to deliver IT service functions has reached the state of constant overwhelm. Simple aggregation only exacerbates the problem so that “data plus data equals more data”.
- Siloed Tools and Data – Most of the tools that exist in the market today are siloed by function (SIEM, ITSM, etc.) meaning no-one has a holistic view of events across the enterprise. In addition, working with multiple cloud providers means more consoles to monitor which in turn means more data and high data egress costs to integrate this data for event correlation.
- Cost of Proprietary Systems – The cost of commercial solutions that scale with your data, for example, also scale in their license expense. This can lead to either trimming data at an arbitrary number of hours, days, weeks, or months to meet budget or deciding that some data isn’t going into the solution at all.
Event Intelligence
The ultimate objective of event management is to help people make smarter decisions about events (such as informational, warning or exception events) or in some cases to intelligently automate these decisions.
To drive action and decision, data first needs to be converted to information and then to intelligence. Intelligence is what allows organizations to move from individual events which must be triaged to orchestrated and holistic responses.
State-of-the-art solutions, therefore, need to be smart about how they address these challenges:
- From Data Overload to Event Intelligence – They need to tackle data overload in an intelligent way that doesn’t necessitate analyzing every single raw event from every single console.
- From Silos to Intelligent Tool and Data Integration – They need to find workarounds for data egress costs, for example, by ingesting the signals and outputs from cloud providers consoles without the need to replicate the entire data set.
- From Proprietary to Open Source for Incomparable ROI – They need to be based on open source technologies so that they deliver incomparable ROI
Introducing hotrock from Technology Spa
Fortunately, there is a solution to assist organizations in solving these growing challenges without breaking the bank. The Open-source solution hotrock (sponsored and created by Technology Spa), provides the following features to turn event data into event intelligence:
- No licensing costs, only pay for compute resources
- On-premise, cloud, or SaaS deployment options
- Hybrid and multi-cloud aggregation
- Highly scalable, multi-region capabilities with end-to-end encryption
- Ready to integrate into most common infrastructure solutions
- Single point of aggregation of events across hybrid and multi-cloud deployments
- Dashboards and out of the box analysis for common IT and security related events
- Integration into ITSM and other event management solutions
- Used to augment and/or replace existing solutions
With digital transformation and the digital customer experience a high priority on most corporate agendas, next generation event management solutions such as hotrock can help you apply event intelligence to navigate data overload, cut through the silos and reduce costs.
For more information about hotrock, please contact us today at support@hotrock.io.
