Tag: anomaly

Operations : Changes Detection and then

Posted on by lechuck park

Process Analysis from “Change Drives Operations” Perspective

Core Philosophy

“No Change, No Operation” – This diagram illustrates the fundamental IT operations principle that operations are driven by change detection.

Change-Centric Operations Framework

1. Change Detection as the Starting Point of All Operations

  • Top-tier monitoring systems continuously detect changes
  • No Changes = No Operations (left gray boxes)
  • Change Detected = Operations Initiated (blue boxes)

2. Operational Strategy Based on Change Characteristics

Change Detection → Operational Need Assessment → Appropriate Response
  • Normal Changes → Standard operational activities
  • Anomalies → Immediate response operations
  • Real-time Events → Emergency operational procedures

3. Cyclical Structure Based on Operational Outcomes

  • Maintenance: Stable operations maintained through proper change management
  • Fault/Big Cost: Increased costs due to inadequate response to changes

Key Insights

“Change Determines Operations”

  1. System without change = No intervention required
  2. System with change = Operational activity mandatory
  3. Early change detection = Efficient operations
  4. Proper change classification = Optimized resource allocation

Operational Paradigm

This diagram demonstrates the evolution from Reactive Operations to Proactive Operations, where:

  • Traditional Approach: Wait for problems → React
  • Modern Approach: Detect changes → Predict → Respond proactively

The framework recognizes change as the trigger for all operational activities, embodying the contemporary IT operations paradigm where:

  • Operations are event-driven rather than schedule-driven
  • Intelligence (AI/Analytics) transforms raw change data into actionable insights
  • Automation ensures appropriate responses to different types of changes

This represents a shift toward Change-Driven Operations Management, where the operational workload directly correlates with the rate and nature of system changes, enabling more efficient resource utilization and better service reliability.

With Claude

Monitoring is from changes

Posted on by lechuck park

Change-Based Monitoring System Analysis

This diagram illustrates a systematic framework for “Monitoring is from changes.” The approach demonstrates a hierarchical structure that begins with simple, certain methods and progresses toward increasingly complex analytical techniques.

Flow of Major Analysis Stages:

  1. One Change Detection:
    • The most fundamental level, identifying simple fluctuations such as numerical changes (5→7).
    • This stage focuses on capturing immediate and clear variations.
  2. Trend Analysis:
    • Recognizes data patterns over time.
    • Moves beyond single changes to understand the directionality and flow of data.
  3. Statistical Analysis:
    • Employs deeper mathematical approaches to interpret data.
    • Utilizes means, variances, correlations, and other statistical measures to derive meaning.
  4. Deep Learning:
    • The most sophisticated analysis stage, using advanced algorithms to discover hidden patterns.
    • Capable of learning complex relationships from large volumes of data.

Evolution Flow of Detection Processes:

  1. Change Detection:
    • The initial stage of detecting basic changes occurring in the system.
    • Identifies numerical variations that deviate from baseline values (e.g., 5→7).
    • Change detection serves as the starting point for the monitoring process and forms the foundation for more complex analyses.
  2. Anomaly Detection:
    • A more advanced form than change detection, identifying abnormal data points that deviate from general patterns or expected ranges.
    • Illustrated in the diagram with a warning icon, representing early signs of potential issues.
    • Utilizes statistical analysis and trend data to detect phenomena outside the normal range.
  3. Abnormal (Error) Detection:
    • The most severe level of detection, identifying actual errors or failures within the system.
    • Shown in the diagram with an X mark, signifying critical issues requiring immediate action.
    • May be classified as a failure when anomaly detection persists or exceeds thresholds.

Supporting Functions:

  • Adding New Relative Data: Continuously collecting relevant data to improve analytical accuracy.
  • Higher Resolution: Utilizing more granular data to enhance analytical precision.

This framework demonstrates a logical progression from simple and certain to gradually more complex analyses. The hierarchical structure of the detection process—from change detection through anomaly detection to error detection—shows how monitoring systems identify and respond to increasingly serious issues.

With Claude

Operation

Posted on by lechuck park

With a Claude’s Help

  1. Normal State:
  • Represented by a gear icon with a green checkmark
  • Indicates system operating under normal conditions
  • Initial state of the monitoring process
  1. Anomaly Detection:
  • Shown with a magnifying glass and graph patterns
  • The graph patterns are more clearly visualized than before
  • Represents the phase where deviations from normal patterns are detected
  1. Abnormal State:
  • Depicted by a human figure with warning indicators
  • Represents confirmed abnormal conditions requiring intervention
  • Links directly to action steps
  1. Analysis and Response Process:
  • Comparison with normal: Shown through A/B document comparison icons
  • Analysis: Data examination phase
  • predictive Action: Now written in lowercase, indicating predicted response measures
  • Recovery Action: Implementation of actual recovery measures
  1. Learning Feedback:
  • Shows how lessons from recovery actions are fed back into the system
  • Creates a continuous improvement loop
  • Connects recovery actions back to normal operations

The workflow continues to effectively illustrate the complete operational cycle, from monitoring and detection through analysis, response, and continuous learning. It demonstrates a systematic approach to handling operational anomalies and maintaining system stability.

Prediction & Detection

Posted on by lechuck park

From Claude with some prompting
This image illustrates a Prediction and Detection system for time series data. Let me break down the key components:

  1. Left Large Box (Learning and Prediction Section):
  • Blue line: Actual Temperature data
  • Red dotted line: Predicted Temperature data
  • Uses time series prediction models like LSTM, ARIMA, and Prophet for learning
  1. Top Right (Threshold-based Anomaly Detection):
  • “Abnormal Detection with Threshold”
  • Detects abnormal temperature changes based on threshold values
  • The area marked with a red circle shows where values exceed the threshold
  • Includes “Warning” and “Critical” threshold levels
  1. Bottom Right (Pattern-based Anomaly Detection):
  • “Anomaly Detection with Predict-Pattern”
  • Compares predicted patterns with actual data to detect anomalies
  • The area marked with a green circle shows where actual data deviates from the predicted pattern

The system detects anomalies in two ways:

  1. When values exceed predetermined thresholds
  2. When actual data significantly deviates from predicted patterns

This type of system is particularly useful in:

  • Industrial monitoring
  • Equipment maintenance
  • Early warning systems
  • Quality control
  • System health monitoring

The combination of prediction and dual detection methods (threshold and pattern-based) provides a robust approach to identifying potential issues before they become critical problems.

Anomaly IP Packet header

Posted on by lechuck park

From Gemini with some prompting
Title: Anomaly Detection (IP Packet Header)

Overview:

The image illustrates the structure of an IP packet header and how to detect anomalous activities based on TCP/IP header information.

Key Elements:

  • IP Address: Identifies the server.
  • TCP/UDP Port: Used to send and receive packets.
  • Service Port Number: Port number used to connect to a specific service.
  • Service Area: IP address range where a specific service is provided.

Indicators of Anomalous Activity:

  • Unknown IP: Packets coming from unknown IP addresses.
  • Foreign IP: Packets coming from overseas IP addresses.
  • Unused Port: Packets sent to unused ports.
  • TCP/UDP Port == 0: Packets with TCP/UDP port number 0.
  • IP/TCP Checksum == 0: Packets with IP/TCP checksum 0.
  • Unused IP Protocol: Packets using unused protocols.
  • Too Large (IP.ttl): Packets with excessively large TTL values.
  • Too Many (TCP Syn): Excessive number of SYN packets.
  • Too Many (IP. fragmented): Excessive number of fragmented packets.

Anomaly Connection Detection #0

Posted on by lechuck park

from DALL-E with some prompting
The image seems to illustrate the concept of anomaly detection in network security. A user is shown with a green line leading to a server, indicating normal interaction, while a red line leading from a network criminal suggests malicious activity. The network architecture is in place to mirror and tap into the data traffic, allowing for the steering of packets for closer inspection. An alert (!!) signifies the detection of an anomaly. Below, details of what is monitored are given: raw or sampled packets, TCP/IP 5-tuples, geographic IP locations, bandwidth, and new detection areas including DNS and HTTP header information. This represents a multifaceted approach to identifying and responding to potential security threats within a network.

Unchanging data

Posted on by lechuck park

from DALL-E with some prompting
The image illustrates a process of monitoring typically unchanging data to detect system malfunctions. The ‘Traffic / User(s)’ data reflects the relative amount of traffic between two connected users, which generally remains constant. The heat generated by the CPU, as well as electrical elements like voltage and current, are also considered unchanging data in a stable state. A fault detection sensor sends an alert when anomalies are detected in these data points. The ‘Detect it!!’ sensor shows no changes under normal conditions but identifies deviations when an event occurs, enabling a response to potential issues.