Tag: machine learning

Operation with AI

Posted on by lechuck park

From Claude with some prompting
This diagram illustrates an integrated approach to modern operational management. The system is divided into three main components: data generation, data processing, and AI application.

The Operation & Biz section shows two primary data sources. First, there’s metric data automatically generated by machines such as servers and network equipment. Second, there’s textual data created by human operators and customer service representatives, primarily through web portals.

These collected data streams then move to the central Data Processing stage. Here, metric data is processed through CPUs and converted into time series data, while textual data is structured via web business services.

Finally, in the AI play stage, different AI models are applied based on data types. For time series data, models like RNN, LSTM, and Auto Encoder are used for predictive analytics. Textual data is processed through a Large Language Model (LLM) to extract insights.

This integrated system effectively utilizes data from various sources to improve operational efficiency, support data-driven decision-making, and enable advanced analysis and prediction through AI. Ultimately, it facilitates easy and effective management even in complex operational environments.

The image emphasizes how different types of data – machine-generated metrics and human-generated text – are processed and analyzed using appropriate AI techniques, all from the perspective of operational management.

Easy Prediction

Posted on by lechuck park

From Claude with some prompting
This image illustrates three main approaches to prediction and pattern recognition.

First, for easy prediction, a linear regression model (Linear Regression, y=ax+b) can be used. This is represented by a simple upward trendline. While a basic concept, it is emphasized that this can cover 90% of cases.

Second, for learning complex patterns that recur over time, an AI model is required. This is depicted by the jagged line shape.

Third, for real-time anomaly detection, sudden spike patterns need to be identified.

Additionally, at the bottom of the image, a new phrase has been added: “More & More & More learning makes More & More & More better AI model.” This conveys the idea that as an AI model learns from more and more data, its performance continues to improve.

In summary, the image highlights a step-by-step approach: starting with simple concepts to build a foundation, then utilizing AI models to learn complex patterns, and continuously improving the models through ongoing data learning and training. The key emphasis is on starting with the basics, while recognizing the potential of advanced AI techniques when combined with extensive learning from data.

Trend & Prediction

Posted on by lechuck park

From Claude with some prompting
The image presents a “Trend & Predictions” process, illustrating a data-driven prediction system. The key aspect is the transition from manual validation to automation.

  1. Data Collection & Storage: Digital data is gathered from various sources and stored in a database.
  2. Manual Selection & Validation: a. User manually selects which metric (data) to use b. User manually chooses which AI model to apply c. Analysis & Confirmation using selected data and model
  3. Transition to Automation:
    • Once optimal metrics and models are confirmed in the manual validation phase, the system learns and switches to automation mode. a. Automatically collects and processes data based on selected metrics b. Automatically applies validated models c. Applies pre-set thresholds to prediction results d. Automatically detects and alerts on significant predictive patterns or anomalies based on thresholds

The core of this process is combining user expertise with system efficiency. Initially, users directly select metrics and models, validating results to “educate” the system. This phase determines which data is meaningful and which models are accurate.

Once this “learning” stage is complete, the system transitions to automation mode. It now automatically collects, processes data, and generates predictions using user-validated metrics and models. Furthermore, it applies preset thresholds to automatically detect significant trend changes or anomalies.

This enables the system to continuously monitor trends, providing alerts to users whenever important changes are detected. This allows users to respond quickly, enhancing both the accuracy of predictions and the efficiency of the system.

Time Series Data in a DC

Posted on by lechuck park

From Claude with some prompting
This image illustrates the concept of time series data analysis in a data center environment. It shows various infrastructure components like IT servers, networking, power and cooling systems, security systems, etc. that generate continuous data streams around the clock (24 hours, 365 days).

This time series data is then processed and analyzed using different machine learning and deep learning techniques such as autoregressive integrated moving average models, generalized autoregressive conditional heteroskedasticity, isolation forest algorithms, support vector machines, local outlier factor, long short-term memory models, and autoencoders.

The goal of this analysis is to gain insights, make predictions, and uncover patterns from the continuous data streams generated by the data center infrastructure components. The analysis results can be further utilized for applications like predictive maintenance, resource optimization, anomaly detection, and other operational efficiency improvements within the data center.

Start Regression (ML)

Posted on by lechuck park

From DALL-E with some prompting

Linear Regression:
Yields a continuous output.
Relates independent variable X with dependent variable Y through a linear relationship.
Uses Mean Squared Error (MSE) as a performance metric.
Can be extended to Multi-linear Regression for multiple independent variables.

Linear & Logistic Regression

  • The process begins with data input, as indicated by “from Data.”
  • Machine learning algorithms then process this data.
  • The outcome of this process branches into two types of regression, as indicated by “get Functions.”

Logistics Regression:
Used for classification tasks, distinguishing between two or more categories.
Outputs a probability percentage (between 0 or 1) indicating the likelihood of belonging to a particular class.
Performance is evaluated using Log Loss or Binary Cross-Entropy metrics.
Can be generalized to Softmax/Multinomial Logistic Regression for multi-class classification problems.

The image also graphically differentiates the two types of regression. Linear Regression is represented with a scatter plot and a trend line indicating the predictive linear equation. Logistic Regression is shown with a sigmoid function curve that distinguishes between two classes, highlighting the model’s ability to classify data points based on the probability threshold.

Event & Alarm

Posted on by lechuck park

From DALL-E with some prompting

The image illustrates the progressive stages of detecting alarm events through data analysis. Here’s a summary:

  1. Internal State: It shows a machine with an ‘ON/OFF’ state, indicating whether the equipment is currently operating.
  2. Numeric & Threshold: A numeric value is monitored against a set threshold, which can trigger an alert if exceeded.
  3. Delta (Changes) & Threshold: A representation of an alert triggered by significant changes or deviations in the equipment’s performance, as compared to a predefined threshold.
  4. Time Series & Analysis: This suggests that analyzing time-series data can identify trends and forecast potential issues.
  5. Machine Learning: Depicts the use of machine learning to interpret data and build predictive models.
  6. More Predictive: The final stage shows the use of machine learning insights to anticipate future events, leading to a more sophisticated alarm system.

Overall, the image conveys the evolution of alarm systems from basic monitoring to advanced prediction using machine learning.


AI 3 Types

Posted on by lechuck park

From DALL-E with some prompting
The image depicts the three stages of AI forming artificial intelligence through repeated classification tasks based on data:

  1. Legacy AI derives statistics from data and transforms them into rule-based programs through human research.
  2. Machine Learning evolves these rules into AI models capable of executing more complex functions.
  3. Deep Learning uses deep neural networks to process data and create complex models that perform cognitive tasks.

In this process, AI leverages extensive data for repetitive classification tasks, and the result is what we refer to as ‘intelligence.’ However, this intelligence is not an emulation of human thought processes but rather a product of data processing and algorithms, which qualifies it as ‘artificial intelligence.’ This underlines that the ‘artificial’ in AI corresponds to intelligence derived artificially rather than naturally through human cognition.