Tag: data

Operation with AI

Posted on by lechuck park

From Claude with some prompting
This diagram illustrates an integrated approach to modern operational management. The system is divided into three main components: data generation, data processing, and AI application.

The Operation & Biz section shows two primary data sources. First, there’s metric data automatically generated by machines such as servers and network equipment. Second, there’s textual data created by human operators and customer service representatives, primarily through web portals.

These collected data streams then move to the central Data Processing stage. Here, metric data is processed through CPUs and converted into time series data, while textual data is structured via web business services.

Finally, in the AI play stage, different AI models are applied based on data types. For time series data, models like RNN, LSTM, and Auto Encoder are used for predictive analytics. Textual data is processed through a Large Language Model (LLM) to extract insights.

This integrated system effectively utilizes data from various sources to improve operational efficiency, support data-driven decision-making, and enable advanced analysis and prediction through AI. Ultimately, it facilitates easy and effective management even in complex operational environments.

The image emphasizes how different types of data – machine-generated metrics and human-generated text – are processed and analyzed using appropriate AI techniques, all from the perspective of operational management.

Why digitalization?

Posted on by lechuck park

From Claude with some prompting
The image depicts the effects of digitalization in three distinct stages:

Stage 1: Long-Term Accumulated Efficiency Gains Initially, efforts towards digitalization, such as standardization, automation, system and data-based work, may not yield visible results for a considerable amount of time. However, during this period, continuous improvement and optimization gradually lead to an accumulation of efficiency gains.

Stage 2: Eventual Leaps Once the efforts from Stage 1 reach a critical point, significant performance improvements and innovative breakthroughs occur, backed by the experience and learning acquired. The previously accumulated data and process improvement know-how enable these sudden leaps forward.

Stage 3: Extensive Huge Upturn with Big Data & AI Through digitalization, big data is built, and when combined with artificial intelligence technologies, unprecedented and massive levels of performance can be achieved. Data-driven predictions and automated decision-making enable disruptive value creation across a wide range of domains.

Therefore, while the initial stage of digital transformation may seem to yield minimal visible gains, persevering with continuous efforts will allow the accumulation of experience and data, eventually opening up opportunities for rapid innovation and large-scale growth. The key is to maintain patience and commitment, as the true potential of digitalization can be unlocked through the combination of data and advanced technologies like AI.

TSDB flow for alerts

Posted on by lechuck park

From Claude with some prompting
This image illustrates the flow and process of a Time Series Database (TSDB) system. The main components are:

Time Series Data: This is the input data stream containing time-stamped values from various sources or metrics.

Counting: It performs change detection on the incoming time series data to capture relevant events or anomalies.

Delta Value: The difference or change observed in the current value compared to a previous reference point, denoted as NOW() – previous value.

Time-series summary Value: Various summary statistics like MAX, MIN, and other aggregations are computed over the time window.

Threshold Checking: The delta values and other aggregations are evaluated against predefined thresholds for anomaly detection.

Alert: If any threshold conditions are violated, an alert is triggered to notify the monitoring system or personnel.

The process also considers correlations with other metrics for improved anomaly detection context. Additionally, AI-based techniques can derive new metrics from the existing data for enhanced monitoring capabilities.

In summary, this flow diagram represents the core functionality of a time series database focused on capturing, analyzing, and alerting on anomalies or deviations from expected patterns in real-time data streams.

Who First

Posted on by lechuck park

From ChatGPT with some prompting
This image explores two potential scenarios related to the advancement of AI (Artificial Intelligence). It raises two main questions:

  1. Exponential Use of Data and Energy: The left side illustrates a scenario where data and energy created by humans are used exponentially by AI. This leads to the concern that data and energy might be depleted. It questions whether we will run out of data and energy first due to this exponential use.
  2. AI’s Self-Sufficiency: The right side presents the possibility that AI might be able to create new data and energy on its own. If AI can generate its own data and energy resources, it could overcome the problem of depletion.

Therefore, the image highlights a dilemma: on one hand, the rapid use of data and energy by AI might lead to their depletion, while on the other hand, AI might potentially find ways to create new data and energy to sustain itself. It questions which of these scenarios will happen first.

Trend & Prediction

Posted on by lechuck park

From Claude with some prompting
The image presents a “Trend & Predictions” process, illustrating a data-driven prediction system. The key aspect is the transition from manual validation to automation.

  1. Data Collection & Storage: Digital data is gathered from various sources and stored in a database.
  2. Manual Selection & Validation: a. User manually selects which metric (data) to use b. User manually chooses which AI model to apply c. Analysis & Confirmation using selected data and model
  3. Transition to Automation:
    • Once optimal metrics and models are confirmed in the manual validation phase, the system learns and switches to automation mode. a. Automatically collects and processes data based on selected metrics b. Automatically applies validated models c. Applies pre-set thresholds to prediction results d. Automatically detects and alerts on significant predictive patterns or anomalies based on thresholds

The core of this process is combining user expertise with system efficiency. Initially, users directly select metrics and models, validating results to “educate” the system. This phase determines which data is meaningful and which models are accurate.

Once this “learning” stage is complete, the system transitions to automation mode. It now automatically collects, processes data, and generates predictions using user-validated metrics and models. Furthermore, it applies preset thresholds to automatically detect significant trend changes or anomalies.

This enables the system to continuously monitor trends, providing alerts to users whenever important changes are detected. This allows users to respond quickly, enhancing both the accuracy of predictions and the efficiency of the system.

Anyway, The probability

Posted on by lechuck park

From Claude with some prompting
Traditional View: AI’s probability-based decisions are seen in contrast to human’s logical, “100% certain” decisions, and this difference could be perceived as problematic.

New Insight: In reality, the concept of human’s “100% certainty” itself might be an illusion. Human judgments are also based on limited data and experiences, making them inherently probabilistic in nature.

Finding Common Ground: Both humans and AI make decisions based on incomplete information. Even human’s logical certainty ultimately stems from restricted data, making it fundamentally probability-based.

Paradigm Shift: This perspective suggests that AI’s probabilistic approach isn’t a flaw but rather a more accurate modeling of human decision-making processes. What we believe to be “100% certainty” is actually a high-probability estimation based on limited information.

Implications: This prompts a reevaluation of the perceived gap between AI and human decision-making styles. AI’s probabilistic approach might not be inferior to human logic; instead, it may more accurately reflect our cognitive processes.

This viewpoint encourages us to see AI’s probabilistic tendencies not as a problem, but as a tool providing deeper insights into human thought processes. It invites us to reconsider how AI and humans collaborate, opening new possibilities to complementarily leverage the strengths of both sides.

The image and your interpretation together challenge the notion that human reasoning is purely logical and certain. Instead, they suggest that both human and AI decisions are fundamentally based on probabilities derived from limited data. This realization can foster a more harmonious and effective partnership between humans and AI, recognizing that our decision-making processes may be more similar than previously thought.

Not Real-Simultaneous Works

Posted on by lechuck park

From Claude with some prompting
The image emphasizes that while it may appear to be simultaneous processing, it is actually very fast serial processing.

From the perspectives of the CPU, LAN, and data processing, each can only handle one unit of work at a time. The CPU can execute one instruction, the network can transmit one packet line, and in data processing, critical sections require mutual exclusion and serialization.

However, due to very fast switching techniques like process/task switching and Ethernet/packet switching, multiple tasks appear to be happening concurrently. But in reality, it is processing single units of work in rapid serial fashion.

So concurrency is achieved through fast serial processing, not parallel processing. Even so, in critical areas, synchronization and serialization are required to maintain data integrity.

In essence, the image highlights that while it looks like simultaneous processing, concurrency is actually implemented through extremely fast serial processing of single work units at a time.