Category: data

Service Development Env.

Posted on by lechuck park

From Claude with some prompting
This image shows a diagram titled “Service Development Env.” (Service Development Environment). It illustrates the stages of a service development process:

  1. Facility: Represented by a building icon, serving as the starting point.
  2. Legacy System: Depicted by a computer screen icon.
  3. Collection: Shown as multiple document icons.
  4. ETL (Extract, Transform, Load): Represented by gear and database icons.
  5. Analysis: Indicated by a magnifying glass icon, including visualization and AI prediction capabilities.
  6. Deploy: Represented by a screen icon with charts, described as “Service = Data + Chart”.

The lower part of the diagram shows additional process steps:

  • Metrics: Includes Configurations.
  • Time Series: Stores data in (id, value, time) format.
  • Tags
  • Roll-Up & TSDB Agg (Time Series Database Aggregation)
  • Prompt with Charts

Overall, this diagram illustrates the entire service development process from data collection to analysis, visualization, and final service deployment. Each stage represents the steps of processing, storing, analyzing data, and ultimately delivering it to end-users.

The flow suggests a progression from legacy systems and facilities, through data collection and processing, to advanced analysis and deployment of data-driven services

Standardization & Platform Why?

Posted on by lechuck park

From Claude with some prompting
This diagram illustrates the importance of standardization and platform development, highlighting two key objectives:

  1. Standardization:
    • Encompasses the stages from real work (machine and processing) through digitization, collecting, and verification.
    • Purpose: “Move on with data trust”
    • Meaning: By establishing standardized processes for data collection and verification, it ensures data reliability. This allows subsequent stages to proceed without concerns about data quality.
  2. Software Development Platform:
    • Includes analysis, improvement, and new development stages.
    • Purpose: “Make easy to improve & go to new”
    • Meaning: Building on standardized data and processes, the platform facilitates easier service improvements and new service development and expansion.

This structure offers several advantages:

  1. Data Reliability: Standardized processes for collection and verification ensure trustworthy data, eliminating concerns about data quality in later stages.
  2. Efficient Improvement and Innovation: With reliable data and a standardized platform, improving existing services or developing new ones becomes more straightforward.
  3. Scalability: The structure provides a foundation for easily adding new services or features.

In conclusion, this diagram visually represents two core strategies: establishing data reliability through standardization and enabling efficient service improvement and expansion through a dedicated platform. It emphasizes how standardization allows teams to trust and focus on using the data, while the platform makes it easier to improve existing services and develop new ones.

“if then” by AI

Posted on by lechuck park

From Claude with some prompting
This image titled “IF THEN” by AI illustrates the evolution from traditional programming to modern AI approaches:

  1. Upper section – “Programming”: This represents the traditional method. Here, programmers collect data, analyze it, and explicitly write “if-then” rules. This process is labeled “Making Rules”.
    • Data collection → Analysis → Setting conditions (IF) → Defining actions (THEN)
  2. Lower section – “AI”: This shows the modern AI approach. It uses “Huge Data” to automatically learn patterns through machine learning algorithms.
    • Large-scale data → Machine Learning → AI model generation

Key differences:

  • Traditional method: Programmers explicitly define rules
  • AI method: Automatically learns patterns from data to create AI models that include basic “if-then” logic

The image effectively diagrams the shift in programming paradigms. It demonstrates how AI can process and learn from massive datasets to automatically generate logic that was previously manually defined by programmers.

This visualization succinctly captures how AI has transformed the approach to problem-solving in computer science, moving from explicit rule-based programming to data-driven, pattern-recognizing models.

Time Series Data ETL

Posted on by lechuck park

From Claude with some prompting
This image illustrates the “Time Series Data ETL” (Extract, Transform, Load) process.

Key components of the image:

  1. Time Series Data structure:
    • Identification (ID): Data identifier
    • Value (Metric): Measured value
    • Time: Timestamp
    • Tags: Additional metadata
  2. ETL Process:
    • Multiple source data points go through the Extract, Transform, Load process to create new transformed data.
  3. Data Transformation:
    • New ID: Generation of a new identifier
    • avg, max, min…: Statistical calculations on values (average, maximum, minimum, etc.)
    • Time Range (Sec, Min): Time range adjustment (in seconds, minutes)
    • all tags: Combination of all tag information

This process demonstrates how raw time series data is collected, transformed as needed, and prepared into a format suitable for analysis or storage. This is a crucial step in large-scale data processing and analysis.

The diagram effectively shows how multiple data points with IDs, values, timestamps, and tags are consolidated and transformed into a new data structure with aggregated information and adjusted time ranges.

Anomaly Detection,Pre-Maintenance,Planning

Posted on by lechuck park

From Claude with some prompting
This image illustrates the concepts of Anomaly Detection, Pre-Maintenance, and Planning in system or equipment management.

Top section:

  1. “Normal Works”: Shows a graph representing normal operational state.
  2. “Threshold Detection”: Depicts the stage where anomalies exceeding a threshold are detected.
  3. “Anomaly Pre-Detection”: Illustrates the stage of detecting anomalies before they reach the threshold.

Bottom section:

  1. “Threshold Detection Anomaly Pre-Detection”: A graph showing both threshold detection and pre-detection of anomalies. It captures anomalies before a real error occurs.
  2. “Pre-Maintenance”: Represents the pre-maintenance stage, where maintenance work is performed after anomalies are detected.
  3. “Maintenance Planning”: Shows the maintenance planning stage, indicating continuous monitoring and scheduled maintenance activities.

The image demonstrates the process of:

  • Detecting anomalies early in normal system operations
  • Implementing pre-maintenance to prevent actual errors
  • Developing systematic maintenance plans

This visual explanation emphasizes the importance of proactive monitoring and maintenance to prevent failures and optimize system performance.

Standardization for DCIM

Posted on by lechuck park

From Claude with some prompting
Data Standardization:

  • Defined a clear process for systematically collecting data from equipment.
  • Proposed an integrated data management approach, including network topology and interfacing between various systems.
  • Emphasized data quality management as a key factor to establish a reliable data foundation.

Service Standardization:

  • Structured the process of connecting data to actual services.
  • Highlighted practical service implementation, including monitoring services and automation tasks.
  • Specified AI service requirements, showing a forward-looking approach.
  • Established a foundation for continuous service improvement by including service analysis and development processes.

Commissioning Standardization:

  • Emphasized verification plans and documentation of results at each stage of design, construction, and operation to enable quality management throughout the entire lifecycle.
  • Prepared an immediate response system for potential operational issues by including real-time data error verification.
  • Considered system scalability and flexibility by incorporating processes for adding facilities and data configurations.

Overall Evaluation:

This DCIM standardization approach comprehensively addresses the core elements of data center infrastructure management. The structured process, from data collection to service delivery and continuous verification, is particularly noteworthy. By emphasizing fundamental data quality management and system stability while considering advanced technologies like AI, the approach is both practical and future-oriented. This comprehensive framework will be a valuable guideline for the implementation and operation of DCIM.

Data Life

Posted on by lechuck park

From ChatGPT with some prompting
reflecting the roles of human research and AI/machine learning in the data process:

Diagram Explanation :

  1. World:
    • Data is collected from the real world. This could be information from the web, sensor data, or other sources.
  2. Raw Data:
    • The collected data is in its raw, unprocessed form. It is prepared for analysis and processing.
  3. Analysis:
    • The data is analyzed to extract important information and patterns. During this process, rules are created.
  4. Rules Creation:
    • This step is driven by human research.
    • The human research process aims for logical and 100% accurate rules.
    • These rules are critical for processing and analyzing data with complete accuracy. For example, creating clear criteria for classifying or making decisions based on the data.
  5. New Data Generation:
    • New data is generated during the analysis process, which can be used for further analysis or to update existing rules.
  6. Machine Learning:
    • In this phase, AI models (rules) are trained using the data.
    • AI/machine learning goes beyond human-defined rules by utilizing vast amounts of data through computing power to achieve over 99% accuracy in predictions.
    • This process relies heavily on computational resources and energy, using probabilistic models to derive results from the data.
    • For instance, AI can identify whether an image contains a cat or a dog with over 99% accuracy based on the data it has learned from.

Overall Flow Summary :

  • Human research establishes logical rules that are 100% accurate, and these rules are essential for precise data processing and analysis.
  • AI/machine learning complements these rules by leveraging massive amounts of data and computing power to find high-probability results. This is done through probabilistic models that continuously improve and refine predictions over time.
  • Together, these two approaches enhance the effectiveness and accuracy of data processing and prediction.

This diagram effectively illustrates how human logical research and AI-driven data learning work together in the data processing lifecycle.