Tag: ai

Humans with numbers

Posted on by lechuck park

From Claude with some prompting
This image depicts the progressive development of human capabilities and knowledge, showcasing how humans have strived to understand and explain the world through the use of numbers, mathematics, and computing technology.

  1. Human Groups: The image represents humans coming together in groups to explore and comprehend the world around them.
  2. Using Math: Humans have leveraged numbers and mathematical calculations in an effort to make sense of the world.
  3. Computing: Building upon their mathematical prowess, the advancement of computing technology has enhanced human analysis and understanding.
  4. High-Speed Infrastructure: The development of cutting-edge technological infrastructure has enabled further evolution of human activities.
  5. AI and Deep Learning: This series of technological advancements has led humans to a point where they may feel they have nearly reached the true essence of reality. However, the image suggests that the emergence of AI and deep learning technologies is now challenging this human-centric perspective, hinting that there may still be an infinite gap to traverse before fully grasping the fundamental nature of the world.

In essence, the image showcases the stepwise progression of human knowledge and capabilities, anchored in numbers, math, and computing, while also highlighting how these efforts are now being disrupted by the rise of advanced AI and deep learning, which may transcend the limitations of human understanding.

Biz AI Arch.

Posted on by lechuck park

From Claude with some prompting
the AI-based enterprise document analysis/conversation service architecture:

Architectural Components:

  1. User Access Layer (On-Premises Private Biz Network)
  • User access through web interface
  • Secure access within corporate internal network environment
  1. Data Management Layer (Local Storage)
  • On-Premises Cloud Deployment support
  • Hybrid cloud environment with AWS outpost, Azure Stack, GCP
  • Secure storage of corporate documents and data
  1. Service Operation Layer (Cloud/AI Infra)
  • Enhanced security through Virtual Private Network
  • Cloud-based AI service integration
  • Document-based AI services like NotebookLM

Key Features and Benefits:

  1. Security
  • Private Network-based operation
  • Minimized data leakage risk
  • Regulatory compliance facilitation
  1. Scalability
  • Hybrid cloud architecture
  • Efficient resource management
  • Expandable to various AI services
  1. Operational Efficiency
  • Centralized data management
  • Unified security policy implementation
  • Easy monitoring and management

Considerations and Improvements:

  1. System Optimization
  • Balance between performance and cost
  • Implementation of caching system
  • Establishment of monitoring framework
  1. Future Extensibility
  • Integration potential for various AI services
  • Multi-cloud strategy development
  • Resource adjustment based on usage patterns

Technical Considerations:

  1. Performance Management
  • Network bandwidth and latency optimization
  • AI model inference response time management
  • Data synchronization between local and cloud storage
  1. Security Measures
  • Data governance and sovereignty
  • Secure data transmission
  • Access control and authentication
  1. Infrastructure Management
  • Resource scaling strategy
  • Service availability monitoring
  • Disaster recovery planning

This architecture provides a framework for implementing document-based AI services securely and efficiently in enterprise environments. It is particularly suitable for organizations where data security and regulatory compliance are critical priorities. The design allows for gradual optimization based on actual usage patterns and performance requirements while maintaining a balance between security and functionality.

This solution effectively combines the benefits of on-premises security with cloud-based AI capabilities, making it an ideal choice for enterprises looking to implement advanced document analysis and conversation services while maintaining strict data control and compliance requirements.

Prediction & Detection

Posted on by lechuck park

From Claude with some prompting
This image illustrates a Prediction and Detection system for time series data. Let me break down the key components:

  1. Left Large Box (Learning and Prediction Section):
  • Blue line: Actual Temperature data
  • Red dotted line: Predicted Temperature data
  • Uses time series prediction models like LSTM, ARIMA, and Prophet for learning
  1. Top Right (Threshold-based Anomaly Detection):
  • “Abnormal Detection with Threshold”
  • Detects abnormal temperature changes based on threshold values
  • The area marked with a red circle shows where values exceed the threshold
  • Includes “Warning” and “Critical” threshold levels
  1. Bottom Right (Pattern-based Anomaly Detection):
  • “Anomaly Detection with Predict-Pattern”
  • Compares predicted patterns with actual data to detect anomalies
  • The area marked with a green circle shows where actual data deviates from the predicted pattern

The system detects anomalies in two ways:

  1. When values exceed predetermined thresholds
  2. When actual data significantly deviates from predicted patterns

This type of system is particularly useful in:

  • Industrial monitoring
  • Equipment maintenance
  • Early warning systems
  • Quality control
  • System health monitoring

The combination of prediction and dual detection methods (threshold and pattern-based) provides a robust approach to identifying potential issues before they become critical problems.

ARIMA

Posted on by lechuck park

From Claude with some prompting
The image depicts the Autoregressive Integrated Moving Average (ARIMA) Integrated Moving Average Model, which is a time series forecasting technique.

The main components are:

  1. AR (Autoregressive):
    • This component models the past pattern in the data.
    • It performs regression analysis on the historical data.
  2. I (Integrated):
    • This component handles the non-stationarity in the time series data.
    • It applies differencing to make the data stationary.
  3. MA (Moving Average):
    • This component uses the past error terms to calculate the current forecast.
    • It applies a moving average to the error terms.

The flow of the model is as follows:

  1. Past Pattern: The historical data patterns are analyzed.
  2. Regression: The past patterns are used to perform regression analysis.
  3. Difference: The non-stationary data is made stationary through differencing.
  4. Applying Weights + Sliding Window: The regression analysis and differencing are combined, with a sliding window used to update the model.
  5. Prediction: The model generates forecasts based on the previous steps.
  6. Stabilization: The forecasts are stabilized and smoothed.
  7. Remove error: The model removes any remaining error from the forecasts, bringing them closer to the true average.

The diagram also includes visual representations of the forecast output, showing both upward and downward trends.

Overall, this ARIMA model integrates autoregressive, differencing, and moving average components to provide accurate time series forecasts while handling non-stationarity in the data.

AI Prediction

Posted on by lechuck park

From Claude with some prompting
This diagram illustrates an AI Prediction System workflow, which is divided into two main sections:

  1. Upper Section (VIEW):
  • Starts with a UI/UX interface
  • Executes queries with tags (metadata)
  • Connects to time series data storage
  • Displays data visualization charts
  • Includes model selection step
  • Finally generates prediction charts
  1. Lower Section (Automation):
  • Selected ID
  • Selected Model
  • Periodic, new tags and additional configuration
  • Batch work processing (consisting of 4 steps):
    1. Registering
    2. Read Data
    3. Generate Predictions
    4. Add Tag
  • Writing new time series data

The system provides two main functionalities:

  1. A user interface for direct data viewing and prediction execution
  2. Automated batch processing for periodic predictions and data updates

Key Components:

  • Time Series Data storage as a central database
  • View Chart for data visualization
  • Model Selection with time selection (learn & predict)
  • Predict Chart as the final output
  • Batch Works system for automated processing

The workflow demonstrates a comprehensive approach to handling both manual and automated AI predictions, combining user interaction with systematic data processing and analysis. The system appears designed to handle time series data efficiently while providing both immediate and scheduled prediction capabilities.

Data with the AI

Posted on by lechuck park

From Claude with some prompting
the key points from the diagram:

  1. Reality of Internet Open Data:
    • Vast amount of open data exists on the internet including:
      • Mobile device data
      • Email communications
      • Video content
      • Location data
    • This open data is utilized by major AI companies for LLM training
    • Key players:
      • OpenAI’s ChatGPT
      • Anthropic’s Claude
      • Google’s Gemini
      • Meta’s LLaMA
  2. Competition Implications:
    • Competition between LLMs trained on similar internet data
    • “Who Winner?” and “A Winner Takes ALL?” suggests potential monopoly in the base LLM market
    • This refers specifically to models trained on public internet data
  3. Market Outlook:
    • While the base LLM market might be dominated by a few players
    • Private enterprise data remains a key differentiator
    • “Still Differentiated and Competitive” indicates ongoing competition through enterprise-specific data
    • Companies can leverage RAG-like technology to combine their private data with LLMs for unique solutions
  4. Key Implications:
    • Base LLM market (trained on internet data) may be dominated by few winners
    • Enterprise competition remains vibrant through:
      • Unique private data assets
      • RAG integration with base LLMs
      • Company-specific implementations
    • Market likely to evolve into dual structure:
      • Foundation LLMs (based on internet data)
      • Enterprise-specific AI services (leveraging private data)

This structure suggests that while base LLM technology might be dominated by a few players, enterprises can maintain competitive advantage through their unique private data assets and specialized implementations using RAG-like technologies.

This creates a market where companies can differentiate themselves even while using the same foundation models, by leveraging their proprietary data and specific use-case implementations.

From Data

Posted on by lechuck park

From Claude with some prompting
following the overall sequence from data collection to AI systems development.

  1. Data Collection and Processing (Upper “From Data” section): a) Collecting data from people worldwide b) “Get Data”: Acquiring raw data c) “Gathering Data”: Converting data into binary format d) “Statistics Analysis”: Performing data analysis e) “Making Rules/Formula”: Generating rules or formulas based on analysis
  2. Evolution of AI Systems (Lower “Human-made AI (Legacy)” section): a) Human-centered analysis:
    • “Combine formulas”: Combining rules and formulas directly created by humans
    b) Machine Learning-based analysis:
    • Rule-based Machine Learning: • Utilizes Big Data • Generates rules/formulas through machine learning • Results evaluated as “True or False”
    • Statistical Machine Learning (e.g., LLM): • Utilizes Big Data • Performs statistical analysis using advanced machine learning • Results evaluated as “Better or Worse”

Key Points Summary:

  1. Data Processing Flow: Illustrates the step-by-step process from raw data collection to rule/formula generation.
  2. AI System Evolution:
    • Begins with human-centered rule-based systems
    • Progresses to machine learning models that learn rules from data
    • Advances to sophisticated statistical models (like LLMs) that recognize complex patterns and provide nuanced results
  3. Shift in Result Interpretation:
    • Moves from simple true/false outcomes
    • To relative and context-dependent “better/worse” evaluations

This image effectively demonstrates the progression of data processing and AI technology, particularly highlighting how AI systems have become more complex and sophisticated. It shows the transition from human-derived rules to data-driven machine learning approaches, culminating in advanced statistical models that can handle nuanced analysis and produce more contextualized results.