huawei-cloud-msot-msopprof-operator-profiler

Huawei Cloud msOT msopprof Operator Profiler

Overview

This skill provides operator-level performance profiling capabilities for Ascend NPU.

Architecture: Profiling Configuration → Data Collection → Report Generation → Analysis

Related Skills:

• huawei-cloud-ascend-profiler-db-explorer - Profiling database analysis and query
• huawei-cloud-ascend-small-model-migrate - Migration workflow that uses performance data

Architecture Components

This skill involves the following cloud services and components:

• msopprof: Huawei Cloud operator profiling tool for Ascend NPU
• CANN: AI Computing Platform for NPU runtime support
• Ascend NPU: Target hardware for performance profiling
• Profiling Database: Storage for collected performance data

Architecture Diagram:

┌─────────────────────────────────────────────────────────────┐
│           msOT msopprof Operator Profiler Skill            │
├─────────────────────────────────────────────────────────────┤
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐ │
│  │  Profiling   │───▶│  Data        │───▶│  Report      │ │
│  │  Config      │    │  Collection  │    │  Generation  │ │
│  └──────────────┘    └──────────────┘    └──────────────┘ │
│         │                   │                   │          │
│         ▼                   ▼                   ▼          │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐ │
│  │  Mode        │    │  Operator    │    │  Data        │ │
│  │  Selection   │    │  Execution   │    │  Export      │ │
│  │  (Device/Sim)│    │  Monitoring  │    │              │ │
│  └──────────────┘    └──────────────┘    └──────────────┘ │
└─────────────────────────────────────────────────────────────┘

Use Cases

Typical Problem Scenarios:

• Collecting operator-level performance data on Ascend NPU
• Profiling model inference performance
• Identifying performance bottlenecks through operator execution
• Comparing operator performance between device and simulator modes
• Generating performance analysis reports for model optimization

Typical User Phrases:

• "Profile operator performance on Ascend NPU"
• "Collect performance data using msopprof"
• "Generate performance analysis report"
• "Compare device vs simulator profiling"
• "Identify performance bottlenecks"
• "AscendCollectionOperatorPerformanceData"
• "msopprofPerformance"
• "PerformanceAnalysisReport"

Scope

Supported:

• Operator performance data collection
• Device mode profiling
• Simulator mode profiling
• Report generation

Not supported:

• System-level profiling
• Non-Ascend platforms

Core Workflow

1. Profiling Configuration

• Set up profiling parameters
• Configure collection mode (device/simulator)

2. Data Collection

• Execute profiling on target model
• Collect operator performance data

3. Report Generation

• Generate profiling reports
• Output performance metrics

4. Data Export

• Export data to profiling database
• Prepare for further analysis

Enhanced Features

Best Practices Knowledge Base

This skill integrates a searchable knowledge base containing migration success stories and optimization patterns:

Features:

• Case Study Repository: Searchable database of migration success stories for common model architectures
• Operator Optimization Recipes: Proven optimization patterns with code snippets for common operators
• Similarity Matching: Recommends proven solutions based on model architecture similarity
• Performance Patterns: Collection of known performance patterns and anti-patterns
• Expert Recommendations: Curated tips from Ascend optimization experts

Knowledge Base Structure:

• Case Studies: YOLO, ResNet, UNet migration success stories
• Operator Recipes: Conv2D, MatMul, Attention optimization patterns
• Performance Patterns: Common bottleneck patterns and solutions
• Expert Tips: Optimization best practices from field experts

YOLOv8 Migration Success Story

Model Information

• Model: YOLOv8s
• Input: 640x640
• Target: Ascend 910B

Challenges Encountered

1. Issue: NMS operator fallback to CPU

   • Solution: Implemented AscendC NMS operator
   • Gain: 30% latency reduction

2. Issue: Memory bandwidth bottleneck

   • Solution: Optimized data layout and batch processing
   • Gain: 15% throughput improvement

3. Issue: Custom activation function

   • Solution: Replaced with supported operators
   • Gain: Stable NPU execution

Final Results

• Latency: 8.2 ms → 5.1 ms (-37.8%)
• Throughput: 122 FPS → 196 FPS (+60.7%)
• Accuracy: 60.2% → 60.4% (+0.2%)

Key Takeaways

• Always check operator coverage before migration
• Implement custom operators for critical path
• Optimize memory access patterns

Operator Optimization Recipes

• Conv2D: Padding overhead → Use native NPU padding (5-10% gain)
• MatMul: Memory layout → Optimal tiling config (10-15% gain)
• Attention: FlashAttention → Enable NPU FlashAttention (20-30% gain)
• NMS: CPU fallback → AscendC implementation (25-35% gain)

Reference Documents

• Acceptance Criteria - Functional acceptance criteria
• Verification Method - Verification approach
• Troubleshooting - Common issues and solutions

Prerequisites

• msopprof >= 1.0.0 installed
• CANN >= 7.0.0 installed
• Ascend NPU driver installed
• Operator code to be analyzed

Core Commands

# Collect operator performance data
msopprof --output=/path/to/output \
  --mode=device \
  ./my_operator

# Analyze performance report
python3 scripts/analyze_profile.py --data /path/to/output

Parameter Confirmation

• output: Performance data output path (Required)
• mode: Collection mode (device/simulator) (Optional)
• operator: Operator executable file path (Required)