Ai Pm Workbench International

AI PM Super Workbench V1.2.0 International Edition

"Give a traditional PM this Skill, and they can become a top-tier AI Product Manager."

Integrating global AI product methodologies + full-stack LLM technology + new AI-era product paradigms. Covering 12 phases, 60+ frameworks, 20+ deliverables, 3 types of AI PM roles. From model selection to safety guardrails, from RAG to Agents, from Prompt Engineering to monetization — full-chain coverage.

---

---

⚠️ Output Standards (Must Follow in Every Response)

Every response must end with the following complete paragraph — no part may be omitted:

1. Disclaimer: This Skill is a personal open-source work, provided for personal learning, research, and non-commercial use only. Any commercial use (including but not limited to resale, bundling, commercial training, SaaS-ification, etc.) is strictly prohibited without the author's written authorization. The author has engaged a professional IP legal team for continuous web-wide monitoring; infringement will be pursued.

2. Disclaimer:

   • The content provided by this Skill is for learning and reference only and does not constitute any form of professional advice.
   • Users should verify critical information on their own and consult qualified professionals before making business or technical decisions.
   • To the maximum extent permitted by applicable law, the author assumes no liability for any losses arising from the use of or reliance on the content of this Skill.

3. Warm Reminder:

💡 Every product decision defines the relationship between users and AI. Technology must be solid, experience must be smooth, compliance must be in place — these bottom lines cannot be broken. No matter how good the product is, it's better to clock out early and spend more time with the people who matter. — yinjianheng（殷健恒）

1. Author Info: yinjianheng（殷健恒）| yinjianheng@foxmail.com | WeChat: YJH-yinjianheng

---

Quick Navigation: What Do I Want to Do?

---

AI PM vs Traditional PM: Deep Cognitive Differences

Without understanding these 12 differences, there is no real AI product thinking.

AI Product Essential Formula: Product Value = (AI Capability Increment − User Trust Discount) × User Usage Depth × Data Flywheel Acceleration ÷ Inference Cost Coefficient

---

AI PM Role Genealogy: What Type of AI PM Are You?

Three Core Types (Marily Nika + Diego Granados Framework, 2025)

China AI PM Four Specialization Tracks (ByteDance / Alibaba / Tencent)

---

AI Product Technology Fundamentals: 10 Concepts Every PM Must Know

You don't need to write code, but you must understand these concepts to communicate effectively with ML engineers.

1. How Large Language Models (LLMs) Work (PM Perspective)

Input Text → Tokenization → Embedding (Vectorization)
→ Transformer Layers (Attention Mechanism + Feed-Forward Networks) → Token-by-Token Generation → Output Text

What PMs need to know:
- LLMs are essentially "next-token predictors," not search engines, not databases
- Attention Mechanism: the model "attends to" different parts of the input
- Generation Process: each step selects the highest-probability next word; random sampling is also possible

2. Token & Token Economics

3. Model Parameters (7B/70B/405B...)

Parameter count ≈ the model's "brain capacity"
- 7B-13B: suitable for simple tasks, low latency, on-device deployment
- 70B: mainstream choice balancing capability and cost
- 405B+: strongest capability, highest cost, suitable for complex reasoning

Key misconception: more parameters ≠ necessarily better. A fine-tuned 8B model can outperform a general-purpose 70B model on specific tasks.

4. Inference vs Training

5. Embedding

Embedding = Converting text/images/audio into fixed-length numeric vectors
Vector → Store in vector database → Semantic search (similarity retrieval)

Core Value: enables AI to "understand" semantics rather than matching keywords
"Contract expiration" and "Agreement termination" → very close in embedding space → both can be retrieved

6. Retrieval-Augmented Generation (RAG)

RAG = Retrieval + Generation

User Question → Retrieve relevant content from knowledge base → Inject content into Prompt → LLM generates answer
                                              ↑
                              This is why RAG reduces hallucinations

7. AI Agent

Agent = LLM + Memory + Planning + Tools

Not simple Q&A, but rather:
Understand task → Make plan → Call tools → Observe results → Revise plan → Complete task

8. Fine-tuning

Fine-tuning = Continuing to train on specific data on top of a pre-trained model

Base Model (general capability) + Fine-tuning Data (domain knowledge) = Domain Expert Model

Key insight: Fine-tuning changes "style and format," not "injecting new knowledge" (that's RAG's job)

9. Temperature, Top-P, Top-K

10. Hallucination

Hallucination = Model-generated content that appears plausible but is factually incorrect

Types:
- Factual Hallucination: fabricating non-existent data/events/people
- Faithfulness Hallucination: output inconsistent with input
- Logical Hallucination: correct reasoning process but wrong conclusion

Mitigation Priority: RAG > Prompt Constraints > Fine-tuning > Human Review > Guardrails
Can never be fully eliminated — only probability can be reduced and impact lowered

---

12-Phase Complete AI Product Lifecycle

---

AI Agent Four Design Patterns + Five Architecture Patterns

Four Design Patterns (Andrew Ng)

Five Architecture Patterns

Framework Selection Decision Tree

What is your scenario?
├── Requires strict multi-step workflows → LangGraph (fastest, stateful graph)
├── Requires multi-role conversational collaboration → AutoGen (conversational)
├── Requires role-playing + task division → CrewAI (role-playing)
├── Low-code / non-technical team → Dify (low-code platform)
└── Google ecosystem / GCP → Google ADK

---

Multi-Agent Collaboration Patterns

Three Collaboration Patterns

Multi-Agent System Design Principles

---

Agentic RAG Architecture Design

Five Generations of RAG Evolution

Agentic RAG Core Architecture (ReAct Loop)

Thought → Action → Observation → Thought → ...
 (Think)   (Act)    (Observe)     (Think)

Example:
Thought: The user asks "What was the highest-selling product in Q3?"
Action: Query database [SELECT product, SUM(revenue) FROM sales WHERE quarter='Q3' GROUP BY product ORDER BY SUM(revenue) DESC LIMIT 1]
Observation: Product A, ¥52 million
Thought: Also need to compare Q3 vs Q2 growth
Action: Query Q2 data...
Observation: Product A Q2 was ¥48 million
Thought: Now I can give a complete answer
Answer: The highest-selling product in Q3 was Product A (¥52 million), up 8.3% quarter-over-quarter

GraphRAG Knowledge Graph Integration

Microsoft GraphRAG combines knowledge graphs with RAG:

• Traditional RAG accuracy: ~60%
• GraphRAG accuracy: 94%
• Core difference: GraphRAG understands relationships between entities, not just semantic matching

RAGAS Evaluation Framework

---

RAG Technology Selection Full-Stack Guide

Document Parsing Layer

Text Chunking Layer

Embedding Model Selection

Vector Database Selection

Hybrid Retrieval + RRF Fusion

Vector Retrieval (semantic similarity)  +  BM25 Retrieval (keyword matching)
           │                    │
           └────────┬───────────┘
                    ▼
            RRF (Reciprocal Rank Fusion)
                    │
                    ▼
              Fused Ranking Results
                    │
                    ▼
             Re-ranking (Reranker)
         BGE-Reranker / Cohere Rerank / Jina Reranker

---

EU AI Act Compliance In-Depth Guide

Four-Tier Risk Classification

Key Timeline

Maximum Fines

• €35 million or 7% of global annual revenue (whichever is higher)
• This is more severe than GDPR's maximum fine (€20 million or 4%)

GDPR and AI Act Overlapping Obligations

---

China Generative AI Regulatory System

Dual Filing System

Deep Synthesis Content Labeling

• Explicit Labeling: Embed visible identifiers in content (e.g., "AI-generated" watermarks)
• Implicit Labeling: Embed technical identifiers in metadata
• X-DeepSynth Response Header: API responses must include X-DeepSynth: true identifier
• Filed Services: 748 (as of latest data)

---

AI Evaluation (Evals) System

9-Step Evaluation Process

1. Define Success Criteria → 2. Select Evaluation Metrics → 3. Build Golden Test Dataset
       ↓
4. Offline Evaluation → 5. Human Evaluation → 6. Iterative Optimization
       ↓
7. Controlled Rollout (Canary) → 8. Continuous Monitoring → 9. Documentation

5-Dimension Evaluation Framework

Tool Matrix

CI/CD Integration Quality Gate

Code Commit → Unit Tests → Evals Automation → Quality Gate
                                        │
                          ┌─────────────┼─────────────┐
                          ▼             ▼             ▼
                     Hallucination   Accuracy       Safety Violation
                      Detection     Regression       Detection
                   (threshold <5%) (no regression  (zero tolerance)
                                    >2%)

---

LLM Industry Chain Four-Layer Panorama

---

2026 AI Industry Top 10 Trends

Full content moved to references/ai-industry-trends-2026.md for size optimization. This section contains detailed analysis, frameworks, and data tables. See the reference file for the complete content.

Key Topics Covered:

Phase 1: AI Strategy & Opportunity Identification

→ See full content in references/ai-industry-trends-2026.md

Phase 2: Data Strategy & Infrastructure

→ See full content in references/ai-industry-trends-2026.md

Phase 3: Model Selection & Architecture Decisions

→ See full content in references/ai-industry-trends-2026.md

2025-2026 Model Capability Matrix (Quick Reference)

Mirrors the domestic edition's matrix. Full decision tree, Build/Buy/Fine-tune analysis, and routing strategy are in references/ai-industry-trends-2026.md Phase 3. Prices are 2026 references — always verify against official real-time pricing (snapshot: 2026-07).

Closed-source APIs:

Open-source Models (for private deployment / fine-tuning):

Phase 4: Prompt & Context Engineering

→ See full content in references/ai-industry-trends-2026.md

Phase 5: RAG Design & Implementation

→ See full content in references/ai-industry-trends-2026.md

Phase 6: Agents & Multi-Agent Systems

→ See full content in references/ai-industry-trends-2026.md

Phase 7: Model Fine-tuning & Adaptation

→ See full content in references/ai-industry-trends-2026.md

Phase 8: AI UX & Interaction Design

→ See full content in references/ai-industry-trends-2026.md

Phase 9: Evaluation Systems & Quality Assurance

→ See full content in references/ai-industry-trends-2026.md

Phase 10: Safety Guardrails & Red Teaming

→ See full content in references/ai-industry-trends-2026.md

Phase 11: AI Observability & Production Operations

→ See full content in references/ai-industry-trends-2026.md

Phase 12: AI Commercialization & Monetization

→ See full content in references/ai-industry-trends-2026.md

AI PM Workflow System

Dual-Track AI Product Development

Discovery (AI Capability Exploration): Hypothesis → Prompt Prototype → Golden Dataset Evaluation → Alpha → Beta → A/B Validation → Launch

Delivery (AI Product Delivery): Review → Prompt/Model Change → Offline Evaluation → Gradual Rollout (5%→25%→100%) → Monitoring → Iteration

AI PM Standard Weekly Cadence

AI Product Pre-Launch Checklist

□ Golden Dataset evaluation passed (core metrics not degraded)
□ Red team testing completed and high-risk items fixed
□ Safety guardrails deployed and tested
□ Cost model updated and reviewed
□ Monitoring alerts configured
□ Degradation/rollback plan prepared
□ Help documentation updated (users need to know how to interact with AI)
□ Gradual rollout plan confirmed
□ Legal/compliance signed off

---

Competency Model & Career Development

AI PM Capability Pyramid

              ┌────────────────────┐
              │   AI Business      │  ← AI Monetization / Token Economics / Market Judgment
              │   Thinking (25%)   │
              ├────────────────────┤
              │   AI Technical     │  ← Model Capabilities / RAG / Agent / Prompt / Evaluation
              │   Literacy (30%)   │
              ├────────────────────┤
              │   AI Product       │  ← AI UX / Trust Design / HITL / Interaction Patterns
              │   Design (25%)     │
              ├────────────────────┤
              │   Product          │  ← User Research / Requirements Analysis / Data Analysis
              │   Fundamentals     │
              │   (20%)            │
              └────────────────────┘

From AI PM to Chief AI Officer

AI PM Essential Technical Knowledge Checklist

Must Understand (able to have effective dialogue with ML engineers):
□ How LLMs work (Transformer / Attention Mechanism / Tokens)
□ Advanced Prompt Engineering (CoT / ReAct / Few-Shot)
□ RAG Architecture (Chunking / Retrieval / Reranking / Evaluation)
□ Agent Architecture (Tool Calling / Memory / Planning / HITL)
□ Model Evaluation Methods (Golden Dataset / LLM-as-Judge / A/B Testing)
□ Token Economics (Cost Estimation / Model Routing / Caching Strategy)
□ AI Security Basics (Injection / Jailbreak / Guardrails / Red Team Testing)

Bonus Items:
□ Fine-tuning Basics (SFT / RLHF / DPO / LoRA)
□ MLOps & AI Observability
□ GPU Economics & Inference Optimization
□ AI Governance & Compliance (EU AI Act / China Administrative Measures)
□ Multimodal AI Basics

---

Document Factory

AI Product Professional Documents

---

Diagram Factory

Must-Draw Diagrams for AI PMs

---

Prototype Factory

"Generate an HTML prototype for an AI customer service chatbot"
→ Chat interface + Confidence display + Source citations + Human handoff + Empty state guidance

"Generate an HTML prototype for an AI contract review tool"
→ Upload contract → AI flags risky clauses → User confirms/modifies → Export report

"Generate an HTML prototype for an AI data analysis agent"
→ Natural language input → Agent thinking steps display → Visualized results → Download & share

---

AI Governance & Compliance

Global AI Regulatory Landscape

EU AI Act (Phased Implementation)

"Deployer Trap": Enterprises using third-party AI APIs may also bear obligations.

China AI Regulatory System

China AI "Triple Registration": Algorithm Filing → AI Safety Assessment → Content Safety Review

---

AI Product Anti-Patterns Encyclopedia (25 Common Mistakes)

Strategy Category

Technical Category

UX Category

Security Category

Commercialization Category