Building Custom AI Operators with CANN TIK and TVM Training Course

Introduction to Custom Operator Development

• The rationale for custom operators: use cases and associated constraints.
• Examining the CANN runtime structure and key integration points for operators.
• An overview of TBE, TIK, and TVM within the broader Huawei AI ecosystem.

Low-Level Operator Programming with TIK

• Exploring the TIK programming model and its supported APIs.
• Strategies for memory management and tiling within TIK.
• The process of creating, compiling, and registering a custom op with CANN.

Testing and Validating Custom Operations

• Conducting unit and integration tests for ops within the graph.
• Diagnosing and resolving kernel-level performance bottlenecks.
• Visualizing operator execution flows and buffer dynamics.

Scheduling and Optimization with TVM

• Understanding TVM as a compiler for tensor operations.
• Crafting schedules for custom ops using TVM.
• Applying TVM for tuning, benchmarking, and code generation targeted at Ascend.

Integration with Frameworks and Models

• Registering custom ops for compatibility with MindSpore and ONNX.
• Ensuring model integrity and managing fallback behaviors.
• Supporting multi-operator graphs that utilize mixed precision.

Case Studies and Specialized Optimizations

• Case study: Achieving high-efficiency convolution for small input shapes.
• Case study: Optimizing memory-aware attention operators.
• Best practices for deploying custom ops across various devices.

Conclusion and Next Steps