Optimize Hexagon ROPE ops with DMA/VTCM#48
Conversation
Improves performance a bit by precomputing things and saving in the context.
- Modified `struct htp_rope_context` to include DMA and VTCM parameters. - Implemented `rope_job_f32` with DMA transfer logic to utilize VTCM for aligned vector operations. - Updated `execute_op_rope_f32` to calculate VTCM requirements and initialize context with proper offsets and sizes. - Added support for both contiguous and strided memory access patterns in ROPE ops. - Removed runtime alignment checks for fallback to scalar path by enforcing alignment via VTCM buffering. Co-authored-by: max-krasnyansky <1380796+max-krasnyansky@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
- Implemented `rope_job_f32` with a double-buffered DMA pipeline and prefetching. - Modified `struct htp_rope_context` to include DMA and VTCM parameters. - Updated `execute_op_rope_f32` to calculate VTCM requirements and initialize context. - Removed legacy scalar fallback and alignment checks by enforcing alignment via VTCM. - Simplified loop to process 1 row at a time (`BLOCK=1`) with efficient pipeline. Co-authored-by: max-krasnyansky <1380796+max-krasnyansky@users.noreply.github.com>
max-krasnyansky
force-pushed
the
master
branch
from
128207f to
f9d5f9d
Compare
1 participant
This change optimizes the RoPE (Rotary Positional Embedding) operation for the Hexagon backend by leveraging the Hexagon VTCM (Vector Tightly Coupled Memory) and DMA engine.
Key changes:
src0(input) anddst(output) tensors are double-buffered in VTCM. This ensures that the data processed by the vector units is always aligned toVLEN(128 bytes), allowing the use of the optimizedhvx_calc_rope_f32kernel unconditionally. Previously, unaligned pointers would force a slow scalar fallback.rope_job_f32to fetch blocks of rows from DDR to VTCM and write results back to DDR. This supports both contiguous tensors (efficient block transfer) and non-contiguous tensors (gather/scatter row-by-row).htp_rope_contextstructure was expanded to hold necessary parameters for DMA and VTCM management, such as aligned row sizes and buffer offsets.thetavalues are also stored in VTCM (theta_cache) for fast access during the rotation calculation.This optimization significantly improves performance for RoPE operations on Hexagon DSPs by maximizing vector unit utilization and memory bandwidth efficiency.
PR created automatically by Jules for task 16848115137368015121 started by @max-krasnyansky