Show HN：智能体式 CUDA Kernel 优化器

一个智能体式 CUDA Kernel 优化器，可通过自动化的代码生成、正确性检查、基准测试和迭代改进循环，将工作负载描述转化为 GPU 实现。该智能体由 LangGraph 驱动，会探索不同的 Kernel 实现和启动配置、查询 GPU 属性，并可检索 NVIDIA 文档以获取优化指导，同时检查 Nsight Compute 计数器，为下一次实验提供依据。每次实验都会被记录，并保留通过验证的最快实现。

模型可以同时修改 Kernel 代码以及每个测试用例的启动配置。独立的 C++ 测试框架通过 NVRTC 编译 Kernel，使用 CUDA Driver API 启动它们，并保存输出结果。Python 负责结果比较和候选方案选择。

1. 加载或生成签名、输入测试用例、参考 Kernel 和初始 Kernel。
2. 运行参考实现并评估初始实现。
3. 提出修改方案，编译后使用 NumPy 比较输出结果，并测量 Kernel 延迟。
4. 将结果反馈至下一次尝试；在迭代预算内修复无效候选方案。
5. 保存通过验证且最快的候选方案、执行历史和耗时热力图。

每个测试用例都必须通过验证。排名依据各性能测试用例延迟的几何平均值；较小的正确性测试用例不会影响得分。计时默认使用 10 次预热启动和 100 次测量启动，并通过 CUDA events 计时。编译时间和分析器重放时间不计入排名。

该项目在配备 RTX 3060 Laptop GPU 的 Windows 环境中开发。需要 Python 3.12+、NVIDIA GPU 及兼容的 CUDA Toolkit/驱动程序、CMake 3.24+、C++17 编译器和 OpenAI API key。以下构建命令使用已安装 C++ 工具的 Visual Studio 2026。

从仓库根目录运行：

python -m venv .venv
.venv\Scripts\python -m pip install -r optimizer_agent/requirements.txt
cmake -S cuda_test_harness -B cuda_test_harness/build -DCMAKE_BUILD_TYPE=Release
cmake --build cuda_test_harness/build --parallel

在仓库根目录创建 .env 文件：

OPENAI_API_KEY=your-key-here

.venv\Scripts\python optimizer_agent/optimizer_agent.py --description "Single-precision GEMM with rectangular matrices." --max-iterations 7

默认模型为 gpt-5-mini，推理强度设为中等。API 使用费用将计入你的账户。使用 -h 查看全部选项，或使用 --config optimizer_agent/example.json 加载附带的配置示例。

可通过 --signature、--reference、--initial-kernel 和 --input-cases 提供自己的工作负载组件。未提供的组件将被推断或生成。要使用之前运行保存的输入继续执行：

.venv\Scripts\python optimizer_agent/optimizer_agent.py --description "Single-precision GEMM with rectangular matrices." --input-cases results/run-001/input_cases.json --reference results/run-001/reference.cu --initial-kernel results/run-001/best.cu --max-iterations 7

此示例假定此前运行已生成 reference.cu；用户提供的参考实现会保存为 supplied-reference.cu。输入清单会保留其二进制数据的路径，因此请确保这些文件仍然可用。

可选参数：

• --use-nsight：在每个有效候选方案之后对一个性能测试用例进行分析，并向模型开放分析器工具。需要安装 Nsight Compute，并拥有访问 GPU 性能计数器的权限。
• --nvidia-research：在生成 Kernel 之前检索 NVIDIA 优化指导。

启用 NVIDIA 资料检索和 Nsight Compute 的情况下，在 RTX 3060 Laptop GPU 上运行 Float32 GEMM。

Run 022	Run 023
优化生成的起始 Kernel。	使用相同输入和参考实现，从 Run 022 的最佳 Kernel 继续优化。

每个会话都会在 results/run-NNN/ 下生成一个目录，其中包含 Kernel 源码、请求、输入/输出数据、模型/工具响应、history.json 和 summary.json。成功的运行会导出 best.cu、每个测试用例的重放请求，以及 heatmap.png/heatmap.svg。启用性能分析时，还会保存 Nsight 报告。

这是一个面向单个 Kernel 的实验性优化器。通过所提供的测试用例并不能证明其具备普遍正确性，而自动生成的参考实现也不是独立的正确性判定依据。性能改进取决于具体工作负载；目前未包含与 cuBLAS 或其他厂商库的对比。比较计时时，请保持 GPU 不运行其他任务。

生成的输入脚本会作为 Python 子进程在本地执行，且不受沙箱保护。生成的 CUDA Kernel 也会在本地 GPU 上运行。

下图由启用 Nsight 性能分析的已编译 LangGraph 工作流渲染而成。虚线边表示条件路由。不使用 --use-nsight 时，评估会直接路由至下一次尝试或最终收尾；除非设置 --nvidia-research，否则会跳过 NVIDIA 资料检索。

An agentic CUDA kernel optimizer that turns workload descriptions into GPU implementations through an automated cycle of code generation, correctness checks, benchmarking, and refinement. Powered by LangGraph, the agent explores kernel implementations and launch configurations, queries GPU properties, and can research NVIDIA documentation for optimization guidance and inspect Nsight Compute counters to inform its next experiment. Each experiment is recorded, and the fastest validated implementation is retained.

The model can change both kernel code and per-case launch configurations. A standalone C++ harness compiles kernels with NVRTC, launches them through the CUDA Driver API, and saves outputs. Python handles comparison and candidate selection.

1. Load or generate a signature, input cases, reference kernel, and initial kernel.
2. Run the reference and evaluate the initial implementation.
3. Propose a change, compile it, compare outputs with NumPy, and measure kernel latency.
4. Feed results back into the next attempt; repair invalid candidates within the iteration budget.
5. Save the fastest validated candidate, execution history, and a timing heatmap.

Every case must pass validation. Ranking uses the geometric mean of latency across performance cases; small correctness cases do not affect the score. Timing defaults to 10 warmup launches and 100 measured launches using CUDA events. Compilation time and profiler replay timings are excluded from ranking.

Developed on Windows with an RTX 3060 Laptop GPU. Requires Python 3.12+, an NVIDIA GPU and compatible CUDA Toolkit/driver, CMake 3.24+, a C++17 compiler, and an OpenAI API key. The build commands below use Visual Studio 2026 with the C++ tools installed.

From the repository root:

python -m venv .venv
.venv\Scripts\python -m pip install -r optimizer_agent/requirements.txt
cmake -S cuda_test_harness -B cuda_test_harness/build -DCMAKE_BUILD_TYPE=Release
cmake --build cuda_test_harness/build --parallel

Create a .env file in the repository root:

OPENAI_API_KEY=your-key-here


.venv\Scripts\python optimizer_agent/optimizer_agent.py --description "Single-precision GEMM with rectangular matrices." --max-iterations 7

The default model is gpt-5-mini with medium reasoning effort. API usage is billed to your account. Use -h for all options, or --config optimizer_agent/example.json for the included configuration example.

Supply your own workload components with --signature, --reference, --initial-kernel, and --input-cases. Omitted components are inferred or generated. To continue from an earlier run using its saved inputs:

.venv\Scripts\python optimizer_agent/optimizer_agent.py --description "Single-precision GEMM with rectangular matrices." --input-cases results/run-001/input_cases.json --reference results/run-001/reference.cu --initial-kernel results/run-001/best.cu --max-iterations 7

This example assumes the earlier run generated reference.cu; supplied references are saved as supplied-reference.cu. Input manifests retain paths to their binary data, so keep those files available.

Optional flags:

• --use-nsight: profile a performance case after each valid candidate and expose profiler tools to the model. Requires Nsight Compute and permission to access GPU performance counters.
• --nvidia-research: retrieve NVIDIA guidance before generating kernels.

Float32 GEMM on an RTX 3060 Laptop GPU, with NVIDIA research and Nsight Compute enabled.

Run 022	Run 023
Optimizing the generated starting kernel.	Continuing from run 022's best kernel with the same inputs and reference.

Each session gets a directory under results/run-NNN/ containing kernel sources, requests, input/output data, model/tool responses, history.json, and summary.json. Successful runs export best.cu, per-case replay requests, and heatmap.png/heatmap.svg. Nsight reports are saved when profiling is enabled.

This is an experimental optimizer for individual kernels. Passing supplied cases does not prove general correctness, and a generated reference is not an independent correctness oracle. Improvements are workload-dependent; no comparison against cuBLAS or other vendor libraries is currently included. Keep the GPU otherwise idle when comparing timings.

Generated input scripts execute locally as Python subprocesses without a sandbox. Generated CUDA kernels also run on the local GPU.

The graph below is rendered from the compiled LangGraph workflow with Nsight profiling enabled. Dashed edges are conditional routes. Without --use-nsight, evaluation routes directly to the next attempt or finalization; NVIDIA research is skipped unless --nvidia-research is set.