Why split an on-device agent into a host and a coprocessor?

An agent has to keep a large model running, and it also has to maintain the KV Cache, tool calls, and system response. The 4 TOPS built into RK3572 is enough for vision inference, but it cannot carry a 7B-class language model, and the host still has to manage the screen, peripherals, and orchestration. RK1828 takes LLM inference off the host.

What are the compute, memory, and bandwidth of RK1828?

The NPU is 20 TOPS (INT8), and it supports INT4, INT8, INT16, FP8, FP16, and BF16. The package integrates 5GB of high-bandwidth DRAM, with bandwidth up to 1 TB/s. There is also a 3-core RISC-V 64-bit processor on chip. It was first launched in July 2025 and goes into mass production in 2026.

How does a 7B model run on RK1828?

W4A16 is recommended. An INT8 7B exceeds 5GB and needs weight sharding, which lowers speed. In the standard test, Qwen2.5-7B (W4A16) has a first token of about 161 ms and generation of about 59 tokens/s, about 2.4–3.3 times a 7B model on the built-in NPU of RK3588 (18–25 tokens/s).

What is the difference between RK3572 + RK1828 and RK3576 + RK1828?

The large model is carried by RK1828 in both cases. RK3576 has 6 TOPS built in, and combined compute after the coprocessor is attached is often described as 6+20. RK3572 is 4 TOPS, with 2 A73 big cores, and the host costs less. Violoop uses RK3576 + RK1828, not RK3572.

What if 5GB on one card cannot hold a larger model?

Up to 4 cards can be cascaded over PCIe, with pipeline parallelism across Transformer layers. 2 cards can run Qwen3.5-9B and gemma-4-12B-it. 4 cards can run Qwen3.5-27B, Qwen3.8-27B, and gemma-4-31B-it. 4 cards at full load are about 40W. RK3572 has only three PCIe 2.1 links, so confirm the lanes before using multiple cards, and note that PCIe and SATA are multiplexed.

RK3572 + RK1828 Host and Coprocessor Architecture | On-Device AI Agents

An on-device agent has to keep a large language model running, and it still has to keep the screen, peripherals, and tool calls responsive. The 4 TOPS on RK3572 cannot carry 7B-class inference. Rockchip's approach is to split the host from a dedicated coprocessor: RK3572 schedules, and RK1828 runs inference.

Diagram: an RK3572 host connects to RK1828 over PCIe 2.1. The host handles the system, screen, peripherals, and 4 TOPS vision preprocessing. The coprocessor provides 20 TOPS, 5GB of on-chip DRAM, and bandwidth up to 1 TB/s. — Figure 1. Dual-chip split. The host sends inference requests to RK1828 over PCIe 2.1 ×1 (5 Gbps per lane), and results return to the host. Numbers in the figure come from this article and are not a separate retest.

1. Why the architecture was proposed: an on-device agent needs the two chips decoupled

On-device AI agents ask something different of the hardware than a traditional one-shot inference. The agent has to keep a large language model running for multi-turn dialogue, maintain the KV Cache, and handle context assembly and I/O for tool calls. The bottleneck is not only peak compute. It is whether memory bandwidth and compute can be supplied continuously.

The 4 TOPS ceiling

The built-in NPU on RK3572 is 4 TOPS (INT8). That is enough for object detection and image classification, but it is hard for it to support a usable inference speed on a 7B-class model.

The host cannot be filled by inference

The host still has to run the system, capture the screen, orchestrate tasks, and control peripherals. If the LLM occupies the host NPU and memory bandwidth, system response and inference quality squeeze each other.

Compute decoupling separates the host SoC and the dedicated AI coprocessor at the physical layer and at the task layer. The host only schedules and does general compute. All LLM inference goes to the coprocessor. RK1828 is the on-device AI coprocessor designed for this architecture.

2. RK1828 coprocessor specifications

2.1 Core architecture: 3D-stacked DRAM breaks the memory wall

RK1828 is the flagship of Rockchip's RK182X series. It was launched worldwide in July 2025 and goes into mass production in 2026. It builds 5GB of high-bandwidth DRAM directly into the package, with memory bandwidth up to 1 TB/s.

Memory-bandwidth comparison: LPDDR4x about 34.1 GB/s, LPDDR5 about 51.2 GB/s, RK1828 on-chip 3D DRAM up to 1 TB/s. The horizontal axis is a logarithmic scale. — Figure 2. External DDR and on-chip DRAM. The horizontal axis uses a logarithmic scale so that 34.1 GB/s and 1 TB/s can be drawn on the same chart. 1 TB/s is counted as 1000 GB/s.

In a conventional design, model weights are read from external DDR. Theoretical bandwidth is about 34.1 GB/s for LPDDR4x and about 51.2 GB/s for LPDDR5. A 7B model walks billions of parameters for every token it generates, and external DDR is not enough. Take a 7B model on the built-in NPU of RK3588: inference speed is about 18–25 tokens/s, and it stalls easily under multiple tasks. RK1828 puts DRAM at 1 TB/s of bandwidth in the same package as the NPU and bypasses that path, so data-access latency is much lower.

2.2 NPU compute and precision

Item	RK1828
NPU	20 TOPS (INT8)
Precision	INT4, INT8, INT16, FP8, FP16, BF16
On-chip processor	3-core RISC-V 64-bit, for scheduling inside the coprocessor
On-chip DRAM	5GB, bandwidth up to 1 TB/s
Recommended 7B quantization	W4A16 (4-bit weights, 16-bit activations)
INT8 7B	Exceeds 5GB, needs weight sharding, and speed drops sharply

Mixed precision decides whether a 7B model fits in 5GB. W4A16 is a comparatively stable pairing of capacity and accuracy. An INT8-quantized 7B crosses the 5GB boundary.

2.3 Hardware forms and interfaces

Two RK1828 forms: the M.2 2280 module uses Key B-M, PCIe 2.1 ×1, and DC 12V. The SO-DIMM module has gold fingers and matches the Jetson Nano/NX size. Both have the same compute and 5GB DRAM. — Figure 3. Two standard forms. NPU compute and DRAM capacity are the same. The difference is how they are installed.

M.2 2280 module: M.2 Key B-M, communicating with the host over PCIe 2.1 ×1 lane, external DC 12V. It suits industrial PCs, single-board computers, and evaluation boards that have an M.2 Key slot, and it is more compact.
SO-DIMM module: gold fingers, compatible with the Jetson Nano/NX size, connected to the host over PCIe. It is easier to attach on a traditional embedded board.

2.4 Power and cooling

RK1828 power: typical 7 to 10 watts, dynamic frequency scaling averages about 7.5 watts with a steady state of about 53 degrees Celsius, and the peak under sustained 7B load is about 29.4 watts. — Figure 4. Power and temperature. Under dynamic frequency scaling, NPU frequency is adjusted between 250–1000 MHz.

Typical power is about 7–10W. Peak power under a sustained 7B load is about 29.4W. Measured average power with ondemand dynamic frequency scaling is about 7.5W, and steady-state temperature is about 53°C. Passive cooling is enough for basic operation. A small fan is recommended for sustained high load. The thermal margin of a fanless edge device is sufficient, but a long full load still needs an air path.

3. How well RK3572 fits as the host

3.1 Interfaces

RK3572 has three PCIe 2.1 controllers, up to 5 Gbps per lane, and it integrates SATA 3.1, USB 3.0, dual Gigabit Ethernet, and DSMC. PCIe 2.1 is the basis of the data path between it and RK1828.

In the RKNN3 SDK, RK3572 can run as an inference platform by itself, with no coprocessor attached. It can also connect to RK1828 over PCIe and enter coprocessor mode. The two paths are configured separately in the SDK.

3.2 How tasks are split

Task split: RK3572 handles the system, peripherals, 4 TOPS vision preprocessing, and tool-call assembly. RK1828 handles LLM, VLM, ASR, and TTS, with weights kept in 5GB of on-chip DRAM. — Figure 5. Resource isolation. Vision detection and large-model dialogue can run at the same time, and no longer compete for the same NPU and memory bandwidth.

The role of RK3572 in this architecture is the same as when RK3576 or RK3588 is the host:

System: Android or Linux, the application framework, and the file system
Screen capture and task orchestration: screen capture, input events, and workflow scheduling
Peripherals: USB, Ethernet, CAN FD, RS485, UART
Light AI: 4 TOPS for object detection, face detection, and light models
Pre- and post-processing: image normalization, the tokenizer, and JSON and context assembly for tool calls

RK1828 is dedicated to heavy tasks such as LLM/VLM, ASR, TTS, Transformers, and vision models. In a single-chip design, the LLM keeps occupying the NPU and memory bandwidth, and screen capture, codec work, and peripherals start to jitter. After the split, the two resource pools do not occupy each other.

3.3 How it differs from RK3576 and RK3588 host designs

RK3576 has a built-in 6 TOPS NPU and an octa-core CPU (4×A72 + 4×A53), and it can do some LLM inference on its own. After RK1828 is attached, combined compute is often written as 6+20=26 TOPS. RK3572 is 4 TOPS, with big cores of only 2×A73 + 6×A53, and both single-core performance and the NPU are below RK3576.

In the dual-chip architecture, the LLM is entirely on RK1828. The host NPU and CPU are used for scheduling, preprocessing, and light inference, and RK3572 is enough for that. The value of this combination is cost: a less expensive host runs the system, and the compute budget sits on the coprocessor. An entry-level AI box or a light intelligent terminal that does not need the RK3576 CPU tier, and that does need an on-device LLM, can take this path. For a fuller comparison of the three hosts, see the RK3572 / RK3576 / RK3588 specification comparison.

4. The RKNN3 software toolchain and deployment

4.1 SDK

Rockchip provides the RKNN3 SDK for RK1820/RK1828. V1.0.0, the formal release, came out in March 2026. It covers the PC development kit, the on-board runtime API, and model-conversion examples, and it supports Android and Linux.

RKNN3 flow: RKNN3-Toolkit on a PC converts and quantizes the model into an RKNN model. Runtime or Toolkit Lite then runs it on the board, and rkllm3-server provides an OpenAI-compatible interface. — Figure 6. From a framework model to an on-board service. Conversion is done on a PC. Inference is done on the board.

RKNN3-Toolkit: model conversion, quantization, and accuracy analysis on a PC
RKNN3 Runtime: on-board inference runtime, C API
RKNN3-Toolkit Lite: a light Python toolkit on the board
rkllm3-server: an OpenAI-compatible model-service interface

The conversion flow is: on a PC, use RKNN3-Toolkit to turn TensorFlow, PyTorch, and ONNX models into RKNN and quantize them, then deploy to the board with Runtime or Toolkit Lite.

4.2 How coprocessor mode communicates

In coprocessor mode, communication between the host and RK1828 is managed by rknn3_transfer_proxy. The host sends the inference request and the input to the coprocessor over PCIe, and sends the result back after the compute finishes. rknn3_transfer_proxy devices can confirm whether recognition succeeded. A device ID looks like 0000:01:00.0 b98e6c51 PCIE.

RKNN3 SDK V1.0.0 supports overlapping data transfer and inference: while this round is still computing, the next batch of input can be prepared, which reduces PCIe waiting.

4.3 Multi-card cascade

Four RK1828 cards relay a pipeline. 4 cards can run Qwen3.5-27B, Qwen3.8-27B, and gemma-4-31B-it. 2 cards can run Qwen3.5-9B and gemma-4-12B-it. 4 cards at full load are about 40 watts. — Figure 7. When 5GB on one card is not enough, the model is cut at Transformer-layer boundaries, with up to 4 cards in relay. Total power of 4 cards at full load is about 40W.

Larger models can be cascaded across 4 cards over PCIe. The RKNN3 SDK uses pipeline parallelism: the model is cut at Transformer-layer boundaries, each RK1828 runs one segment, and the cards relay the full inference.

Adapted setups include Qwen3.5-27B, Qwen3.8-27B, and gemma-4-31B-it on 4 cards, and Qwen3.5-9B and gemma-4-12B-it on 2 cards. Total power of 4 cards at full load is held at about 40W.

5. Measured performance

5.1 LLM inference

Test conditions: RKNN3 SDK V1.0.0, Input Tokens = 128, New Tokens = 128.

Model	Parameters	TTFT (first token)	Decode
Qwen2.5-0.5B	0.5B	21.89 ms	215.86 tokens/s
Qwen2.5-3B	3B	—	102.01 tokens/s
Qwen2.5-7B (W4A16)	7B	161 ms	59 tokens/s
Qwen3-8B	8B	—	61.11 tokens/s

Decode speed: Qwen2.5-0.5B is 215.86, Qwen2.5-3B is 102.01, Qwen2.5-Omni-3B is 102.63, Qwen3-VL-4B is close to 90, Qwen3-8B is 61.11, Qwen2.5-7B W4A16 is 59, Qwen2.5-VL-7B is 58.94, and a 7B model on the RK3588 built-in NPU is 18 to 25 tokens/s. — Figure 8. Generation speed on one scale. The source text for Qwen3-VL-4B is "close to 90," and the chart draws it as 90. Amber is the comparison upper bound for the RK3588 built-in NPU, not an RK1828 result.

Decode at the 3B size is above 100 tokens/s, and Qwen2.5-3B is 102.01. TTFT for 0.5B is only 21.89 ms, TPOT is 4.63 ms, and Decode is 215.86 tokens/s. Under W4A16, 7B has a TTFT of about 161 ms and generation of about 59 tokens/s, which is usable for everyday chat, summaries, and instruction calls. A 7B model on the built-in NPU of a pure RK3588 is about 18–25 tokens/s. A single RK1828 card is about 2.4–3.3 times that.

5.2 VLM and multimodal models

LLM Decode for Qwen3-VL-4B is close to 90 tokens/s. TTFT for Qwen2.5-VL-7B is about 161 ms, and generation is about 58.94 tokens/s. Qwen2.5-Omni-3B runs the full audio, vision, and language path on RK1828, decoding at 102.63 tokens/s. Time stays steady at a 392×392 vision resolution, and audio inference is 98.91 ms.

5.3 CNN vision models

YOLOv5s inference time on RK1828 stays within 31 ms, and CPU usage is low. The RKNN3 SDK also optimizes multi-batch and multi-core inference. In multi-core mode the frame rate can increase by multiples, which suits intelligent monitoring and industrial inspection.

6. Typical application cases

Violoop: a screen-aware agent

This device uses an RK3576 host + RK1828, not RK3572. On Kickstarter it raised more than USD 1 million within 20 hours. RK3576 handles the system, screen capture, and orchestration. The 20 TOPS on RK1828 runs the local large model. The device sits between a computer and a display over HDMI and USB. It reads the screen, understands it, waits for the user to confirm, and then acts across applications over USB-HID.

Offline office agent

The offline local AI office agent from Rockchip and Yici Wanglian carries RK1828. The dual-chip architecture can support dense models up to 27B. Models, files, and conversations stay local. It can connect to existing collaborative documents, a private network drive, and OA, and it has been deployed in some provinces.

Vehicles and robotics. A vehicle AI BOX based on RK3576M + RK1828 runs Qwen3.5-Omni and fuses in-cabin voice, vision, and sensing locally, with data staying on the device. On the robotics side, RK182X can already run agents that understand a graphical interface, interact locally without a wake word, and operate across applications and across systems.

7. Deployment points and engineering constraints

7.1 Memory boundary

5GB on one card is a hard boundary. A W4A16 7B fits in that 5GB. Anything larger needs multiple cards, up to 4, and 4 cards correspond to 27B–31B. Set the memory budget when selecting the design, so inference does not have to shard on the fly.

7.2 Cooling

The M.2 form can run stably under passive cooling, typically about 10W, with a dynamic-frequency-scaling steady state of about 53°C. For long multi-turn dialogue or high-concurrency vision inference, leave an air path or add a small fan.

7.3 PCIe link

RK3572 has three PCIe 2.1 links, 1 lane each. One RK1828 takes one link, and the rest can connect NVMe or Wi-Fi. Confirm that there are enough lanes before using multiple cards. PCIe0 and SATA0 are multiplexed, and that has to be decided at the schematic stage.

7.4 Software ecosystem

The RKNN3 SDK is already adapted for Qwen2.5, Qwen3, Qwen3.5, Gemma4, and DeepSeek, and it supports mRoPE and Function Call. A custom model has to be quantized and have its operators converted with RKNN3-Toolkit. When selecting a model, verify first that the operators can pass.

8. Architecture summary

The architecture can be closed in one sentence: use the host to keep system behavior deterministic, and the coprocessor to keep inference deterministic.

The 4 TOPS and 2×A73 + 6×A53 of RK3572 are a stable back end inside the dual-chip design. Screen capture, peripherals, orchestration, and preprocessing are pulsed and predictable. The 20 TOPS and 1 TB/s on-chip bandwidth of RK1828 are a continuous inference engine, with long loads that consume bandwidth. After the two kinds of load are separated physically, latency jitter gets smaller, and the agent is no longer held back by the host's everyday tasks.

Looking at the 2026 product cadence, Violoop, the vehicle AI BOX, and the offline office agent show that "host + coprocessor" is already being shipped. Rockchip treats this route as a long-term chip direction, and RK182X uses 3D-stacked DRAM to solve bandwidth. When an on-device LLM is required and cost, power, or cooling is constrained, RK3572 + RK1828 is an engineering path that handles compute, energy efficiency, and system response separately.

Related reading

For how to choose when the host runs the model itself, see the RK3572 / RK3576 / RK3588 specification comparison and AI agent deployment. This article discusses the case where the host does not carry the LLM alone and hands inference to RK1828.

Host + Coprocessor: In-Depth Technical Analysis of the RK3572 + RK1828 On-Device AI Agent Architecture