Extropic says it has adapted transformer-like models for its probabilistic Z1 chip, an early attempt to make sparse model architecture and nontraditional silicon work together. In a September 4 technical report, the company describes Z1T as a research direction rather than a finished replacement for GPU inference.
The project starts with a mismatch between mainstream AI models and Extropic’s hardware. Transformers are usually built around dense matrix multiplication, while Z1 is a probabilistic chip with a fixed, sparse connection graph. Extropic says each Z1 chip contains 269,568 probabilistic bits, or pbits, and 2,135,904 coupling edges. Each node has degree 16, which limits how many connections can be placed directly into the silicon.
That constraint shapes Z1T. Extropic uses four-bit-style probabilistic encodings and sparse operations that can map onto the chip’s physical topology. It also changes parts of the transformer block, replacing standard normalization with a dynamic tanh operation and using gated convolutional attention instead of dense softmax attention. An FPGA handles operations that do not fit naturally on Z1, creating a hybrid path rather than a standalone accelerator.
A different tradeoff for inference
Extropic’s report separates the cost of computation from the amount of work required to achieve a given model quality. Its sparse Z1T models need roughly an order of magnitude more training FLOPs than a GPT-2 model to reach a comparable loss in the company’s extrapolation. That is a disadvantage in conventional compute terms, but Extropic argues that sparse operations can cost much less energy on hardware designed around in-memory probabilistic updates.
The company’s energy comparison is theoretical and excludes the final dense vocabulary readout. In its model, a Z1T token path uses an estimated 294.52 nan joules for the included Z1 and FPGA work. Extropic compares that with an H100 estimate that varies with utilization, reporting ratios of about 14 times at 100 percent utilization, 28 times at 50 percent, and 139 times at 10 percent. Those figures are projections based on chip-energy estimates and a small model, not measurements of a production Z1T system.
That distinction matters when reading the result alongside our look at memory bandwidth as an AI hardware bottleneck. Extropic is not claiming that a sparse probabilistic chip wins every workload. Its own analysis says dense models remain more efficient per FLOP, while the proposed advantage comes from the energy cost of each sparse operation and from matching the model to the substrate.
Extropic estimates a conservative 58.8 microseconds per token for a serial Z1T path, or about 17,000 tokens per second, excluding the final vocabulary computation. Its H100 comparison lists about 1,425 tokens per second for eager PyTorch and 9,764 for torch.compile on the same narrow, batch-one setup. The company warns that batching would make the GPU substantially more efficient and recommends the hybrid Z1 plus XPU design for decode rather than prefill at this stage.
Why the model matters as much as the chip
Z1T is also an argument for co-design. Extropic open-sourced training recipes and weights for a larger run, allowing developers to inspect how fixed connectivity, sparsity, and model size interact. Its experiments vary connectivity from four to 128 incoming links and compare those models with a dense baseline. The goal is not simply to shrink a transformer, but to learn which architectures can make a sparse physical graph useful.
For readers comparing local AI options, our NPU, GPU, and CPU guide covers the more familiar division of labor. Z1T points toward a less settled category, where the hardware may force changes in attention, precision, and deployment. Extropic’s results are promising as a design study, but the next test is whether the approach scales beyond a small research model, accounts for data movement and readout costs, and survives workloads that are not tailored to its topology.
