Imec Says AI Inference Pushes Optical I/O Closer To Chips

Imec researchers say AI inference is pushing the optical I/O roadmap closer to processors, because large models now need faster movement of data between accelerators as well as more compute.

The AI inference optical I/O roadmap described by imec starts with co-packaged optics, and the research group says future systems may need 2.5D and 3D integration to control bandwidth, latency and power.

Imec Projects 250 Tb/s Bandwidth Demand For Future AI Processors

Peter Ossieur, portfolio director at imec and professor at Ghent University, told EE Times that inference needs to be reactive because it is the moment when users interact with a model.

EE Times said those workloads can combine extended prompts, retrieval tools, reasoning chains, multimodal inputs and agent-style systems.

Imec described scale-up networking as the more difficult domain.

Scale-out networks connect racks and systems across a data centre, while scale-up links GPUs or other accelerators tightly enough that applications can treat them as one larger processor.

Today's scale-up fabric is often copper-based and mostly constrained to the rack, according to imec.

That model becomes harder as AI systems move towards hundreds, thousands or tens of thousands of accelerators working together.

Co-Packaged Optics Is Not The End Of Imec's Roadmap

Imene Jadli, portfolio manager for optical interconnect at imec, told the report that co-packaged optics is a logical next step and not the end of the roadmap.

Co-packaged optics moves optical engines into the package and shortens the electrical path between the optical module and the processor or switch.

The power limit is the main constraint in the figures discussed by imec.

One future-system projection used by imec puts processor input and output bandwidth at roughly 250 Tb/s.

Anticipated co-packaged optics approaches could require about 1.25 kW for the optics alone, on top of processors that may already consume several kilowatts.

Ossieur told the report that imec needs an approach that is better than co-packaged optics.

The research group's proposed path is 2.5D optical I/O, where optics is integrated at the interposer or substrate level to shorten the electrical path and lower the power needed to move data.

2.5D Optical I/O Could Cut Optical Power Below 200 W

Ossieur said the same imec projection showed 2.5D optical I/O reducing optical power from about 1.25 kW to below 200 W. Imec describes the approach as "wide and slow", using many lanes at moderate speeds instead of a smaller number of very high-speed lanes with heavier signal processing.

The materials work remains broad.

Co-packaged optics still needs compact, efficient and high-bandwidth devices, including electro-absorption modulators and high-speed photodetectors.

Imec is exploring barium titanate and III-V compounds alongside silicon photonics because silicon remains attractive for CMOS manufacturing and may not provide every property needed for future optical I/O.

Imec's longer-term endpoint is 3D optical I/O, where optics becomes part of the 3D compute stack.

That roadmap could move optical links into advanced packages for processor-to-processor and memory-to-processor data movement, including high-bandwidth memory and stacked processors.

A commercial product timetable, named AI-system customers, manufacturing-yield data, optical-coupling results and cooling test results for 3D optical I/O remain outside the public record.