Skip to main content

EmbeddingGemma-2 INT8

A multimodal embedding model from Google that maps text, images, video and audio into one 768-dimensional space for cross-modal search, optimized for the Torq NPU.

Supported processors

  • SL2619

Performance

Torq Compiler/Runtime v2.3.0 · audio_1120

2188.848ms
Inference Time
377.5MB
DRAM Footprint

Torq Compiler/Runtime v2.3.0 · audio_140

326.629ms
Inference Time
350MB
DRAM Footprint

Torq Compiler/Runtime v2.3.0 · audio_280

490.968ms
Inference Time
352.5MB
DRAM Footprint

Torq Compiler/Runtime v2.3.0 · audio_560

990.621ms
Inference Time
358.5MB
DRAM Footprint

Torq Compiler/Runtime v2.3.0 · text_body_s128

416.327ms
Inference Time
194.6MB
DRAM Footprint

Torq Compiler/Runtime v2.3.0 · text_body_s512

2300.843ms
Inference Time
214.1MB
DRAM Footprint

Torq Compiler/Runtime v2.3.0 · vision_384x384

2930.231ms
Inference Time
325.6MB
DRAM Footprint

EmbeddingGemma-2

Model Overview​

EmbeddingGemma-2 is a multimodal embedding model from Google. It maps text, images, video and audio into one 768-dimensional embedding space (Matryoshka: 768, 512, 256 or 128 dimensions), so a text query can be compared directly with photos, video frames and sound clips.

EmbeddingGemma-2 has been optimized for the Synaptics Astra® SL2610-Series processors with Torq NPU. This build uses 8-bit weights (int8, block size 32) with bf16 activations, and produces embeddings close to the original model (cosine similarity 0.997 to fp32 over a media library).

Model Features​

  • Model Type: Multimodal embedding model
  • Input: Text, image (384x384), video (sampled frames), audio (16 kHz mono)
  • Output: 768-dimensional embedding (truncatable to 512, 256 or 128 dimensions and L2-normalized)

Deployment​

The compiled model files are available for download on Huggingface at Synaptics/Google-EmbeddingGemma-2.

The 8-bit build (about 2.3 GB) is on the main branch. A smaller 4-bit build (1.1 GB, lower accuracy) is on the q4 branch and a bf16 build (3.6 GB) is on the bf16 branch.

Usage Tutorial​

A usage example is provided at Torq Examples / EmbeddingGemma-2. Check out the README for instructions.

License​

Both the source model and the compiled model for on-device deployment are licensed under Apache License 2.0.

Learn More​