README.md
| 1 | --- |
| 2 | license: apache-2.0 |
| 3 | library_name: ced.cpp |
| 4 | pipeline_tag: audio-classification |
| 5 | tags: |
| 6 | - audio-classification |
| 7 | - sound-event-detection |
| 8 | - audio-tagging |
| 9 | - audioset |
| 10 | - ggml |
| 11 | - gguf |
| 12 | - ced |
| 13 | base_model: |
| 14 | - mispeech/ced-tiny |
| 15 | - mispeech/ced-mini |
| 16 | - mispeech/ced-small |
| 17 | - mispeech/ced-base |
| 18 | --- |
| 19 | |
| 20 | # CED (GGUF) for ced.cpp / LocalAI |
| 21 | |
| 22 | GGUF quantizations of the **CED** family (Consistent Ensemble Distillation, |
| 23 | Xiaomi) - SOTA-tier audio-tagging models that classify everyday sounds (baby |
| 24 | cry, footsteps, glass breaking, alarms, dog bark, ...) into the 527-class |
| 25 | [AudioSet](https://research.google.com/audioset/) ontology. |
| 26 | |
| 27 | These files run with [**ced.cpp**](https://github.com/mudler/ced.cpp), a |
| 28 | standalone C++/[ggml](https://github.com/ggml-org/ggml) port (no Python, no |
| 29 | PyTorch at inference), and with [**LocalAI**](https://github.com/mudler/LocalAI) |
| 30 | via the `ced` backend. Converted from the `mispeech/ced-*` checkpoints |
| 31 | (Apache-2.0). CED is a plain AST/DeiT Vision Transformer over a log-mel |
| 32 | spectrogram; the port is numerically equal to the PyTorch reference. |
| 33 | |
| 34 | ## Files |
| 35 | |
| 36 | One self-contained GGUF per size + quant (config, 527 labels, and the mel |
| 37 | filterbank/window are all embedded). Pick by your accuracy/size budget: |
| 38 | |
| 39 | | size | params | f16 | q8_0 | f32 | |
| 40 | |------|--------|-----|------|-----| |
| 41 | | **tiny** | 5.5M | `ced-tiny-f16.gguf` (11 MB) | `ced-tiny-q8_0.gguf` (6 MB) | - | |
| 42 | | **mini** | 9.6M | `ced-mini-f16.gguf` (19 MB) | `ced-mini-q8_0.gguf` (11 MB) | - | |
| 43 | | **small** | 22M | `ced-small-f16.gguf` (42 MB) | `ced-small-q8_0.gguf` (23 MB)| - | |
| 44 | | **base** | 86M | `ced-base-f16.gguf` (165 MB) | `ced-base-q8_0.gguf` (88 MB) | `ced-base-f32.gguf` (328 MB) | |
| 45 | |
| 46 | `tiny`/`q8_0` (6 MB) is ideal for Raspberry-Pi-class CPUs; `base`/`f16` is the |
| 47 | accuracy default. |
| 48 | |
| 49 | ## Parity vs PyTorch (ced-base, end-to-end probs) |
| 50 | |
| 51 | | quant | max abs diff | top-5 tags | |
| 52 | |-------|--------------|------------| |
| 53 | | f32 | 1.7e-7 | identical | |
| 54 | | f16 | 6.4e-5 | identical | |
| 55 | | q8_0 | 6.0e-3 | identical | |
| 56 | |
| 57 | ## Performance (CPU, ced-base, 10s clip, Ryzen 9 9950X3D, 4 threads) |
| 58 | |
| 59 | | | latency | realtime factor | peak RSS | |
| 60 | |---|---|---|---| |
| 61 | | PyTorch (transformers, f32) | 155.7 ms | 65x | 717 MB | |
| 62 | | ced.cpp f16 | 100.6 ms | 100x | 189 MB | |
| 63 | | ced.cpp q8_0 | 117.2 ms | 86x | 111 MB | |
| 64 | |
| 65 | ced.cpp f16 is ~1.55x faster than the PyTorch reference; q8_0 uses ~6.5x less |
| 66 | memory. |
| 67 | |
| 68 | ## Usage |
| 69 | |
| 70 | ```sh |
| 71 | ced-cli classify ced-base-f16.gguf clip.wav --top-k 5 |
| 72 | # 0.87 Baby cry, infant cry |
| 73 | # 0.12 Crying, sobbing |
| 74 | ``` |
| 75 | |
| 76 | In LocalAI: install the `ced` backend, configure a model with one of these |
| 77 | GGUFs, then call `POST /v1/audio/classification` (or stream over the realtime |
| 78 | websocket API for live recognition). |
| 79 | |
| 80 | ## License |
| 81 | |
| 82 | Model weights: **Apache-2.0** (© Xiaomi Corporation; from the `mispeech/ced-*` |
| 83 | checkpoints). AudioSet labels are CC-BY-4.0. The ced.cpp inference code is MIT. |
| 84 | |