README.md
21.1 KB · 398 lines · markdown Raw
1 ---
2 license: apache-2.0
3 license_link: https://ai.google.dev/gemma/docs/gemma_4_license
4 language:
5 - en
6 base_model:
7 - google/gemma-4-26B-A4B-it
8 base_model_relation: quantized
9 tags:
10 - telecom
11 - telecommunications
12 - open-telco-ai
13 - classification
14 - typed-decisions
15 - decision-engine
16 - system-one
17 - gemma4
18 - mixture-of-experts
19 - awq
20 - w4a16
21 - compressed-tensors
22 - 4-bit
23 - amd
24 - rocm
25 - nvidia
26 - cuda
27 - h100
28 - vllm
29 - gguf
30 - llama.cpp
31 pipeline_tag: text-generation
32 ---
33
34 # OTel-2.0-Decision-26B-A4B-4bit
35
36 > **Versioning notice:** This card describes the current OTel-2.0-Decision-26B-A4B-4bit checkpoint. The model weights may be updated over time. For reproducible evaluation or production deployment, pin a specific model revision, checkpoint hash, or release tag rather than relying on the floating latest version.
37
38 Details on the training data, training method and configuration, data-processing pipeline, and training infrastructure will be added soon.
39
40 **OTel-2.0-Decision-26B-A4B-4bit** is a 4-bit W4A16 build of Google's Gemma 4 26B-A4B-it for typed decisions: classification, Yes/No questions, and ordered levels. Questions are supplied at request time, and decisions are read from answer-label probabilities rather than generated prose.
41
42 ## Intended Use
43
44 Classify messages, call transcripts, tickets, and documents, or ask multiple questions about the same text. Use label probabilities for confidence thresholds and routing. This release is not intended for open-ended chat, retrieval, or reranking.
45
46 ## Evaluation
47
48 Benchmarks use the typed-decision readout (`level=L0`, no prior, shared-prefix execution, adaptive Yes/No ordering with margin 4.0). All runs reported zero errors and `answer_mass` of 1.00.
49
50 | Benchmark | AMD MI350P | AMD Ryzen AI MAX+ PRO 395 (Strix Halo) |
51 |---|---:|---:|
52 | JevBench public (231 tasks) | **86.58% to 87.45%** (200 to 202 of 231) | **87.45%** (202/231) |
53
54 JevBench by tier: 100% on easy, 94.44% to 95.83% on original, and 74.77% to 76.58% on hard tasks. The MoE kernel is not bit-deterministic, so results vary on a few low-margin tasks across runs. The Strix Halo run used vLLM 0.26.0 on ROCm 7.13 (48/48 easy, 71/72 original, 83/111 hard).
55
56 Use a system message that asks for a single label (see *Prompting and Label Probabilities*). Without one, the model sometimes answers hard tasks in prose instead of a label, which lowers hard-tier accuracy by about 6 points.
57
58 ## Inference Recipe
59
60 This repository holds the same model in two formats. Pick the recipe that matches your engine:
61
62 | Weights | Engine | API | Section |
63 |---|---|---|---|
64 | Safetensors, W4A16 (repository root) | vLLM 0.26 or 0.30 on ROCm or CUDA | `/v1/chat/completions` with logprobs | Options 1 and 2, *NVIDIA H100*, *Strix Halo* |
65 | Safetensors, W4A16 (repository root) | vLLM nightly | `/v1/systemone` | *System One API* |
66 | GGUF Q8_0 or Q4_K_M (`gguf/`) | llama.cpp `llama-server` (Vulkan, CUDA and other backends) | `/v1/chat/completions` with logprobs | *Inference with llama.cpp* |
67 | GGUF Q8_0_systemone or Q4_K_M_systemone (`gguf/`) | llama.cpp `llama-server` | `/v1/systemone` | *`/v1/systemone` with llama.cpp* |
68
69 Serve on one AMD MI350P with vLLM 0.30.0. Both options use the same weights and API. For NVIDIA GPUs, see *NVIDIA H100: vLLM on CUDA* below.
70
71 | Option | Image | Throughput |
72 |---|---|---:|
73 | 1. Official vLLM | `vllm/vllm-openai-rocm:v0.30.0` | 22.7k tok/s |
74 | 2. Prebuilt image | `farbodatdocker/otel-decision-vllm-rocm:v0.30.0` | 32.3k tok/s (42% more) |
75
76 Throughput figures are peak aggregate output tokens per second. The highest throughput is reached only when the GPU runs near maximum utilization, with many concurrent requests in flight. A single request runs much slower, so batch or parallelize requests to reach these numbers.
77
78 ### Download
79
80 ```bash
81 pip install -U huggingface_hub
82 hf download farbodtavakkoli/OTel-2.0-Decision-26B-A4B-4bit \
83 --exclude "gguf/*" --local-dir ./OTel-2.0-Decision-26B-A4B-4bit
84 ```
85
86 `--exclude "gguf/*"` skips the four GGUF files (87.3 GB), which only `llama-server` needs.
87
88 ### Option 1: Official vLLM
89
90 ```bash
91 docker run --rm --name otel_decision \
92 --device /dev/kfd --device /dev/dri --group-add video \
93 --ipc=host --security-opt seccomp=unconfined -p 8000:8000 \
94 -v "$PWD/OTel-2.0-Decision-26B-A4B-4bit:/model:ro" \
95 vllm/vllm-openai-rocm:v0.30.0 \
96 /model \
97 --served-model-name OTel-2.0-Decision-26B-A4B-4bit \
98 --max-model-len 16384 \
99 --enable-prefix-caching \
100 --max-logprobs 64 \
101 --port 8000
102 ```
103
104 With vLLM 0.30.0 already installed, the equivalent is:
105
106 ```bash
107 vllm serve ./OTel-2.0-Decision-26B-A4B-4bit \
108 --served-model-name OTel-2.0-Decision-26B-A4B-4bit \
109 --max-model-len 16384 --enable-prefix-caching --max-logprobs 64 --port 8000
110 ```
111
112 ### Option 2: Prebuilt Image
113
114 The official image with a corrected Gemma 4 MoE kernel and MI350P tile configuration.
115
116 ```bash
117 docker run --rm --name otel_decision \
118 --device /dev/kfd --device /dev/dri --group-add video \
119 --ipc=host --security-opt seccomp=unconfined -p 8000:8000 \
120 -v "$PWD/OTel-2.0-Decision-26B-A4B-4bit:/model:ro" \
121 farbodatdocker/otel-decision-vllm-rocm:v0.30.0 \
122 /model \
123 --served-model-name OTel-2.0-Decision-26B-A4B-4bit \
124 --max-model-len 16384 \
125 --enable-prefix-caching \
126 --max-logprobs 64 \
127 --moe-backend flydsl \
128 --port 8000
129 ```
130
131 Use `--moe-backend flydsl` only with Option 2. The first start takes 4 to 10 minutes.
132
133 ### System One API (`/v1/systemone`)
134
135 vLLM serves this model on its System One endpoint with no conversion. One request carries a state and any number of named questions (choice, Yes/No, score), and the reply gives a probability for every option. The endpoint is on vLLM main and in nightly builds, not yet in a release. Tested on one NVIDIA H100 80GB with `0.33.0rc1.dev613+g44965a109`, the vLLM main build of commit `44965a1`; the command below installs exactly that build. A newer nightly or release that includes `/v1/systemone` should also work: replace the pinned install line with `uv pip install -U vllm --torch-backend=auto --extra-index-url https://wheels.vllm.ai/nightly`.
136
137 ```bash
138 uv venv --python 3.12 --seed && source .venv/bin/activate
139 uv pip install "vllm==0.33.0rc1.dev613+g44965a109" --torch-backend=auto \
140 --extra-index-url https://wheels.vllm.ai/44965a10974ee417f6f937eee207409462b2a0ab
141
142 vllm serve farbodtavakkoli/OTel-2.0-Decision-26B-A4B-4bit \
143 --served-model-name OTel-2.0-Decision-26B-A4B-4bit \
144 --max-model-len 16384 --enable-prefix-caching \
145 --logprobs-mode raw_logprobs --port 8000
146 ```
147
148 `--logprobs-mode raw_logprobs` is required. With prefix caching, the state is prefilled once and each question only adds its own tokens.
149
150 ```bash
151 curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
152 "model": "OTel-2.0-Decision-26B-A4B-4bit",
153 "instructions": "You are a decision function. You will be given a state and one question. Reply with the answer label only: no words, no punctuation, no explanation.",
154 "state": "Caller: There is a 14 dollar charge on my bill I do not recognise. I want it refunded. I already rebooted the router twice.",
155 "questions": {
156 "refund_requested": {"type": "noul", "instructions": "Did the caller explicitly ask for a refund or credit?"},
157 "route": {"type": "choice", "instructions": "Which team should handle this call?",
158 "criteria": {"billing": "Charges, refunds and payments", "technical": "Internet, TV and equipment faults", "sales": "New plans and upgrades"}},
159 "urgency": {"type": "score", "instructions": "How urgent is this call?", "criteria": ["Routine", "Soon", "Today"]}
160 }
161 }'
162 ```
163
164 Each answer has `probabilities` for every option and `confidence`; a Yes/No answer's `noul` is the probability of yes. `diagnostics.<id>.label_mass` is the probability the model put on the answer labels and should be close to 1.0. Pass the system prompt in `instructions` as above: without it, `label_mass` falls close to zero on some hard multiple choice questions.
165
166 | Measured on `/v1/systemone` (one request at a time) | Accuracy | Latency mean / median / p95 |
167 |---|---:|---:|
168 | JevBench public, 231 tasks, one question per request | 85.28% (197/231) | 0.030 / 0.019 / 0.064 s |
169 | Synthetic call transcripts, 150 transcripts with 27 Yes/No questions each, one request per transcript | 99.80% | 0.114 / 0.112 / 0.148 s per transcript |
170
171 JevBench is lower than in *Evaluation* above because the endpoint writes its own prompt and reads each question once, without the adaptive Yes/No ordering.
172
173 ### Inference with llama.cpp (GGUF)
174
175 The `gguf/` folder holds two GGUF files for `llama-server` chat completions, made from the BF16 weights with llama.cpp's own converter. Use llama.cpp release v0.9.0 or newer.
176
177 | File | Size | When to use |
178 |---|---:|---|
179 | `OTel-2.0-Decision-26B-A4B-Q8_0.gguf` | 26.9 GB | Recommended: about the same accuracy as the safetensors |
180 | `OTel-2.0-Decision-26B-A4B-Q4_K_M.gguf` | 16.8 GB | Less memory, about 2.6 points lower on JevBench |
181
182 ```bash
183 llama-server -hf farbodtavakkoli/OTel-2.0-Decision-26B-A4B-4bit:Q8_0 \
184 -ngl 999 -c 16384 --jinja --host 0.0.0.0 --port 8080
185 ```
186
187 `-hf` downloads only the chosen GGUF, not the safetensors; use `:Q4_K_M` for the smaller file. Get `llama-server` for Vulkan from the llama.cpp releases page, or build it with `cmake -B build -DGGML_VULKAN=ON && cmake --build build -j` (needs the Vulkan SDK, or `libvulkan-dev` and `glslc`); the binary is `build/bin/llama-server`. To use a downloaded file, replace `-hf ...` with `-m OTel-2.0-Decision-26B-A4B-Q8_0.gguf`.
188
189 Read the label probabilities from `/v1/chat/completions`, as in *Prompting and Label Probabilities* below, with `max_tokens` 1, `temperature` 0, `top_logprobs` 64 and `"chat_template_kwargs": {"enable_thinking": false}`. For `/v1/systemone`, use the `_systemone` files below.
190
191 ```bash
192 curl -s localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{
193 "messages": [
194 {"role": "system", "content": "You are a decision function. You will be given a state and one question. Reply with the answer label only: no words, no punctuation, no explanation."},
195 {"role": "user", "content": "State:\nCaller: There is a 14 dollar charge on my bill I do not recognise. I want it refunded.\n\nQuestion: Did the caller explicitly ask for a refund or credit?\nAnswer Yes or No."}
196 ],
197 "max_tokens": 1, "temperature": 0, "logprobs": true, "top_logprobs": 64,
198 "chat_template_kwargs": {"enable_thinking": false}
199 }'
200 ```
201
202 The reply's `top_logprobs` should put almost all probability on `Yes`.
203
204 | `llama-server` on 1x H100 (CUDA), one request at a time | JevBench, 231 tasks (median per task) | Synthetic transcripts, 27 Yes/No questions each (mean per transcript) |
205 |---|---:|---:|
206 | Q8_0 | 87.01% (201/231), 0.072 s | 99.85%, 2.753 s |
207 | Q4_K_M | 84.42% (195/231), 0.072 s | 99.75%, 2.195 s |
208
209 On 70 JevBench Yes/No tasks whose criteria are labelled `true`/`false`, the model answers `true` or `false`, so `Yes`/`No` carry little probability and the decision is read from those small values, as in the published evaluation. Each transcript is 27 separate requests here, which is why it is slower than on vLLM.
210
211 On AMD Ryzen AI MAX+ PRO 395 (Radeon 8060S, Vulkan, llama.cpp v0.9.0), both files gave the expected answer to the request above on 20 of 20 runs, with a median of 0.077 s (Q8_0, 27.1 GiB of GPU memory) and 0.058 s (Q4_K_M, 17.6 GiB) per request after the first. Validate on your own labelled data.
212
213 #### `/v1/systemone` with llama.cpp
214
215 The `gguf/` folder also holds `OTel-2.0-Decision-26B-A4B-Q8_0_systemone.gguf` and `OTel-2.0-Decision-26B-A4B-Q4_K_M_systemone.gguf`: the same weights with metadata that turns on `llama-server`'s own `/v1/systemone`. They take the same request as the vLLM *System One API* above and answer every question of a request in one call. These files answer decisions only (chat returns 501), and their system prompt is built in, so a request's `instructions` field has no effect.
216
217 ```bash
218 llama-server -hf farbodtavakkoli/OTel-2.0-Decision-26B-A4B-4bit:Q8_0_systemone \
219 -ngl 999 -c 65536 -np 8 -fa on --swa-full --host 0.0.0.0 --port 8081
220 ```
221
222 Use `:Q4_K_M_systemone` for the smaller file. `--swa-full` is required: without it the questions of one request do not share the state, and answers can change between identical requests.
223
224 ```bash
225 curl -s localhost:8081/v1/systemone -H 'content-type: application/json' -d '{
226 "state": "Caller: There is a 14 dollar charge on my bill I do not recognise. I want it refunded. I already rebooted the router twice.",
227 "questions": {
228 "refund_requested": {"type": "noul", "instructions": "Did the caller explicitly ask for a refund or credit?"},
229 "route": {"type": "choice", "instructions": "Which team should handle this call?",
230 "criteria": {"billing": "Charges, refunds and payments", "technical": "Internet, TV and equipment faults", "sales": "New plans and upgrades"}},
231 "urgency": {"type": "score", "instructions": "How urgent is this call?", "criteria": ["Routine", "Soon", "Today"]}
232 }
233 }'
234 ```
235
236 A Yes/No answer is `{"type": "noul", "noul": <probability of yes>}`; choice and score answers have `probabilities` for every option, as on vLLM. The reply has no `diagnostics` field.
237
238 | `/v1/systemone` with `llama-server`, one request at a time | JevBench, 231 tasks (median per task) | Synthetic transcripts, 27 Yes/No questions each (mean per transcript) |
239 |---|---:|---:|
240 | Q8_0_systemone, 1x H100 (CUDA) | 87.88% (203/231), 0.031 s | 99.75%, 0.588 s |
241 | Q4_K_M_systemone, 1x H100 (CUDA) | 86.58% (200/231), 0.032 s | 99.78%, 0.589 s |
242
243 On AMD Ryzen AI MAX+ PRO 395 (Radeon 8060S, Vulkan, llama.cpp v0.9.0), both files gave the expected answers to a five question request on 20 of 20 runs, with a median of 0.515 s (Q8_0, 39.7 GiB of GPU memory) and 0.472 s (Q4_K_M, 30.1 GiB) per request.
244
245 ### NVIDIA H100: vLLM on CUDA
246
247 Serve on one NVIDIA H100 80GB (SXM, HBM3) with the official CUDA image of vLLM 0.26.0. vLLM selects the Marlin W4A16 kernels for both the dense and MoE layers automatically, and no config shim or extra flags are needed.
248
249 ```bash
250 docker run --rm --name otel_decision \
251 --gpus '"device=0"' \
252 --ipc=host -p 8000:8000 \
253 -v "$PWD/OTel-2.0-Decision-26B-A4B-4bit:/model:ro" \
254 vllm/vllm-openai:v0.26.0 \
255 /model \
256 --served-model-name OTel-2.0-Decision-26B-A4B-4bit \
257 --max-model-len 16384 \
258 --enable-prefix-caching \
259 --max-logprobs 64 \
260 --port 8000
261 ```
262
263 With vLLM 0.26.0 already installed (`pip install vllm==0.26.0`), the equivalent is:
264
265 ```bash
266 CUDA_VISIBLE_DEVICES=0 vllm serve ./OTel-2.0-Decision-26B-A4B-4bit \
267 --served-model-name OTel-2.0-Decision-26B-A4B-4bit \
268 --max-model-len 16384 --enable-prefix-caching --max-logprobs 64 --port 8000
269 ```
270
271 | Measured (1x H100 80GB, vLLM 0.26.0, 512 output tokens per request) | Value |
272 |---|---:|
273 | Peak aggregate output, 256 concurrent requests | 10.6k tok/s |
274 | 1,024 concurrent requests | 10.1k tok/s |
275 | Single request | 223 tok/s |
276
277 The weights use about 17 GB, which leaves about 51 GB for KV cache (about 242k tokens) at the default `--gpu-memory-utilization 0.92`. The first start takes about 4 minutes because of torch.compile and CUDA graph capture. Later starts reuse the compile cache. To serve several replicas on one multi-GPU node, add `--data-parallel-size N`.
278
279 ### AMD Ryzen AI MAX+ PRO 395 (Strix Halo): vLLM on ROCm
280
281 Serve on one AMD Ryzen AI MAX+ PRO 395 (Strix Halo: Radeon 8060S GPU, 128 GB unified memory, configured as 96 GB GPU and 32 GB NPU) with vLLM 0.26.0 and ROCm 7.13 for gfx1151.
282
283 ```bash
284 # 1. ROCm 7.13 + vLLM for gfx1151 (managed by the ROCm CLI)
285 rocm engines install vllm
286 PY=~/.rocm/runtimes/wheel/release-wheel-gfx1151-7-13-0/bin/python
287
288 # 2. Config shim: transformers 5.19 drops Gemma 4's global head sizes that vLLM 0.26 reads
289 mkdir -p shim && cat > shim/sitecustomize.py <<'EOF'
290 from transformers import PretrainedConfig
291 from transformers.models.gemma4 import configuration_gemma4 as g4
292 PretrainedConfig.allow_global_per_layer_attribute_access = True
293 _init = g4.Gemma4TextConfig.__init__
294 def init(self, *a, **k):
295 _init(self, *a, **k)
296 lt, pl = list(getattr(self, "layer_types", []) or []), getattr(self, "per_layer_config", None)
297 if pl is not None and "full_attention" in lt:
298 full = pl[lt.index("full_attention")]
299 for dst, src in (("global_head_dim", "head_dim"), ("num_global_key_value_heads", "num_key_value_heads")):
300 if dst not in self.__dict__ and getattr(full, src, None) is not None:
301 object.__setattr__(self, dst, getattr(full, src))
302 g4.Gemma4TextConfig.__init__ = init
303 EOF
304
305 # 3. Serve (the torch wheel links OpenMPI 4's libmpi_cxx.so.40; point at its lib dir)
306 LD_LIBRARY_PATH=/path/to/openmpi4/lib PYTHONPATH=$PWD/shim \
307 $PY -m vllm.entrypoints.cli.main serve ./OTel-2.0-Decision-26B-A4B-4bit \
308 --served-model-name OTel-2.0-Decision-26B-A4B-4bit \
309 --max-model-len 16384 --enable-prefix-caching --max-logprobs 64 \
310 --gpu-memory-utilization 0.55 --max-num-seqs 256 --port 8000
311 ```
312
313 | Measured (AC power, performance profile) | Value |
314 |---|---:|
315 | Peak aggregate output, 256 concurrent requests | 377 tok/s |
316 | 64 concurrent requests | 257 tok/s |
317 | Single request | 30 tok/s |
318 | JevBench public, 8 concurrent | 1.27 tasks/s |
319
320 Peak throughput again needs near-maximum utilization: aggregate output keeps rising up to 256 concurrent requests. Keep HIP graphs on (the default). `--enforce-eager` cuts single-request speed to about 19 tok/s. The shim is not needed with vLLM releases that read Gemma 4's per-layer config directly. torch fails to import without an OpenMPI 4.x runtime. Distributions that ship only OpenMPI 5 (for example Ubuntu 26.04) lack `libmpi_cxx.so.40`, so extract an OpenMPI 4 package into a local directory and use that directory; no `sudo` is needed.
321
322 ### Prompting and Label Probabilities
323
324 Put the text first, then the question and labels. Use letters (`A`, `B`, …) for options or Yes/No, and numbers for ordered levels. Add a system message that asks for exactly one label, and request only the label through the chat endpoint, which applies the repository's template with thinking disabled.
325
326 ```python
327 import math
328 from openai import OpenAI
329
330 client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
331 question = """State:
332 Order #1182. Status: shipped on 3 September. Carrier: DHL.
333
334 Question: Has the order been shipped? Answer strictly from the facts stated in the text.
335 Options:
336 A. Yes
337 B. No
338 Answer with the letter only."""
339 labels = ["A", "B"]
340
341 r = client.chat.completions.create(
342 model="OTel-2.0-Decision-26B-A4B-4bit",
343 temperature=0,
344 max_tokens=1,
345 logprobs=True,
346 top_logprobs=64,
347 messages=[
348 {"role": "system", "content": "You are a typed decision engine. Reply with exactly one answer label "
349 "(a single letter or number) and nothing else. Do not explain."},
350 {"role": "user", "content": question},
351 ],
352 )
353 top = {t.token: t.logprob for t in r.choices[0].logprobs.content[0].top_logprobs}
354 probs = {label: math.exp(top[label]) if label in top else 0.0 for label in labels}
355 print("probabilities:", probs)
356 print("decision:", max(probs, key=probs.get))
357 print("answer_mass:", sum(probs.values()))
358 # Expected: decision A, answer_mass approximately 1.0
359 ```
360
361 Choose the label with the highest probability. `answer_mass` is the summed probability of the answer labels and should be close to 1.0; a low value indicates an invalid decision readout. For multiple questions, keep the shared text byte-identical so prefix caching can reuse it.
362
363 ### Verify the Deployment
364
365 Check that `http://localhost:8000/v1/models` lists the branded model name, then run the example above and confirm decision `A` with `answer_mass` near 1.0. For `llama-server`, use port 8080 and the request in the llama.cpp section instead; `/v1/models` lists the GGUF path given to `-m` (or a name set with `--alias`), and the answer should be `Yes`. Validate representative prompts against a reference run before deployment.
366
367 ## Limitations and Responsible Use
368
369 Evaluation covers English text only. JevBench is small and does not guarantee production accuracy. Validate on your own human-labeled data, and do not assume label probabilities are calibrated for your workload. Keep human review for consequential decisions.
370
371 The recipe starts an unauthenticated server. Add authentication before exposing it beyond your machine.
372
373 ## Resources
374
375 - [Project Page](https://huggingface.co/farbodtavakkoli)
376 - [OTel code](https://github.com/farbodtavakkoli/OTel)
377 - [OTel Selected Coverage](https://github.com/farbodtavakkoli/OTel/blob/main/docs/OTel-selected-coverage.md)
378 - [OTel Paper](https://arxiv.org/pdf/2610.07766)
379 - [OTel 1.0 Coverage](https://github.com/farbodtavakkoli/OTel/blob/main/docs/OTel-1.0-coverage.md)
380 - [OTel 2.0 Coverage](https://github.com/farbodtavakkoli/OTel/blob/main/docs/OTel-2.0-coverage.md)
381 - [OTel Docker](https://hub.docker.com/u/farbodatdocker)
382
383 ## Citation
384
385 ```bibtex
386 @inproceedings{tavakkoli2026otel2,
387 title = {OTel 2.0: Open Telco AI Datasets, Benchmarks, and Models},
388 author = {Tavakkoli, Farbod and others},
389 booktitle = {TBD},
390 year = {2026},
391 url = {https://huggingface.co/farbodtavakkoli}
392 }
393 ```
394
395 ## Contact
396
397 If you have any technical questions, please feel free to reach out to [mark.austin@att.com](mailto:mark.austin@att.com) or [farbod.tavakkoli@att.com](mailto:farbod.tavakkoli@att.com) or [farbodtavakoli@gmail.com](mailto:farbodtavakoli@gmail.com)
398