README.md · bge-base-en

README.md

88.0 KB · 3011 lines · markdown Raw

1	`---`
2	`tags:`
3	`- mteb`
4	`model-index:`
5	`- name: bge-base-en`
6	`results:`
7	`- task:`
8	`type: Classification`
9	`dataset:`
10	`type: mteb/amazon_counterfactual`
11	`name: MTEB AmazonCounterfactualClassification (en)`
12	`config: en`
13	`split: test`
14	`revision: e8379541af4e31359cca9fbcf4b00f2671dba205`
15	`metrics:`
16	`- type: accuracy`
17	`value: 75.73134328358209`
18	`- type: ap`
19	`value: 38.97277232632892`
20	`- type: f1`
21	`value: 69.81740361139785`
22	`- task:`
23	`type: Classification`
24	`dataset:`
25	`type: mteb/amazon_polarity`
26	`name: MTEB AmazonPolarityClassification`
27	`config: default`
28	`split: test`
29	`revision: e2d317d38cd51312af73b3d32a06d1a08b442046`
30	`metrics:`
31	`- type: accuracy`
32	`value: 92.56522500000001`
33	`- type: ap`
34	`value: 88.88821771869553`
35	`- type: f1`
36	`value: 92.54817512659696`
37	`- task:`
38	`type: Classification`
39	`dataset:`
40	`type: mteb/amazon_reviews_multi`
41	`name: MTEB AmazonReviewsClassification (en)`
42	`config: en`
43	`split: test`
44	`revision: 1399c76144fd37290681b995c656ef9b2e06e26d`
45	`metrics:`
46	`- type: accuracy`
47	`value: 46.91`
48	`- type: f1`
49	`value: 46.28536394320311`
50	`- task:`
51	`type: Retrieval`
52	`dataset:`
53	`type: arguana`
54	`name: MTEB ArguAna`
55	`config: default`
56	`split: test`
57	`revision: None`
58	`metrics:`
59	`- type: map_at_1`
60	`value: 38.834`
61	`- type: map_at_10`
62	`value: 53.564`
63	`- type: map_at_100`
64	`value: 54.230000000000004`
65	`- type: map_at_1000`
66	`value: 54.235`
67	`- type: map_at_3`
68	`value: 49.49`
69	`- type: map_at_5`
70	`value: 51.784`
71	`- type: mrr_at_1`
72	`value: 39.26`
73	`- type: mrr_at_10`
74	`value: 53.744`
75	`- type: mrr_at_100`
76	`value: 54.410000000000004`
77	`- type: mrr_at_1000`
78	`value: 54.415`
79	`- type: mrr_at_3`
80	`value: 49.656`
81	`- type: mrr_at_5`
82	`value: 52.018`
83	`- type: ndcg_at_1`
84	`value: 38.834`
85	`- type: ndcg_at_10`
86	`value: 61.487`
87	`- type: ndcg_at_100`
88	`value: 64.303`
89	`- type: ndcg_at_1000`
90	`value: 64.408`
91	`- type: ndcg_at_3`
92	`value: 53.116`
93	`- type: ndcg_at_5`
94	`value: 57.248`
95	`- type: precision_at_1`
96	`value: 38.834`
97	`- type: precision_at_10`
98	`value: 8.663`
99	`- type: precision_at_100`
100	`value: 0.989`
101	`- type: precision_at_1000`
102	`value: 0.1`
103	`- type: precision_at_3`
104	`value: 21.218999999999998`
105	`- type: precision_at_5`
106	`value: 14.737`
107	`- type: recall_at_1`
108	`value: 38.834`
109	`- type: recall_at_10`
110	`value: 86.629`
111	`- type: recall_at_100`
112	`value: 98.86200000000001`
113	`- type: recall_at_1000`
114	`value: 99.644`
115	`- type: recall_at_3`
116	`value: 63.656`
117	`- type: recall_at_5`
118	`value: 73.68400000000001`
119	`- task:`
120	`type: Clustering`
121	`dataset:`
122	`type: mteb/arxiv-clustering-p2p`
123	`name: MTEB ArxivClusteringP2P`
124	`config: default`
125	`split: test`
126	`revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d`
127	`metrics:`
128	`- type: v_measure`
129	`value: 48.88475477433035`
130	`- task:`
131	`type: Clustering`
132	`dataset:`
133	`type: mteb/arxiv-clustering-s2s`
134	`name: MTEB ArxivClusteringS2S`
135	`config: default`
136	`split: test`
137	`revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53`
138	`metrics:`
139	`- type: v_measure`
140	`value: 42.85053138403176`
141	`- task:`
142	`type: Reranking`
143	`dataset:`
144	`type: mteb/askubuntudupquestions-reranking`
145	`name: MTEB AskUbuntuDupQuestions`
146	`config: default`
147	`split: test`
148	`revision: 2000358ca161889fa9c082cb41daa8dcfb161a54`
149	`metrics:`
150	`- type: map`
151	`value: 62.23221013208242`
152	`- type: mrr`
153	`value: 74.64857318735436`
154	`- task:`
155	`type: STS`
156	`dataset:`
157	`type: mteb/biosses-sts`
158	`name: MTEB BIOSSES`
159	`config: default`
160	`split: test`
161	`revision: d3fb88f8f02e40887cd149695127462bbcf29b4a`
162	`metrics:`
163	`- type: cos_sim_pearson`
164	`value: 87.4403443247284`
165	`- type: cos_sim_spearman`
166	`value: 85.5326718115169`
167	`- type: euclidean_pearson`
168	`value: 86.0114007449595`
169	`- type: euclidean_spearman`
170	`value: 86.05979225604875`
171	`- type: manhattan_pearson`
172	`value: 86.05423806568598`
173	`- type: manhattan_spearman`
174	`value: 86.02485170086835`
175	`- task:`
176	`type: Classification`
177	`dataset:`
178	`type: mteb/banking77`
179	`name: MTEB Banking77Classification`
180	`config: default`
181	`split: test`
182	`revision: 0fd18e25b25c072e09e0d92ab615fda904d66300`
183	`metrics:`
184	`- type: accuracy`
185	`value: 86.44480519480518`
186	`- type: f1`
187	`value: 86.41301900941988`
188	`- task:`
189	`type: Clustering`
190	`dataset:`
191	`type: mteb/biorxiv-clustering-p2p`
192	`name: MTEB BiorxivClusteringP2P`
193	`config: default`
194	`split: test`
195	`revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40`
196	`metrics:`
197	`- type: v_measure`
198	`value: 40.17547250880036`
199	`- task:`
200	`type: Clustering`
201	`dataset:`
202	`type: mteb/biorxiv-clustering-s2s`
203	`name: MTEB BiorxivClusteringS2S`
204	`config: default`
205	`split: test`
206	`revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908`
207	`metrics:`
208	`- type: v_measure`
209	`value: 37.74514172687293`
210	`- task:`
211	`type: Retrieval`
212	`dataset:`
213	`type: BeIR/cqadupstack`
214	`name: MTEB CQADupstackAndroidRetrieval`
215	`config: default`
216	`split: test`
217	`revision: None`
218	`metrics:`
219	`- type: map_at_1`
220	`value: 32.096000000000004`
221	`- type: map_at_10`
222	`value: 43.345`
223	`- type: map_at_100`
224	`value: 44.73`
225	`- type: map_at_1000`
226	`value: 44.85`
227	`- type: map_at_3`
228	`value: 39.956`
229	`- type: map_at_5`
230	`value: 41.727`
231	`- type: mrr_at_1`
232	`value: 38.769999999999996`
233	`- type: mrr_at_10`
234	`value: 48.742000000000004`
235	`- type: mrr_at_100`
236	`value: 49.474000000000004`
237	`- type: mrr_at_1000`
238	`value: 49.513`
239	`- type: mrr_at_3`
240	`value: 46.161`
241	`- type: mrr_at_5`
242	`value: 47.721000000000004`
243	`- type: ndcg_at_1`
244	`value: 38.769999999999996`
245	`- type: ndcg_at_10`
246	`value: 49.464999999999996`
247	`- type: ndcg_at_100`
248	`value: 54.632000000000005`
249	`- type: ndcg_at_1000`
250	`value: 56.52`
251	`- type: ndcg_at_3`
252	`value: 44.687`
253	`- type: ndcg_at_5`
254	`value: 46.814`
255	`- type: precision_at_1`
256	`value: 38.769999999999996`
257	`- type: precision_at_10`
258	`value: 9.471`
259	`- type: precision_at_100`
260	`value: 1.4909999999999999`
261	`- type: precision_at_1000`
262	`value: 0.194`
263	`- type: precision_at_3`
264	`value: 21.268`
265	`- type: precision_at_5`
266	`value: 15.079`
267	`- type: recall_at_1`
268	`value: 32.096000000000004`
269	`- type: recall_at_10`
270	`value: 60.99099999999999`
271	`- type: recall_at_100`
272	`value: 83.075`
273	`- type: recall_at_1000`
274	`value: 95.178`
275	`- type: recall_at_3`
276	`value: 47.009`
277	`- type: recall_at_5`
278	`value: 53.348`
279	`- task:`
280	`type: Retrieval`
281	`dataset:`
282	`type: BeIR/cqadupstack`
283	`name: MTEB CQADupstackEnglishRetrieval`
284	`config: default`
285	`split: test`
286	`revision: None`
287	`metrics:`
288	`- type: map_at_1`
289	`value: 32.588`
290	`- type: map_at_10`
291	`value: 42.251`
292	`- type: map_at_100`
293	`value: 43.478`
294	`- type: map_at_1000`
295	`value: 43.617`
296	`- type: map_at_3`
297	`value: 39.381`
298	`- type: map_at_5`
299	`value: 41.141`
300	`- type: mrr_at_1`
301	`value: 41.21`
302	`- type: mrr_at_10`
303	`value: 48.765`
304	`- type: mrr_at_100`
305	`value: 49.403000000000006`
306	`- type: mrr_at_1000`
307	`value: 49.451`
308	`- type: mrr_at_3`
309	`value: 46.73`
310	`- type: mrr_at_5`
311	`value: 47.965999999999994`
312	`- type: ndcg_at_1`
313	`value: 41.21`
314	`- type: ndcg_at_10`
315	`value: 47.704`
316	`- type: ndcg_at_100`
317	`value: 51.916`
318	`- type: ndcg_at_1000`
319	`value: 54.013999999999996`
320	`- type: ndcg_at_3`
321	`value: 44.007000000000005`
322	`- type: ndcg_at_5`
323	`value: 45.936`
324	`- type: precision_at_1`
325	`value: 41.21`
326	`- type: precision_at_10`
327	`value: 8.885`
328	`- type: precision_at_100`
329	`value: 1.409`
330	`- type: precision_at_1000`
331	`value: 0.189`
332	`- type: precision_at_3`
333	`value: 21.274`
334	`- type: precision_at_5`
335	`value: 15.045`
336	`- type: recall_at_1`
337	`value: 32.588`
338	`- type: recall_at_10`
339	`value: 56.333`
340	`- type: recall_at_100`
341	`value: 74.251`
342	`- type: recall_at_1000`
343	`value: 87.518`
344	`- type: recall_at_3`
345	`value: 44.962`
346	`- type: recall_at_5`
347	`value: 50.609`
348	`- task:`
349	`type: Retrieval`
350	`dataset:`
351	`type: BeIR/cqadupstack`
352	`name: MTEB CQADupstackGamingRetrieval`
353	`config: default`
354	`split: test`
355	`revision: None`
356	`metrics:`
357	`- type: map_at_1`
358	`value: 40.308`
359	`- type: map_at_10`
360	`value: 53.12`
361	`- type: map_at_100`
362	`value: 54.123`
363	`- type: map_at_1000`
364	`value: 54.173`
365	`- type: map_at_3`
366	`value: 50.017999999999994`
367	`- type: map_at_5`
368	`value: 51.902`
369	`- type: mrr_at_1`
370	`value: 46.394999999999996`
371	`- type: mrr_at_10`
372	`value: 56.531`
373	`- type: mrr_at_100`
374	`value: 57.19800000000001`
375	`- type: mrr_at_1000`
376	`value: 57.225`
377	`- type: mrr_at_3`
378	`value: 54.368`
379	`- type: mrr_at_5`
380	`value: 55.713`
381	`- type: ndcg_at_1`
382	`value: 46.394999999999996`
383	`- type: ndcg_at_10`
384	`value: 58.811`
385	`- type: ndcg_at_100`
386	`value: 62.834`
387	`- type: ndcg_at_1000`
388	`value: 63.849999999999994`
389	`- type: ndcg_at_3`
390	`value: 53.88699999999999`
391	`- type: ndcg_at_5`
392	`value: 56.477999999999994`
393	`- type: precision_at_1`
394	`value: 46.394999999999996`
395	`- type: precision_at_10`
396	`value: 9.398`
397	`- type: precision_at_100`
398	`value: 1.2309999999999999`
399	`- type: precision_at_1000`
400	`value: 0.136`
401	`- type: precision_at_3`
402	`value: 24.221999999999998`
403	`- type: precision_at_5`
404	`value: 16.539`
405	`- type: recall_at_1`
406	`value: 40.308`
407	`- type: recall_at_10`
408	`value: 72.146`
409	`- type: recall_at_100`
410	`value: 89.60900000000001`
411	`- type: recall_at_1000`
412	`value: 96.733`
413	`- type: recall_at_3`
414	`value: 58.91499999999999`
415	`- type: recall_at_5`
416	`value: 65.34299999999999`
417	`- task:`
418	`type: Retrieval`
419	`dataset:`
420	`type: BeIR/cqadupstack`
421	`name: MTEB CQADupstackGisRetrieval`
422	`config: default`
423	`split: test`
424	`revision: None`
425	`metrics:`
426	`- type: map_at_1`
427	`value: 27.383000000000003`
428	`- type: map_at_10`
429	`value: 35.802`
430	`- type: map_at_100`
431	`value: 36.756`
432	`- type: map_at_1000`
433	`value: 36.826`
434	`- type: map_at_3`
435	`value: 32.923`
436	`- type: map_at_5`
437	`value: 34.577999999999996`
438	`- type: mrr_at_1`
439	`value: 29.604999999999997`
440	`- type: mrr_at_10`
441	`value: 37.918`
442	`- type: mrr_at_100`
443	`value: 38.732`
444	`- type: mrr_at_1000`
445	`value: 38.786`
446	`- type: mrr_at_3`
447	`value: 35.198`
448	`- type: mrr_at_5`
449	`value: 36.808`
450	`- type: ndcg_at_1`
451	`value: 29.604999999999997`
452	`- type: ndcg_at_10`
453	`value: 40.836`
454	`- type: ndcg_at_100`
455	`value: 45.622`
456	`- type: ndcg_at_1000`
457	`value: 47.427`
458	`- type: ndcg_at_3`
459	`value: 35.208`
460	`- type: ndcg_at_5`
461	`value: 38.066`
462	`- type: precision_at_1`
463	`value: 29.604999999999997`
464	`- type: precision_at_10`
465	`value: 6.226`
466	`- type: precision_at_100`
467	`value: 0.9079999999999999`
468	`- type: precision_at_1000`
469	`value: 0.11`
470	`- type: precision_at_3`
471	`value: 14.463000000000001`
472	`- type: precision_at_5`
473	`value: 10.35`
474	`- type: recall_at_1`
475	`value: 27.383000000000003`
476	`- type: recall_at_10`
477	`value: 54.434000000000005`
478	`- type: recall_at_100`
479	`value: 76.632`
480	`- type: recall_at_1000`
481	`value: 90.25`
482	`- type: recall_at_3`
483	`value: 39.275`
484	`- type: recall_at_5`
485	`value: 46.225`
486	`- task:`
487	`type: Retrieval`
488	`dataset:`
489	`type: BeIR/cqadupstack`
490	`name: MTEB CQADupstackMathematicaRetrieval`
491	`config: default`
492	`split: test`
493	`revision: None`
494	`metrics:`
495	`- type: map_at_1`
496	`value: 17.885`
497	`- type: map_at_10`
498	`value: 25.724000000000004`
499	`- type: map_at_100`
500	`value: 26.992`
501	`- type: map_at_1000`
502	`value: 27.107999999999997`
503	`- type: map_at_3`
504	`value: 23.04`
505	`- type: map_at_5`
506	`value: 24.529`
507	`- type: mrr_at_1`
508	`value: 22.264`
509	`- type: mrr_at_10`
510	`value: 30.548`
511	`- type: mrr_at_100`
512	`value: 31.593`
513	`- type: mrr_at_1000`
514	`value: 31.657999999999998`
515	`- type: mrr_at_3`
516	`value: 27.756999999999998`
517	`- type: mrr_at_5`
518	`value: 29.398999999999997`
519	`- type: ndcg_at_1`
520	`value: 22.264`
521	`- type: ndcg_at_10`
522	`value: 30.902`
523	`- type: ndcg_at_100`
524	`value: 36.918`
525	`- type: ndcg_at_1000`
526	`value: 39.735`
527	`- type: ndcg_at_3`
528	`value: 25.915`
529	`- type: ndcg_at_5`
530	`value: 28.255999999999997`
531	`- type: precision_at_1`
532	`value: 22.264`
533	`- type: precision_at_10`
534	`value: 5.634`
535	`- type: precision_at_100`
536	`value: 0.9939999999999999`
537	`- type: precision_at_1000`
538	`value: 0.13699999999999998`
539	`- type: precision_at_3`
540	`value: 12.396`
541	`- type: precision_at_5`
542	`value: 9.055`
543	`- type: recall_at_1`
544	`value: 17.885`
545	`- type: recall_at_10`
546	`value: 42.237`
547	`- type: recall_at_100`
548	`value: 68.489`
549	`- type: recall_at_1000`
550	`value: 88.721`
551	`- type: recall_at_3`
552	`value: 28.283`
553	`- type: recall_at_5`
554	`value: 34.300000000000004`
555	`- task:`
556	`type: Retrieval`
557	`dataset:`
558	`type: BeIR/cqadupstack`
559	`name: MTEB CQADupstackPhysicsRetrieval`
560	`config: default`
561	`split: test`
562	`revision: None`
563	`metrics:`
564	`- type: map_at_1`
565	`value: 29.737000000000002`
566	`- type: map_at_10`
567	`value: 39.757`
568	`- type: map_at_100`
569	`value: 40.992`
570	`- type: map_at_1000`
571	`value: 41.102`
572	`- type: map_at_3`
573	`value: 36.612`
574	`- type: map_at_5`
575	`value: 38.413000000000004`
576	`- type: mrr_at_1`
577	`value: 35.804`
578	`- type: mrr_at_10`
579	`value: 45.178000000000004`
580	`- type: mrr_at_100`
581	`value: 45.975`
582	`- type: mrr_at_1000`
583	`value: 46.021`
584	`- type: mrr_at_3`
585	`value: 42.541000000000004`
586	`- type: mrr_at_5`
587	`value: 44.167`
588	`- type: ndcg_at_1`
589	`value: 35.804`
590	`- type: ndcg_at_10`
591	`value: 45.608`
592	`- type: ndcg_at_100`
593	`value: 50.746`
594	`- type: ndcg_at_1000`
595	`value: 52.839999999999996`
596	`- type: ndcg_at_3`
597	`value: 40.52`
598	`- type: ndcg_at_5`
599	`value: 43.051`
600	`- type: precision_at_1`
601	`value: 35.804`
602	`- type: precision_at_10`
603	`value: 8.104`
604	`- type: precision_at_100`
605	`value: 1.256`
606	`- type: precision_at_1000`
607	`value: 0.161`
608	`- type: precision_at_3`
609	`value: 19.121`
610	`- type: precision_at_5`
611	`value: 13.532`
612	`- type: recall_at_1`
613	`value: 29.737000000000002`
614	`- type: recall_at_10`
615	`value: 57.66`
616	`- type: recall_at_100`
617	`value: 79.121`
618	`- type: recall_at_1000`
619	`value: 93.023`
620	`- type: recall_at_3`
621	`value: 43.13`
622	`- type: recall_at_5`
623	`value: 49.836000000000006`
624	`- task:`
625	`type: Retrieval`
626	`dataset:`
627	`type: BeIR/cqadupstack`
628	`name: MTEB CQADupstackProgrammersRetrieval`
629	`config: default`
630	`split: test`
631	`revision: None`
632	`metrics:`
633	`- type: map_at_1`
634	`value: 26.299`
635	`- type: map_at_10`
636	`value: 35.617`
637	`- type: map_at_100`
638	`value: 36.972`
639	`- type: map_at_1000`
640	`value: 37.096000000000004`
641	`- type: map_at_3`
642	`value: 32.653999999999996`
643	`- type: map_at_5`
644	`value: 34.363`
645	`- type: mrr_at_1`
646	`value: 32.877`
647	`- type: mrr_at_10`
648	`value: 41.423`
649	`- type: mrr_at_100`
650	`value: 42.333999999999996`
651	`- type: mrr_at_1000`
652	`value: 42.398`
653	`- type: mrr_at_3`
654	`value: 39.193`
655	`- type: mrr_at_5`
656	`value: 40.426`
657	`- type: ndcg_at_1`
658	`value: 32.877`
659	`- type: ndcg_at_10`
660	`value: 41.271`
661	`- type: ndcg_at_100`
662	`value: 46.843`
663	`- type: ndcg_at_1000`
664	`value: 49.366`
665	`- type: ndcg_at_3`
666	`value: 36.735`
667	`- type: ndcg_at_5`
668	`value: 38.775999999999996`
669	`- type: precision_at_1`
670	`value: 32.877`
671	`- type: precision_at_10`
672	`value: 7.580000000000001`
673	`- type: precision_at_100`
674	`value: 1.192`
675	`- type: precision_at_1000`
676	`value: 0.158`
677	`- type: precision_at_3`
678	`value: 17.541999999999998`
679	`- type: precision_at_5`
680	`value: 12.443`
681	`- type: recall_at_1`
682	`value: 26.299`
683	`- type: recall_at_10`
684	`value: 52.256`
685	`- type: recall_at_100`
686	`value: 75.919`
687	`- type: recall_at_1000`
688	`value: 93.185`
689	`- type: recall_at_3`
690	`value: 39.271`
691	`- type: recall_at_5`
692	`value: 44.901`
693	`- task:`
694	`type: Retrieval`
695	`dataset:`
696	`type: BeIR/cqadupstack`
697	`name: MTEB CQADupstackRetrieval`
698	`config: default`
699	`split: test`
700	`revision: None`
701	`metrics:`
702	`- type: map_at_1`
703	`value: 27.05741666666667`
704	`- type: map_at_10`
705	`value: 36.086416666666665`
706	`- type: map_at_100`
707	`value: 37.26916666666667`
708	`- type: map_at_1000`
709	`value: 37.38191666666666`
710	`- type: map_at_3`
711	`value: 33.34225`
712	`- type: map_at_5`
713	`value: 34.86425`
714	`- type: mrr_at_1`
715	`value: 32.06008333333333`
716	`- type: mrr_at_10`
717	`value: 40.36658333333333`
718	`- type: mrr_at_100`
719	`value: 41.206500000000005`
720	`- type: mrr_at_1000`
721	`value: 41.261083333333325`
722	`- type: mrr_at_3`
723	`value: 38.01208333333334`
724	`- type: mrr_at_5`
725	`value: 39.36858333333333`
726	`- type: ndcg_at_1`
727	`value: 32.06008333333333`
728	`- type: ndcg_at_10`
729	`value: 41.3535`
730	`- type: ndcg_at_100`
731	`value: 46.42066666666666`
732	`- type: ndcg_at_1000`
733	`value: 48.655166666666666`
734	`- type: ndcg_at_3`
735	`value: 36.78041666666667`
736	`- type: ndcg_at_5`
737	`value: 38.91783333333334`
738	`- type: precision_at_1`
739	`value: 32.06008333333333`
740	`- type: precision_at_10`
741	`value: 7.169833333333332`
742	`- type: precision_at_100`
743	`value: 1.1395`
744	`- type: precision_at_1000`
745	`value: 0.15158333333333332`
746	`- type: precision_at_3`
747	`value: 16.852`
748	`- type: precision_at_5`
749	`value: 11.8645`
750	`- type: recall_at_1`
751	`value: 27.05741666666667`
752	`- type: recall_at_10`
753	`value: 52.64491666666666`
754	`- type: recall_at_100`
755	`value: 74.99791666666667`
756	`- type: recall_at_1000`
757	`value: 90.50524999999999`
758	`- type: recall_at_3`
759	`value: 39.684000000000005`
760	`- type: recall_at_5`
761	`value: 45.37225`
762	`- task:`
763	`type: Retrieval`
764	`dataset:`
765	`type: BeIR/cqadupstack`
766	`name: MTEB CQADupstackStatsRetrieval`
767	`config: default`
768	`split: test`
769	`revision: None`
770	`metrics:`
771	`- type: map_at_1`
772	`value: 25.607999999999997`
773	`- type: map_at_10`
774	`value: 32.28`
775	`- type: map_at_100`
776	`value: 33.261`
777	`- type: map_at_1000`
778	`value: 33.346`
779	`- type: map_at_3`
780	`value: 30.514999999999997`
781	`- type: map_at_5`
782	`value: 31.415`
783	`- type: mrr_at_1`
784	`value: 28.988000000000003`
785	`- type: mrr_at_10`
786	`value: 35.384`
787	`- type: mrr_at_100`
788	`value: 36.24`
789	`- type: mrr_at_1000`
790	`value: 36.299`
791	`- type: mrr_at_3`
792	`value: 33.717000000000006`
793	`- type: mrr_at_5`
794	`value: 34.507`
795	`- type: ndcg_at_1`
796	`value: 28.988000000000003`
797	`- type: ndcg_at_10`
798	`value: 36.248000000000005`
799	`- type: ndcg_at_100`
800	`value: 41.034`
801	`- type: ndcg_at_1000`
802	`value: 43.35`
803	`- type: ndcg_at_3`
804	`value: 32.987`
805	`- type: ndcg_at_5`
806	`value: 34.333999999999996`
807	`- type: precision_at_1`
808	`value: 28.988000000000003`
809	`- type: precision_at_10`
810	`value: 5.506`
811	`- type: precision_at_100`
812	`value: 0.853`
813	`- type: precision_at_1000`
814	`value: 0.11199999999999999`
815	`- type: precision_at_3`
816	`value: 14.11`
817	`- type: precision_at_5`
818	`value: 9.417`
819	`- type: recall_at_1`
820	`value: 25.607999999999997`
821	`- type: recall_at_10`
822	`value: 45.344`
823	`- type: recall_at_100`
824	`value: 67.132`
825	`- type: recall_at_1000`
826	`value: 84.676`
827	`- type: recall_at_3`
828	`value: 36.02`
829	`- type: recall_at_5`
830	`value: 39.613`
831	`- task:`
832	`type: Retrieval`
833	`dataset:`
834	`type: BeIR/cqadupstack`
835	`name: MTEB CQADupstackTexRetrieval`
836	`config: default`
837	`split: test`
838	`revision: None`
839	`metrics:`
840	`- type: map_at_1`
841	`value: 18.44`
842	`- type: map_at_10`
843	`value: 25.651000000000003`
844	`- type: map_at_100`
845	`value: 26.735`
846	`- type: map_at_1000`
847	`value: 26.86`
848	`- type: map_at_3`
849	`value: 23.409`
850	`- type: map_at_5`
851	`value: 24.604`
852	`- type: mrr_at_1`
853	`value: 22.195`
854	`- type: mrr_at_10`
855	`value: 29.482000000000003`
856	`- type: mrr_at_100`
857	`value: 30.395`
858	`- type: mrr_at_1000`
859	`value: 30.471999999999998`
860	`- type: mrr_at_3`
861	`value: 27.409`
862	`- type: mrr_at_5`
863	`value: 28.553`
864	`- type: ndcg_at_1`
865	`value: 22.195`
866	`- type: ndcg_at_10`
867	`value: 30.242`
868	`- type: ndcg_at_100`
869	`value: 35.397`
870	`- type: ndcg_at_1000`
871	`value: 38.287`
872	`- type: ndcg_at_3`
873	`value: 26.201`
874	`- type: ndcg_at_5`
875	`value: 28.008`
876	`- type: precision_at_1`
877	`value: 22.195`
878	`- type: precision_at_10`
879	`value: 5.372`
880	`- type: precision_at_100`
881	`value: 0.9259999999999999`
882	`- type: precision_at_1000`
883	`value: 0.135`
884	`- type: precision_at_3`
885	`value: 12.228`
886	`- type: precision_at_5`
887	`value: 8.727`
888	`- type: recall_at_1`
889	`value: 18.44`
890	`- type: recall_at_10`
891	`value: 40.325`
892	`- type: recall_at_100`
893	`value: 63.504000000000005`
894	`- type: recall_at_1000`
895	`value: 83.909`
896	`- type: recall_at_3`
897	`value: 28.925`
898	`- type: recall_at_5`
899	`value: 33.641`
900	`- task:`
901	`type: Retrieval`
902	`dataset:`
903	`type: BeIR/cqadupstack`
904	`name: MTEB CQADupstackUnixRetrieval`
905	`config: default`
906	`split: test`
907	`revision: None`
908	`metrics:`
909	`- type: map_at_1`
910	`value: 26.535999999999998`
911	`- type: map_at_10`
912	`value: 35.358000000000004`
913	`- type: map_at_100`
914	`value: 36.498999999999995`
915	`- type: map_at_1000`
916	`value: 36.597`
917	`- type: map_at_3`
918	`value: 32.598`
919	`- type: map_at_5`
920	`value: 34.185`
921	`- type: mrr_at_1`
922	`value: 31.25`
923	`- type: mrr_at_10`
924	`value: 39.593`
925	`- type: mrr_at_100`
926	`value: 40.443`
927	`- type: mrr_at_1000`
928	`value: 40.498`
929	`- type: mrr_at_3`
930	`value: 37.018`
931	`- type: mrr_at_5`
932	`value: 38.492`
933	`- type: ndcg_at_1`
934	`value: 31.25`
935	`- type: ndcg_at_10`
936	`value: 40.71`
937	`- type: ndcg_at_100`
938	`value: 46.079`
939	`- type: ndcg_at_1000`
940	`value: 48.287`
941	`- type: ndcg_at_3`
942	`value: 35.667`
943	`- type: ndcg_at_5`
944	`value: 38.080000000000005`
945	`- type: precision_at_1`
946	`value: 31.25`
947	`- type: precision_at_10`
948	`value: 6.847`
949	`- type: precision_at_100`
950	`value: 1.079`
951	`- type: precision_at_1000`
952	`value: 0.13699999999999998`
953	`- type: precision_at_3`
954	`value: 16.262`
955	`- type: precision_at_5`
956	`value: 11.455`
957	`- type: recall_at_1`
958	`value: 26.535999999999998`
959	`- type: recall_at_10`
960	`value: 52.92099999999999`
961	`- type: recall_at_100`
962	`value: 76.669`
963	`- type: recall_at_1000`
964	`value: 92.096`
965	`- type: recall_at_3`
966	`value: 38.956`
967	`- type: recall_at_5`
968	`value: 45.239000000000004`
969	`- task:`
970	`type: Retrieval`
971	`dataset:`
972	`type: BeIR/cqadupstack`
973	`name: MTEB CQADupstackWebmastersRetrieval`
974	`config: default`
975	`split: test`
976	`revision: None`
977	`metrics:`
978	`- type: map_at_1`
979	`value: 24.691`
980	`- type: map_at_10`
981	`value: 33.417`
982	`- type: map_at_100`
983	`value: 35.036`
984	`- type: map_at_1000`
985	`value: 35.251`
986	`- type: map_at_3`
987	`value: 30.646`
988	`- type: map_at_5`
989	`value: 32.177`
990	`- type: mrr_at_1`
991	`value: 30.04`
992	`- type: mrr_at_10`
993	`value: 37.905`
994	`- type: mrr_at_100`
995	`value: 38.929`
996	`- type: mrr_at_1000`
997	`value: 38.983000000000004`
998	`- type: mrr_at_3`
999	`value: 35.276999999999994`
1000	`- type: mrr_at_5`
1001	`value: 36.897000000000006`
1002	`- type: ndcg_at_1`
1003	`value: 30.04`
1004	`- type: ndcg_at_10`
1005	`value: 39.037`
1006	`- type: ndcg_at_100`
1007	`value: 44.944`
1008	`- type: ndcg_at_1000`
1009	`value: 47.644`
1010	`- type: ndcg_at_3`
1011	`value: 34.833999999999996`
1012	`- type: ndcg_at_5`
1013	`value: 36.83`
1014	`- type: precision_at_1`
1015	`value: 30.04`
1016	`- type: precision_at_10`
1017	`value: 7.4510000000000005`
1018	`- type: precision_at_100`
1019	`value: 1.492`
1020	`- type: precision_at_1000`
1021	`value: 0.234`
1022	`- type: precision_at_3`
1023	`value: 16.337`
1024	`- type: precision_at_5`
1025	`value: 11.897`
1026	`- type: recall_at_1`
1027	`value: 24.691`
1028	`- type: recall_at_10`
1029	`value: 49.303999999999995`
1030	`- type: recall_at_100`
1031	`value: 76.20400000000001`
1032	`- type: recall_at_1000`
1033	`value: 93.30000000000001`
1034	`- type: recall_at_3`
1035	`value: 36.594`
1036	`- type: recall_at_5`
1037	`value: 42.41`
1038	`- task:`
1039	`type: Retrieval`
1040	`dataset:`
1041	`type: BeIR/cqadupstack`
1042	`name: MTEB CQADupstackWordpressRetrieval`
1043	`config: default`
1044	`split: test`
1045	`revision: None`
1046	`metrics:`
1047	`- type: map_at_1`
1048	`value: 23.118`
1049	`- type: map_at_10`
1050	`value: 30.714999999999996`
1051	`- type: map_at_100`
1052	`value: 31.656000000000002`
1053	`- type: map_at_1000`
1054	`value: 31.757`
1055	`- type: map_at_3`
1056	`value: 28.355000000000004`
1057	`- type: map_at_5`
1058	`value: 29.337000000000003`
1059	`- type: mrr_at_1`
1060	`value: 25.323`
1061	`- type: mrr_at_10`
1062	`value: 32.93`
1063	`- type: mrr_at_100`
1064	`value: 33.762`
1065	`- type: mrr_at_1000`
1066	`value: 33.829`
1067	`- type: mrr_at_3`
1068	`value: 30.775999999999996`
1069	`- type: mrr_at_5`
1070	`value: 31.774`
1071	`- type: ndcg_at_1`
1072	`value: 25.323`
1073	`- type: ndcg_at_10`
1074	`value: 35.408`
1075	`- type: ndcg_at_100`
1076	`value: 40.083`
1077	`- type: ndcg_at_1000`
1078	`value: 42.542`
1079	`- type: ndcg_at_3`
1080	`value: 30.717`
1081	`- type: ndcg_at_5`
1082	`value: 32.385000000000005`
1083	`- type: precision_at_1`
1084	`value: 25.323`
1085	`- type: precision_at_10`
1086	`value: 5.564`
1087	`- type: precision_at_100`
1088	`value: 0.843`
1089	`- type: precision_at_1000`
1090	`value: 0.116`
1091	`- type: precision_at_3`
1092	`value: 13.001`
1093	`- type: precision_at_5`
1094	`value: 8.834999999999999`
1095	`- type: recall_at_1`
1096	`value: 23.118`
1097	`- type: recall_at_10`
1098	`value: 47.788000000000004`
1099	`- type: recall_at_100`
1100	`value: 69.37`
1101	`- type: recall_at_1000`
1102	`value: 87.47399999999999`
1103	`- type: recall_at_3`
1104	`value: 34.868`
1105	`- type: recall_at_5`
1106	`value: 39.001999999999995`
1107	`- task:`
1108	`type: Retrieval`
1109	`dataset:`
1110	`type: climate-fever`
1111	`name: MTEB ClimateFEVER`
1112	`config: default`
1113	`split: test`
1114	`revision: None`
1115	`metrics:`
1116	`- type: map_at_1`
1117	`value: 14.288`
1118	`- type: map_at_10`
1119	`value: 23.256`
1120	`- type: map_at_100`
1121	`value: 25.115`
1122	`- type: map_at_1000`
1123	`value: 25.319000000000003`
1124	`- type: map_at_3`
1125	`value: 20.005`
1126	`- type: map_at_5`
1127	`value: 21.529999999999998`
1128	`- type: mrr_at_1`
1129	`value: 31.401`
1130	`- type: mrr_at_10`
1131	`value: 42.251`
1132	`- type: mrr_at_100`
1133	`value: 43.236999999999995`
1134	`- type: mrr_at_1000`
1135	`value: 43.272`
1136	`- type: mrr_at_3`
1137	`value: 39.164`
1138	`- type: mrr_at_5`
1139	`value: 40.881`
1140	`- type: ndcg_at_1`
1141	`value: 31.401`
1142	`- type: ndcg_at_10`
1143	`value: 31.615`
1144	`- type: ndcg_at_100`
1145	`value: 38.982`
1146	`- type: ndcg_at_1000`
1147	`value: 42.496`
1148	`- type: ndcg_at_3`
1149	`value: 26.608999999999998`
1150	`- type: ndcg_at_5`
1151	`value: 28.048000000000002`
1152	`- type: precision_at_1`
1153	`value: 31.401`
1154	`- type: precision_at_10`
1155	`value: 9.536999999999999`
1156	`- type: precision_at_100`
1157	`value: 1.763`
1158	`- type: precision_at_1000`
1159	`value: 0.241`
1160	`- type: precision_at_3`
1161	`value: 19.153000000000002`
1162	`- type: precision_at_5`
1163	`value: 14.228`
1164	`- type: recall_at_1`
1165	`value: 14.288`
1166	`- type: recall_at_10`
1167	`value: 36.717`
1168	`- type: recall_at_100`
1169	`value: 61.9`
1170	`- type: recall_at_1000`
1171	`value: 81.676`
1172	`- type: recall_at_3`
1173	`value: 24.203`
1174	`- type: recall_at_5`
1175	`value: 28.793999999999997`
1176	`- task:`
1177	`type: Retrieval`
1178	`dataset:`
1179	`type: dbpedia-entity`
1180	`name: MTEB DBPedia`
1181	`config: default`
1182	`split: test`
1183	`revision: None`
1184	`metrics:`
1185	`- type: map_at_1`
1186	`value: 9.019`
1187	`- type: map_at_10`
1188	`value: 19.963`
1189	`- type: map_at_100`
1190	`value: 28.834`
1191	`- type: map_at_1000`
1192	`value: 30.537999999999997`
1193	`- type: map_at_3`
1194	`value: 14.45`
1195	`- type: map_at_5`
1196	`value: 16.817999999999998`
1197	`- type: mrr_at_1`
1198	`value: 65.75`
1199	`- type: mrr_at_10`
1200	`value: 74.646`
1201	`- type: mrr_at_100`
1202	`value: 74.946`
1203	`- type: mrr_at_1000`
1204	`value: 74.95100000000001`
1205	`- type: mrr_at_3`
1206	`value: 72.625`
1207	`- type: mrr_at_5`
1208	`value: 74.012`
1209	`- type: ndcg_at_1`
1210	`value: 54`
1211	`- type: ndcg_at_10`
1212	`value: 42.014`
1213	`- type: ndcg_at_100`
1214	`value: 47.527`
1215	`- type: ndcg_at_1000`
1216	`value: 54.911`
1217	`- type: ndcg_at_3`
1218	`value: 46.586`
1219	`- type: ndcg_at_5`
1220	`value: 43.836999999999996`
1221	`- type: precision_at_1`
1222	`value: 65.75`
1223	`- type: precision_at_10`
1224	`value: 33.475`
1225	`- type: precision_at_100`
1226	`value: 11.16`
1227	`- type: precision_at_1000`
1228	`value: 2.145`
1229	`- type: precision_at_3`
1230	`value: 50.083`
1231	`- type: precision_at_5`
1232	`value: 42.55`
1233	`- type: recall_at_1`
1234	`value: 9.019`
1235	`- type: recall_at_10`
1236	`value: 25.558999999999997`
1237	`- type: recall_at_100`
1238	`value: 53.937999999999995`
1239	`- type: recall_at_1000`
1240	`value: 77.67399999999999`
1241	`- type: recall_at_3`
1242	`value: 15.456`
1243	`- type: recall_at_5`
1244	`value: 19.259`
1245	`- task:`
1246	`type: Classification`
1247	`dataset:`
1248	`type: mteb/emotion`
1249	`name: MTEB EmotionClassification`
1250	`config: default`
1251	`split: test`
1252	`revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37`
1253	`metrics:`
1254	`- type: accuracy`
1255	`value: 52.635`
1256	`- type: f1`
1257	`value: 47.692783881403926`
1258	`- task:`
1259	`type: Retrieval`
1260	`dataset:`
1261	`type: fever`
1262	`name: MTEB FEVER`
1263	`config: default`
1264	`split: test`
1265	`revision: None`
1266	`metrics:`
1267	`- type: map_at_1`
1268	`value: 76.893`
1269	`- type: map_at_10`
1270	`value: 84.897`
1271	`- type: map_at_100`
1272	`value: 85.122`
1273	`- type: map_at_1000`
1274	`value: 85.135`
1275	`- type: map_at_3`
1276	`value: 83.88`
1277	`- type: map_at_5`
1278	`value: 84.565`
1279	`- type: mrr_at_1`
1280	`value: 83.003`
1281	`- type: mrr_at_10`
1282	`value: 89.506`
1283	`- type: mrr_at_100`
1284	`value: 89.574`
1285	`- type: mrr_at_1000`
1286	`value: 89.575`
1287	`- type: mrr_at_3`
1288	`value: 88.991`
1289	`- type: mrr_at_5`
1290	`value: 89.349`
1291	`- type: ndcg_at_1`
1292	`value: 83.003`
1293	`- type: ndcg_at_10`
1294	`value: 88.351`
1295	`- type: ndcg_at_100`
1296	`value: 89.128`
1297	`- type: ndcg_at_1000`
1298	`value: 89.34100000000001`
1299	`- type: ndcg_at_3`
1300	`value: 86.92`
1301	`- type: ndcg_at_5`
1302	`value: 87.78200000000001`
1303	`- type: precision_at_1`
1304	`value: 83.003`
1305	`- type: precision_at_10`
1306	`value: 10.517999999999999`
1307	`- type: precision_at_100`
1308	`value: 1.115`
1309	`- type: precision_at_1000`
1310	`value: 0.11499999999999999`
1311	`- type: precision_at_3`
1312	`value: 33.062999999999995`
1313	`- type: precision_at_5`
1314	`value: 20.498`
1315	`- type: recall_at_1`
1316	`value: 76.893`
1317	`- type: recall_at_10`
1318	`value: 94.374`
1319	`- type: recall_at_100`
1320	`value: 97.409`
1321	`- type: recall_at_1000`
1322	`value: 98.687`
1323	`- type: recall_at_3`
1324	`value: 90.513`
1325	`- type: recall_at_5`
1326	`value: 92.709`
1327	`- task:`
1328	`type: Retrieval`
1329	`dataset:`
1330	`type: fiqa`
1331	`name: MTEB FiQA2018`
1332	`config: default`
1333	`split: test`
1334	`revision: None`
1335	`metrics:`
1336	`- type: map_at_1`
1337	`value: 20.829`
1338	`- type: map_at_10`
1339	`value: 32.86`
1340	`- type: map_at_100`
1341	`value: 34.838`
1342	`- type: map_at_1000`
1343	`value: 35.006`
1344	`- type: map_at_3`
1345	`value: 28.597`
1346	`- type: map_at_5`
1347	`value: 31.056`
1348	`- type: mrr_at_1`
1349	`value: 41.358`
1350	`- type: mrr_at_10`
1351	`value: 49.542`
1352	`- type: mrr_at_100`
1353	`value: 50.29900000000001`
1354	`- type: mrr_at_1000`
1355	`value: 50.334999999999994`
1356	`- type: mrr_at_3`
1357	`value: 46.579`
1358	`- type: mrr_at_5`
1359	`value: 48.408`
1360	`- type: ndcg_at_1`
1361	`value: 41.358`
1362	`- type: ndcg_at_10`
1363	`value: 40.758`
1364	`- type: ndcg_at_100`
1365	`value: 47.799`
1366	`- type: ndcg_at_1000`
1367	`value: 50.589`
1368	`- type: ndcg_at_3`
1369	`value: 36.695`
1370	`- type: ndcg_at_5`
1371	`value: 38.193`
1372	`- type: precision_at_1`
1373	`value: 41.358`
1374	`- type: precision_at_10`
1375	`value: 11.142000000000001`
1376	`- type: precision_at_100`
1377	`value: 1.8350000000000002`
1378	`- type: precision_at_1000`
1379	`value: 0.234`
1380	`- type: precision_at_3`
1381	`value: 24.023`
1382	`- type: precision_at_5`
1383	`value: 17.963`
1384	`- type: recall_at_1`
1385	`value: 20.829`
1386	`- type: recall_at_10`
1387	`value: 47.467999999999996`
1388	`- type: recall_at_100`
1389	`value: 73.593`
1390	`- type: recall_at_1000`
1391	`value: 90.122`
1392	`- type: recall_at_3`
1393	`value: 32.74`
1394	`- type: recall_at_5`
1395	`value: 39.608`
1396	`- task:`
1397	`type: Retrieval`
1398	`dataset:`
1399	`type: hotpotqa`
1400	`name: MTEB HotpotQA`
1401	`config: default`
1402	`split: test`
1403	`revision: None`
1404	`metrics:`
1405	`- type: map_at_1`
1406	`value: 40.324`
1407	`- type: map_at_10`
1408	`value: 64.183`
1409	`- type: map_at_100`
1410	`value: 65.037`
1411	`- type: map_at_1000`
1412	`value: 65.094`
1413	`- type: map_at_3`
1414	`value: 60.663`
1415	`- type: map_at_5`
1416	`value: 62.951`
1417	`- type: mrr_at_1`
1418	`value: 80.648`
1419	`- type: mrr_at_10`
1420	`value: 86.005`
1421	`- type: mrr_at_100`
1422	`value: 86.157`
1423	`- type: mrr_at_1000`
1424	`value: 86.162`
1425	`- type: mrr_at_3`
1426	`value: 85.116`
1427	`- type: mrr_at_5`
1428	`value: 85.703`
1429	`- type: ndcg_at_1`
1430	`value: 80.648`
1431	`- type: ndcg_at_10`
1432	`value: 72.351`
1433	`- type: ndcg_at_100`
1434	`value: 75.279`
1435	`- type: ndcg_at_1000`
1436	`value: 76.357`
1437	`- type: ndcg_at_3`
1438	`value: 67.484`
1439	`- type: ndcg_at_5`
1440	`value: 70.31500000000001`
1441	`- type: precision_at_1`
1442	`value: 80.648`
1443	`- type: precision_at_10`
1444	`value: 15.103`
1445	`- type: precision_at_100`
1446	`value: 1.7399999999999998`
1447	`- type: precision_at_1000`
1448	`value: 0.188`
1449	`- type: precision_at_3`
1450	`value: 43.232`
1451	`- type: precision_at_5`
1452	`value: 28.165000000000003`
1453	`- type: recall_at_1`
1454	`value: 40.324`
1455	`- type: recall_at_10`
1456	`value: 75.517`
1457	`- type: recall_at_100`
1458	`value: 86.982`
1459	`- type: recall_at_1000`
1460	`value: 94.072`
1461	`- type: recall_at_3`
1462	`value: 64.848`
1463	`- type: recall_at_5`
1464	`value: 70.41199999999999`
1465	`- task:`
1466	`type: Classification`
1467	`dataset:`
1468	`type: mteb/imdb`
1469	`name: MTEB ImdbClassification`
1470	`config: default`
1471	`split: test`
1472	`revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7`
1473	`metrics:`
1474	`- type: accuracy`
1475	`value: 91.4`
1476	`- type: ap`
1477	`value: 87.4422032289312`
1478	`- type: f1`
1479	`value: 91.39249564302281`
1480	`- task:`
1481	`type: Retrieval`
1482	`dataset:`
1483	`type: msmarco`
1484	`name: MTEB MSMARCO`
1485	`config: default`
1486	`split: dev`
1487	`revision: None`
1488	`metrics:`
1489	`- type: map_at_1`
1490	`value: 22.03`
1491	`- type: map_at_10`
1492	`value: 34.402`
1493	`- type: map_at_100`
1494	`value: 35.599`
1495	`- type: map_at_1000`
1496	`value: 35.648`
1497	`- type: map_at_3`
1498	`value: 30.603`
1499	`- type: map_at_5`
1500	`value: 32.889`
1501	`- type: mrr_at_1`
1502	`value: 22.679`
1503	`- type: mrr_at_10`
1504	`value: 35.021`
1505	`- type: mrr_at_100`
1506	`value: 36.162`
1507	`- type: mrr_at_1000`
1508	`value: 36.205`
1509	`- type: mrr_at_3`
1510	`value: 31.319999999999997`
1511	`- type: mrr_at_5`
1512	`value: 33.562`
1513	`- type: ndcg_at_1`
1514	`value: 22.692999999999998`
1515	`- type: ndcg_at_10`
1516	`value: 41.258`
1517	`- type: ndcg_at_100`
1518	`value: 46.967`
1519	`- type: ndcg_at_1000`
1520	`value: 48.175000000000004`
1521	`- type: ndcg_at_3`
1522	`value: 33.611000000000004`
1523	`- type: ndcg_at_5`
1524	`value: 37.675`
1525	`- type: precision_at_1`
1526	`value: 22.692999999999998`
1527	`- type: precision_at_10`
1528	`value: 6.5089999999999995`
1529	`- type: precision_at_100`
1530	`value: 0.936`
1531	`- type: precision_at_1000`
1532	`value: 0.104`
1533	`- type: precision_at_3`
1534	`value: 14.413`
1535	`- type: precision_at_5`
1536	`value: 10.702`
1537	`- type: recall_at_1`
1538	`value: 22.03`
1539	`- type: recall_at_10`
1540	`value: 62.248000000000005`
1541	`- type: recall_at_100`
1542	`value: 88.524`
1543	`- type: recall_at_1000`
1544	`value: 97.714`
1545	`- type: recall_at_3`
1546	`value: 41.617`
1547	`- type: recall_at_5`
1548	`value: 51.359`
1549	`- task:`
1550	`type: Classification`
1551	`dataset:`
1552	`type: mteb/mtop_domain`
1553	`name: MTEB MTOPDomainClassification (en)`
1554	`config: en`
1555	`split: test`
1556	`revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf`
1557	`metrics:`
1558	`- type: accuracy`
1559	`value: 94.36844505243957`
1560	`- type: f1`
1561	`value: 94.12408743818202`
1562	`- task:`
1563	`type: Classification`
1564	`dataset:`
1565	`type: mteb/mtop_intent`
1566	`name: MTEB MTOPIntentClassification (en)`
1567	`config: en`
1568	`split: test`
1569	`revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba`
1570	`metrics:`
1571	`- type: accuracy`
1572	`value: 76.43410852713177`
1573	`- type: f1`
1574	`value: 58.501855709435624`
1575	`- task:`
1576	`type: Classification`
1577	`dataset:`
1578	`type: mteb/amazon_massive_intent`
1579	`name: MTEB MassiveIntentClassification (en)`
1580	`config: en`
1581	`split: test`
1582	`revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7`
1583	`metrics:`
1584	`- type: accuracy`
1585	`value: 76.04909213180902`
1586	`- type: f1`
1587	`value: 74.1800860395823`
1588	`- task:`
1589	`type: Classification`
1590	`dataset:`
1591	`type: mteb/amazon_massive_scenario`
1592	`name: MTEB MassiveScenarioClassification (en)`
1593	`config: en`
1594	`split: test`
1595	`revision: 7d571f92784cd94a019292a1f45445077d0ef634`
1596	`metrics:`
1597	`- type: accuracy`
1598	`value: 79.76126429051781`
1599	`- type: f1`
1600	`value: 79.85705217473232`
1601	`- task:`
1602	`type: Clustering`
1603	`dataset:`
1604	`type: mteb/medrxiv-clustering-p2p`
1605	`name: MTEB MedrxivClusteringP2P`
1606	`config: default`
1607	`split: test`
1608	`revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73`
1609	`metrics:`
1610	`- type: v_measure`
1611	`value: 34.70119520292863`
1612	`- task:`
1613	`type: Clustering`
1614	`dataset:`
1615	`type: mteb/medrxiv-clustering-s2s`
1616	`name: MTEB MedrxivClusteringS2S`
1617	`config: default`
1618	`split: test`
1619	`revision: 35191c8c0dca72d8ff3efcd72aa802307d469663`
1620	`metrics:`
1621	`- type: v_measure`
1622	`value: 32.33544316467486`
1623	`- task:`
1624	`type: Reranking`
1625	`dataset:`
1626	`type: mteb/mind_small`
1627	`name: MTEB MindSmallReranking`
1628	`config: default`
1629	`split: test`
1630	`revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69`
1631	`metrics:`
1632	`- type: map`
1633	`value: 30.75499243990726`
1634	`- type: mrr`
1635	`value: 31.70602251821063`
1636	`- task:`
1637	`type: Retrieval`
1638	`dataset:`
1639	`type: nfcorpus`
1640	`name: MTEB NFCorpus`
1641	`config: default`
1642	`split: test`
1643	`revision: None`
1644	`metrics:`
1645	`- type: map_at_1`
1646	`value: 6.451999999999999`
1647	`- type: map_at_10`
1648	`value: 13.918`
1649	`- type: map_at_100`
1650	`value: 17.316000000000003`
1651	`- type: map_at_1000`
1652	`value: 18.747`
1653	`- type: map_at_3`
1654	`value: 10.471`
1655	`- type: map_at_5`
1656	`value: 12.104`
1657	`- type: mrr_at_1`
1658	`value: 46.749`
1659	`- type: mrr_at_10`
1660	`value: 55.717000000000006`
1661	`- type: mrr_at_100`
1662	`value: 56.249`
1663	`- type: mrr_at_1000`
1664	`value: 56.288000000000004`
1665	`- type: mrr_at_3`
1666	`value: 53.818`
1667	`- type: mrr_at_5`
1668	`value: 55.103`
1669	`- type: ndcg_at_1`
1670	`value: 45.201`
1671	`- type: ndcg_at_10`
1672	`value: 35.539`
1673	`- type: ndcg_at_100`
1674	`value: 32.586`
1675	`- type: ndcg_at_1000`
1676	`value: 41.486000000000004`
1677	`- type: ndcg_at_3`
1678	`value: 41.174`
1679	`- type: ndcg_at_5`
1680	`value: 38.939`
1681	`- type: precision_at_1`
1682	`value: 46.749`
1683	`- type: precision_at_10`
1684	`value: 25.944`
1685	`- type: precision_at_100`
1686	`value: 8.084`
1687	`- type: precision_at_1000`
1688	`value: 2.076`
1689	`- type: precision_at_3`
1690	`value: 38.7`
1691	`- type: precision_at_5`
1692	`value: 33.56`
1693	`- type: recall_at_1`
1694	`value: 6.451999999999999`
1695	`- type: recall_at_10`
1696	`value: 17.302`
1697	`- type: recall_at_100`
1698	`value: 32.14`
1699	`- type: recall_at_1000`
1700	`value: 64.12`
1701	`- type: recall_at_3`
1702	`value: 11.219`
1703	`- type: recall_at_5`
1704	`value: 13.993`
1705	`- task:`
1706	`type: Retrieval`
1707	`dataset:`
1708	`type: nq`
1709	`name: MTEB NQ`
1710	`config: default`
1711	`split: test`
1712	`revision: None`
1713	`metrics:`
1714	`- type: map_at_1`
1715	`value: 32.037`
1716	`- type: map_at_10`
1717	`value: 46.565`
1718	`- type: map_at_100`
1719	`value: 47.606`
1720	`- type: map_at_1000`
1721	`value: 47.636`
1722	`- type: map_at_3`
1723	`value: 42.459`
1724	`- type: map_at_5`
1725	`value: 44.762`
1726	`- type: mrr_at_1`
1727	`value: 36.181999999999995`
1728	`- type: mrr_at_10`
1729	`value: 49.291000000000004`
1730	`- type: mrr_at_100`
1731	`value: 50.059`
1732	`- type: mrr_at_1000`
1733	`value: 50.078`
1734	`- type: mrr_at_3`
1735	`value: 45.829`
1736	`- type: mrr_at_5`
1737	`value: 47.797`
1738	`- type: ndcg_at_1`
1739	`value: 36.153`
1740	`- type: ndcg_at_10`
1741	`value: 53.983000000000004`
1742	`- type: ndcg_at_100`
1743	`value: 58.347`
1744	`- type: ndcg_at_1000`
1745	`value: 59.058`
1746	`- type: ndcg_at_3`
1747	`value: 46.198`
1748	`- type: ndcg_at_5`
1749	`value: 50.022`
1750	`- type: precision_at_1`
1751	`value: 36.153`
1752	`- type: precision_at_10`
1753	`value: 8.763`
1754	`- type: precision_at_100`
1755	`value: 1.123`
1756	`- type: precision_at_1000`
1757	`value: 0.11900000000000001`
1758	`- type: precision_at_3`
1759	`value: 20.751`
1760	`- type: precision_at_5`
1761	`value: 14.646999999999998`
1762	`- type: recall_at_1`
1763	`value: 32.037`
1764	`- type: recall_at_10`
1765	`value: 74.008`
1766	`- type: recall_at_100`
1767	`value: 92.893`
1768	`- type: recall_at_1000`
1769	`value: 98.16`
1770	`- type: recall_at_3`
1771	`value: 53.705999999999996`
1772	`- type: recall_at_5`
1773	`value: 62.495`
1774	`- task:`
1775	`type: Retrieval`
1776	`dataset:`
1777	`type: quora`
1778	`name: MTEB QuoraRetrieval`
1779	`config: default`
1780	`split: test`
1781	`revision: None`
1782	`metrics:`
1783	`- type: map_at_1`
1784	`value: 71.152`
1785	`- type: map_at_10`
1786	`value: 85.104`
1787	`- type: map_at_100`
1788	`value: 85.745`
1789	`- type: map_at_1000`
1790	`value: 85.761`
1791	`- type: map_at_3`
1792	`value: 82.175`
1793	`- type: map_at_5`
1794	`value: 84.066`
1795	`- type: mrr_at_1`
1796	`value: 82.03`
1797	`- type: mrr_at_10`
1798	`value: 88.115`
1799	`- type: mrr_at_100`
1800	`value: 88.21`
1801	`- type: mrr_at_1000`
1802	`value: 88.211`
1803	`- type: mrr_at_3`
1804	`value: 87.19200000000001`
1805	`- type: mrr_at_5`
1806	`value: 87.85`
1807	`- type: ndcg_at_1`
1808	`value: 82.03`
1809	`- type: ndcg_at_10`
1810	`value: 88.78`
1811	`- type: ndcg_at_100`
1812	`value: 89.96300000000001`
1813	`- type: ndcg_at_1000`
1814	`value: 90.056`
1815	`- type: ndcg_at_3`
1816	`value: 86.051`
1817	`- type: ndcg_at_5`
1818	`value: 87.63499999999999`
1819	`- type: precision_at_1`
1820	`value: 82.03`
1821	`- type: precision_at_10`
1822	`value: 13.450000000000001`
1823	`- type: precision_at_100`
1824	`value: 1.5310000000000001`
1825	`- type: precision_at_1000`
1826	`value: 0.157`
1827	`- type: precision_at_3`
1828	`value: 37.627`
1829	`- type: precision_at_5`
1830	`value: 24.784`
1831	`- type: recall_at_1`
1832	`value: 71.152`
1833	`- type: recall_at_10`
1834	`value: 95.649`
1835	`- type: recall_at_100`
1836	`value: 99.58200000000001`
1837	`- type: recall_at_1000`
1838	`value: 99.981`
1839	`- type: recall_at_3`
1840	`value: 87.767`
1841	`- type: recall_at_5`
1842	`value: 92.233`
1843	`- task:`
1844	`type: Clustering`
1845	`dataset:`
1846	`type: mteb/reddit-clustering`
1847	`name: MTEB RedditClustering`
1848	`config: default`
1849	`split: test`
1850	`revision: 24640382cdbf8abc73003fb0fa6d111a705499eb`
1851	`metrics:`
1852	`- type: v_measure`
1853	`value: 56.48713646277477`
1854	`- task:`
1855	`type: Clustering`
1856	`dataset:`
1857	`type: mteb/reddit-clustering-p2p`
1858	`name: MTEB RedditClusteringP2P`
1859	`config: default`
1860	`split: test`
1861	`revision: 282350215ef01743dc01b456c7f5241fa8937f16`
1862	`metrics:`
1863	`- type: v_measure`
1864	`value: 63.394940772438545`
1865	`- task:`
1866	`type: Retrieval`
1867	`dataset:`
1868	`type: scidocs`
1869	`name: MTEB SCIDOCS`
1870	`config: default`
1871	`split: test`
1872	`revision: None`
1873	`metrics:`
1874	`- type: map_at_1`
1875	`value: 5.043`
1876	`- type: map_at_10`
1877	`value: 12.949`
1878	`- type: map_at_100`
1879	`value: 15.146`
1880	`- type: map_at_1000`
1881	`value: 15.495000000000001`
1882	`- type: map_at_3`
1883	`value: 9.333`
1884	`- type: map_at_5`
1885	`value: 11.312999999999999`
1886	`- type: mrr_at_1`
1887	`value: 24.9`
1888	`- type: mrr_at_10`
1889	`value: 35.958`
1890	`- type: mrr_at_100`
1891	`value: 37.152`
1892	`- type: mrr_at_1000`
1893	`value: 37.201`
1894	`- type: mrr_at_3`
1895	`value: 32.667`
1896	`- type: mrr_at_5`
1897	`value: 34.567`
1898	`- type: ndcg_at_1`
1899	`value: 24.9`
1900	`- type: ndcg_at_10`
1901	`value: 21.298000000000002`
1902	`- type: ndcg_at_100`
1903	`value: 29.849999999999998`
1904	`- type: ndcg_at_1000`
1905	`value: 35.506`
1906	`- type: ndcg_at_3`
1907	`value: 20.548`
1908	`- type: ndcg_at_5`
1909	`value: 18.064`
1910	`- type: precision_at_1`
1911	`value: 24.9`
1912	`- type: precision_at_10`
1913	`value: 10.9`
1914	`- type: precision_at_100`
1915	`value: 2.331`
1916	`- type: precision_at_1000`
1917	`value: 0.367`
1918	`- type: precision_at_3`
1919	`value: 19.267`
1920	`- type: precision_at_5`
1921	`value: 15.939999999999998`
1922	`- type: recall_at_1`
1923	`value: 5.043`
1924	`- type: recall_at_10`
1925	`value: 22.092`
1926	`- type: recall_at_100`
1927	`value: 47.323`
1928	`- type: recall_at_1000`
1929	`value: 74.553`
1930	`- type: recall_at_3`
1931	`value: 11.728`
1932	`- type: recall_at_5`
1933	`value: 16.188`
1934	`- task:`
1935	`type: STS`
1936	`dataset:`
1937	`type: mteb/sickr-sts`
1938	`name: MTEB SICK-R`
1939	`config: default`
1940	`split: test`
1941	`revision: a6ea5a8cab320b040a23452cc28066d9beae2cee`
1942	`metrics:`
1943	`- type: cos_sim_pearson`
1944	`value: 83.7007085938325`
1945	`- type: cos_sim_spearman`
1946	`value: 80.0171084446234`
1947	`- type: euclidean_pearson`
1948	`value: 81.28133218355893`
1949	`- type: euclidean_spearman`
1950	`value: 79.99291731740131`
1951	`- type: manhattan_pearson`
1952	`value: 81.22926922327846`
1953	`- type: manhattan_spearman`
1954	`value: 79.94444878127038`
1955	`- task:`
1956	`type: STS`
1957	`dataset:`
1958	`type: mteb/sts12-sts`
1959	`name: MTEB STS12`
1960	`config: default`
1961	`split: test`
1962	`revision: a0d554a64d88156834ff5ae9920b964011b16384`
1963	`metrics:`
1964	`- type: cos_sim_pearson`
1965	`value: 85.7411883252923`
1966	`- type: cos_sim_spearman`
1967	`value: 77.93462937801245`
1968	`- type: euclidean_pearson`
1969	`value: 83.00858563882404`
1970	`- type: euclidean_spearman`
1971	`value: 77.82717362433257`
1972	`- type: manhattan_pearson`
1973	`value: 82.92887645790769`
1974	`- type: manhattan_spearman`
1975	`value: 77.78807488222115`
1976	`- task:`
1977	`type: STS`
1978	`dataset:`
1979	`type: mteb/sts13-sts`
1980	`name: MTEB STS13`
1981	`config: default`
1982	`split: test`
1983	`revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca`
1984	`metrics:`
1985	`- type: cos_sim_pearson`
1986	`value: 82.04222459361023`
1987	`- type: cos_sim_spearman`
1988	`value: 83.85931509330395`
1989	`- type: euclidean_pearson`
1990	`value: 83.26916063876055`
1991	`- type: euclidean_spearman`
1992	`value: 83.98621985648353`
1993	`- type: manhattan_pearson`
1994	`value: 83.14935679184327`
1995	`- type: manhattan_spearman`
1996	`value: 83.87938828586304`
1997	`- task:`
1998	`type: STS`
1999	`dataset:`
2000	`type: mteb/sts14-sts`
2001	`name: MTEB STS14`
2002	`config: default`
2003	`split: test`
2004	`revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375`
2005	`metrics:`
2006	`- type: cos_sim_pearson`
2007	`value: 81.41136639535318`
2008	`- type: cos_sim_spearman`
2009	`value: 81.51200091040481`
2010	`- type: euclidean_pearson`
2011	`value: 81.45382456114775`
2012	`- type: euclidean_spearman`
2013	`value: 81.46201181707931`
2014	`- type: manhattan_pearson`
2015	`value: 81.37243088439584`
2016	`- type: manhattan_spearman`
2017	`value: 81.39828421893426`
2018	`- task:`
2019	`type: STS`
2020	`dataset:`
2021	`type: mteb/sts15-sts`
2022	`name: MTEB STS15`
2023	`config: default`
2024	`split: test`
2025	`revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3`
2026	`metrics:`
2027	`- type: cos_sim_pearson`
2028	`value: 85.71942451732227`
2029	`- type: cos_sim_spearman`
2030	`value: 87.33044482064973`
2031	`- type: euclidean_pearson`
2032	`value: 86.58580899365178`
2033	`- type: euclidean_spearman`
2034	`value: 87.09206723832895`
2035	`- type: manhattan_pearson`
2036	`value: 86.47460784157013`
2037	`- type: manhattan_spearman`
2038	`value: 86.98367656583076`
2039	`- task:`
2040	`type: STS`
2041	`dataset:`
2042	`type: mteb/sts16-sts`
2043	`name: MTEB STS16`
2044	`config: default`
2045	`split: test`
2046	`revision: 4d8694f8f0e0100860b497b999b3dbed754a0513`
2047	`metrics:`
2048	`- type: cos_sim_pearson`
2049	`value: 83.55868078863449`
2050	`- type: cos_sim_spearman`
2051	`value: 85.38299230074065`
2052	`- type: euclidean_pearson`
2053	`value: 84.64715256244595`
2054	`- type: euclidean_spearman`
2055	`value: 85.49112229604047`
2056	`- type: manhattan_pearson`
2057	`value: 84.60814346792462`
2058	`- type: manhattan_spearman`
2059	`value: 85.44886026766822`
2060	`- task:`
2061	`type: STS`
2062	`dataset:`
2063	`type: mteb/sts17-crosslingual-sts`
2064	`name: MTEB STS17 (en-en)`
2065	`config: en-en`
2066	`split: test`
2067	`revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d`
2068	`metrics:`
2069	`- type: cos_sim_pearson`
2070	`value: 84.99292526370614`
2071	`- type: cos_sim_spearman`
2072	`value: 85.58139465695983`
2073	`- type: euclidean_pearson`
2074	`value: 86.51325066734084`
2075	`- type: euclidean_spearman`
2076	`value: 85.56736418284562`
2077	`- type: manhattan_pearson`
2078	`value: 86.48190836601357`
2079	`- type: manhattan_spearman`
2080	`value: 85.51616256224258`
2081	`- task:`
2082	`type: STS`
2083	`dataset:`
2084	`type: mteb/sts22-crosslingual-sts`
2085	`name: MTEB STS22 (en)`
2086	`config: en`
2087	`split: test`
2088	`revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80`
2089	`metrics:`
2090	`- type: cos_sim_pearson`
2091	`value: 64.54124715078807`
2092	`- type: cos_sim_spearman`
2093	`value: 65.32134275948374`
2094	`- type: euclidean_pearson`
2095	`value: 67.09791698300816`
2096	`- type: euclidean_spearman`
2097	`value: 65.79468982468465`
2098	`- type: manhattan_pearson`
2099	`value: 67.13304723693966`
2100	`- type: manhattan_spearman`
2101	`value: 65.68439995849283`
2102	`- task:`
2103	`type: STS`
2104	`dataset:`
2105	`type: mteb/stsbenchmark-sts`
2106	`name: MTEB STSBenchmark`
2107	`config: default`
2108	`split: test`
2109	`revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831`
2110	`metrics:`
2111	`- type: cos_sim_pearson`
2112	`value: 83.4231099581624`
2113	`- type: cos_sim_spearman`
2114	`value: 85.95475815226862`
2115	`- type: euclidean_pearson`
2116	`value: 85.00339401999706`
2117	`- type: euclidean_spearman`
2118	`value: 85.74133081802971`
2119	`- type: manhattan_pearson`
2120	`value: 85.00407987181666`
2121	`- type: manhattan_spearman`
2122	`value: 85.77509596397363`
2123	`- task:`
2124	`type: Reranking`
2125	`dataset:`
2126	`type: mteb/scidocs-reranking`
2127	`name: MTEB SciDocsRR`
2128	`config: default`
2129	`split: test`
2130	`revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab`
2131	`metrics:`
2132	`- type: map`
2133	`value: 87.25666719585716`
2134	`- type: mrr`
2135	`value: 96.32769917083642`
2136	`- task:`
2137	`type: Retrieval`
2138	`dataset:`
2139	`type: scifact`
2140	`name: MTEB SciFact`
2141	`config: default`
2142	`split: test`
2143	`revision: None`
2144	`metrics:`
2145	`- type: map_at_1`
2146	`value: 57.828`
2147	`- type: map_at_10`
2148	`value: 68.369`
2149	`- type: map_at_100`
2150	`value: 68.83399999999999`
2151	`- type: map_at_1000`
2152	`value: 68.856`
2153	`- type: map_at_3`
2154	`value: 65.38000000000001`
2155	`- type: map_at_5`
2156	`value: 67.06299999999999`
2157	`- type: mrr_at_1`
2158	`value: 61`
2159	`- type: mrr_at_10`
2160	`value: 69.45400000000001`
2161	`- type: mrr_at_100`
2162	`value: 69.785`
2163	`- type: mrr_at_1000`
2164	`value: 69.807`
2165	`- type: mrr_at_3`
2166	`value: 67`
2167	`- type: mrr_at_5`
2168	`value: 68.43299999999999`
2169	`- type: ndcg_at_1`
2170	`value: 61`
2171	`- type: ndcg_at_10`
2172	`value: 73.258`
2173	`- type: ndcg_at_100`
2174	`value: 75.173`
2175	`- type: ndcg_at_1000`
2176	`value: 75.696`
2177	`- type: ndcg_at_3`
2178	`value: 68.162`
2179	`- type: ndcg_at_5`
2180	`value: 70.53399999999999`
2181	`- type: precision_at_1`
2182	`value: 61`
2183	`- type: precision_at_10`
2184	`value: 9.8`
2185	`- type: precision_at_100`
2186	`value: 1.087`
2187	`- type: precision_at_1000`
2188	`value: 0.11299999999999999`
2189	`- type: precision_at_3`
2190	`value: 27`
2191	`- type: precision_at_5`
2192	`value: 17.666999999999998`
2193	`- type: recall_at_1`
2194	`value: 57.828`
2195	`- type: recall_at_10`
2196	`value: 87.122`
2197	`- type: recall_at_100`
2198	`value: 95.667`
2199	`- type: recall_at_1000`
2200	`value: 99.667`
2201	`- type: recall_at_3`
2202	`value: 73.139`
2203	`- type: recall_at_5`
2204	`value: 79.361`
2205	`- task:`
2206	`type: PairClassification`
2207	`dataset:`
2208	`type: mteb/sprintduplicatequestions-pairclassification`
2209	`name: MTEB SprintDuplicateQuestions`
2210	`config: default`
2211	`split: test`
2212	`revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46`
2213	`metrics:`
2214	`- type: cos_sim_accuracy`
2215	`value: 99.85247524752475`
2216	`- type: cos_sim_ap`
2217	`value: 96.25640197639723`
2218	`- type: cos_sim_f1`
2219	`value: 92.37851662404091`
2220	`- type: cos_sim_precision`
2221	`value: 94.55497382198953`
2222	`- type: cos_sim_recall`
2223	`value: 90.3`
2224	`- type: dot_accuracy`
2225	`value: 99.76138613861386`
2226	`- type: dot_ap`
2227	`value: 93.40295864389073`
2228	`- type: dot_f1`
2229	`value: 87.64267990074441`
2230	`- type: dot_precision`
2231	`value: 86.99507389162562`
2232	`- type: dot_recall`
2233	`value: 88.3`
2234	`- type: euclidean_accuracy`
2235	`value: 99.85049504950496`
2236	`- type: euclidean_ap`
2237	`value: 96.24254350525462`
2238	`- type: euclidean_f1`
2239	`value: 92.32323232323232`
2240	`- type: euclidean_precision`
2241	`value: 93.26530612244898`
2242	`- type: euclidean_recall`
2243	`value: 91.4`
2244	`- type: manhattan_accuracy`
2245	`value: 99.85346534653465`
2246	`- type: manhattan_ap`
2247	`value: 96.2635334753325`
2248	`- type: manhattan_f1`
2249	`value: 92.37899073120495`
2250	`- type: manhattan_precision`
2251	`value: 95.22292993630573`
2252	`- type: manhattan_recall`
2253	`value: 89.7`
2254	`- type: max_accuracy`
2255	`value: 99.85346534653465`
2256	`- type: max_ap`
2257	`value: 96.2635334753325`
2258	`- type: max_f1`
2259	`value: 92.37899073120495`
2260	`- task:`
2261	`type: Clustering`
2262	`dataset:`
2263	`type: mteb/stackexchange-clustering`
2264	`name: MTEB StackExchangeClustering`
2265	`config: default`
2266	`split: test`
2267	`revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259`
2268	`metrics:`
2269	`- type: v_measure`
2270	`value: 65.83905786483794`
2271	`- task:`
2272	`type: Clustering`
2273	`dataset:`
2274	`type: mteb/stackexchange-clustering-p2p`
2275	`name: MTEB StackExchangeClusteringP2P`
2276	`config: default`
2277	`split: test`
2278	`revision: 815ca46b2622cec33ccafc3735d572c266efdb44`
2279	`metrics:`
2280	`- type: v_measure`
2281	`value: 35.031896152126436`
2282	`- task:`
2283	`type: Reranking`
2284	`dataset:`
2285	`type: mteb/stackoverflowdupquestions-reranking`
2286	`name: MTEB StackOverflowDupQuestions`
2287	`config: default`
2288	`split: test`
2289	`revision: e185fbe320c72810689fc5848eb6114e1ef5ec69`
2290	`metrics:`
2291	`- type: map`
2292	`value: 54.551326709447146`
2293	`- type: mrr`
2294	`value: 55.43758222986165`
2295	`- task:`
2296	`type: Summarization`
2297	`dataset:`
2298	`type: mteb/summeval`
2299	`name: MTEB SummEval`
2300	`config: default`
2301	`split: test`
2302	`revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c`
2303	`metrics:`
2304	`- type: cos_sim_pearson`
2305	`value: 30.305688567308874`
2306	`- type: cos_sim_spearman`
2307	`value: 29.27135743434515`
2308	`- type: dot_pearson`
2309	`value: 30.336741878796563`
2310	`- type: dot_spearman`
2311	`value: 30.513365725895937`
2312	`- task:`
2313	`type: Retrieval`
2314	`dataset:`
2315	`type: trec-covid`
2316	`name: MTEB TRECCOVID`
2317	`config: default`
2318	`split: test`
2319	`revision: None`
2320	`metrics:`
2321	`- type: map_at_1`
2322	`value: 0.245`
2323	`- type: map_at_10`
2324	`value: 1.92`
2325	`- type: map_at_100`
2326	`value: 10.519`
2327	`- type: map_at_1000`
2328	`value: 23.874000000000002`
2329	`- type: map_at_3`
2330	`value: 0.629`
2331	`- type: map_at_5`
2332	`value: 1.0290000000000001`
2333	`- type: mrr_at_1`
2334	`value: 88`
2335	`- type: mrr_at_10`
2336	`value: 93.5`
2337	`- type: mrr_at_100`
2338	`value: 93.5`
2339	`- type: mrr_at_1000`
2340	`value: 93.5`
2341	`- type: mrr_at_3`
2342	`value: 93`
2343	`- type: mrr_at_5`
2344	`value: 93.5`
2345	`- type: ndcg_at_1`
2346	`value: 84`
2347	`- type: ndcg_at_10`
2348	`value: 76.447`
2349	`- type: ndcg_at_100`
2350	`value: 56.516`
2351	`- type: ndcg_at_1000`
2352	`value: 48.583999999999996`
2353	`- type: ndcg_at_3`
2354	`value: 78.877`
2355	`- type: ndcg_at_5`
2356	`value: 79.174`
2357	`- type: precision_at_1`
2358	`value: 88`
2359	`- type: precision_at_10`
2360	`value: 80.60000000000001`
2361	`- type: precision_at_100`
2362	`value: 57.64`
2363	`- type: precision_at_1000`
2364	`value: 21.227999999999998`
2365	`- type: precision_at_3`
2366	`value: 82`
2367	`- type: precision_at_5`
2368	`value: 83.6`
2369	`- type: recall_at_1`
2370	`value: 0.245`
2371	`- type: recall_at_10`
2372	`value: 2.128`
2373	`- type: recall_at_100`
2374	`value: 13.767`
2375	`- type: recall_at_1000`
2376	`value: 44.958`
2377	`- type: recall_at_3`
2378	`value: 0.654`
2379	`- type: recall_at_5`
2380	`value: 1.111`
2381	`- task:`
2382	`type: Retrieval`
2383	`dataset:`
2384	`type: webis-touche2020`
2385	`name: MTEB Touche2020`
2386	`config: default`
2387	`split: test`
2388	`revision: None`
2389	`metrics:`
2390	`- type: map_at_1`
2391	`value: 2.5170000000000003`
2392	`- type: map_at_10`
2393	`value: 10.915`
2394	`- type: map_at_100`
2395	`value: 17.535`
2396	`- type: map_at_1000`
2397	`value: 19.042`
2398	`- type: map_at_3`
2399	`value: 5.689`
2400	`- type: map_at_5`
2401	`value: 7.837`
2402	`- type: mrr_at_1`
2403	`value: 34.694`
2404	`- type: mrr_at_10`
2405	`value: 49.547999999999995`
2406	`- type: mrr_at_100`
2407	`value: 50.653000000000006`
2408	`- type: mrr_at_1000`
2409	`value: 50.653000000000006`
2410	`- type: mrr_at_3`
2411	`value: 44.558`
2412	`- type: mrr_at_5`
2413	`value: 48.333`
2414	`- type: ndcg_at_1`
2415	`value: 32.653`
2416	`- type: ndcg_at_10`
2417	`value: 26.543`
2418	`- type: ndcg_at_100`
2419	`value: 38.946`
2420	`- type: ndcg_at_1000`
2421	`value: 49.406`
2422	`- type: ndcg_at_3`
2423	`value: 29.903000000000002`
2424	`- type: ndcg_at_5`
2425	`value: 29.231`
2426	`- type: precision_at_1`
2427	`value: 34.694`
2428	`- type: precision_at_10`
2429	`value: 23.265`
2430	`- type: precision_at_100`
2431	`value: 8.102`
2432	`- type: precision_at_1000`
2433	`value: 1.5`
2434	`- type: precision_at_3`
2435	`value: 31.293`
2436	`- type: precision_at_5`
2437	`value: 29.796`
2438	`- type: recall_at_1`
2439	`value: 2.5170000000000003`
2440	`- type: recall_at_10`
2441	`value: 16.88`
2442	`- type: recall_at_100`
2443	`value: 49.381`
2444	`- type: recall_at_1000`
2445	`value: 81.23899999999999`
2446	`- type: recall_at_3`
2447	`value: 6.965000000000001`
2448	`- type: recall_at_5`
2449	`value: 10.847999999999999`
2450	`- task:`
2451	`type: Classification`
2452	`dataset:`
2453	`type: mteb/toxic_conversations_50k`
2454	`name: MTEB ToxicConversationsClassification`
2455	`config: default`
2456	`split: test`
2457	`revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c`
2458	`metrics:`
2459	`- type: accuracy`
2460	`value: 71.5942`
2461	`- type: ap`
2462	`value: 13.92074156956546`
2463	`- type: f1`
2464	`value: 54.671999698839066`
2465	`- task:`
2466	`type: Classification`
2467	`dataset:`
2468	`type: mteb/tweet_sentiment_extraction`
2469	`name: MTEB TweetSentimentExtractionClassification`
2470	`config: default`
2471	`split: test`
2472	`revision: d604517c81ca91fe16a244d1248fc021f9ecee7a`
2473	`metrics:`
2474	`- type: accuracy`
2475	`value: 59.39728353140916`
2476	`- type: f1`
2477	`value: 59.68980496759517`
2478	`- task:`
2479	`type: Clustering`
2480	`dataset:`
2481	`type: mteb/twentynewsgroups-clustering`
2482	`name: MTEB TwentyNewsgroupsClustering`
2483	`config: default`
2484	`split: test`
2485	`revision: 6125ec4e24fa026cec8a478383ee943acfbd5449`
2486	`metrics:`
2487	`- type: v_measure`
2488	`value: 52.11181870104935`
2489	`- task:`
2490	`type: PairClassification`
2491	`dataset:`
2492	`type: mteb/twittersemeval2015-pairclassification`
2493	`name: MTEB TwitterSemEval2015`
2494	`config: default`
2495	`split: test`
2496	`revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1`
2497	`metrics:`
2498	`- type: cos_sim_accuracy`
2499	`value: 86.46957143708649`
2500	`- type: cos_sim_ap`
2501	`value: 76.16120197845457`
2502	`- type: cos_sim_f1`
2503	`value: 69.69919295671315`
2504	`- type: cos_sim_precision`
2505	`value: 64.94986326344576`
2506	`- type: cos_sim_recall`
2507	`value: 75.19788918205805`
2508	`- type: dot_accuracy`
2509	`value: 83.0780234845324`
2510	`- type: dot_ap`
2511	`value: 64.21717343541934`
2512	`- type: dot_f1`
2513	`value: 59.48375497624245`
2514	`- type: dot_precision`
2515	`value: 57.94345759319489`
2516	`- type: dot_recall`
2517	`value: 61.108179419525065`
2518	`- type: euclidean_accuracy`
2519	`value: 86.6543482148179`
2520	`- type: euclidean_ap`
2521	`value: 76.4527555010203`
2522	`- type: euclidean_f1`
2523	`value: 70.10156056477584`
2524	`- type: euclidean_precision`
2525	`value: 66.05975723622782`
2526	`- type: euclidean_recall`
2527	`value: 74.67018469656992`
2528	`- type: manhattan_accuracy`
2529	`value: 86.66030875603504`
2530	`- type: manhattan_ap`
2531	`value: 76.40304567255436`
2532	`- type: manhattan_f1`
2533	`value: 70.05275426328058`
2534	`- type: manhattan_precision`
2535	`value: 65.4666360926393`
2536	`- type: manhattan_recall`
2537	`value: 75.32981530343008`
2538	`- type: max_accuracy`
2539	`value: 86.66030875603504`
2540	`- type: max_ap`
2541	`value: 76.4527555010203`
2542	`- type: max_f1`
2543	`value: 70.10156056477584`
2544	`- task:`
2545	`type: PairClassification`
2546	`dataset:`
2547	`type: mteb/twitterurlcorpus-pairclassification`
2548	`name: MTEB TwitterURLCorpus`
2549	`config: default`
2550	`split: test`
2551	`revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf`
2552	`metrics:`
2553	`- type: cos_sim_accuracy`
2554	`value: 88.42123646524624`
2555	`- type: cos_sim_ap`
2556	`value: 85.15431437761646`
2557	`- type: cos_sim_f1`
2558	`value: 76.98069301530742`
2559	`- type: cos_sim_precision`
2560	`value: 72.9314502239063`
2561	`- type: cos_sim_recall`
2562	`value: 81.50600554357868`
2563	`- type: dot_accuracy`
2564	`value: 86.70974502270346`
2565	`- type: dot_ap`
2566	`value: 80.77621563599457`
2567	`- type: dot_f1`
2568	`value: 73.87058697285117`
2569	`- type: dot_precision`
2570	`value: 68.98256396552877`
2571	`- type: dot_recall`
2572	`value: 79.50415768401602`
2573	`- type: euclidean_accuracy`
2574	`value: 88.46392672798541`
2575	`- type: euclidean_ap`
2576	`value: 85.20370297495491`
2577	`- type: euclidean_f1`
2578	`value: 77.01372369624886`
2579	`- type: euclidean_precision`
2580	`value: 73.39052800446397`
2581	`- type: euclidean_recall`
2582	`value: 81.01324299353249`
2583	`- type: manhattan_accuracy`
2584	`value: 88.43481973066325`
2585	`- type: manhattan_ap`
2586	`value: 85.16318289864545`
2587	`- type: manhattan_f1`
2588	`value: 76.90884877182597`
2589	`- type: manhattan_precision`
2590	`value: 74.01737396753062`
2591	`- type: manhattan_recall`
2592	`value: 80.03541730828458`
2593	`- type: max_accuracy`
2594	`value: 88.46392672798541`
2595	`- type: max_ap`
2596	`value: 85.20370297495491`
2597	`- type: max_f1`
2598	`value: 77.01372369624886`
2599	`license: mit`
2600	`language:`
2601	`- en`
2602	`---`
2603
2604
2605	`Recommend switching to newest [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5), which has more reasonable similarity distribution and same method of usage.`
2606
2607	`<h1 align="center">FlagEmbedding</h1>`
2608
2609
2610	`<h4 align="center">`
2611	`<p>`
2612	`<a href=#model-list>Model List</a> \|`
2613	`<a href=#frequently-asked-questions>FAQ</a> \|`
2614	`<a href=#usage>Usage</a> \|`
2615	`<a href="#evaluation">Evaluation</a> \|`
2616	`<a href="#train">Train</a> \|`
2617	`<a href="#contact">Contact</a> \|`
2618	`<a href="#citation">Citation</a> \|`
2619	`<a href="#license">License</a>`
2620	`<p>`
2621	`</h4>`
2622
2623	`More details please refer to our Github: [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding).`
2624
2625
2626	`[English](README.md) \| [中文](https://github.com/FlagOpen/FlagEmbedding/blob/master/README_zh.md)`
2627
2628	`FlagEmbedding can map any text to a low-dimensional dense vector which can be used for tasks like retrieval, classification, clustering, or semantic search.`
2629	`And it also can be used in vector databases for LLMs.`
2630
2631	`*********** 🌟Updates🌟 ***********`
2632	`- 10/12/2023: Release [LLM-Embedder](./FlagEmbedding/llm_embedder/README.md), a unified embedding model to support diverse retrieval augmentation needs for LLMs. [Paper](https://arxiv.org/pdf/2310.07554.pdf) :fire:`
2633	`- 09/15/2023: The [technical report](https://arxiv.org/pdf/2309.07597.pdf) of BGE has been released`
2634	`- 09/15/2023: The [masive training data](https://data.baai.ac.cn/details/BAAI-MTP) of BGE has been released`
2635	`- 09/12/2023: New models:`
2636	- New reranker model: release cross-encoder models `BAAI/bge-reranker-base` and `BAAI/bge-reranker-large`, which are more powerful than embedding model. We recommend to use/fine-tune them to re-rank top-k documents returned by embedding models.
2637	- update embedding model: release `bge-*-v1.5` embedding model to alleviate the issue of the similarity distribution, and enhance its retrieval ability without instruction.
2638
2639
2640	`<details>`
2641	`<summary>More</summary>`
2642	`<!-- ### More -->`
2643
2644	`- 09/07/2023: Update [fine-tune code](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md): Add script to mine hard negatives and support adding instruction during fine-tuning.`
2645	`- 08/09/2023: BGE Models are integrated into Langchain, you can use it like [this](#using-langchain); C-MTEB leaderboard is [available](https://huggingface.co/spaces/mteb/leaderboard).`
2646	`- 08/05/2023: Release base-scale and small-scale models, best performance among the models of the same size 🤗`
2647	- 08/02/2023: Release `bge-large-`(short for BAAI General Embedding) Models, rank 1st on MTEB and C-MTEB benchmark!* :tada: :tada:
2648	`- 08/01/2023: We release the [Chinese Massive Text Embedding Benchmark](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB) (C-MTEB), consisting of 31 test dataset.`
2649
2650	`</details>`
2651
2652
2653	`## Model List`
2654
2655	`bge` is short for `BAAI general embedding`.
2656
2657	`\| Model \| Language \| \| Description \| query instruction for retrieval [1] \|`
2658	`\|:-------------------------------\|:--------:\| :--------:\| :--------:\|:--------:\|`
2659	`\| [BAAI/llm-embedder](https://huggingface.co/BAAI/llm-embedder) \| English \| [Inference](./FlagEmbedding/llm_embedder/README.md) [Fine-tune](./FlagEmbedding/llm_embedder/README.md) \| a unified embedding model to support diverse retrieval augmentation needs for LLMs \| See [README](./FlagEmbedding/llm_embedder/README.md) \|`
2660	`\| [BAAI/bge-reranker-large](https://huggingface.co/BAAI/bge-reranker-large) \| Chinese and English \| [Inference](#usage-for-reranker) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/reranker) \| a cross-encoder model which is more accurate but less efficient [2] \| \|`
2661	`\| [BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base) \| Chinese and English \| [Inference](#usage-for-reranker) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/reranker) \| a cross-encoder model which is more accurate but less efficient [2] \| \|`
2662	\| [BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5) \| English \| [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) \| version 1.5 with more reasonable similarity distribution \| `Represent this sentence for searching relevant passages: ` \|
2663	\| [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) \| English \| [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) \| version 1.5 with more reasonable similarity distribution \| `Represent this sentence for searching relevant passages: ` \|
2664	\| [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) \| English \| [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) \| version 1.5 with more reasonable similarity distribution \| `Represent this sentence for searching relevant passages: ` \|
2665	\| [BAAI/bge-large-zh-v1.5](https://huggingface.co/BAAI/bge-large-zh-v1.5) \| Chinese \| [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) \| version 1.5 with more reasonable similarity distribution \| `为这个句子生成表示以用于检索相关文章：` \|
2666	\| [BAAI/bge-base-zh-v1.5](https://huggingface.co/BAAI/bge-base-zh-v1.5) \| Chinese \| [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) \| version 1.5 with more reasonable similarity distribution \| `为这个句子生成表示以用于检索相关文章：` \|
2667	\| [BAAI/bge-small-zh-v1.5](https://huggingface.co/BAAI/bge-small-zh-v1.5) \| Chinese \| [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) \| version 1.5 with more reasonable similarity distribution \| `为这个句子生成表示以用于检索相关文章：` \|
2668	\| [BAAI/bge-large-en](https://huggingface.co/BAAI/bge-large-en) \| English \| [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) \| :trophy: rank 1st in [MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard \| `Represent this sentence for searching relevant passages: ` \|
2669	\| [BAAI/bge-base-en](https://huggingface.co/BAAI/bge-base-en) \| English \| [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) \| a base-scale model but with similar ability to `bge-large-en` \| `Represent this sentence for searching relevant passages: ` \|
2670	\| [BAAI/bge-small-en](https://huggingface.co/BAAI/bge-small-en) \| English \| [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) \|a small-scale model but with competitive performance \| `Represent this sentence for searching relevant passages: ` \|
2671	\| [BAAI/bge-large-zh](https://huggingface.co/BAAI/bge-large-zh) \| Chinese \| [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) \| :trophy: rank 1st in [C-MTEB](https://github.com/FlagOpen/FlagEmbedding/tree/master/C_MTEB) benchmark \| `为这个句子生成表示以用于检索相关文章：` \|
2672	\| [BAAI/bge-base-zh](https://huggingface.co/BAAI/bge-base-zh) \| Chinese \| [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) \| a base-scale model but with similar ability to `bge-large-zh` \| `为这个句子生成表示以用于检索相关文章：` \|
2673	\| [BAAI/bge-small-zh](https://huggingface.co/BAAI/bge-small-zh) \| Chinese \| [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) \| a small-scale model but with competitive performance \| `为这个句子生成表示以用于检索相关文章：` \|
2674
2675
2676	`[1\]: If you need to search the relevant passages to a query, we suggest to add the instruction to the query; in other cases, no instruction is needed, just use the original query directly. In all cases, no instruction needs to be added to passages.`
2677
2678	`[2\]: Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding. To balance the accuracy and time cost, cross-encoder is widely used to re-rank top-k documents retrieved by other simple models.`
2679	`For examples, use bge embedding model to retrieve top 100 relevant documents, and then use bge reranker to re-rank the top 100 document to get the final top-3 results.`
2680
2681	`All models have been uploaded to Huggingface Hub, and you can see them at https://huggingface.co/BAAI.`
2682	`If you cannot open the Huggingface Hub, you also can download the models at https://model.baai.ac.cn/models .`
2683
2684
2685	`## Frequently asked questions`
2686
2687	`<details>`
2688	`<summary>1. How to fine-tune bge embedding model?</summary>`
2689
2690	`<!-- ### How to fine-tune bge embedding model? -->`
2691	`Following this [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) to prepare data and fine-tune your model.`
2692	`Some suggestions:`
2693	`- Mine hard negatives following this [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune#hard-negatives), which can improve the retrieval performance.`
2694	`- If you pre-train bge on your data, the pre-trained model cannot be directly used to calculate similarity, and it must be fine-tuned with contrastive learning before computing similarity.`
2695	`- If the accuracy of the fine-tuned model is still not high, it is recommended to use/fine-tune the cross-encoder model (bge-reranker) to re-rank top-k results. Hard negatives also are needed to fine-tune reranker.`
2696
2697
2698	`</details>`
2699
2700	`<details>`
2701	`<summary>2. The similarity score between two dissimilar sentences is higher than 0.5</summary>`
2702
2703	`<!-- ### The similarity score between two dissimilar sentences is higher than 0.5 -->`
2704	`Suggest to use bge v1.5, which alleviates the issue of the similarity distribution.`
2705
2706	`Since we finetune the models by contrastive learning with a temperature of 0.01,`
2707	`the similarity distribution of the current BGE model is about in the interval \[0.6, 1\].`
2708	`So a similarity score greater than 0.5 does not indicate that the two sentences are similar.`
2709
2710	`For downstream tasks, such as passage retrieval or semantic similarity,`
2711	`what matters is the relative order of the scores, not the absolute value.`
2712	`If you need to filter similar sentences based on a similarity threshold,`
2713	`please select an appropriate similarity threshold based on the similarity distribution on your data (such as 0.8, 0.85, or even 0.9).`
2714
2715	`</details>`
2716
2717	`<details>`
2718	`<summary>3. When does the query instruction need to be used</summary>`
2719
2720	`<!-- ### When does the query instruction need to be used -->`
2721
2722	For the `bge-*-v1.5`, we improve its retrieval ability when not using instruction.
2723	`No instruction only has a slight degradation in retrieval performance compared with using instruction.`
2724	`So you can generate embedding without instruction in all cases for convenience.`
2725
2726	`For a retrieval task that uses short queries to find long related documents,`
2727	`it is recommended to add instructions for these short queries.`
2728	`The best method to decide whether to add instructions for queries is choosing the setting that achieves better performance on your task.`
2729	`In all cases, the documents/passages do not need to add the instruction.`
2730
2731	`</details>`
2732
2733
2734	`## Usage`
2735
2736	`### Usage for Embedding Model`
2737
2738	Here are some examples for using `bge` models with
2739	`[FlagEmbedding](#using-flagembedding), [Sentence-Transformers](#using-sentence-transformers), [Langchain](#using-langchain), or [Huggingface Transformers](#using-huggingface-transformers).`
2740
2741	`#### Using FlagEmbedding`
2742	```
2743	`pip install -U FlagEmbedding`
2744	```
2745	`If it doesn't work for you, you can see [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md) for more methods to install FlagEmbedding.`
2746
2747	```python
2748	`from FlagEmbedding import FlagModel`
2749	`sentences_1 = ["样例数据-1", "样例数据-2"]`
2750	`sentences_2 = ["样例数据-3", "样例数据-4"]`
2751	`model = FlagModel('BAAI/bge-large-zh-v1.5',`
2752	`query_instruction_for_retrieval="为这个句子生成表示以用于检索相关文章：",`
2753	`use_fp16=True) # Setting use_fp16 to True speeds up computation with a slight performance degradation`
2754	`embeddings_1 = model.encode(sentences_1)`
2755	`embeddings_2 = model.encode(sentences_2)`
2756	`similarity = embeddings_1 @ embeddings_2.T`
2757	`print(similarity)`
2758
2759	`# for s2p(short query to long passage) retrieval task, suggest to use encode_queries() which will automatically add the instruction to each query`
2760	`# corpus in retrieval task can still use encode() or encode_corpus(), since they don't need instruction`
2761	`queries = ['query_1', 'query_2']`
2762	`passages = ["样例文档-1", "样例文档-2"]`
2763	`q_embeddings = model.encode_queries(queries)`
2764	`p_embeddings = model.encode(passages)`
2765	`scores = q_embeddings @ p_embeddings.T`
2766	```
2767	For the value of the argument `query_instruction_for_retrieval`, see [Model List](https://github.com/FlagOpen/FlagEmbedding/tree/master#model-list).
2768
2769	By default, FlagModel will use all available GPUs when encoding. Please set `os.environ["CUDA_VISIBLE_DEVICES"]` to select specific GPUs.
2770	You also can set `os.environ["CUDA_VISIBLE_DEVICES"]=""` to make all GPUs unavailable.
2771
2772
2773	`#### Using Sentence-Transformers`
2774
2775	You can also use the `bge` models with [sentence-transformers](https://www.SBERT.net):
2776
2777	```
2778	`pip install -U sentence-transformers`
2779	```
2780	```python
2781	`from sentence_transformers import SentenceTransformer`
2782	`sentences_1 = ["样例数据-1", "样例数据-2"]`
2783	`sentences_2 = ["样例数据-3", "样例数据-4"]`
2784	`model = SentenceTransformer('BAAI/bge-large-zh-v1.5')`
2785	`embeddings_1 = model.encode(sentences_1, normalize_embeddings=True)`
2786	`embeddings_2 = model.encode(sentences_2, normalize_embeddings=True)`
2787	`similarity = embeddings_1 @ embeddings_2.T`
2788	`print(similarity)`
2789	```
2790	`For s2p(short query to long passage) retrieval task,`
2791	`each short query should start with an instruction (instructions see [Model List](https://github.com/FlagOpen/FlagEmbedding/tree/master#model-list)).`
2792	`But the instruction is not needed for passages.`
2793	```python
2794	`from sentence_transformers import SentenceTransformer`
2795	`queries = ['query_1', 'query_2']`
2796	`passages = ["样例文档-1", "样例文档-2"]`
2797	`instruction = "为这个句子生成表示以用于检索相关文章："`
2798
2799	`model = SentenceTransformer('BAAI/bge-large-zh-v1.5')`
2800	`q_embeddings = model.encode([instruction+q for q in queries], normalize_embeddings=True)`
2801	`p_embeddings = model.encode(passages, normalize_embeddings=True)`
2802	`scores = q_embeddings @ p_embeddings.T`
2803	```
2804
2805	`#### Using Langchain`
2806
2807	You can use `bge` in langchain like this:
2808	```python
2809	`from langchain.embeddings import HuggingFaceBgeEmbeddings`
2810	`model_name = "BAAI/bge-large-en-v1.5"`
2811	`model_kwargs = {'device': 'cuda'}`
2812	`encode_kwargs = {'normalize_embeddings': True} # set True to compute cosine similarity`
2813	`model = HuggingFaceBgeEmbeddings(`
2814	`model_name=model_name,`
2815	`model_kwargs=model_kwargs,`
2816	`encode_kwargs=encode_kwargs,`
2817	`query_instruction="为这个句子生成表示以用于检索相关文章："`
2818	`)`
2819	`model.query_instruction = "为这个句子生成表示以用于检索相关文章："`
2820	```
2821
2822
2823	`#### Using HuggingFace Transformers`
2824
2825	`With the transformers package, you can use the model like this: First, you pass your input through the transformer model, then you select the last hidden state of the first token (i.e., [CLS]) as the sentence embedding.`
2826
2827	```python
2828	`from transformers import AutoTokenizer, AutoModel`
2829	`import torch`
2830	`# Sentences we want sentence embeddings for`
2831	`sentences = ["样例数据-1", "样例数据-2"]`
2832
2833	`# Load model from HuggingFace Hub`
2834	`tokenizer = AutoTokenizer.from_pretrained('BAAI/bge-large-zh-v1.5')`
2835	`model = AutoModel.from_pretrained('BAAI/bge-large-zh-v1.5')`
2836	`model.eval()`
2837
2838	`# Tokenize sentences`
2839	`encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')`
2840	`# for s2p(short query to long passage) retrieval task, add an instruction to query (not add instruction for passages)`
2841	`# encoded_input = tokenizer([instruction + q for q in queries], padding=True, truncation=True, return_tensors='pt')`
2842
2843	`# Compute token embeddings`
2844	`with torch.no_grad():`
2845	`model_output = model(**encoded_input)`
2846	`# Perform pooling. In this case, cls pooling.`
2847	`sentence_embeddings = model_output[0][:, 0]`
2848	`# normalize embeddings`
2849	`sentence_embeddings = torch.nn.functional.normalize(sentence_embeddings, p=2, dim=1)`
2850	`print("Sentence embeddings:", sentence_embeddings)`
2851	```
2852
2853	`### Usage for Reranker`
2854
2855	`Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding.`
2856	`You can get a relevance score by inputting query and passage to the reranker.`
2857	`The reranker is optimized based cross-entropy loss, so the relevance score is not bounded to a specific range.`
2858
2859
2860	`#### Using FlagEmbedding`
2861	```
2862	`pip install -U FlagEmbedding`
2863	```
2864
2865	`Get relevance scores (higher scores indicate more relevance):`
2866	```python
2867	`from FlagEmbedding import FlagReranker`
2868	`reranker = FlagReranker('BAAI/bge-reranker-large', use_fp16=True) # Setting use_fp16 to True speeds up computation with a slight performance degradation`
2869
2870	`score = reranker.compute_score(['query', 'passage'])`
2871	`print(score)`
2872
2873	`scores = reranker.compute_score([['what is panda?', 'hi'], ['what is panda?', 'The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.']])`
2874	`print(scores)`
2875	```
2876
2877
2878	`#### Using Huggingface transformers`
2879
2880	```python
2881	`import torch`
2882	`from transformers import AutoModelForSequenceClassification, AutoTokenizer`
2883
2884	`tokenizer = AutoTokenizer.from_pretrained('BAAI/bge-reranker-large')`
2885	`model = AutoModelForSequenceClassification.from_pretrained('BAAI/bge-reranker-large')`
2886	`model.eval()`
2887
2888	`pairs = [['what is panda?', 'hi'], ['what is panda?', 'The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.']]`
2889	`with torch.no_grad():`
2890	`inputs = tokenizer(pairs, padding=True, truncation=True, return_tensors='pt', max_length=512)`
2891	`scores = model(**inputs, return_dict=True).logits.view(-1, ).float()`
2892	`print(scores)`
2893	```
2894
2895	`## Evaluation`
2896
2897	`baai-general-embedding` models achieve state-of-the-art performance on both MTEB and C-MTEB leaderboard!
2898	`For more details and evaluation tools see our [scripts](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB/README.md).`
2899
2900	`- MTEB:`
2901
2902	`\| Model Name \| Dimension \| Sequence Length \| Average (56) \| Retrieval (15) \|Clustering (11) \| Pair Classification (3) \| Reranking (4) \| STS (10) \| Summarization (1) \| Classification (12) \|`
2903	`\|:----:\|:---:\|:---:\|:---:\|:---:\|:---:\|:---:\|:---:\|:---:\|:---:\|:---:\|`
2904	`\| [BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5) \| 1024 \| 512 \| 64.23 \| 54.29 \| 46.08 \| 87.12 \| 60.03 \| 83.11 \| 31.61 \| 75.97 \|`
2905	`\| [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) \| 768 \| 512 \| 63.55 \| 53.25 \| 45.77 \| 86.55 \| 58.86 \| 82.4 \| 31.07 \| 75.53 \|`
2906	`\| [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) \| 384 \| 512 \| 62.17 \|51.68 \| 43.82 \| 84.92 \| 58.36 \| 81.59 \| 30.12 \| 74.14 \|`
2907	`\| [bge-large-en](https://huggingface.co/BAAI/bge-large-en) \| 1024 \| 512 \| 63.98 \| 53.9 \| 46.98 \| 85.8 \| 59.48 \| 81.56 \| 32.06 \| 76.21 \|`
2908	`\| [bge-base-en](https://huggingface.co/BAAI/bge-base-en) \| 768 \| 512 \| 63.36 \| 53.0 \| 46.32 \| 85.86 \| 58.7 \| 81.84 \| 29.27 \| 75.27 \|`
2909	`\| [gte-large](https://huggingface.co/thenlper/gte-large) \| 1024 \| 512 \| 63.13 \| 52.22 \| 46.84 \| 85.00 \| 59.13 \| 83.35 \| 31.66 \| 73.33 \|`
2910	`\| [gte-base](https://huggingface.co/thenlper/gte-base) \| 768 \| 512 \| 62.39 \| 51.14 \| 46.2 \| 84.57 \| 58.61 \| 82.3 \| 31.17 \| 73.01 \|`
2911	`\| [e5-large-v2](https://huggingface.co/intfloat/e5-large-v2) \| 1024\| 512 \| 62.25 \| 50.56 \| 44.49 \| 86.03 \| 56.61 \| 82.05 \| 30.19 \| 75.24 \|`
2912	`\| [bge-small-en](https://huggingface.co/BAAI/bge-small-en) \| 384 \| 512 \| 62.11 \| 51.82 \| 44.31 \| 83.78 \| 57.97 \| 80.72 \| 30.53 \| 74.37 \|`
2913	`\| [instructor-xl](https://huggingface.co/hkunlp/instructor-xl) \| 768 \| 512 \| 61.79 \| 49.26 \| 44.74 \| 86.62 \| 57.29 \| 83.06 \| 32.32 \| 61.79 \|`
2914	`\| [e5-base-v2](https://huggingface.co/intfloat/e5-base-v2) \| 768 \| 512 \| 61.5 \| 50.29 \| 43.80 \| 85.73 \| 55.91 \| 81.05 \| 30.28 \| 73.84 \|`
2915	`\| [gte-small](https://huggingface.co/thenlper/gte-small) \| 384 \| 512 \| 61.36 \| 49.46 \| 44.89 \| 83.54 \| 57.7 \| 82.07 \| 30.42 \| 72.31 \|`
2916	`\| [text-embedding-ada-002](https://platform.openai.com/docs/guides/embeddings) \| 1536 \| 8192 \| 60.99 \| 49.25 \| 45.9 \| 84.89 \| 56.32 \| 80.97 \| 30.8 \| 70.93 \|`
2917	`\| [e5-small-v2](https://huggingface.co/intfloat/e5-base-v2) \| 384 \| 512 \| 59.93 \| 49.04 \| 39.92 \| 84.67 \| 54.32 \| 80.39 \| 31.16 \| 72.94 \|`
2918	`\| [sentence-t5-xxl](https://huggingface.co/sentence-transformers/sentence-t5-xxl) \| 768 \| 512 \| 59.51 \| 42.24 \| 43.72 \| 85.06 \| 56.42 \| 82.63 \| 30.08 \| 73.42 \|`
2919	`\| [all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) \| 768 \| 514 \| 57.78 \| 43.81 \| 43.69 \| 83.04 \| 59.36 \| 80.28 \| 27.49 \| 65.07 \|`
2920	`\| [sgpt-bloom-7b1-msmarco](https://huggingface.co/bigscience/sgpt-bloom-7b1-msmarco) \| 4096 \| 2048 \| 57.59 \| 48.22 \| 38.93 \| 81.9 \| 55.65 \| 77.74 \| 33.6 \| 66.19 \|`
2921
2922
2923
2924	`- C-MTEB:`
2925	`We create the benchmark C-MTEB for Chinese text embedding which consists of 31 datasets from 6 tasks.`
2926	`Please refer to [C_MTEB](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB/README.md) for a detailed introduction.`
2927
2928	`\| Model \| Embedding dimension \| Avg \| Retrieval \| STS \| PairClassification \| Classification \| Reranking \| Clustering \|`
2929	`\|:-------------------------------\|:--------:\|:--------:\|:--------:\|:--------:\|:--------:\|:--------:\|:--------:\|:--------:\|`
2930	`\| [BAAI/bge-large-zh-v1.5](https://huggingface.co/BAAI/bge-large-zh-v1.5) \| 1024 \| 64.53 \| 70.46 \| 56.25 \| 81.6 \| 69.13 \| 65.84 \| 48.99 \|`
2931	`\| [BAAI/bge-base-zh-v1.5](https://huggingface.co/BAAI/bge-base-zh-v1.5) \| 768 \| 63.13 \| 69.49 \| 53.72 \| 79.75 \| 68.07 \| 65.39 \| 47.53 \|`
2932	`\| [BAAI/bge-small-zh-v1.5](https://huggingface.co/BAAI/bge-small-zh-v1.5) \| 512 \| 57.82 \| 61.77 \| 49.11 \| 70.41 \| 63.96 \| 60.92 \| 44.18 \|`
2933	`\| [BAAI/bge-large-zh](https://huggingface.co/BAAI/bge-large-zh) \| 1024 \| 64.20 \| 71.53 \| 54.98 \| 78.94 \| 68.32 \| 65.11 \| 48.39 \|`
2934	`\| [bge-large-zh-noinstruct](https://huggingface.co/BAAI/bge-large-zh-noinstruct) \| 1024 \| 63.53 \| 70.55 \| 53 \| 76.77 \| 68.58 \| 64.91 \| 50.01 \|`
2935	`\| [BAAI/bge-base-zh](https://huggingface.co/BAAI/bge-base-zh) \| 768 \| 62.96 \| 69.53 \| 54.12 \| 77.5 \| 67.07 \| 64.91 \| 47.63 \|`
2936	`\| [multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) \| 1024 \| 58.79 \| 63.66 \| 48.44 \| 69.89 \| 67.34 \| 56.00 \| 48.23 \|`
2937	`\| [BAAI/bge-small-zh](https://huggingface.co/BAAI/bge-small-zh) \| 512 \| 58.27 \| 63.07 \| 49.45 \| 70.35 \| 63.64 \| 61.48 \| 45.09 \|`
2938	`\| [m3e-base](https://huggingface.co/moka-ai/m3e-base) \| 768 \| 57.10 \| 56.91 \| 50.47 \| 63.99 \| 67.52 \| 59.34 \| 47.68 \|`
2939	`\| [m3e-large](https://huggingface.co/moka-ai/m3e-large) \| 1024 \| 57.05 \| 54.75 \| 50.42 \| 64.3 \| 68.2 \| 59.66 \| 48.88 \|`
2940	`\| [multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) \| 768 \| 55.48 \| 61.63 \| 46.49 \| 67.07 \| 65.35 \| 54.35 \| 40.68 \|`
2941	`\| [multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small) \| 384 \| 55.38 \| 59.95 \| 45.27 \| 66.45 \| 65.85 \| 53.86 \| 45.26 \|`
2942	`\| [text-embedding-ada-002(OpenAI)](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings) \| 1536 \| 53.02 \| 52.0 \| 43.35 \| 69.56 \| 64.31 \| 54.28 \| 45.68 \|`
2943	`\| [luotuo](https://huggingface.co/silk-road/luotuo-bert-medium) \| 1024 \| 49.37 \| 44.4 \| 42.78 \| 66.62 \| 61 \| 49.25 \| 44.39 \|`
2944	`\| [text2vec-base](https://huggingface.co/shibing624/text2vec-base-chinese) \| 768 \| 47.63 \| 38.79 \| 43.41 \| 67.41 \| 62.19 \| 49.45 \| 37.66 \|`
2945	`\| [text2vec-large](https://huggingface.co/GanymedeNil/text2vec-large-chinese) \| 1024 \| 47.36 \| 41.94 \| 44.97 \| 70.86 \| 60.66 \| 49.16 \| 30.02 \|`
2946
2947
2948	`- Reranking:`
2949	`See [C_MTEB](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB/) for evaluation script.`
2950
2951	`\| Model \| T2Reranking \| T2RerankingZh2En\* \| T2RerankingEn2Zh\* \| MMarcoReranking \| CMedQAv1 \| CMedQAv2 \| Avg \|`
2952	`\|:-------------------------------\|:--------:\|:--------:\|:--------:\|:--------:\|:--------:\|:--------:\|:--------:\|`
2953	`\| text2vec-base-multilingual \| 64.66 \| 62.94 \| 62.51 \| 14.37 \| 48.46 \| 48.6 \| 50.26 \|`
2954	`\| multilingual-e5-small \| 65.62 \| 60.94 \| 56.41 \| 29.91 \| 67.26 \| 66.54 \| 57.78 \|`
2955	`\| multilingual-e5-large \| 64.55 \| 61.61 \| 54.28 \| 28.6 \| 67.42 \| 67.92 \| 57.4 \|`
2956	`\| multilingual-e5-base \| 64.21 \| 62.13 \| 54.68 \| 29.5 \| 66.23 \| 66.98 \| 57.29 \|`
2957	`\| m3e-base \| 66.03 \| 62.74 \| 56.07 \| 17.51 \| 77.05 \| 76.76 \| 59.36 \|`
2958	`\| m3e-large \| 66.13 \| 62.72 \| 56.1 \| 16.46 \| 77.76 \| 78.27 \| 59.57 \|`
2959	`\| bge-base-zh-v1.5 \| 66.49 \| 63.25 \| 57.02 \| 29.74 \| 80.47 \| 84.88 \| 63.64 \|`
2960	`\| bge-large-zh-v1.5 \| 65.74 \| 63.39 \| 57.03 \| 28.74 \| 83.45 \| 85.44 \| 63.97 \|`
2961	`\| [BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base) \| 67.28 \| 63.95 \| 60.45 \| 35.46 \| 81.26 \| 84.1 \| 65.42 \|`
2962	`\| [BAAI/bge-reranker-large](https://huggingface.co/BAAI/bge-reranker-large) \| 67.6 \| 64.03 \| 61.44 \| 37.16 \| 82.15 \| 84.18 \| 66.09 \|`
2963
2964	`\* : T2RerankingZh2En and T2RerankingEn2Zh are cross-language retrieval tasks`
2965
2966	`## Train`
2967
2968	`### BAAI Embedding`
2969
2970	`We pre-train the models using [retromae](https://github.com/staoxiao/RetroMAE) and train them on large-scale pairs data using contrastive learning.`
2971	`You can fine-tune the embedding model on your data following our [examples](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune).`
2972	`We also provide a [pre-train example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/pretrain).`
2973	`Note that the goal of pre-training is to reconstruct the text, and the pre-trained model cannot be used for similarity calculation directly, it needs to be fine-tuned.`
2974	`More training details for bge see [baai_general_embedding](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md).`
2975
2976
2977
2978	`### BGE Reranker`
2979
2980	`Cross-encoder will perform full-attention over the input pair,`
2981	`which is more accurate than embedding model (i.e., bi-encoder) but more time-consuming than embedding model.`
2982	`Therefore, it can be used to re-rank the top-k documents returned by embedding model.`
2983	`We train the cross-encoder on a multilingual pair data,`
2984	`The data format is the same as embedding model, so you can fine-tune it easily following our [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/reranker).`
2985	`More details please refer to [./FlagEmbedding/reranker/README.md](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/reranker)`
2986
2987
2988	`## Contact`
2989	`If you have any question or suggestion related to this project, feel free to open an issue or pull request.`
2990	`You also can email Shitao Xiao(stxiao@baai.ac.cn) and Zheng Liu(liuzheng@baai.ac.cn).`
2991
2992
2993	`## Citation`
2994
2995	`If you find this repository useful, please consider giving a star :star: and citation`
2996
2997	```
2998	`@misc{bge_embedding,`
2999	`title={C-Pack: Packaged Resources To Advance General Chinese Embedding},`
3000	`author={Shitao Xiao and Zheng Liu and Peitian Zhang and Niklas Muennighoff},`
3001	`year={2023},`
3002	`eprint={2309.07597},`
3003	`archivePrefix={arXiv},`
3004	`primaryClass={cs.CL}`
3005	`}`
3006	```
3007
3008	`## License`
3009	`FlagEmbedding is licensed under the [MIT License](https://github.com/FlagOpen/FlagEmbedding/blob/master/LICENSE). The released models can be used for commercial purposes free of charge.`
3010
3011