README.md
32.2 KB · 357 lines · markdown Raw
1 ---
2 tags:
3 - sentence-transformers
4 - sentence-similarity
5 - feature-extraction
6 - generated_from_trainer
7 - dataset_size:45
8 - loss:MultipleNegativesRankingLoss
9 base_model: BAAI/bge-base-en-v1.5
10 pipeline_tag: sentence-similarity
11 library_name: sentence-transformers
12 ---
13
14 # SentenceTransformer based on BAAI/bge-base-en-v1.5
15
16 This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
17
18 ## Model Details
19
20 ### Model Description
21 - **Model Type:** Sentence Transformer
22 - **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a -->
23 - **Maximum Sequence Length:** 512 tokens
24 - **Output Dimensionality:** 768 tokens
25 - **Similarity Function:** Cosine Similarity
26 <!-- - **Training Dataset:** Unknown -->
27 <!-- - **Language:** Unknown -->
28 <!-- - **License:** Unknown -->
29
30 ### Model Sources
31
32 - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
33 - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
34 - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
35
36 ### Full Model Architecture
37
38 ```
39 SentenceTransformer(
40 (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel
41 (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
42 (2): Normalize()
43 )
44 ```
45
46 ## Usage
47
48 ### Direct Usage (Sentence Transformers)
49
50 First install the Sentence Transformers library:
51
52 ```bash
53 pip install -U sentence-transformers
54 ```
55
56 Then you can load this model and run inference.
57 ```python
58 from sentence_transformers import SentenceTransformer
59
60 # Download from the 🤗 Hub
61 model = SentenceTransformer("datasocietyco/bge-base-en-v1.5-course-recommender-v5")
62 # Run inference
63 sentences = [
64 'The weather is lovely today.',
65 "It's so sunny outside!",
66 'He drove to the stadium.',
67 ]
68 embeddings = model.encode(sentences)
69 print(embeddings.shape)
70 # [3, 768]
71
72 # Get the similarity scores for the embeddings
73 similarities = model.similarity(embeddings, embeddings)
74 print(similarities.shape)
75 # [3, 3]
76 ```
77
78 <!--
79 ### Direct Usage (Transformers)
80
81 <details><summary>Click to see the direct usage in Transformers</summary>
82
83 </details>
84 -->
85
86 <!--
87 ### Downstream Usage (Sentence Transformers)
88
89 You can finetune this model on your own dataset.
90
91 <details><summary>Click to expand</summary>
92
93 </details>
94 -->
95
96 <!--
97 ### Out-of-Scope Use
98
99 *List how the model may foreseeably be misused and address what users ought not to do with the model.*
100 -->
101
102 <!--
103 ## Bias, Risks and Limitations
104
105 *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
106 -->
107
108 <!--
109 ### Recommendations
110
111 *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
112 -->
113
114 ## Training Details
115
116 ### Training Dataset
117
118 #### Unnamed Dataset
119
120
121 * Size: 45 training samples
122 * Columns: <code>anchor</code> and <code>positive</code>
123 * Approximate statistics based on the first 45 samples:
124 | | anchor | positive |
125 |:--------|:--------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------|
126 | type | string | string |
127 | details | <ul><li>min: 143 tokens</li><li>mean: 178.76 tokens</li><li>max: 258 tokens</li></ul> | <ul><li>min: 141 tokens</li><li>mean: 176.76 tokens</li><li>max: 256 tokens</li></ul> |
128 * Samples:
129 | anchor | positive |
130 |:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
131 | <code>Creating and Querying Data in SQL. This course builds on foundational data skills to teach learners how to effectively manipulate data in Structured Query Language (SQL). By the end of this course, learners will be able to describe database structures, import data into a database, and combine and manipulate data within a single table. Learners will also have gained exposure to working with data of different types, including string, numerical, and temporal.. tags: 'DML', 'analytics', 'ERD', 'SQL', 'functions', 'DQL', 'DDL'. Languages: Course language: TBD. Prerequisites: No prerequisite course required. Target audience: Professionals with limited or no experience in SQL or similar languages. Junior analysts or analysts familiar with other similar programming languages and frameworks.</code> | <code>Course Name:Creating and Querying Data in SQL|Course Description:This course builds on foundational data skills to teach learners how to effectively manipulate data in Structured Query Language (SQL). By the end of this course, learners will be able to describe database structures, import data into a database, and combine and manipulate data within a single table. Learners will also have gained exposure to working with data of different types, including string, numerical, and temporal.|Tags:'DML', 'analytics', 'ERD', 'SQL', 'functions', 'DQL', 'DDL'|Course language: TBD|Target Audience:Professionals with limited or no experience in SQL or similar languages. Junior analysts or analysts familiar with other similar programming languages and frameworks|No prerequisite course required</code> |
132 | <code>Clustering Categorical and Mixed Data in Python. In this course, learners will prepare data for, implement, and optimize three advanced clustering models in Python while comparing their different use cases. In particular, this course focuses on the suitability of different clustering methods for different kinds of data: numerical, categorical, and mixed. Learners will distinguish between k-modes, mean shift, and k-prototypes models, developing their understanding of when each model will best meet their needs.. tags: 'k-prototypes', 'mean-shift', 'clustering', 'k-modes'. Languages: Course language: Python. Prerequisites: No prerequisite course required. Target audience: This is an introductory level course for data scientists who want to learn to detect and visualize underlying patterns and groups in unlabelled data and how to handle different types of data..</code> | <code>Course Name:Clustering Categorical and Mixed Data in Python|Course Description:In this course, learners will prepare data for, implement, and optimize three advanced clustering models in Python while comparing their different use cases. In particular, this course focuses on the suitability of different clustering methods for different kinds of data: numerical, categorical, and mixed. Learners will distinguish between k-modes, mean shift, and k-prototypes models, developing their understanding of when each model will best meet their needs.|Tags:'k-prototypes', 'mean-shift', 'clustering', 'k-modes'|Course language: Python|Target Audience:This is an introductory level course for data scientists who want to learn to detect and visualize underlying patterns and groups in unlabelled data and how to handle different types of data.|No prerequisite course required</code> |
133 | <code>Hierarchical and Density-Based Clustering in Python. In this course, learners will encounter more sophisticated methods for generating clusters within unlabeled data using Python. The first method, hierarchical clustering, creates easy-to-read, tree branch-like clusters in order of increasing specificity. The second method, DBSCAN (Density-Based Spatial Clustering of Applications with Noise), creates groups based on the concentration of data points within a region, facilitating analysis of irregularly shaped data. By the end of this course, learners will prepare data for, implement, and optimize these models.. tags: 'Hierarchical', 'clustering', 'DBSCAN'. Languages: Course language: Python. Prerequisites: No prerequisite course required. Target audience: This is an introductory level course for data scientists who want to learn to detect and visualize underlying patterns and groups in unlabelled data and how to handle different types of data..</code> | <code>Course Name:Hierarchical and Density-Based Clustering in Python|Course Description:In this course, learners will encounter more sophisticated methods for generating clusters within unlabeled data using Python. The first method, hierarchical clustering, creates easy-to-read, tree branch-like clusters in order of increasing specificity. The second method, DBSCAN (Density-Based Spatial Clustering of Applications with Noise), creates groups based on the concentration of data points within a region, facilitating analysis of irregularly shaped data. By the end of this course, learners will prepare data for, implement, and optimize these models.|Tags:'Hierarchical', 'clustering', 'DBSCAN'|Course language: Python|Target Audience:This is an introductory level course for data scientists who want to learn to detect and visualize underlying patterns and groups in unlabelled data and how to handle different types of data.|No prerequisite course required</code> |
134 * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
135 ```json
136 {
137 "scale": 20.0,
138 "similarity_fct": "cos_sim"
139 }
140 ```
141
142 ### Evaluation Dataset
143
144 #### Unnamed Dataset
145
146
147 * Size: 5 evaluation samples
148 * Columns: <code>anchor</code> and <code>positive</code>
149 * Approximate statistics based on the first 5 samples:
150 | | anchor | positive |
151 |:--------|:-------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
152 | type | string | string |
153 | details | <ul><li>min: 167 tokens</li><li>mean: 211.2 tokens</li><li>max: 272 tokens</li></ul> | <ul><li>min: 165 tokens</li><li>mean: 209.2 tokens</li><li>max: 270 tokens</li></ul> |
154 * Samples:
155 | anchor | positive |
156 |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
157 | <code>Interactive Visualization with Bokeh in Python. Bokeh is a Python library designed for building complex, multi-layered statistical plots. In this course, learners will discuss the advantages of interactive visualizations using this library. They will first generate a series of simple interactive plots, such as bar charts, scatter plots, and histograms. Then, they will clean and wrangle geospatial data to create an interactive choropleth map. They will also discuss how to export and integrate Bokeh visualizations into web sites as HTML widgets.<br>. tags: 'data story telling', 'analytics', 'interactive visualization', 'visualization', 'python', 'bokeh'. Languages: Course language: Python. Prerequisites: No prerequisite course required. Target audience: Professionals some Python experience who would like to expand their skill set to more advanced Python visualization techniques and tools..</code> | <code>Course Name:Interactive Visualization with Bokeh in Python|Course Description:Bokeh is a Python library designed for building complex, multi-layered statistical plots. In this course, learners will discuss the advantages of interactive visualizations using this library. They will first generate a series of simple interactive plots, such as bar charts, scatter plots, and histograms. Then, they will clean and wrangle geospatial data to create an interactive choropleth map. They will also discuss how to export and integrate Bokeh visualizations into web sites as HTML widgets.<br>|Tags:'data story telling', 'analytics', 'interactive visualization', 'visualization', 'python', 'bokeh'|Course language: Python|Target Audience:Professionals some Python experience who would like to expand their skill set to more advanced Python visualization techniques and tools.|No prerequisite course required</code> |
158 | <code>Data Visualization with ggplot2 in R. In this course, learners will take their R skills to the next level by preparing data for exploratory analysis and creating basic, static visualizations. Learners will begin by discussing the purpose of conducting exploratory data analysis (EDA) and best practices for profiling a dataset. Then, using both base R and tidyverse packages, learners will generate bar charts, scatter plots, histograms, and other common visualizations to better understand the shape, structure, and features of a sample dataset.<br>. tags: 'ggplot', 'analytics', 'R', 'visualization', 'tidyverse', 'EDA'. Languages: Course language: TBD. Prerequisites: Prerequisite course required: Data Wrangling in R. Target audience: Creating visualizations is a critical means of exploring data and revealing insights. In this course, learners will take their R skills to the next level by preparing data for exploratory analysis and creating basic, static visualizations. Using both base R and tidyverse packages, learners will generate bar charts, scatter plots, histograms, and other common visualizations to better understand the shape, structure, and features of a sample dataset..</code> | <code>Course Name:Data Visualization with ggplot2 in R|Course Description:In this course, learners will take their R skills to the next level by preparing data for exploratory analysis and creating basic, static visualizations. Learners will begin by discussing the purpose of conducting exploratory data analysis (EDA) and best practices for profiling a dataset. Then, using both base R and tidyverse packages, learners will generate bar charts, scatter plots, histograms, and other common visualizations to better understand the shape, structure, and features of a sample dataset.<br>|Tags:'ggplot', 'analytics', 'R', 'visualization', 'tidyverse', 'EDA'|Course language: TBD|Target Audience:Creating visualizations is a critical means of exploring data and revealing insights. In this course, learners will take their R skills to the next level by preparing data for exploratory analysis and creating basic, static visualizations. Using both base R and tidyverse packages, learners will generate bar charts, scatter plots, histograms, and other common visualizations to better understand the shape, structure, and features of a sample dataset.|Prerequisite course required: Data Wrangling in R</code> |
159 | <code>Outlier Detection for Time Series in Python. While many outlier detection techniques are suitable for general-purpose datasets, time series data involves complex dependencies that, if properly modeled, allow past values to predict future values. In this course, learners will explore how to detect outliers in time series data based on their divergence from predicted values over a particular period. They will begin by developing an Autoregressive Integrated Moving Average (ARIMA) model to extrapolate key statistical properties used in forecasting based on a historical dataset. They will then identify outliers by comparing the model's predictions against actuals as the basis of an anomaly detection model.. tags: 'anomaly detection', 'outlier detection', 'AR', 'ARIMA', 'time series', 'MA'. Languages: Course language: Python. Prerequisites: No prerequisite course required. Target audience: Professionals with some Python experience who would like to expand their skills to learn about various outlier detection techniques.</code> | <code>Course Name:Outlier Detection for Time Series in Python|Course Description:While many outlier detection techniques are suitable for general-purpose datasets, time series data involves complex dependencies that, if properly modeled, allow past values to predict future values. In this course, learners will explore how to detect outliers in time series data based on their divergence from predicted values over a particular period. They will begin by developing an Autoregressive Integrated Moving Average (ARIMA) model to extrapolate key statistical properties used in forecasting based on a historical dataset. They will then identify outliers by comparing the model's predictions against actuals as the basis of an anomaly detection model.|Tags:'anomaly detection', 'outlier detection', 'AR', 'ARIMA', 'time series', 'MA'|Course language: Python|Target Audience:Professionals with some Python experience who would like to expand their skills to learn about various outlier detection techniques|No prerequisite course required</code> |
160 * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
161 ```json
162 {
163 "scale": 20.0,
164 "similarity_fct": "cos_sim"
165 }
166 ```
167
168 ### Training Hyperparameters
169 #### Non-Default Hyperparameters
170
171 - `eval_strategy`: steps
172 - `per_device_train_batch_size`: 16
173 - `per_device_eval_batch_size`: 16
174 - `learning_rate`: 3e-06
175 - `max_steps`: 24
176 - `warmup_ratio`: 0.1
177 - `batch_sampler`: no_duplicates
178
179 #### All Hyperparameters
180 <details><summary>Click to expand</summary>
181
182 - `overwrite_output_dir`: False
183 - `do_predict`: False
184 - `eval_strategy`: steps
185 - `prediction_loss_only`: True
186 - `per_device_train_batch_size`: 16
187 - `per_device_eval_batch_size`: 16
188 - `per_gpu_train_batch_size`: None
189 - `per_gpu_eval_batch_size`: None
190 - `gradient_accumulation_steps`: 1
191 - `eval_accumulation_steps`: None
192 - `torch_empty_cache_steps`: None
193 - `learning_rate`: 3e-06
194 - `weight_decay`: 0.0
195 - `adam_beta1`: 0.9
196 - `adam_beta2`: 0.999
197 - `adam_epsilon`: 1e-08
198 - `max_grad_norm`: 1.0
199 - `num_train_epochs`: 3.0
200 - `max_steps`: 24
201 - `lr_scheduler_type`: linear
202 - `lr_scheduler_kwargs`: {}
203 - `warmup_ratio`: 0.1
204 - `warmup_steps`: 0
205 - `log_level`: passive
206 - `log_level_replica`: warning
207 - `log_on_each_node`: True
208 - `logging_nan_inf_filter`: True
209 - `save_safetensors`: True
210 - `save_on_each_node`: False
211 - `save_only_model`: False
212 - `restore_callback_states_from_checkpoint`: False
213 - `no_cuda`: False
214 - `use_cpu`: False
215 - `use_mps_device`: False
216 - `seed`: 42
217 - `data_seed`: None
218 - `jit_mode_eval`: False
219 - `use_ipex`: False
220 - `bf16`: False
221 - `fp16`: False
222 - `fp16_opt_level`: O1
223 - `half_precision_backend`: auto
224 - `bf16_full_eval`: False
225 - `fp16_full_eval`: False
226 - `tf32`: None
227 - `local_rank`: 0
228 - `ddp_backend`: None
229 - `tpu_num_cores`: None
230 - `tpu_metrics_debug`: False
231 - `debug`: []
232 - `dataloader_drop_last`: False
233 - `dataloader_num_workers`: 0
234 - `dataloader_prefetch_factor`: None
235 - `past_index`: -1
236 - `disable_tqdm`: False
237 - `remove_unused_columns`: True
238 - `label_names`: None
239 - `load_best_model_at_end`: False
240 - `ignore_data_skip`: False
241 - `fsdp`: []
242 - `fsdp_min_num_params`: 0
243 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
244 - `fsdp_transformer_layer_cls_to_wrap`: None
245 - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
246 - `deepspeed`: None
247 - `label_smoothing_factor`: 0.0
248 - `optim`: adamw_torch
249 - `optim_args`: None
250 - `adafactor`: False
251 - `group_by_length`: False
252 - `length_column_name`: length
253 - `ddp_find_unused_parameters`: None
254 - `ddp_bucket_cap_mb`: None
255 - `ddp_broadcast_buffers`: False
256 - `dataloader_pin_memory`: True
257 - `dataloader_persistent_workers`: False
258 - `skip_memory_metrics`: True
259 - `use_legacy_prediction_loop`: False
260 - `push_to_hub`: False
261 - `resume_from_checkpoint`: None
262 - `hub_model_id`: None
263 - `hub_strategy`: every_save
264 - `hub_private_repo`: False
265 - `hub_always_push`: False
266 - `gradient_checkpointing`: False
267 - `gradient_checkpointing_kwargs`: None
268 - `include_inputs_for_metrics`: False
269 - `eval_do_concat_batches`: True
270 - `fp16_backend`: auto
271 - `push_to_hub_model_id`: None
272 - `push_to_hub_organization`: None
273 - `mp_parameters`:
274 - `auto_find_batch_size`: False
275 - `full_determinism`: False
276 - `torchdynamo`: None
277 - `ray_scope`: last
278 - `ddp_timeout`: 1800
279 - `torch_compile`: False
280 - `torch_compile_backend`: None
281 - `torch_compile_mode`: None
282 - `dispatch_batches`: None
283 - `split_batches`: None
284 - `include_tokens_per_second`: False
285 - `include_num_input_tokens_seen`: False
286 - `neftune_noise_alpha`: None
287 - `optim_target_modules`: None
288 - `batch_eval_metrics`: False
289 - `eval_on_start`: False
290 - `use_liger_kernel`: False
291 - `eval_use_gather_object`: False
292 - `batch_sampler`: no_duplicates
293 - `multi_dataset_batch_sampler`: proportional
294
295 </details>
296
297 ### Training Logs
298 | Epoch | Step | Training Loss | loss |
299 |:------:|:----:|:-------------:|:------:|
300 | 6.6667 | 20 | 0.0651 | 0.0005 |
301
302
303 ### Framework Versions
304 - Python: 3.12.8
305 - Sentence Transformers: 3.1.1
306 - Transformers: 4.45.2
307 - PyTorch: 2.2.2
308 - Accelerate: 1.2.1
309 - Datasets: 3.2.0
310 - Tokenizers: 0.20.3
311
312 ## Citation
313
314 ### BibTeX
315
316 #### Sentence Transformers
317 ```bibtex
318 @inproceedings{reimers-2019-sentence-bert,
319 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
320 author = "Reimers, Nils and Gurevych, Iryna",
321 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
322 month = "11",
323 year = "2019",
324 publisher = "Association for Computational Linguistics",
325 url = "https://arxiv.org/abs/1908.10084",
326 }
327 ```
328
329 #### MultipleNegativesRankingLoss
330 ```bibtex
331 @misc{henderson2017efficient,
332 title={Efficient Natural Language Response Suggestion for Smart Reply},
333 author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
334 year={2017},
335 eprint={1705.00652},
336 archivePrefix={arXiv},
337 primaryClass={cs.CL}
338 }
339 ```
340
341 <!--
342 ## Glossary
343
344 *Clearly define terms in order to be accessible across audiences.*
345 -->
346
347 <!--
348 ## Model Card Authors
349
350 *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
351 -->
352
353 <!--
354 ## Model Card Contact
355
356 *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
357 -->