README.md
5.6 KB · 147 lines · markdown Raw
1 ---
2 language: en
3 license: mit
4 tags:
5 - sagemaker
6 - bart
7 - summarization
8 datasets:
9 - samsum
10 widget:
11 - text: "Jeff: Can I train a \U0001F917 Transformers model on Amazon SageMaker? \n\
12 Philipp: Sure you can use the new Hugging Face Deep Learning Container. \nJeff:\
13 \ ok.\nJeff: and how can I get started? \nJeff: where can I find documentation?\
14 \ \nPhilipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face\n"
15 model-index:
16 - name: bart-large-cnn-samsum
17 results:
18 - task:
19 type: summarization
20 name: Summarization
21 dataset:
22 name: 'SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization'
23 type: samsum
24 metrics:
25 - type: rogue-1
26 value: 42.621
27 name: Validation ROGUE-1
28 - type: rogue-2
29 value: 21.9825
30 name: Validation ROGUE-2
31 - type: rogue-l
32 value: 33.034
33 name: Validation ROGUE-L
34 - type: rogue-1
35 value: 41.3174
36 name: Test ROGUE-1
37 - type: rogue-2
38 value: 20.8716
39 name: Test ROGUE-2
40 - type: rogue-l
41 value: 32.1337
42 name: Test ROGUE-L
43 - task:
44 type: summarization
45 name: Summarization
46 dataset:
47 name: samsum
48 type: samsum
49 config: samsum
50 split: test
51 metrics:
52 - type: rouge
53 value: 41.3282
54 name: ROUGE-1
55 verified: true
56 verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZTYzNzZkZDUzOWQzNGYxYTJhNGE4YWYyZjA0NzMyOWUzMDNhMmVhYzY1YTM0ZTJhYjliNGE4MDZhMjhhYjRkYSIsInZlcnNpb24iOjF9.OOM6l3v5rJCndmUIJV-2SDh2NjbPo5IgQOSL-Ju1Gwbi1voL5amsDEDOelaqlUBE3n55KkUsMLZhyn66yWxZBQ
57 - type: rouge
58 value: 20.8755
59 name: ROUGE-2
60 verified: true
61 verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMWZiODFiYWQzY2NmOTc5YjA3NTI0YzQ1MzQ0ODk2NjgyMmVlMjA5MjZiNTJkMGRmZGEzN2M3MDNkMjkxMDVhYSIsInZlcnNpb24iOjF9.b8cPk2-IL24La3Vd0hhtii4tRXujh5urAwy6IVeTWHwYfXaURyC2CcQOWtlOx5bdO5KACeaJFrFBCGgjk-VGCQ
62 - type: rouge
63 value: 32.1353
64 name: ROUGE-L
65 verified: true
66 verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYWNmYzdiYWQ2ZWRkYzRiMGMxNWUwODgwZTdkY2NjZTc1NWE5NTFiMzU0OTU1N2JjN2ExYWQ2NGZkNjk5OTc4YSIsInZlcnNpb24iOjF9.Fzv4p-TEVicljiCqsBJHK1GsnE_AwGqamVmxTPI0WBNSIhZEhliRGmIL_z1pDq6WOzv3GN2YUGvhowU7GxnyAQ
67 - type: rouge
68 value: 38.401
69 name: ROUGE-LSUM
70 verified: true
71 verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGI4MWY0NWMxMmQ0ODQ5MDhiNDczMDAzYzJkODBiMzgzYWNkMWM2YTZkZDJmNWJiOGQ3MmNjMGViN2UzYWI2ZSIsInZlcnNpb24iOjF9.7lw3h5k5lJ7tYFLZGUtLyDabFYd00l6ByhmvkW4fykocBy9Blyin4tdw4Xps4DW-pmrdMLgidHxBWz5MrSx1Bw
72 - type: loss
73 value: 1.4297215938568115
74 name: loss
75 verified: true
76 verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzI0ZWNhNDM5YTViZDMyZGJjMDA1ZWFjYzNhOTdlOTFiNzhhMDBjNmM2MjA3ZmRkZjJjMjEyMGY3MzcwOTI2NyIsInZlcnNpb24iOjF9.oNaZsAtUDqGAqoZWJavlcW7PKx1AWsnkbhaQxadpOKk_u7ywJJabvTtzyx_DwEgZslgDETCf4MM-JKitZKjiDA
77 - type: gen_len
78 value: 60.0757
79 name: gen_len
80 verified: true
81 verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTgwYWYwMDRkNTJkMDM5N2I2MWNmYzQ3OWM1NDJmODUyZGViMGE4ZTdkNmIwYWM2N2VjZDNmN2RiMDE4YTYyYiIsInZlcnNpb24iOjF9.PbXTcNYX_SW-BuRQEcqyc21M7uKrOMbffQSAK6k2GLzTVRrzZxsDC57ktKL68zRY8fSiRGsnknOwv-nAR6YBCQ
82 ---
83
84 ## `bart-large-cnn-samsum`
85
86 > If you want to use the model you should try a newer fine-tuned FLAN-T5 version [philschmid/flan-t5-base-samsum](https://huggingface.co/philschmid/flan-t5-base-samsum) out socring the BART version with `+6` on `ROGUE1` achieving `47.24`.
87
88 # TRY [philschmid/flan-t5-base-samsum](https://huggingface.co/philschmid/flan-t5-base-samsum)
89
90
91 This model was trained using Amazon SageMaker and the new Hugging Face Deep Learning container.
92
93 For more information look at:
94 - [🤗 Transformers Documentation: Amazon SageMaker](https://huggingface.co/transformers/sagemaker.html)
95 - [Example Notebooks](https://github.com/huggingface/notebooks/tree/master/sagemaker)
96 - [Amazon SageMaker documentation for Hugging Face](https://docs.aws.amazon.com/sagemaker/latest/dg/hugging-face.html)
97 - [Python SDK SageMaker documentation for Hugging Face](https://sagemaker.readthedocs.io/en/stable/frameworks/huggingface/index.html)
98 - [Deep Learning Container](https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-training-containers)
99
100 ## Hyperparameters
101 ```json
102 {
103 "dataset_name": "samsum",
104 "do_eval": true,
105 "do_predict": true,
106 "do_train": true,
107 "fp16": true,
108 "learning_rate": 5e-05,
109 "model_name_or_path": "facebook/bart-large-cnn",
110 "num_train_epochs": 3,
111 "output_dir": "/opt/ml/model",
112 "per_device_eval_batch_size": 4,
113 "per_device_train_batch_size": 4,
114 "predict_with_generate": true,
115 "seed": 7
116 }
117 ```
118
119 ## Usage
120 ```python
121 from transformers import pipeline
122 summarizer = pipeline("summarization", model="philschmid/bart-large-cnn-samsum")
123
124 conversation = '''Jeff: Can I train a 🤗 Transformers model on Amazon SageMaker?
125 Philipp: Sure you can use the new Hugging Face Deep Learning Container.
126 Jeff: ok.
127 Jeff: and how can I get started?
128 Jeff: where can I find documentation?
129 Philipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face
130 '''
131 summarizer(conversation)
132 ```
133
134 ## Results
135
136 | key | value |
137 | --- | ----- |
138 | eval_rouge1 | 42.621 |
139 | eval_rouge2 | 21.9825 |
140 | eval_rougeL | 33.034 |
141 | eval_rougeLsum | 39.6783 |
142 | test_rouge1 | 41.3174 |
143 | test_rouge2 | 20.8716 |
144 | test_rougeL | 32.1337 |
145 | test_rougeLsum | 38.4149 |
146
147