README.md
1.9 KB · 55 lines · markdown Raw
1 ---
2 license: "cc-by-nc-4.0"
3 tags:
4 - vision
5 - video-classification
6 ---
7
8 # TimeSformer (base-sized model, fine-tuned on Kinetics-600)
9
10 TimeSformer model pre-trained on [Kinetics-600](https://www.deepmind.com/open-source/kinetics). It was introduced in the paper [TimeSformer: Is Space-Time Attention All You Need for Video Understanding?](https://arxiv.org/abs/2102.05095) by Tong et al. and first released in [this repository](https://github.com/facebookresearch/TimeSformer).
11
12 Disclaimer: The team releasing TimeSformer did not write a model card for this model so this model card has been written by [fcakyon](https://github.com/fcakyon).
13
14 ## Intended uses & limitations
15
16 You can use the raw model for video classification into one of the 600 possible Kinetics-600 labels.
17
18 ### How to use
19
20 Here is how to use this model to classify a video:
21
22 ```python
23 from transformers import AutoImageProcessor, TimesformerForVideoClassification
24 import numpy as np
25 import torch
26
27 video = list(np.random.randn(8, 3, 224, 224))
28
29 processor = AutoImageProcessor.from_pretrained("facebook/timesformer-base-finetuned-k600")
30 model = TimesformerForVideoClassification.from_pretrained("facebook/timesformer-base-finetuned-k600")
31
32 inputs = processor(images=video, return_tensors="pt")
33
34 with torch.no_grad():
35 outputs = model(**inputs)
36 logits = outputs.logits
37
38 predicted_class_idx = logits.argmax(-1).item()
39 print("Predicted class:", model.config.id2label[predicted_class_idx])
40 ```
41
42 For more code examples, we refer to the [documentation](https://huggingface.co/transformers/main/model_doc/timesformer.html#).
43
44 ### BibTeX entry and citation info
45
46 ```bibtex
47 @inproceedings{bertasius2021space,
48 title={Is Space-Time Attention All You Need for Video Understanding?},
49 author={Bertasius, Gedas and Wang, Heng and Torresani, Lorenzo},
50 booktitle={International Conference on Machine Learning},
51 pages={813--824},
52 year={2021},
53 organization={PMLR}
54 }
55 ```