목록Beit (2)
On the journey of
[논문읽기] BEiT : BERT Pre-training of Image Transformers
Original Paper ) BEiT: https://arxiv.org/abs/2106.08254 BEiT: BERT Pre-Training of Image Transformers We introduce a self-supervised vision representation model BEiT, which stands for Bidirectional Encoder representation from Image Transformers. Following BERT developed in the natural language processing area, we propose a masked image modeling task to pre arxiv.org Contribution BERT의 Masked Lan..
[CSE URP]BEiT: BERT Pre-Training of Image Transformers (1)
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao, Li Dong, Songhao Piao, Furu Wei * https://github.com/microsoft/unilm/tree/master/beit Abstract 논문의 제목인 BEIT (Bidirectional Encoder representation from Image Transformers)는 BERT 모델에서 차용한 것 입력 이미지를 2가지 방법을 통해 masked image modeling(MIM) 학습 VIT + Blockwise Masking : image patches (such as 16 x 16 pixles) DALL-E Tokenizer : visual toekns (i.e...