led-base-16384
LED base 16384 es el modelo Longformer Encoder-Decoder de Ai2 para resumir y procesar documentos de hasta 16 384 tokens.
Tarjeta del Modelo
Introduction
Allenai's Longformer Encoder-Decoder (LED).
As described in Longformer: The Long-Document Transformer by Iz Beltagy, Matthew E. Peters, Arman Cohan, led-base-16384 was initialized from bart-base since both models share the exact same architecture. To be able to process 16K tokens, bart-base's position embedding matrix was simply copied 16 times.
This model is especially interesting for long-range summarization and question answering.
Fine-tuning for down-stream task
This notebook shows how led-base-16384 can effectively be fine-tuned on a downstream task.
Regístrate para leer casos de estudio completos, acceder a métricas detalladas y recibir todos los reportes.
Regístrate para leer casos de estudio completos, acceder a métricas detalladas y recibir todos los reportes.