how will you get the embeddings of long sentences/paragraphs that transformer models like BERT truncate? how will you go about using BERT for such sentences? will you use sentence embeddings or word embeddings for this? AND name models that can handle such long inputs if you know any?

Question

Accepted Answer

To get embeddings of long sentences/paragraphs truncated by BERT, we can use pooling techniques like mean/max pooling.

We can use pooling techniques like mean/max pooling to get embeddings of truncated sentences/paragraphs.
We can also use sliding window approach to get embeddings of overlapping segments of the long input.
For using BERT on such long inputs, we can use sentence embeddings or word embeddings depending on the task.
Models like Longformer and Reformer can handle long inputs without truncation.

Asked in Chubb

how will you get the embeddings of long sentences/paragraphs that transformer models like BERT truncate? how will you go about using BERT for such sentences? will you use sentence embeddings or word embeddings for this? AND name models that can handle such long inputs if you know any?

Top Data Scientist Interview Questions Asked at Chubb

Interview Questions Asked to Data Scientist at Other Companies

Top Skill-Based Questions for Chubb Data Scientist