how will you get the embeddings of long sentences/paragraphs that transformer models like BERT truncate? how will you go about using BERT for such sentences? will you use sentence embeddings or word embeddings for this? AND name models that can handle such long inputs if you know any?

Question

Accepted Answer

To get embeddings of long sentences/paragraphs truncated by BERT, we can use pooling techniques like mean/max pooling.

We can use pooling techniques like mean/max pooling to get embeddings of truncated sentences/paragraphs.
We can also use sliding window approach to get embeddings of overlapping segments of the long input.
For using BERT on such long inputs, we can use sentence embeddings or word embeddings depending on the task.
Models like Longformer and Reformer can handle long inputs without truncation.

how will you get the embeddings of long sentences/paragraphs that transformer models like BERT truncate? how will you go about using BERT for such sentences? will you use sentence embeddings or word embeddings for this? AND name models that can handle such long inputs if you know any?

Top Chubb Data Scientist interview questions & answers

Popular interview questions of Data Scientist

Top HR questions asked in Chubb Data Scientist