LayoutReader: A Powerful Tool for Reading Order Detection

LayoutReader is an innovative tool used for reading order detection that takes advantage of both textual and layout information. The tool leverages layout-aware language models like LayoutLM as an encoder. Simply put, LayoutReader is a sequence-to-sequence model that modifies the generation stage of the encoder-decoder structure to generate the reading order sequence.

Encoding Stage of LayoutReader

In the encoding stage, LayoutReader employs a unique approach that involves packing a pair of source and target sequences into a continuous input sequence using a LayoutLM. The input sequence is carefully designed with self-attention masks to control the visibility between tokens. This means that LayoutReader allows tokens in the source segment to attend to each other while preventing tokens in the target segment from attending to the rightward context. The self-attention mask is defined in detail as follows:

Here, i and j are indices in the packed input sequence, which may be from the source or target segments, and i, j ∈ src indicates that both tokens come from the source segment.

Decoding Stage of LayoutReader

In the decoding stage, the source and target segments are reordered sequences, and prediction candidates can be constrained to the source segment. Hence, LayoutReader asks the model to predict the indices in the source sequence. The probability of prediction is calculated using the following formula:

Here, i is an index in the source segment, while ei and ej are the i-th and j-th input embeddings of the source segment, respectively. hk represents the hidden states at the k-th time step, while bk is the bias at the k-th time step.

All in all, LayoutReader is an extremely valuable tool for reading order detection that combines textual and layout information. It's flexible and precise, making it an essential tool for researchers or data scientists alike.

Great! Next, complete checkout for full access to SERP AI.
Welcome back! You've successfully signed in.
You've successfully subscribed to SERP AI.
Success! Your account is fully activated, you now have access to all content.
Success! Your billing info has been updated.
Your billing was not updated.