cerebras.modelzoo.data_preparation.nlp.bert.bertsum_data_processor.RougeBasedLabelsFormatter#

class cerebras.modelzoo.data_preparation.nlp.bert.bertsum_data_processor.RougeBasedLabelsFormatter[source]#

Bases: object

Based on the reference n-grams, RougeBasedLabelsFormatter selects sentences from the input with the highest rouge-score calculated between them and the reference. This is needed since we solve extractive summarization task, where target summarization is the subset of the input sentences in contrast to abstractive summarization, where summarized text is generated by the system without relying on the input text.

Methods

process

__init__()[source]#

Based on the reference n-grams, RougeBasedLabelsFormatter selects sentences from the input with the highest rouge-score calculated between them and the reference. This is needed since we solve extractive summarization task, where target summarization is the subset of the input sentences in contrast to abstractive summarization, where summarized text is generated by the system without relying on the input text.

previous

cerebras.modelzoo.data_preparation.nlp.bert.bertsum_data_processor.JsonConverter

next

cerebras.modelzoo.data_preparation.nlp.bert.bertsum_data_processor.Tokenizer