cerebras.modelzoo.data.nlp.bert.bert_utils.Vocab#

class cerebras.modelzoo.data.nlp.bert.bert_utils.Vocab[source]#

Bases: dict

Class to store vocab related attributes.

Methods

backward

clear

copy

forward

fromkeys

Create a new dictionary with keys from iterable and values set to value.

get

Return the value for key if key is in the dictionary, else default.

items

keys

pop

If key is not found, default is returned if given, otherwise KeyError is raised

popitem

Remove and return a (key, value) pair as a 2-tuple.

setdefault

Insert key with a value of default if key is not in the dictionary.

update

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values

__init__(inp_list, oov_id=- 1)[source]#
update([E, ]**F) → None.  Update D from dict/iterable E and F.[source]#

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

__new__(**kwargs)#
clear() → None.  Remove all items from D.#
copy() → a shallow copy of D#
fromkeys(value=None, /)#

Create a new dictionary with keys from iterable and values set to value.

get(key, default=None, /)#

Return the value for key if key is in the dictionary, else default.

items() → a set-like object providing a view on D's items#
keys() → a set-like object providing a view on D's keys#
pop(k[, d]) → v, remove specified key and return the corresponding value.#

If key is not found, default is returned if given, otherwise KeyError is raised

popitem()#

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

setdefault(key, default=None, /)#

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

values() → an object providing a view on D's values#

previous

cerebras.modelzoo.data.nlp.bert.bert_utils.shard_and_shuffle_data

next

cerebras.modelzoo.data.nlp.dpo