mojolearn.LanguageModelConfig#

mojolearn.LanguageModelConfig#

Methods

to_dict()

The fields as a dict, suitable for ByteLanguageModelConfig(**d).

Attributes

`batch`
`chunked_lm_head_v2`
`d_model`
`head_dim`
`intermediate`
`length`
`n_heads`
`n_kv`
`n_layers`
`n_tensors`	`2 + 9 * n_layers`.
`n_total`	Total number of parameters.
`native_shape`	The dimensions as a tuple of ints, in field order.
`offsets`	Start offset of every parameter tensor in the flat parameter vector, plus the total.
`parameter_names`	The name of every parameter tensor, in packing order.
`parameter_shapes`	The shape of every parameter tensor, in packing order.
`profile`	The name of the training profile this shape selects.
`vocab_size`