Home
last modified time | relevance | path

Searched defs:tokenizer_class (Results 1 – 4 of 4) sorted by relevance

/aosp_15_r20/external/executorch/examples/mediatek/aot_utils/llm_utils/
H A Dutils.py483 LlamaTokenizer as tokenizer_class, unknown
488 LlamaTokenizer as tokenizer_class, unknown
492 PreTrainedTokenizerFast as tokenizer_class, unknown
/aosp_15_r20/external/executorch/examples/mediatek/models/llm_models/weights/llama3-8B-instruct/
H A Dtokenizer_config.json2061 "tokenizer_class": "PreTrainedTokenizerFast" string
/aosp_15_r20/external/executorch/examples/mediatek/models/llm_models/weights/Llama-3.2-1B-Instruct/
H A Dtokenizer_config.json2061 "tokenizer_class": "PreTrainedTokenizerFast" string
/aosp_15_r20/external/executorch/examples/mediatek/models/llm_models/weights/Llama-3.2-3B-Instruct/
H A Dtokenizer_config.json2061 "tokenizer_class": "PreTrainedTokenizerFast" string