initial commit of project

This commit is contained in:
2021-04-11 19:51:12 +02:00
commit a21a8186d9
110 changed files with 16326178 additions and 0 deletions

15
train_model/README.md Normal file
View File

@@ -0,0 +1,15 @@
#Vocab
To create vocab.txt file, run **make_new_vocab.py**
# Prep dataset
**prep_dataset_training**: Format and split dataset, so it can be used for training. Adapt which dataset version to make!
# train German FoodBERT
**language_modeling**
#Vocab Files:
**bert-base-german-cased_tokenizer.json**: original bert-base-german-cased tokenizer file
**bert_vocab.txt**: original bert-base-german-cased vocab
**used_ingredients**: all ingredients in dataset
**vocab.txt**: German FoodBERT vocabulary