Commit 3bcb988e authored by afshinrahimi's avatar afshinrahimi Committed by GitHub
Browse files

update reader._file_to_word_ids

_file_to_word_ids throws a key error if there is a word in the validation or test data that doesn't exist in the training data.

The unknown words won't exist in word_to_id dictionary and so will throw a key error while converting validation and test files into ids.

Changed to way the file id sequence is built by checking if the word exists in the word_to_id dictionary to prevent the key error.
parent 9725395b
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment