Training Issue

[visualizeMath]

Hi. First of all thank you very much for your help. You have saved my life at least several times :) My question is that I have experinced some problems while training word2vec with large data corpus. The data i'd like to use for training process is almost 4 Gb. I wonder whether if it's possible or not. I tried to train word2vec with 2 Gb data and it didn't work too.Shall i increase the heap-size or something like that ?

[eabdullin]

Can you share your training data? I'll try to train vectors :)

[CaCTuCaTu4ECKuu]

I find out where is this issue and #1
I use some 100mb internet data and it was surprise that there is exception, but ther i understand that when I do StreamReader.ReadLine() I read a whole file which is storing with only spaces and thats cause an exception. And actually I dont even sure what to do to save same performance, because there is threads and seek, but you cant just seek through single line so

[CaCTuCaTu4ECKuu]

I solve this by preprocessing train file and separating some amount of words in single line because solid line cause issues even when opening with notepad++ when opening processed files occurs instantly