diff --git a/StreamingASR/CMakeLists.txt b/StreamingASR/CMakeLists.txt deleted file mode 100644 index bfc11b57..00000000 --- a/StreamingASR/CMakeLists.txt +++ /dev/null @@ -1,6 +0,0 @@ -cmake_minimum_required(VERSION 3.4.1) - -project(StreamingASR) - -add_subdirectory(external/eigen) -add_subdirectory(StreamingASR/app/src/main/cpp) diff --git a/StreamingASR/README.md b/StreamingASR/README.md index 33ac5d39..4958338c 100644 --- a/StreamingASR/README.md +++ b/StreamingASR/README.md @@ -6,9 +6,9 @@ In the Speech Recognition Android [demo app](https://github.com/pytorch/android- ## Prerequisites -* PyTorch 1.11 and torchaudio 0.11 or above (Optional) +* PyTorch 1.12 and torchaudio 0.12 or above (Optional) * Python 3.8 (Optional) -* Android Pytorch library org.pytorch:pytorch_android_lite:1.11.0 +* Android Pytorch library org.pytorch:pytorch_android_lite:1.12.2 * Android Studio 4.0.1 or later ## Quick Start @@ -22,39 +22,32 @@ git clone https://github.com/pytorch/android-demo-app cd android-demo-app/StreamingASR ``` -If you don't have PyTorch 1.11 and torchaudio 0.11 installed or want to have a quick try of the demo app, you can download the optimized scripted model file [streaming_asr.ptl](https://drive.google.com/file/d/1awT_1S6H5IXSOOqpFLmpeg0B-kQVWG2y/view?usp=sharing), then drag and drop it to the `StreamingASR/app/src/main/assets` folder inside `android-demo-app/StreamingASR`, and continue to Step 3. - -Also you need to download [Eigen](https://eigen.tuxfamily.org/), a C++ template library for linear algebra, for Android NDK build required to run the app (see last section of this README for more info): -``` -mkdir external; cd external -git clone https://github.com/jeffxtang/eigen -``` +If you don't have PyTorch 1.12 and torchaudio 0.12 installed or want to have a quick try of the demo app, you can download the optimized scripted model file [streaming_asrv2.ptl](https://drive.google.com/file/d/1XRCAFpMqOSz5e7VP0mhiACMGCCcYfpk-/view?usp=sharing), then drag and drop it to the `StreamingASR/app/src/main/assets` folder inside `android-demo-app/StreamingASR`, and continue to Step 3. ### 2. Test and Prepare the Model -To install PyTorch 1.11, torchaudio 0.11, and other required Python packages (numpy and pyaudio), do something like this: +To install PyTorch 1.12, torchaudio 0.12, and other required packages (numpy, pyaudio, and fairseq), do something like this: ``` -conda create -n pt1.11 python=3.8.5 -conda activate pt1.11 -pip install torch torchaudio numpy pyaudio +conda create -n pt1.12 python=3.8.5 +conda activate pt1.12 +pip install torch torchaudio numpy pyaudio fairseq ``` -Now download the streaming ASR model file -[scripted_wrapper_tuple_no_transform.pt](https://drive.google.com/file/d/1_49DwHS_a3p3THGdHZj3TXmjNJj60AhP/view?usp=sharing) to the `android-demo-app/StreamingASR` directory. +First, create the model file `scripted_wrapper_tuple.pt` by running `python generate_ts.py`. -To test the model, run `python run_sasr.py`. After you see: +Then, to test the model, run `python run_sasr.py`. After you see: ``` Initializing model... Initialization complete. ``` -say something like "good afternoon happy new year", and you'll likely see the streaming recognition results `▁good ▁afternoon ▁happy ▁new ▁year` while you speak. Hit Ctrl-C to end. +say something like "good afternoon happy new year", and you'll likely see the streaming recognition results `good afternoon happy new year` while you speak. Hit Ctrl-C to end. -To optimize and convert the model to the format that can run on Android, run the following commands: +Finally, to optimize and convert the model to the format that can run on Android, run the following commands: ``` mkdir -p StreamingASR/app/src/main/assets python save_model_for_mobile.py -mv streaming_asr.ptl StreamingASR/app/src/main/assets +mv streaming_asrv2.ptl StreamingASR/app/src/main/assets ``` ### 3. Build and run with Android Studio @@ -67,10 +60,6 @@ Start Android Studio, open the project located in `android-demo-app/StreamingASR ## Librosa C++, Eigen, and JNI -Note that this demo uses a [C++ port](https://github.com/ewan-xu/LibrosaCpp/) of [Librosa](https://librosa.org), a popular audio processing library in Python, to perform the MelSpectrogram transform. In the Python script `run_sasr.py` above, the torchaudio's [MelSpectrogram](https://pytorch.org/audio/stable/transforms.html#melspectrogram) is used, but you can achieve the same transform result by replacing `spectrogram = transform(tensor).transpose(1, 0)`, line 46 of run_sasr.py with: -``` -mel = librosa.feature.melspectrogram(np_array, sr=16000, n_fft=400, n_mels=80, hop_length=160) -spectrogram = torch.tensor(mel).transpose(1, 0) -``` +The first version of this demo uses a [C++ port](https://github.com/ewan-xu/LibrosaCpp/) of [Librosa](https://librosa.org), a popular audio processing library in Python, to perform the MelSpectrogram transform, because torchaudio before version 0.11 doesn't support fft on Android (see [here](https://github.com/pytorch/audio/issues/408)). Using the Librosa C++ port and [JNI](https://developer.android.com/training/articles/perf-jni) (Java Native Interface) on Android makes the MelSpectrogram possible on Android. Furthermore, the Librosa C++ port requires [Eigen](https://eigen.tuxfamily.org/), a C++ template library for linear algebra, so both the port and the Eigen library are included in the first version of the demo app and built as JNI. -Because torchaudio currently doesn't support fft on Android (see [here](https://github.com/pytorch/audio/issues/408)), using the Librosa C++ port and [JNI](https://developer.android.com/training/articles/perf-jni) (Java Native Interface) on Android makes the MelSpectrogram possible on Android. Furthermore, the Librosa C++ port requires [Eigen](https://eigen.tuxfamily.org/), a C++ template library for linear algebra, so both the port and the Eigen library are included in the demo app and built as JNI, using the `CMakeLists.txt` and `MainActivityJNI.cpp` in `StreamingASR/app/src/main/cpp`. +See [here](https://github.com/jeffxtang/android-demo-app/tree/librosa_jni/StreamingASR) for the first version of the demo if interested in an example of using native C++ to expand operations not yet supported in PyTorch or one of its domain libraries. diff --git a/StreamingASR/StreamingASR/app/build.gradle b/StreamingASR/StreamingASR/app/build.gradle index 288702a7..1bda3c67 100644 --- a/StreamingASR/StreamingASR/app/build.gradle +++ b/StreamingASR/StreamingASR/app/build.gradle @@ -13,13 +13,6 @@ android { versionName "1.0" testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner" - - externalNativeBuild { - cmake { - cppFlags "" - arguments "-DLOGGER_BUILD_HEADER_LIB=ON", "-DBUILD_TESTING=OFF" - } - } } buildTypes { @@ -32,13 +25,6 @@ android { sourceCompatibility JavaVersion.VERSION_1_8 targetCompatibility JavaVersion.VERSION_1_8 } - - externalNativeBuild { - cmake { - path "../../CMakeLists.txt" - version "3.10.2" - } - } } dependencies { @@ -50,5 +36,5 @@ dependencies { androidTestImplementation 'androidx.test.ext:junit:1.1.3' androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0' - implementation 'org.pytorch:pytorch_android_lite:1.11' + implementation 'org.pytorch:pytorch_android_lite:1.12.2' } \ No newline at end of file diff --git a/StreamingASR/StreamingASR/app/src/main/cpp/CMakeLists.txt b/StreamingASR/StreamingASR/app/src/main/cpp/CMakeLists.txt deleted file mode 100644 index d52e785d..00000000 --- a/StreamingASR/StreamingASR/app/src/main/cpp/CMakeLists.txt +++ /dev/null @@ -1,5 +0,0 @@ -cmake_minimum_required(VERSION 3.4.1) -project(CTest LANGUAGES C CXX) - -add_library( MainActivityJNI SHARED MainActivityJNI.cpp ) -target_link_libraries( MainActivityJNI Eigen3::Eigen) diff --git a/StreamingASR/StreamingASR/app/src/main/cpp/MainActivityJNI.cpp b/StreamingASR/StreamingASR/app/src/main/cpp/MainActivityJNI.cpp deleted file mode 100644 index ff5086d4..00000000 --- a/StreamingASR/StreamingASR/app/src/main/cpp/MainActivityJNI.cpp +++ /dev/null @@ -1,92 +0,0 @@ -#include -#include -#include -#include "librosa/librosa.h" -#include -#include -#include -#include -#include - -#include -#include -#include -#include - -using namespace std; - -using Eigen::MatrixXd; - -extern "C" JNIEXPORT jobject JNICALL -Java_org_pytorch_demo_streamingasr_MainActivity_melSpectrogram(JNIEnv* env, jobject obj, - jdoubleArray data) { - int len = env -> GetArrayLength(data); - std::vector x; - jdouble *element = env->GetDoubleArrayElements(data, 0); - for(int i=0; i>> X = librosa::Feature::stft(x, n_fft, n_hop, "hann", true, "reflect"); - - std::vector> mels = librosa::Feature::melspectrogram(x, sr, n_fft, n_hop, "hann", true, "reflect", 2.f, n_mel, fmin, fmax); - - jclass vectorClass = env->FindClass("java/util/Vector"); - if(vectorClass == NULL) { - return NULL; - } - - jclass floatClass = env->FindClass("java/lang/Float"); - if(floatClass == NULL) { - return NULL; - } - - jmethodID vectorConstructorID = env->GetMethodID( - vectorClass, "", "()V"); - if(vectorConstructorID == NULL) { - return NULL; - } - - jmethodID addMethodID = env->GetMethodID( - vectorClass, "add", "(Ljava/lang/Object;)Z" ); - if(addMethodID == NULL) { - return NULL; - } - - jmethodID floatConstructorID = env->GetMethodID(floatClass, "", "(F)V"); - if(floatConstructorID == NULL) { - return NULL; - } - - jobject outerVector = env->NewObject(vectorClass, vectorConstructorID); - if(outerVector == NULL) { - return NULL; - } - - for(vector i : mels) { - jobject innerVector = env->NewObject(vectorClass, vectorConstructorID); - - for(float f : i) { - jobject floatValue = env->NewObject(floatClass, floatConstructorID, f); - if(floatValue == NULL) { - return NULL; - } - - env->CallBooleanMethod(innerVector, addMethodID, floatValue); - } - - env->CallBooleanMethod(outerVector, addMethodID, innerVector); - } - - env->DeleteLocalRef(vectorClass); - env->DeleteLocalRef(floatClass); - - return outerVector; -} diff --git a/StreamingASR/StreamingASR/app/src/main/cpp/librosa/librosa.h b/StreamingASR/StreamingASR/app/src/main/cpp/librosa/librosa.h deleted file mode 100644 index 30a2f9e7..00000000 --- a/StreamingASR/StreamingASR/app/src/main/cpp/librosa/librosa.h +++ /dev/null @@ -1,265 +0,0 @@ -/* ------------------------------------------------------------------ -* Copyright (C) 2020 ewan xu -* -* Licensed under the Apache License, Version 2.0 (the "License"); -* you may not use this file except in compliance with the License. -* You may obtain a copy of the License at -* -* http://www.apache.org/licenses/LICENSE-2.0 -* -* Unless required by applicable law or agreed to in writing, software -* distributed under the License is distributed on an "AS IS" BASIS, -* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either -* express or implied. -* See the License for the specific language governing permissions -* and limitations under the License. -* ------------------------------------------------------------------- -*/ - -#ifndef LIBROSA_H_ -#define LIBROSA_H_ - -#include "Eigen/Core" -#include "unsupported/Eigen/FFT" - -#include -#include -#include - -/// -/// \brief c++ implemention of librosa -/// -namespace librosa{ - -#ifndef M_PI -#define M_PI 3.14159265358979323846 -#endif // !M_PI - -typedef Eigen::Matrix Vectorf; -typedef Eigen::Matrix, 1, Eigen::Dynamic, Eigen::RowMajor> Vectorcf; -typedef Eigen::Matrix Matrixf; -typedef Eigen::Matrix, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor> Matrixcf; - -namespace internal{ - -static Vectorf pad(Vectorf &x, int left, int right, const std::string &mode, float value){ - Vectorf x_paded = Vectorf::Constant(left+x.size()+right, value); - x_paded.segment(left, x.size()) = x; - - if (mode.compare("reflect") == 0){ - for (int i = 0; i < left; ++i){ - x_paded[i] = x[left-i]; - } - for (int i = left; i < left+right; ++i){ - x_paded[i+x.size()] = x[x.size()-2-i+left]; - } - } - - if (mode.compare("symmetric") == 0){ - for (int i = 0; i < left; ++i){ - x_paded[i] = x[left-i-1]; - } - for (int i = left; i < left+right; ++i){ - x_paded[i+x.size()] = x[x.size()-1-i+left]; - } - } - - if (mode.compare("edge") == 0){ - for (int i = 0; i < left; ++i){ - x_paded[i] = x[0]; - } - for (int i = left; i < left+right; ++i){ - x_paded[i+x.size()] = x[x.size()-1]; - } - } - return x_paded; -} - -static Matrixcf stft(Vectorf &x, int n_fft, int n_hop, const std::string &win, bool center, const std::string &mode){ - // hanning - Vectorf window = 0.5*(1.f-(Vectorf::LinSpaced(n_fft, 0.f, static_cast(n_fft-1))*2.f*M_PI/n_fft).array().cos()); - - int pad_len = center ? n_fft / 2 : 0; - Vectorf x_paded = pad(x, pad_len, pad_len, mode, 0.f); - - int n_f = n_fft/2+1; - int n_frames = 1+(x_paded.size()-n_fft) / n_hop; - Matrixcf X(n_frames, n_fft); - Eigen::FFT fft; - - for (int i = 0; i < n_frames; ++i){ - Vectorf x_frame = window.array()*x_paded.segment(i*n_hop, n_fft).array(); - X.row(i) = fft.fwd(x_frame); - } - return X.leftCols(n_f); -} - -static Matrixf spectrogram(Matrixcf &X, float power = 1.f){ - return X.cwiseAbs().array().pow(power); -} - -static Matrixf melfilter(int sr, int n_fft, int n_mels, int fmin, int fmax){ - int n_f = n_fft/2+1; - Vectorf fft_freqs = (Vectorf::LinSpaced(n_f, 0.f, static_cast(n_f-1))*sr)/n_fft; - - float f_min = 0.f; - float f_sp = 200.f/3.f; - float min_log_hz = 1000.f; - float min_log_mel = (min_log_hz-f_min)/f_sp; - float logstep = logf(6.4f)/27.f; - - auto hz_to_mel = [=](int hz, bool htk = false) -> float { - if (htk){ - return 2595.0f*log10f(1.0f+hz/700.0f); - } - float mel = (hz-f_min)/f_sp; - if (hz >= min_log_hz){ - mel = min_log_mel+logf(hz/min_log_hz)/logstep; - } - return mel; - }; - auto mel_to_hz = [=](Vectorf &mels, bool htk = false) -> Vectorf { - if (htk){ - return 700.0f*(Vectorf::Constant(n_mels+2, 10.f).array().pow(mels.array()/2595.0f)-1.0f); - } - return (mels.array()>min_log_mel).select(((mels.array()-min_log_mel)*logstep).exp()*min_log_hz, (mels*f_sp).array()+f_min); - }; - - float min_mel = hz_to_mel(fmin); - float max_mel = hz_to_mel(fmax); - Vectorf mels = Vectorf::LinSpaced(n_mels+2, min_mel, max_mel); - Vectorf mel_f = mel_to_hz(mels); - Vectorf fdiff = mel_f.segment(1, mel_f.size() - 1) - mel_f.segment(0, mel_f.size() - 1); - Matrixf ramps = mel_f.replicate(n_f, 1).transpose().array() - fft_freqs.replicate(n_mels + 2, 1).array(); - - Matrixf lower = -ramps.topRows(n_mels).array()/fdiff.segment(0, n_mels).transpose().replicate(1, n_f).array(); - Matrixf upper = ramps.bottomRows(n_mels).array()/fdiff.segment(1, n_mels).transpose().replicate(1, n_f).array(); - Matrixf weights = (lower.array()(N-1)); - // type 2 - Matrixf coeff = 2*(M_PI*xi.transpose().array()/N*(xi.array()+0.5)).cos(); - Matrixf dct = x*coeff.transpose(); - // ortho - if (norm) { - Vectorf ortho = Vectorf::Constant(N, std::sqrtf(0.5f/N)); - ortho[0] = std::sqrtf(0.25f/N); - dct = dct*ortho.asDiagonal(); - } - return dct; -} - -} // namespace internal - -class Feature -{ -public: - /// \brief short-time fourier transform similar with librosa.feature.stft - /// \param x input audio signal - /// \param n_fft length of the FFT size - /// \param n_hop number of samples between successive frames - /// \param win window function. currently only supports 'hann' - /// \param center same as librosa - /// \param mode pad mode. support "reflect","symmetric","edge" - /// \return complex-valued matrix of short-time fourier transform coefficients. - static std::vector>> stft(std::vector &x, - int n_fft, int n_hop, - const std::string &win, bool center, - const std::string &mode){ - Vectorf map_x = Eigen::Map(x.data(), x.size()); - Matrixcf X = internal::stft(map_x, n_fft, n_hop, win, center, mode); - std::vector>> X_vector(X.rows(), std::vector>(X.cols(), 0)); - for (int i = 0; i < X.rows(); ++i){ - auto &row = X_vector[i]; - Eigen::Map(row.data(), row.size()) = X.row(i); - } - return X_vector; - } - - /// \brief compute mel spectrogram similar with librosa.feature.melspectrogram - /// \param x input audio signal - /// \param sr sample rate of 'x' - /// \param n_fft length of the FFT size - /// \param n_hop number of samples between successive frames - /// \param win window function. currently only supports 'hann' - /// \param center same as librosa - /// \param mode pad mode. support "reflect","symmetric","edge" - /// \param power exponent for the magnitude melspectrogram - /// \param n_mels number of mel bands - /// \param f_min lowest frequency (in Hz) - /// \param f_max highest frequency (in Hz) - /// \return mel spectrogram matrix - static std::vector> melspectrogram(std::vector &x, int sr, - int n_fft, int n_hop, const std::string &win, bool center, const std::string &mode, - float power, int n_mels, int fmin, int fmax){ - Vectorf map_x = Eigen::Map(x.data(), x.size()); - Matrixf mel = internal::melspectrogram(map_x, sr, n_fft, n_hop, win, center, mode, power, n_mels, fmin, fmax).transpose(); - std::vector> mel_vector(mel.rows(), std::vector(mel.cols(), 0.f)); - for (int i = 0; i < mel.rows(); ++i){ - auto &row = mel_vector[i]; - Eigen::Map(row.data(), row.size()) = mel.row(i); - } - return mel_vector; - } - - /// \brief compute mfcc similar with librosa.feature.mfcc - /// \param x input audio signal - /// \param sr sample rate of 'x' - /// \param n_fft length of the FFT size - /// \param n_hop number of samples between successive frames - /// \param win window function. currently only supports 'hann' - /// \param center same as librosa - /// \param mode pad mode. support "reflect","symmetric","edge" - /// \param power exponent for the magnitude melspectrogram - /// \param n_mels number of mel bands - /// \param f_min lowest frequency (in Hz) - /// \param f_max highest frequency (in Hz) - /// \param n_mfcc number of mfccs - /// \param norm ortho-normal dct basis - /// \param type dct type. currently only supports 'type-II' - /// \return mfcc matrix - static std::vector> mfcc(std::vector &x, int sr, - int n_fft, int n_hop, const std::string &win, bool center, const std::string &mode, - float power, int n_mels, int fmin, int fmax, - int n_mfcc, bool norm, int type) { - Vectorf map_x = Eigen::Map(x.data(), x.size()); - Matrixf mel = internal::melspectrogram(map_x, sr, n_fft, n_hop, win, center, mode, power, n_mels, fmin, fmax).transpose(); - Matrixf mel_db = internal::power2db(mel); - Matrixf dct = internal::dct(mel_db, norm, type).leftCols(n_mfcc); - std::vector> mfcc_vector(dct.rows(), std::vector(dct.cols(), 0.f)); - for (int i = 0; i < dct.rows(); ++i) { - auto &row = mfcc_vector[i]; - Eigen::Map(row.data(), row.size()) = dct.row(i); - } - return mfcc_vector; - } -}; - -} // namespace librosa - -#endif diff --git a/StreamingASR/StreamingASR/app/src/main/java/org/pytorch/demo/streamingasr/MainActivity.java b/StreamingASR/StreamingASR/app/src/main/java/org/pytorch/demo/streamingasr/MainActivity.java index 12abe2ec..59999930 100644 --- a/StreamingASR/StreamingASR/app/src/main/java/org/pytorch/demo/streamingasr/MainActivity.java +++ b/StreamingASR/StreamingASR/app/src/main/java/org/pytorch/demo/streamingasr/MainActivity.java @@ -24,7 +24,6 @@ import java.io.InputStream; import java.io.OutputStream; import java.nio.FloatBuffer; -import java.util.Vector; import org.pytorch.LiteModuleLoader; @@ -41,187 +40,11 @@ public class MainActivity extends AppCompatActivity implements Runnable { private final static int SAMPLE_RATE = 16000; private final static int CHUNK_TO_READ = 5; private final static int CHUNK_SIZE = 640; - private final static int SPECTROGRAM_X = 21; - private final static int SPECTROGRAM_Y = 80; + private final static int INPUT_SIZE = 3200; private IValue hypo = null; private IValue state = null; - private static final double[] MEAN = { - 16.462461471557617, - 17.020158767700195, - 17.27733039855957, - 17.273637771606445, - 17.78028678894043, - 18.112783432006836, - 18.322141647338867, - 18.3536319732666, - 18.220436096191406, - 17.93610191345215, - 17.650646209716797, - 17.505868911743164, - 17.450956344604492, - 17.420780181884766, - 17.36254119873047, - 17.24843978881836, - 17.073762893676758, - 16.893953323364258, - 16.62371826171875, - 16.279895782470703, - 16.046218872070312, - 15.789617538452148, - 15.458984375, - 15.335075378417969, - 15.103074073791504, - 14.993032455444336, - 14.818647384643555, - 14.713132858276367, - 14.576343536376953, - 14.482580184936523, - 14.431093215942383, - 14.392385482788086, - 14.357626914978027, - 14.335031509399414, - 14.344644546508789, - 14.341029167175293, - 14.338135719299316, - 14.311485290527344, - 14.266831398010254, - 14.205205917358398, - 14.159194946289062, - 14.07589054107666, - 14.02244758605957, - 13.954248428344727, - 13.897454261779785, - 13.856722831726074, - 13.80321216583252, - 13.75955867767334, - 13.718783378601074, - 13.67695426940918, - 13.626880645751953, - 13.554975509643555, - 13.465453147888184, - 13.372663497924805, - 13.269320487976074, - 13.184920310974121, - 13.094778060913086, - 12.998514175415039, - 12.891039848327637, - 12.765382766723633, - 12.638651847839355, - 12.50733470916748, - 12.345802307128906, - 12.195826530456543, - 12.019110679626465, - 11.842704772949219, - 11.680868148803711, - 11.518675804138184, - 11.37252426147461, - 11.252099990844727, - 11.12936019897461, - 11.029287338256836, - 10.927411079406738, - 10.825841903686523, - 10.717211723327637, - 10.499553680419922, - 9.722028732299805, - 8.256664276123047, - 7.897761344909668, - 7.252806663513184 - }; - - private static final double[] INVSTDDEV = { - 0.2532021571066031, - 0.2597563367511928, - 0.2579079373215276, - 0.2416085222005694, - 0.23003407153886749, - 0.21714598348479108, - 0.20868966256973892, - 0.20397882792073063, - 0.20346486748979434, - 0.20568288111895272, - 0.20795624145573485, - 0.20848980415063503, - 0.20735096423640872, - 0.2060772210458722, - 0.20577174595523076, - 0.20655349986725383, - 0.2080547906859301, - 0.21015748217276387, - 0.2127639989370032, - 0.2156462785763535, - 0.21848300746868443, - 0.22174608140608748, - 0.22541974458780933, - 0.22897465119671973, - 0.23207484606149037, - 0.2353556049061462, - 0.23820711835547867, - 0.24016651485087528, - 0.24200318561465783, - 0.2435905301766702, - 0.24527147180928432, - 0.2493368450351618, - 0.25120444993308483, - 0.2521961451825939, - 0.25358032484699955, - 0.25349767201088286, - 0.2534676894845623, - 0.25149125467665234, - 0.25001929593946776, - 0.25064096375066197, - 0.25194505955280033, - 0.25270402089338095, - 0.2535205901701615, - 0.25363568106276674, - 0.2535307075541985, - 0.25315144026701186, - 0.2523683857532224, - 0.25200854739575596, - 0.2516561583169735, - 0.25147053419035553, - 0.25187638352086095, - 0.25176343344798546, - 0.25256615785525305, - 0.25310796555079107, - 0.2535568871416053, - 0.2542411936874833, - 0.2544978632482573, - 0.2553210332506536, - 0.2567248511819892, - 0.2559665595456875, - 0.2564729970835735, - 0.2585267417223537, - 0.2573770145474615, - 0.2585495460828127, - 0.2593605768768532, - 0.25906572100606984, - 0.26026752519153573, - 0.2609952847918467, - 0.26222905157170767, - 0.26395874733435604, - 0.26404203898769246, - 0.26501581381370537, - 0.2666259054856709, - 0.2676190865432322, - 0.26813030555166134, - 0.26873271506658997, - 0.2624062353014993, - 0.2289515918968408, - 0.22755587298227964, - 0.24719513536827162 - }; - - private static final double DECIBEL = 2 * 20 * Math.log10(32767); - private static final double GAIN = Math.pow(10, 0.05 * DECIBEL); - - public native Vector> melSpectrogram(double[] data); - - static { - System.loadLibrary("MainActivityJNI"); - } - @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); @@ -231,7 +54,7 @@ protected void onCreate(Bundle savedInstanceState) { mTextView = findViewById(R.id.tvResult); if (mModuleEncoder == null) { - mModuleEncoder = LiteModuleLoader.load(assetFilePath(getApplicationContext(), "streaming_asr.ptl")); + mModuleEncoder = LiteModuleLoader.load(assetFilePath(getApplicationContext(), "streaming_asrv2.ptl")); } mButton.setOnClickListener(new View.OnClickListener() { @@ -341,36 +164,12 @@ public void run() { } private String recognize(double[] inputBuffer) { - - double[][] spectrogram = new double[SPECTROGRAM_X][SPECTROGRAM_Y]; - Vector> result = melSpectrogram(inputBuffer); - for (int i = 0; i < result.size(); i++) { - for (int j = 0; j < result.get(i).size(); j++) { - spectrogram[i][j] = result.get(i).get(j); - } - } - - for (int i = 0; i < spectrogram.length; i++) { - for (int j = 0; j < spectrogram[i].length; j++) { - spectrogram[i][j] *= GAIN; - if (spectrogram[i][j] > Math.E) - spectrogram[i][j] = Math.log(spectrogram[i][j]); - else - spectrogram[i][j] /= Math.E; - } - } - - FloatBuffer inTensorBuffer = Tensor.allocateFloatBuffer((SPECTROGRAM_X-1) * SPECTROGRAM_Y); - // get rid of the last row and transform the others - for (int i = 0; i < spectrogram.length - 1; i++) { - for (int j = 0; j < spectrogram[i].length; j++) { - spectrogram[i][j] -= MEAN[j]; - spectrogram[i][j] *= INVSTDDEV[j]; - inTensorBuffer.put((float) spectrogram[i][j]); - } + FloatBuffer inTensorBuffer = Tensor.allocateFloatBuffer(INPUT_SIZE); + for (int i = 0; i < inputBuffer.length - 1; i++) { + inTensorBuffer.put((float) inputBuffer[i]); } - final Tensor inTensor = Tensor.fromBlob(inTensorBuffer, new long[]{1, SPECTROGRAM_X - 1, SPECTROGRAM_Y}); + final Tensor inTensor = Tensor.fromBlob(inTensorBuffer, new long[]{INPUT_SIZE}); final long startTime = SystemClock.elapsedRealtime(); IValue[] outputTuple; if (hypo == null && state == null) diff --git a/StreamingASR/generate_ts.py b/StreamingASR/generate_ts.py new file mode 100644 index 00000000..e6c19eae --- /dev/null +++ b/StreamingASR/generate_ts.py @@ -0,0 +1,100 @@ +from typing import Dict, List, Optional, Tuple +import json +import math + +from fairseq.data import Dictionary +import torch +import torchaudio +from torchaudio.pipelines import EMFORMER_RNNT_BASE_LIBRISPEECH +from torchaudio.models import Hypothesis + + +def get_hypo_tokens(hypo: Hypothesis) -> List[int]: + return hypo[0] + + +def get_hypo_score(hypo: Hypothesis) -> float: + return hypo[3] + + +def to_string(input: List[int], tgt_dict: List[str], bos_idx: int = 0, eos_idx: int = 2, separator: str = "",) -> str: + # torchscript dislikes sets + extra_symbols_to_ignore: Dict[int, int] = {} + extra_symbols_to_ignore[eos_idx] = 1 + extra_symbols_to_ignore[bos_idx] = 1 + + # it also dislikes comprehensions with conditionals + filtered_idx: List[int] = [] + for idx in input: + if idx not in extra_symbols_to_ignore: + filtered_idx.append(idx) + + return separator.join([tgt_dict[idx] for idx in filtered_idx]).replace("\u2581", " ") + + +def post_process_hypos( + hypos: List[Hypothesis], tgt_dict: List[str], +) -> List[Tuple[str, List[float], List[int]]]: + post_process_remove_list = [ + 3, # unk + 2, # eos + 1, # pad + ] + hypos_str: List[str] = [] + for h in hypos: + filtered_tokens: List[int] = [] + for token_index in get_hypo_tokens(h)[1:]: + if token_index not in post_process_remove_list: + filtered_tokens.append(token_index) + string = to_string(filtered_tokens, tgt_dict) + hypos_str.append(string) + + hypos_ids = [get_hypo_tokens(h)[1:] for h in hypos] + hypos_score = [[math.exp(get_hypo_score(h))] for h in hypos] + + nbest_batch = list(zip(hypos_str, hypos_score, hypos_ids)) + + return nbest_batch + + +def _piecewise_linear_log(x): + x[x > math.e] = torch.log(x[x > math.e]) + x[x <= math.e] = x[x <= math.e] / math.e + return x + + +class ModelWrapper(torch.nn.Module): + def __init__(self, tgt_dict: List[str]): + super().__init__() + self.transform = torchaudio.transforms.MelSpectrogram(sample_rate=16000, n_fft=400, n_mels=80, hop_length=160) + + self.decoder = EMFORMER_RNNT_BASE_LIBRISPEECH.get_decoder() + + self.tgt_dict = tgt_dict + + with open("global_stats.json") as f: + blob = json.loads(f.read()) + + self.mean = torch.tensor(blob["mean"]) + self.invstddev = torch.tensor(blob["invstddev"]) + + self.decibel = 2 * 20 * math.log10(32767) + self.gain = pow(10, 0.05 * self.decibel) + + def forward( + self, input: torch.Tensor, prev_hypo: Optional[Hypothesis], prev_state: Optional[List[List[torch.Tensor]]] + ) -> Tuple[str, Hypothesis, Optional[List[List[torch.Tensor]]]]: + spectrogram = self.transform(input).transpose(1, 0) + features = _piecewise_linear_log(spectrogram * self.gain).unsqueeze(0)[:, :-1] + features = (features - self.mean) * self.invstddev + length = torch.tensor([features.shape[1]]) + + hypotheses, state = self.decoder.infer(features, length, 10, state=prev_state, hypothesis=prev_hypo) + transcript = post_process_hypos(hypotheses[:1], self.tgt_dict)[0][0] + return transcript, hypotheses[0], state + + +tgt_dict = Dictionary.load("spm_bpe_4096_fairseq.dict") +wrapper = ModelWrapper(tgt_dict.symbols) +wrapper = torch.jit.script(wrapper) +wrapper.save("scripted_wrapper_tuple.pt") diff --git a/StreamingASR/run_sasr.py b/StreamingASR/run_sasr.py index 68276a7e..06042049 100644 --- a/StreamingASR/run_sasr.py +++ b/StreamingASR/run_sasr.py @@ -1,5 +1,3 @@ -import math -import json import pyaudio import queue import numpy as np @@ -8,53 +6,40 @@ def get_demo_wrapper(): - wrapper = torch.jit.load("scripted_wrapper_tuple_no_transform.pt") + wrapper = torch.jit.load("scripted_wrapper_tuple.pt") return wrapper + wrapper = get_demo_wrapper() + + +################################################################ + + data_queue = queue.Queue() + def callback(in_data, frame_count, time_info, status): global data_queue data_queue.put(in_data) return in_data, pyaudio.paContinue -def _piecewise_linear_log(x): - x[x > math.e] = torch.log(x[x > math.e]) - x[x <= math.e] = x[x <= math.e] / math.e - return x - -transform = torchaudio.transforms.MelSpectrogram( - sample_rate=16000, n_fft=400, n_mels=80, hop_length=160, -) - -with open("global_stats.json") as f: - blob = json.loads(f.read()) - -_mean = torch.tensor(blob["mean"]) -_invstddev = torch.tensor(blob["invstddev"]) - -_decibel = 2 * 20 * math.log10(32767) -_gain = pow(10, 0.05 * _decibel) state = None hypo = None + def transcribe(np_array, should_print=True): global state, hypo tensor = torch.tensor(np_array) - spectrogram = transform(tensor).transpose(1, 0) - - features = _piecewise_linear_log(spectrogram * _gain) - features = features.unsqueeze(0)[:, :-1] - features = (features - _mean) * _invstddev - - transcript, hypo, state = wrapper(features, hypo, state) + transcript, hypo, state = wrapper(tensor, hypo, state) if should_print and transcript: - print(transcript, end=" ", flush=True) + print(transcript, end="", flush=True) + previous_right_context = None + def process(should_print=True): global previous_right_context if previous_right_context is None: diff --git a/StreamingASR/save_model_for_mobile.py b/StreamingASR/save_model_for_mobile.py index f6412417..9bac252a 100644 --- a/StreamingASR/save_model_for_mobile.py +++ b/StreamingASR/save_model_for_mobile.py @@ -3,11 +3,11 @@ from torch.utils.mobile_optimizer import optimize_for_mobile def get_demo_wrapper(): - wrapper = torch.jit.load("scripted_wrapper_tuple_no_transform.pt") + wrapper = torch.jit.load("scripted_wrapper_tuple.pt") return wrapper wrapper = get_demo_wrapper() scripted_model = torch.jit.script(wrapper) optimized_model = optimize_for_mobile(scripted_model) -optimized_model._save_for_lite_interpreter("streaming_asr.ptl") +optimized_model._save_for_lite_interpreter("streaming_asrv2.ptl") print("Done _save_for_lite_interpreter") diff --git a/StreamingASR/spm_bpe_4096_fairseq.dict b/StreamingASR/spm_bpe_4096_fairseq.dict new file mode 100644 index 00000000..2480c719 --- /dev/null +++ b/StreamingASR/spm_bpe_4096_fairseq.dict @@ -0,0 +1,4092 @@ +▁t 14697 +he 5328 +▁a 209134 +▁the 572568 +in 19274 +▁s 9248 +▁w 10474 +▁o 9631 +re 13300 +nd 6197 +▁b 17700 +▁h 8270 +er 28595 +▁m 11262 +▁i 139138 +ou 3036 +▁c 14295 +▁f 11805 +at 11182 +ed 56863 +▁and 321677 +en 14298 +▁to 262805 +▁of 283235 +on 19881 +is 11436 +▁d 17294 +ing 51145 +▁th 3267 +▁p 19150 +▁he 130145 +or 15624 +▁l 13203 +es 21091 +▁in 160308 +ll 12577 +it 13886 +ar 15342 +as 8677 +an 16493 +▁n 10256 +▁g 13713 +om 8259 +▁be 60181 +▁ha 5156 +▁e 7960 +le 14970 +ot 7567 +▁y 2073 +ut 5571 +ow 5259 +ic 14481 +▁wh 3596 +▁it 111248 +ld 1227 +ve 8977 +▁that 121714 +ly 37023 +▁was 117423 +id 8944 +se 7808 +st 6828 +▁on 54663 +gh 830 +ent 12794 +▁re 11479 +▁you 77607 +im 7743 +ce 6481 +▁u 1141 +ver 5618 +ion 13834 +▁as 77546 +et 9449 +▁for 74445 +ay 5312 +▁we 34586 +▁his 90961 +ith 1301 +al 20814 +ir 5524 +▁r 16904 +▁with 77315 +▁st 11079 +ad 7797 +ur 7567 +ght 15 +▁an 29971 +▁her 68089 +▁not 62109 +▁had 71556 +▁is 68716 +ter 6785 +her 3010 +ac 6601 +am 9048 +▁at 57846 +oo 2352 +▁but 62954 +ould 496 +▁she 61550 +▁k 5691 +▁se 6281 +▁sa 3228 +▁sh 9704 +▁fr 4324 +▁him 46882 +▁so 41623 +ill 3076 +▁me 37983 +ain 6067 +▁su 2840 +ight 1771 +ch 11568 +red 4904 +ct 2765 +all 3524 +ro 4794 +ke 4298 +ess 3688 +il 10312 +ore 3514 +▁de 6700 +▁they 46096 +▁my 40748 +▁whe 684 +▁all 43796 +ich 1187 +▁ne 2045 +ri 6595 +▁by 42736 +▁have 42689 +ome 1401 +pp 1235 +▁this 41805 +▁li 3403 +▁do 21879 +▁con 4332 +us 8234 +▁which 40719 +▁ch 10631 +ul 5635 +qu 4537 +▁j 8280 +▁up 23971 +▁said 38483 +▁from 38395 +ard 6083 +ge 6353 +▁or 31151 +▁v 6799 +▁one 36062 +th 5318 +▁no 31439 +▁ex 4069 +▁were 35533 +▁there 32360 +pe 3907 +and 4103 +est 12158 +▁man 23530 +▁who 26205 +ble 3868 +ant 9571 +ie 7597 +▁al 4452 +res 2439 +ous 7309 +ust 4150 +very 1155 +ation 14993 +▁fe 4127 +▁them 27431 +lf 1385 +▁when 29017 +nt 1642 +ind 2148 +ame 2123 +ra 3673 +▁go 12987 +ers 13013 +ast 4107 +fe 1912 +ood 1145 +▁kn 2005 +▁int 2742 +ist 7306 +art 2879 +▁are 28467 +out 1859 +▁would 27073 +▁le 5075 +os 5199 +▁their 27633 +ong 1096 +▁what 26179 +our 5357 +▁if 26950 +ound 2201 +▁com 1558 +▁ab 4619 +▁out 24985 +▁wor 1289 +em 4268 +▁will 24249 +ak 2660 +▁mis 2348 +ate 12478 +ol 8628 +um 6734 +un 5356 +itt 1634 +ough 1005 +ked 3014 +ap 5057 +ig 5602 +one 3098 +▁been 23285 +own 2087 +ive 5876 +▁then 22755 +▁br 7893 +ven 3017 +if 3490 +▁ar 4196 +▁tr 6992 +self 274 +▁pl 5573 +▁ro 5149 +ther 519 +▁pr 3290 +reat 538 +▁un 8844 +▁af 915 +▁sp 6543 +▁qu 4036 +▁pro 3054 +ity 9883 +hed 1877 +▁tw 2228 +▁ag 2788 +▁could 19595 +ost 1305 +ace 2057 +ort 2072 +ure 6993 +ake 2988 +ack 2756 +▁am 11779 +▁any 14986 +▁some 17928 +▁your 17343 +▁more 19162 +▁can 16402 +au 2101 +▁tim 733 +ep 2263 +▁en 3795 +ag 5128 +ck 2866 +▁cl 7806 +▁into 18895 +ry 4907 +hing 2403 +▁now 18452 +nder 1658 +are 3473 +▁very 17938 +▁gr 4729 +el 6757 +ose 2285 +▁loo 305 +▁bo 3409 +ved 3328 +op 2922 +▁other 13325 +▁did 14932 +ance 6595 +▁than 15276 +ittle 67 +▁little 16976 +ine 8344 +ies 9473 +way 2079 +ite 3891 +▁like 15819 +ide 2819 +ass 2977 +▁bl 5623 +able 9291 +▁lo 4285 +urn 617 +ought 1021 +▁know 12722 +other 322 +▁time 16223 +▁im 1438 +▁dis 4688 +▁us 11629 +▁co 4978 +fore 313 +▁te 3674 +▁how 11922 +ence 5156 +▁day 11890 +▁ad 3226 +ade 2717 +▁about 15699 +ice 3567 +▁see 13317 +▁over 15579 +pt 3300 +cc 921 +▁too 8750 +ink 2772 +▁fl 8537 +wn 1125 +▁great 12115 +▁after 12658 +pl 2957 +de 2326 +▁per 5360 +▁again 9854 +ment 8142 +▁upon 14708 +▁hand 9375 +ab 3416 +ree 1220 +▁has 14536 +ish 6424 +ci 2909 +▁only 14339 +ally 2408 +▁well 14172 +▁should 12521 +▁po 3764 +▁mar 4978 +ress 2374 +▁say 9459 +▁good 13305 +ather 910 +▁two 13668 +ings 6309 +▁pe 4156 +ount 1119 +▁our 12687 +ire 4075 +ving 4187 +▁down 13267 +ars 4148 +ert 2384 +we 1154 +▁before 12947 +ile 2835 +▁app 2783 +ves 3500 +▁every 8675 +▁its 12677 +▁old 12572 +▁thr 1739 +▁mu 1508 +▁made 12467 +ick 4326 +ied 3615 +▁long 10338 +te 3470 +age 5277 +ft 2682 +▁where 12048 +▁never 11300 +ang 2975 +▁pre 4626 +▁must 11923 +▁sm 2103 +▁such 11825 +ull 972 +ful 7890 +▁str 3253 +ions 9070 +▁sc 7571 +▁off 8111 +▁came 11699 +ious 4009 +ue 3193 +▁miss 5345 +ward 2948 +▁fir 358 +ild 1155 +▁even 8803 +▁under 7045 +▁these 11335 +act 1842 +▁come 11295 +▁part 6636 +▁fo 1603 +ated 7587 +ness 8547 +▁rem 2019 +ord 1728 +▁bec 32 +▁may 11044 +ty 3278 +▁think 8945 +▁much 11007 +per 2389 +▁way 10030 +▁mister 10898 +led 8312 +orn 2151 +▁ey 502 +▁let 7523 +▁cont 3371 +▁gl 4897 +▁thought 9247 +▁look 6001 +▁spe 1966 +ect 98 +▁back 10335 +ise 3764 +▁bet 733 +▁ye 2054 +ady 765 +ach 3321 +ans 3610 +▁just 9398 +▁first 10108 +▁here 10099 +ren 1742 +▁ho 1870 +▁des 2255 +▁ob 1739 +▁own 9956 +ried 629 +ud 4343 +ary 4757 +▁went 9782 +▁himself 9736 +▁mo 2875 +cl 2087 +▁men 9698 +air 1106 +ave 1790 +ath 2682 +▁sl 4762 +ff 2336 +co 1899 +▁cr 4657 +llow 353 +▁res 3143 +▁might 8779 +ily 4579 +▁seem 1981 +int 1880 +ip 3920 +▁beg 1339 +ouse 1031 +anc 970 +▁wat 344 +▁through 8392 +▁comp 3895 +ber 3761 +▁car 4203 +▁away 8706 +▁em 3937 +▁get 7246 +▁imp 3164 +▁head 7633 +oss 1222 +▁don 8515 +▁bel 1025 +▁life 8486 +▁without 8459 +▁pass 2681 +▁most 8394 +▁make 8342 +ened 3152 +▁cons 4142 +▁som 299 +▁turn 2716 +av 2813 +ng 237 +▁shall 8124 +▁those 8105 +▁eyes 8093 +▁pres 1762 +▁acc 2079 +▁house 8073 +iz 1540 +▁somet 101 +▁jo 2661 +▁still 7903 +▁call 2844 +hes 2885 +▁op 320 +▁night 7878 +ause 614 +▁wom 29 +less 5479 +▁last 7812 +ks 4091 +ared 3443 +▁comm 3013 +▁nothing 7700 +▁ent 1569 +▁tell 6955 +▁new 6091 +▁take 7620 +ign 1798 +▁being 7603 +▁many 7549 +▁word 3481 +▁found 7438 +ons 4574 +▁ret 1768 +ase 3363 +▁while 7371 +▁ear 1255 +▁att 2854 +ory 2817 +▁saw 7273 +ix 1052 +oth 2352 +▁put 6508 +ne 2776 +▁ser 1715 +▁peop 149 +iend 131 +▁wr 1910 +ark 1691 +▁young 7132 +dy 1403 +aking 1532 +les 5666 +▁la 1787 +▁once 7084 +ens 4707 +▁count 2890 +pect 1218 +▁friend 3532 +▁people 7035 +ible 3061 +ors 5004 +▁mat 2089 +fect 595 +ince 468 +▁room 6332 +ered 3129 +▁three 6928 +▁yet 6906 +ail 2214 +▁same 6844 +▁father 6778 +▁right 6774 +▁child 3660 +igh 580 +▁cour 611 +▁another 6708 +▁place 6706 +ult 2249 +iv 2115 +▁though 6672 +ition 2892 +▁ind 2973 +▁want 4120 +▁nor 5324 +▁far 5958 +▁king 5837 +▁end 5378 +▁happ 984 +▁heart 5942 +▁face 6534 +▁ever 6528 +▁nat 1334 +get 855 +thing 78 +▁took 6497 +▁hu 673 +▁dist 1675 +▁love 5691 +ew 297 +ever 406 +▁arm 2993 +ian 4527 +▁inst 1309 +man 3943 +▁work 5659 +▁light 5372 +▁set 4822 +▁ple 469 +ict 2538 +▁looked 6259 +▁char 2209 +▁ac 1031 +▁missus 6235 +▁mind 5685 +▁inte 449 +▁rep 3494 +▁asked 6168 +▁supp 2011 +cess 1046 +▁yes 5484 +ently 2192 +▁left 6096 +ertain 530 +gg 1866 +▁ke 675 +ished 3338 +▁things 6045 +ub 2454 +▁pers 835 +ways 303 +▁mom 24 +irl 414 +alk 378 +▁sir 5970 +▁moment 5286 +▁wa 1758 +ations 5375 +▁sat 3975 +sel 779 +▁find 5117 +ia 5875 +ower 625 +rew 1740 +▁world 5867 +ject 711 +vent 779 +▁give 5792 +▁gen 1205 +▁cap 1453 +so 475 +▁sw 3894 +▁gu 3388 +▁why 5739 +lt 1607 +ling 5212 +▁always 5714 +▁mother 5712 +dd 644 +pped 1614 +▁soon 5109 +▁ans 351 +▁act 4028 +▁form 3018 +▁el 1970 +▁heard 5645 +der 1456 +ret 2066 +▁thing 5608 +▁seemed 5596 +▁something 5584 +ange 1060 +▁door 5571 +▁sub 2382 +▁girl 4147 +ced 2585 +ither 701 +▁appe 837 +▁wind 2580 +▁mon 3563 +▁because 5491 +▁dif 355 +ss 967 +▁told 5463 +▁going 5456 +orm 1625 +▁home 5431 +▁war 4077 +ained 3445 +▁got 5414 +aught 662 +▁gi 714 +▁god 4701 +▁eng 1270 +▁sur 1571 +land 2245 +ning 2982 +▁hands 4479 +▁woman 5288 +aut 526 +▁vo 868 +▁poss 418 +▁follow 1866 +▁feel 2628 +ched 2534 +▁rel 2979 +ph 3781 +ple 1720 +ical 4076 +▁return 2156 +ook 1333 +▁boy 3764 +▁knew 5122 +▁reg 1935 +ner 1342 +▁each 5100 +▁rest 4609 +▁ma 1973 +▁kind 4017 +▁exp 1795 +▁cle 1103 +iver 1825 +▁oh 5040 +▁hel 932 +▁sil 1257 +ual 2957 +▁water 4997 +ting 2886 +▁del 2208 +▁ass 2998 +▁inf 1620 +▁wo 1373 +▁bre 1173 +▁certain 3043 +▁against 4943 +▁conf 3396 +cept 768 +▁belie 424 +▁hard 3391 +row 1384 +▁unt 531 +▁years 4865 +▁quite 4855 +iness 1532 +▁near 3225 +▁ph 2072 +ined 3307 +▁side 4846 +▁hor 999 +▁four 4205 +ired 2142 +ters 2230 +ool 944 +▁few 4783 +ier 2424 +rest 1369 +▁done 4765 +most 774 +▁half 4751 +▁che 2310 +ited 2564 +▁better 4743 +▁tre 1780 +▁min 2001 +ock 2484 +ps 2866 +▁also 4662 +uck 1788 +▁care 3285 +oub 448 +▁began 4639 +ully 435 +ised 1850 +▁having 4593 +ru 1568 +▁enough 4585 +▁gener 1139 +▁dra 1268 +▁seen 4554 +▁lady 4552 +▁pur 1577 +aps 648 +ott 706 +▁hum 2140 +ross 298 +aken 1205 +ying 1801 +▁ter 1407 +ank 2027 +▁inde 389 +▁called 4463 +▁hour 2685 +ial 3612 +ason 579 +▁beh 782 +▁whole 4418 +▁does 3699 +▁morn 199 +▁ste 2680 +▁pleas 723 +▁turned 4377 +ib 2066 +▁ref 2919 +ense 1935 +▁ins 3494 +ream 317 +▁occ 54 +▁course 4346 +gether 0 +▁both 4337 +▁gave 4336 +uth 581 +▁cur 2964 +▁sou 322 +een 456 +▁add 1903 +▁read 3530 +ween 16 +▁col 2753 +selves 192 +▁between 4273 +▁among 4259 +ular 1287 +▁keep 3597 +▁beaut 329 +▁inc 3020 +▁poor 4238 +▁sure 3345 +▁morning 4210 +▁white 4209 +ged 3693 +▁dear 4192 +▁name 4192 +▁toward 1804 +▁whom 4174 +▁small 4170 +▁sk 1682 +▁repl 360 +▁lar 418 +ute 2021 +▁felt 4121 +osed 2322 +bo 1235 +ating 4105 +▁open 3348 +▁six 2824 +▁myself 4089 +ond 1210 +▁however 4080 +xt 396 +▁bu 1227 +▁inter 2111 +▁high 4060 +▁herself 4060 +aint 872 +▁fore 2072 +ction 1857 +▁stood 4040 +▁tra 2390 +▁hund 2 +▁hundred 4035 +▁ev 801 +▁sent 4015 +aster 1189 +▁sim 1127 +▁show 2819 +ife 103 +▁round 3992 +▁point 2789 +▁almost 3980 +▁days 3974 +▁words 3972 +vel 1879 +▁gra 1321 +ale 2831 +▁dr 2590 +▁gre 1177 +▁eight 1831 +ents 2833 +dden 102 +ates 2768 +▁bus 992 +▁fam 157 +ces 1836 +▁land 3342 +▁stand 2364 +ung 2065 +▁sun 3881 +▁ed 1690 +haps 13 +ird 389 +▁mean 2567 +ned 2772 +▁perhaps 3862 +ures 2737 +iet 781 +▁since 3847 +▁sudden 1202 +▁sle 862 +▁best 3808 +▁dark 2835 +iss 1630 +▁replied 3794 +▁voice 3791 +▁bar 2739 +▁met 2941 +▁till 3771 +▁anything 3770 +▁until 3741 +▁underst 6 +its 3096 +▁black 3730 +oud 671 +▁bro 346 +aring 2581 +▁looking 3721 +ins 3191 +▁cried 3708 +amp 1838 +▁prin 303 +▁fact 3174 +▁next 3680 +▁less 3677 +▁law 2807 +▁lay 3673 +up 1110 +▁power 3055 +▁prop 2744 +▁brought 3642 +not 371 +enty 2 +ately 2232 +rent 1411 +▁country 3601 +▁help 3120 +med 1730 +▁vis 1448 +▁sn 2257 +als 2762 +▁air 3578 +fully 2746 +▁quest 136 +▁together 3576 +▁spo 686 +▁adv 973 +▁need 2981 +▁person 2745 +▁use 3029 +▁indeed 3546 +▁contin 1618 +oney 416 +ows 2699 +▁gent 308 +▁present 2932 +▁par 2755 +▁unc 1770 +ured 3000 +▁run 2318 +▁aw 1732 +▁full 3500 +▁rather 3496 +▁ide 484 +nded 1815 +▁feet 3471 +tain 928 +▁cond 765 +▁sy 957 +▁lat 695 +be 3450 +▁fall 1960 +du 280 +▁five 3447 +eter 755 +▁har 2041 +▁fin 1546 +cei 707 +▁bed 3420 +▁mil 757 +▁doct 454 +▁interest 2135 +oc 1206 +▁gone 3383 +▁matter 3383 +ressed 1361 +▁lord 3376 +▁wife 3373 +▁pat 3372 +▁es 333 +ering 2764 +fort 891 +▁serv 502 +▁death 3338 +oubt 283 +▁adm 1277 +▁taken 3289 +ash 2113 +▁art 2634 +▁hope 3281 +▁others 3279 +▁tri 2649 +▁talk 2029 +▁ext 1021 +az 1359 +▁dire 68 +ief 285 +▁along 3260 +▁cannot 3255 +▁speak 3252 +▁themselves 3251 +ove 594 +pr 1135 +▁lau 142 +▁cou 508 +▁bes 451 +ten 394 +uring 751 +▁pol 1509 +mb 844 +▁large 3227 +▁gentle 738 +▁mor 1433 +▁cat 2503 +▁fire 3199 +app 757 +vers 1756 +ention 104 +▁swe 1314 +▁thus 3182 +▁sec 777 +▁play 1983 +ield 859 +▁real 3153 +ments 3152 +ains 1865 +▁prom 897 +wered 43 +ousand 2 +▁reason 3135 +▁thousand 3134 +iting 1075 +akes 1151 +ached 606 +▁money 3120 +▁brother 2437 +▁dep 1286 +▁behind 3108 +ont 1368 +▁true 3104 +▁children 3102 +▁remem 239 +oy 1012 +▁answered 3097 +used 1138 +▁sound 2602 +▁whose 3090 +ison 1485 +▁twenty 3088 +▁wood 2354 +od 1442 +ene 486 +ants 2323 +sh 1398 +▁wonder 2009 +ably 1913 +itu 209 +▁believe 3066 +▁inv 2528 +▁dec 1998 +▁thou 3062 +ton 3056 +▁ra 1933 +ued 1130 +▁hold 2181 +▁ele 1683 +▁doubt 3048 +▁suff 571 +▁hon 648 +▁hear 3043 +▁passed 3043 +▁cor 2528 +▁opp 1307 +▁diffe 150 +ities 3022 +me 1296 +▁rec 1910 +▁nature 3013 +▁mer 2432 +▁alone 3000 +ided 1934 +▁sing 1181 +lic 1283 +▁dead 2983 +ken 823 +aces 1085 +▁fur 1225 +▁pri 906 +▁gold 1899 +▁hur 714 +▁close 2444 +og 1347 +▁red 2955 +▁doctor 2955 +▁start 1692 +ances 1975 +▁ground 2950 +▁bur 1558 +▁sea 2946 +ets 2231 +▁ask 2390 +ery 2932 +▁son 2931 +▁strong 2931 +▁leave 2928 +illed 504 +wer 686 +ept 466 +▁compan 94 +cy 1726 +▁obs 406 +▁ce 1796 +ides 696 +▁mount 930 +tle 1242 +body 44 +▁fell 2879 +cond 65 +▁sign 2871 +▁given 2871 +▁fair 2269 +ane 1979 +▁ir 1770 +iful 389 +▁deep 2850 +▁therefore 2850 +▁body 2843 +ys 1705 +▁short 2839 +▁often 2838 +fic 727 +ceed 0 +▁pap 145 +unt 1264 +▁tem 800 +▁earth 2819 +▁master 2818 +▁fort 1110 +bed 2808 +▁second 2807 +▁horse 2802 +▁mad 1805 +▁stre 222 +▁fa 533 +idd 607 +gth 11 +owed 1310 +ually 952 +vered 660 +▁const 1501 +▁pa 671 +▁ten 2768 +▁dro 564 +▁ca 1989 +▁order 2755 +▁fif 495 +▁cir 232 +▁wish 2753 +▁eas 332 +hen 1281 +aim 279 +▁really 2735 +read 482 +ceived 487 +▁fear 2710 +▁spir 166 +▁abs 1234 +▁ill 2698 +▁spr 805 +▁list 444 +aced 1378 +osition 1267 +▁question 2688 +▁understand 2200 +▁chr 484 +▁wall 1715 +anger 104 +▁coming 2672 +▁aff 751 +▁everything 2669 +ider 526 +aughter 639 +ching 1929 +▁above 2653 +▁prince 2648 +▁expl 872 +ready 7 +▁least 2640 +▁step 1584 +▁ru 2034 +▁already 2634 +▁itself 2628 +▁used 2626 +▁reco 898 +▁necess 435 +▁suddenly 2617 +hn 92 +ger 1560 +ister 984 +▁around 2610 +▁case 2608 +▁general 2606 +▁num 918 +▁wal 877 +▁soul 2598 +▁mag 976 +▁evening 2596 +ized 1868 +▁known 2589 +▁lad 1509 +▁mus 1413 +imes 261 +▁lear 555 +▁quick 1233 +▁sch 718 +▁whether 2559 +ific 1503 +men 1710 +gn 371 +▁within 2553 +▁live 2544 +▁times 2540 +▁captain 2539 +▁expect 1434 +▁sort 2530 +▁women 2529 +▁friends 2529 +▁john 2524 +▁bring 2524 +▁dri 1313 +▁circ 1031 +▁sum 1487 +iled 1573 +vern 446 +▁table 2510 +▁meet 1746 +▁returned 2507 +▁held 2505 +▁state 2505 +▁exc 810 +▁big 2493 +▁else 2491 +▁obser 781 +▁perfect 1430 +▁lea 442 +▁during 2479 +▁hus 411 +ident 732 +▁beautiful 2469 +▁says 2467 +▁town 2462 +▁clear 1861 +▁either 2459 +ted 2458 +▁sight 2451 +▁cal 1435 +▁lost 2446 +▁foot 2441 +▁kept 2440 +▁cut 2438 +▁ran 2435 +▁means 2433 +ience 1403 +▁prof 1137 +▁sleep 2425 +tered 1826 +ety 707 +▁past 2414 +▁dream 1868 +▁fellow 1933 +here 825 +▁became 2401 +ages 2397 +▁pret 407 +ution 2392 +▁bad 2390 +▁disc 1678 +▁making 2388 +▁object 1874 +▁low 2386 +▁mist 1128 +▁strange 2379 +▁towards 2379 +▁dev 1803 +▁year 2373 +▁manner 2370 +▁human 2365 +ient 1628 +ught 489 +old 1240 +ines 1630 +▁sever 159 +ches 2338 +airs 407 +▁est 1419 +▁city 2326 +▁dem 1102 +▁sometimes 2322 +ility 1017 +▁ann 2318 +▁rose 2318 +▁walk 1487 +▁pal 1054 +▁road 2309 +gs 672 +▁ready 2302 +lish 487 +▁stra 837 +cious 175 +ians 1711 +iew 484 +▁cold 2286 +▁corn 1068 +iously 2272 +mer 1240 +▁bat 1113 +cri 963 +owing 1549 +bert 1539 +br 2264 +lled 1755 +▁squ 1511 +▁truth 2258 +▁pie 178 +▁thy 2253 +▁didn 2251 +▁ship 1672 +▁thir 814 +▁soci 173 +▁business 2248 +▁bit 1263 +▁mur 897 +lly 2243 +▁subject 2239 +▁won 2238 +▁appro 1707 +▁river 2235 +▁answer 2232 +▁sto 736 +▁appear 1041 +▁free 2226 +rie 1071 +ingly 1718 +▁arms 2220 +▁excl 678 +▁family 2220 +▁hair 2219 +ern 1694 +ony 2217 +▁ut 1305 +▁cast 1301 +ape 488 +▁jud 842 +▁letter 2202 +▁become 2201 +ouble 30 +▁christ 1477 +ote 1291 +▁possible 2194 +▁different 2194 +▁val 1446 +ffect 102 +▁several 2185 +▁lead 1572 +▁express 1022 +▁natur 14 +li 1446 +▁fine 2169 +▁forg 317 +itting 732 +▁arri 495 +▁sus 1414 +▁blood 2156 +▁idea 2155 +ah 2154 +▁quiet 1330 +oon 993 +▁glad 2146 +▁able 2142 +rence 516 +itted 1658 +▁consider 1551 +ster 1473 +▁charac 280 +▁tou 473 +▁begin 995 +▁win 952 +▁eye 2127 +▁across 2125 +▁equ 812 +▁except 2124 +band 62 +▁chur 214 +▁story 2120 +▁pen 2119 +eth 1002 +ative 2116 +llect 5 +▁fre 461 +▁partic 376 +▁surpr 227 +▁diffic 0 +▁window 2109 +▁direct 1382 +▁disco 497 +▁grand 1602 +▁prov 2106 +▁cent 1584 +isf 168 +ey 784 +teen 700 +▁conc 1163 +aw 2095 +▁hen 553 +▁soft 1474 +▁flo 778 +▁sor 145 +by 2084 +▁wild 2083 +▁effect 2083 +▁sold 489 +▁longer 2078 +▁pub 367 +▁followed 2076 +▁cross 1519 +ased 939 +▁govern 277 +antly 1401 +ards 2063 +▁husband 2061 +▁respect 2061 +ts 2055 +▁suppose 2051 +▁comple 781 +▁hast 2045 +▁taking 2041 +▁ap 1070 +▁rich 2035 +▁sal 1461 +▁happy 2034 +▁heav 518 +▁creat 556 +une 1006 +▁stay 2024 +▁thee 2021 +▁daughter 2020 +▁du 1262 +▁bird 1111 +▁spoke 2018 +▁wee 53 +▁requ 1393 +▁ve 2012 +▁mere 956 +▁green 2006 +▁din 617 +▁anim 306 +alth 222 +▁possess 697 +ny 2000 +empt 93 +▁gard 350 +anged 955 +▁reached 1992 +ination 1418 +▁appeared 1990 +▁pretty 1988 +ov 1986 +▁exist 1045 +▁common 1982 +▁opened 1982 +▁remember 1979 +utes 389 +▁slow 613 +▁book 1976 +ames 703 +▁hea 134 +ract 1196 +▁imag 694 +ley 1965 +▁tom 1964 +▁occas 337 +king 1963 +▁rus 910 +ices 1362 +▁saying 1955 +ump 1142 +ms 1310 +▁fav 418 +ising 1203 +▁exper 413 +▁bright 1937 +▁break 1041 +▁accord 325 +▁satisf 571 +▁month 775 +▁sense 1929 +▁scar 519 +▁lu 1927 +▁succ 135 +▁secret 1927 +tern 1350 +use 1431 +▁stri 1416 +▁continued 1921 +▁carried 1917 +nding 1916 +eric 264 +▁cry 1415 +▁church 1908 +▁certainly 1903 +▁wanted 1899 +▁spirit 1894 +▁command 1893 +▁move 1198 +▁sens 1890 +oun 1120 +▁tree 1888 +▁dest 1145 +ham 1389 +cul 1046 +▁rock 1270 +ots 1880 +▁sweet 1878 +▁account 1878 +riage 10 +▁decl 984 +▁purp 346 +▁ought 1870 +outh 222 +▁def 1869 +▁eat 1869 +▁prep 1088 +▁street 1866 +▁esc 379 +owers 119 +▁draw 956 +▁breat 680 +▁stud 1120 +medi 77 +▁character 1854 +ime 1288 +▁school 1844 +▁regard 1275 +▁feeling 1842 +▁heaven 1837 +ement 1240 +itude 1836 +▁sad 1835 +▁worth 1195 +▁mot 896 +▁trans 1828 +▁hill 1132 +aimed 282 +▁dut 694 +▁smo 645 +▁disapp 850 +▁hung 1326 +▁blue 1817 +▁pain 1817 +ury 1296 +▁quar 1045 +▁hot 1272 +ney 789 +▁english 1809 +▁delight 1264 +idence 507 +▁view 1803 +▁produ 1150 +▁dress 1799 +▁laugh 1055 +umb 1798 +▁bra 1012 +son 1797 +▁seven 1796 +▁mill 1794 +▁port 1794 +▁hat 1792 +▁fast 1792 +▁lived 1792 +▁success 1792 +▁terri 672 +▁neither 1789 +▁princess 1789 +▁fight 1269 +▁court 1787 +▁div 1291 +▁marg 471 +▁self 1781 +▁immedi 397 +▁hours 1771 +▁var 872 +ze 1768 +▁ang 897 +▁sol 1766 +▁carry 1140 +▁exam 1146 +▁import 218 +to 992 +▁tried 1762 +ric 1761 +ledge 346 +▁disp 1753 +▁uncle 1752 +▁entered 1752 +▁further 1752 +▁len 278 +▁cause 1747 +ask 561 +▁forward 1746 +▁ge 680 +▁none 1743 +▁danger 1195 +▁det 1223 +▁pay 1740 +▁tor 1740 +▁front 1737 +▁offic 449 +ror 529 +▁particular 1097 +▁deal 1732 +▁led 1730 +▁added 1728 +lect 845 +▁acqu 716 +▁chief 1723 +▁happened 1722 +▁walked 1721 +iar 213 +▁prot 786 +▁eff 486 +▁bene 708 +▁maj 168 +▁ord 266 +▁public 1709 +▁later 1708 +▁alth 3 +▁pray 1707 +▁joy 1706 +▁affect 893 +▁although 1704 +▁sho 585 +▁fig 580 +▁prob 542 +ening 1202 +gr 1700 +iam 133 +▁comfort 1216 +day 1059 +resh 461 +uct 648 +ead 207 +ality 1684 +▁fail 1060 +▁number 1682 +ship 1164 +▁yourself 1676 +▁mem 913 +▁ah 1673 +▁doing 1673 +▁seems 1672 +▁living 1672 +side 479 +oof 321 +arent 146 +▁shad 771 +▁bey 1 +▁field 1082 +▁beyond 1664 +▁desp 952 +ize 1104 +▁received 1655 +▁queen 1650 +▁boat 1649 +▁garden 1649 +inct 239 +▁mouth 1647 +▁party 1647 +▁americ 209 +▁je 1134 +▁line 1644 +▁bear 1641 +▁visit 1641 +▁gentleman 1641 +▁phil 914 +▁entire 554 +▁laid 1634 +ism 1633 +rig 612 +▁clot 458 +▁hall 1633 +▁age 1632 +▁repe 627 +raid 53 +irt 674 +▁occasion 1627 +uff 1033 +▁nearly 1626 +▁sister 1626 +ously 1621 +▁accom 139 +▁whis 815 +▁trees 1615 +iff 1128 +iling 889 +▁vi 325 +▁lab 989 +ming 1082 +wards 579 +▁eighteen 1605 +▁prison 962 +bit 694 +ones 916 +anced 1005 +▁forth 1601 +▁princi 452 +ilt 780 +▁early 1594 +▁thinking 1593 +▁enem 1 +▁tal 848 +ug 945 +▁vill 694 +▁try 1588 +▁minutes 1588 +▁da 451 +▁chance 1584 +▁england 1583 +▁crow 187 +▁med 1581 +▁mary 1581 +▁cre 1579 +where 582 +▁anx 719 +▁late 1575 +▁afraid 1575 +▁smile 1574 +▁fle 891 +lling 1570 +▁wait 1568 +▁pict 122 +▁news 1031 +▁wis 1057 +▁stren 255 +▁aunt 1561 +▁seeing 1560 +iment 1559 +▁pleasure 1559 +oint 798 +▁neigh 19 +▁super 990 +▁glass 1549 +▁miles 1549 +▁trouble 1549 +rench 133 +▁bill 1078 +▁silence 1546 +▁rob 1543 +▁exclaimed 1542 +ply 491 +pend 300 +▁plain 1059 +▁faith 1538 +▁drew 1537 +els 1535 +▁neighb 419 +▁chap 386 +gy 1058 +aret 214 +▁change 1529 +▁purpose 1526 +▁necessary 1526 +▁standing 1525 +▁pract 1524 +▁mine 1519 +▁deter 58 +▁heavy 1514 +▁sevent 313 +▁inqu 420 +▁top 1509 +fered 120 +▁shar 486 +icked 315 +▁caught 1506 +▁bell 1500 +▁natural 1500 +▁bott 575 +▁hardly 1498 +ives 1491 +▁stopped 1491 +▁fing 72 +▁circum 166 +wh 69 +▁auth 138 +rupt 671 +▁wit 839 +ounded 947 +▁grew 1481 +▁opin 282 +ament 1480 +aining 1480 +▁av 703 +▁instead 1478 +mon 1476 +▁island 1476 +▁knight 1472 +▁length 1472 +ls 1470 +▁ago 1468 +▁hig 139 +ush 1467 +inary 229 +uc 670 +▁loved 1463 +head 954 +ically 1461 +▁third 1460 +▁straight 1459 +▁determ 516 +▁priv 613 +form 544 +the 791 +▁inn 683 +ues 1454 +▁thor 784 +ining 865 +▁beauty 1451 +▁peace 1450 +ler 1447 +pper 829 +▁cra 1447 +ior 886 +ancy 638 +asp 878 +▁infl 313 +▁convers 260 +rage 573 +▁conv 958 +▁besides 1438 +▁thirty 1437 +▁den 1435 +▁dog 1435 +ored 1434 +▁sitting 1432 +▁attention 1429 +ager 268 +▁bab 582 +▁rom 1427 +▁hist 374 +▁position 1424 +tend 683 +aged 733 +▁inform 347 +▁hop 410 +▁comes 1420 +▁lie 1418 +▁pull 726 +▁rid 1417 +▁please 1417 +iter 751 +ustom 23 +ma 886 +▁build 685 +▁grace 1415 +▁french 1414 +▁ast 524 +▁whatever 1413 +▁resol 819 +▁struck 1412 +▁force 1411 +▁girls 1411 +▁scarce 288 +▁chair 1408 +ghed 103 +▁knowledge 1407 +▁wrong 1406 +hood 1405 +▁rap 311 +▁fifty 1403 +▁attempt 1401 +▁ver 1399 +▁south 1398 +▁fat 1397 +▁thoughts 1397 +▁burn 881 +▁watch 1396 +▁crowd 1395 +▁arr 785 +▁box 1393 +▁getting 1393 +▁save 1389 +▁thick 1389 +▁slight 1389 +▁dinner 1388 +▁enj 257 +▁die 1385 +▁company 1384 +▁rev 505 +▁class 1382 +▁plan 1381 +▁exact 434 +▁immediately 1381 +iers 504 +usion 1376 +▁maid 1376 +▁treat 1376 +pping 1373 +▁incl 1373 +ius 1372 +▁born 1372 +▁drink 1371 +▁meant 1371 +iced 541 +▁forest 1367 +▁sugg 6 +▁tears 1364 +▁slowly 1364 +▁boys 1362 +ists 1361 +ndon 33 +▁lim 1359 +▁fool 812 +▁suggest 793 +▁compl 1356 +▁result 1356 +▁saint 1355 +▁accept 834 +▁died 1354 +ceive 653 +▁stone 1353 +▁post 1352 +▁fright 622 +▁ty 778 +▁author 666 +▁paper 1349 +▁belong 1349 +cient 475 +▁warm 1348 +▁enc 1347 +▁youth 1347 +▁beside 1344 +▁send 1343 +▁bound 1343 +▁conse 262 +▁quickly 1343 +▁tone 1341 +▁vict 1340 +▁er 1339 +▁leg 1339 +▁jour 315 +▁imposs 92 +▁brown 1335 +▁contr 630 +▁fix 404 +ified 1333 +▁camp 1333 +val 836 +ics 1330 +▁desire 1329 +▁filled 1327 +▁moon 1326 +▁london 1326 +▁snow 1324 +▁circumst 349 +▁distance 1324 +▁obl 222 +ging 1321 +▁lips 1321 +ift 564 +▁north 1319 +▁forget 1319 +▁makes 1318 +▁mid 762 +▁pock 309 +▁placed 1315 +▁margaret 1315 +ooks 225 +▁food 1313 +ann 1312 +▁floor 1311 +ries 1310 +▁sac 584 +▁hors 27 +itions 727 +▁spring 1308 +▁strength 1308 +▁id 1307 +▁ball 1307 +▁following 1307 +arch 603 +▁jack 1305 +▁descri 1305 +▁remained 1305 +▁laughed 1304 +bs 1303 +▁safe 1302 +▁grow 1300 +itten 217 +▁broken 1297 +▁nar 403 +ively 1296 +▁profess 737 +aded 1294 +amed 627 +oured 1294 +▁coun 1294 +▁depart 1293 +▁und 1292 +▁government 1291 +ats 1289 +▁viol 1287 +▁easy 1284 +▁waiting 1284 +▁knows 1283 +▁horses 1282 +▁proceed 756 +dom 521 +ges 1279 +▁thank 1279 +erv 736 +▁anne 1276 +▁dread 680 +ented 1274 +▁piece 1274 +▁gall 1271 +ugg 287 +▁une 270 +▁shout 654 +▁gaz 1268 +▁neck 1267 +dered 520 +▁complete 698 +▁van 1265 +▁guard 1263 +asure 103 +▁wide 1262 +▁started 1262 +ope 564 +▁doll 510 +▁sail 1260 +▁palace 1260 +cially 168 +isted 1258 +ires 1256 +▁lift 509 +ences 1254 +▁ger 456 +acy 1247 +▁talking 1247 +▁imm 760 +let 1244 +▁service 1244 +▁impossible 1244 +▁week 1243 +und 1240 +▁deg 464 +▁mess 715 +▁fun 1237 +▁stop 1237 +▁gar 1236 +▁fresh 1235 +▁difficult 1234 +▁hy 710 +reme 343 +▁effort 1232 +▁society 1232 +▁grave 1229 +well 1227 +▁weak 1225 +aching 1224 +▁incre 716 +▁proper 1224 +▁demand 618 +▁army 1217 +▁shut 1217 +iddle 173 +▁cloud 709 +▁corner 1214 +▁nine 1211 +▁turning 1210 +▁sky 1209 +ivid 282 +▁mass 1205 +▁below 1204 +▁sit 1202 +▁single 1202 +uss 489 +▁fly 705 +▁opinion 1199 +▁aftern 41 +▁lang 129 +▁occup 624 +▁trust 1194 +▁recogn 1194 +▁gun 1193 +▁outside 1192 +oom 467 +ises 1189 +▁nin 634 +▁beat 1189 +▁kiss 681 +cell 174 +uit 570 +▁couldn 1187 +▁evil 1186 +▁appearance 1186 +par 205 +▁mention 578 +▁running 1183 +▁ant 1182 +arer 471 +▁bank 1181 +▁conn 207 +▁conversation 1181 +pic 432 +▁west 1179 +▁breath 1179 +▁espe 86 +▁greater 1177 +ency 1176 +▁showed 1176 +▁winter 1176 +▁fash 51 +▁clothes 1175 +▁morrow 1173 +ases 560 +▁coll 652 +▁arrived 1172 +▁sand 1171 +▁music 1171 +ork 465 +▁kne 540 +▁peter 1169 +cial 440 +hered 381 +▁married 1168 +▁presence 1168 +▁emb 1166 +▁pun 1166 +▁usual 1163 +vere 618 +▁mark 1162 +▁suffic 426 +▁neg 1161 +▁former 1161 +▁giving 1161 +▁wished 1161 +ushed 680 +▁eager 1160 +▁probably 1160 +▁gro 1159 +aped 677 +▁pet 1158 +▁sym 344 +itation 1157 +▁battle 1157 +▁simple 1157 +▁mountain 1157 +▁hunt 609 +▁months 1156 +▁expression 1156 +▁covered 1155 +▁afternoon 1155 +▁event 558 +▁relig 79 +rief 0 +ories 659 +str 1149 +▁moved 1149 +▁states 1149 +▁chapter 1148 +▁cab 550 +▁shore 1147 +▁concl 1146 +▁pop 638 +▁silent 1145 +▁loc 1144 +▁speaking 1143 +gar 1138 +▁instant 1137 +amb 1136 +▁phys 601 +▁occur 509 +▁beginning 1136 +▁tur 1135 +light 1135 +▁polit 584 +▁stream 1134 +▁easily 1133 +unity 343 +▁remark 586 +ald 1131 +▁pale 1131 +▁sick 1131 +▁z 1130 +▁remain 1130 +▁enjoy 1129 +▁influ 253 +▁duty 1128 +▁pleasant 1127 +▁path 1126 +▁twel 1 +▁twelve 1125 +cho 1124 +▁fashion 1124 +uel 364 +▁vol 1122 +▁figure 1122 +▁raised 1122 +▁scarcely 1121 +che 1119 +▁star 1119 +▁terrible 1117 +▁conscious 595 +▁minute 1114 +▁kill 1113 +▁leaving 1113 +▁fut 128 +▁threw 1109 +▁marry 1108 +olog 1107 +yal 221 +▁tall 1104 +▁expected 1104 +▁forty 1103 +▁according 1103 +ature 1102 +▁sword 1102 +rib 1101 +▁flowers 1101 +inal 630 +▁guess 1100 +▁oblig 267 +▁stick 1100 +zz 1099 +▁trying 1099 +▁iron 1093 +▁rapid 533 +▁ben 1092 +▁learned 1091 +▁especially 1091 +▁killed 1090 +▁places 1090 +▁books 1089 +▁isn 1088 +ipped 1088 +▁office 1088 +ulty 211 +orted 1087 +▁silver 1087 +▁purs 1085 +▁surprise 1085 +▁direction 1085 +▁entirely 1084 +wise 449 +▁cho 583 +▁bread 1083 +▁tender 1082 +ech 182 +▁consequ 1081 +orth 1080 +▁written 1080 +ibly 442 +▁exh 605 +▁write 1078 +▁ladies 1078 +▁condition 1078 +▁action 1077 +mit 1076 +eness 1076 +pered 276 +▁suc 44 +▁happiness 1073 +ounds 477 +▁enter 1072 +▁note 1070 +aled 1069 +tering 1069 +jo 328 +▁fal 272 +oop 1066 +▁langu 328 +▁golden 1065 +▁decided 1065 +▁wonderful 1065 +ume 1064 +▁geor 209 +▁unless 1064 +fer 466 +▁perfectly 1060 +▁shook 1057 +ndered 539 +uous 1054 +▁broad 1054 +▁steps 1054 +▁closed 1054 +atic 1053 +tended 1052 +▁merely 1052 +idered 133 +▁enemy 1051 +▁history 1051 +osing 1050 +▁wise 1050 +▁latter 1050 +▁understood 1050 +▁jew 1048 +▁simply 1048 +▁sole 369 +▁spot 1045 +ishing 1045 +▁wouldn 1044 +onel 174 +▁middle 1042 +mn 364 +llig 121 +▁admir 1041 +ital 522 +▁shot 1038 +▁conduct 1038 +▁pit 1037 +▁sav 1037 +▁tast 300 +▁changed 1037 +gan 417 +▁reply 1035 +fast 142 +▁board 1032 +▁succeed 421 +▁bow 1030 +▁summer 1030 +▁grass 1029 +retched 521 +▁experience 1029 +sy 1028 +gress 428 +uted 1027 +ax 1026 +io 1026 +▁afterwards 1026 +▁lying 1025 +▁repres 371 +▁finally 1025 +▁marriage 1025 +▁broke 1024 +▁fortune 1024 +▁fit 1023 +▁situ 257 +▁observed 1023 +▁sharp 1022 +▁flat 1021 +▁orig 15 +▁trave 1021 +▁journey 1021 +▁mir 1020 +osp 443 +▁lean 1019 +▁persons 1016 +▁earn 193 +▁calm 1014 +▁faint 1014 +▁excell 354 +unk 1013 +▁clim 1011 +▁listen 1008 +▁acquaint 516 +olute 57 +▁beneath 1007 +ouses 87 +▁ring 1006 +▁pocket 1006 +ests 1005 +▁oper 1005 +cience 452 +▁lives 1005 +▁repeated 1005 +bered 122 +▁tail 1004 +▁passion 1003 +▁escape 1000 +▁fish 998 +▁ven 997 +clock 997 +▁gate 997 +▁mut 996 +▁prev 996 +▁madame 995 +▁lou 994 +▁famil 318 +▁becom 20 +▁reach 992 +▁empl 32 +▁sorry 990 +▁allowed 990 +oses 989 +ples 989 +▁dick 989 +▁ign 272 +ivers 334 +▁bless 987 +▁health 987 +bing 986 +▁furn 986 +▁temper 986 +semb 479 +ission 985 +▁firm 984 +▁rain 984 +▁bitter 984 +▁ban 983 +▁future 983 +▁strugg 983 +▁spite 982 +ina 981 +▁cand 981 +umed 979 +▁passing 979 +kes 978 +▁separ 978 +hel 977 +iding 977 +▁sett 232 +▁pleased 975 +▁circumstances 975 +▁loud 974 +▁magn 458 +▁connect 974 +▁letters 973 +▁civ 117 +▁kit 258 +idden 376 +▁nice 972 +▁disappe 315 +▁darkness 972 +▁tea 971 +come 403 +▁yellow 970 +▁important 970 +▁presently 968 +▁walls 967 +▁spread 967 +▁dressed 967 +▁discovered 967 +▁apart 966 +▁worse 966 +▁wand 965 +▁nation 964 +▁legs 963 +▁delic 474 +my 962 +▁blind 961 +▁remar 247 +▁dim 959 +▁spec 423 +▁employ 958 +▁dam 957 +▁rub 957 +otten 128 +idently 86 +▁dry 953 +▁cost 953 +▁virt 441 +▁sorrow 953 +ec 192 +rel 952 +urs 952 +velop 150 +▁game 951 +▁opport 162 +▁absolute 377 +▁heads 948 +oved 434 +▁exactly 947 +▁ears 946 +▁host 946 +▁beast 946 +ference 946 +▁subst 945 +▁dropped 945 +▁chee 474 +▁knowing 943 +▁bag 942 +▁determined 942 +▁ves 106 +▁seat 941 +▁smiled 941 +ils 940 +▁creature 940 +▁existence 938 +▁resp 937 +▁major 937 +roy 195 +▁conce 936 +▁storm 936 +▁notice 936 +▁promise 936 +▁concer 935 +▁motion 935 +▁pros 463 +ishment 934 +▁protect 933 +▁cous 155 +que 931 +anging 931 +▁press 931 +▁arg 930 +▁fixed 930 +▁learn 930 +▁leaves 930 +ours 928 +▁prevent 928 +▁vent 926 +▁assist 925 +lend 30 +ument 924 +▁bottom 924 +▁hes 923 +▁pick 923 +▁individ 35 +▁nobody 922 +iod 38 +▁intellig 440 +iety 919 +acked 919 +▁houses 919 +ashed 918 +erable 417 +▁considered 918 +▁dom 916 +action 306 +▁castle 915 +▁empt 167 +▁stir 914 +▁suit 914 +▁advant 230 +▁disg 913 +ancing 913 +▁utter 912 +▁perform 912 +rect 401 +▁tired 911 +ruct 910 +istr 909 +▁habit 909 +zen 360 +▁cheer 908 +▁birds 907 +▁estab 907 +▁frequ 378 +▁wrote 907 +idge 423 +▁stro 906 +▁dar 905 +▁ones 905 +▁luck 904 +▁drawing 904 +inite 421 +▁vall 174 +▁honor 902 +▁pra 901 +▁impress 402 +▁propert 77 +ell 900 +▁song 900 +▁clean 899 +▁speech 899 +ays 898 +▁pure 898 +▁various 897 +▁shadow 896 +▁mal 895 +▁noble 895 +▁village 895 +▁paid 894 +▁splend 238 +▁ur 893 +rodu 118 +▁narrow 893 +▁allow 892 +▁declared 892 +▁breakfast 892 +▁wear 891 +▁extra 326 +▁believed 891 +▁train 890 +▁somewh 2 +▁aston 889 +▁extreme 889 +▁somewhat 888 +▁individual 888 +ayed 887 +▁judge 887 +▁moder 324 +▁ham 886 +▁lower 886 +▁parts 886 +enth 885 +▁jane 885 +▁agree 885 +▁royal 885 +go 882 +▁period 882 +▁confess 882 +▁remembered 882 +pan 124 +▁reflect 881 +▁cott 322 +▁animal 879 +▁custom 879 +na 878 +▁greatest 878 +▁revol 877 +▁scene 877 +iny 876 +▁drawn 876 +▁soldiers 876 +▁influence 876 +▁difficulty 876 +▁mel 875 +▁ways 874 +▁surely 874 +▁desert 873 +▁ancient 873 +▁likely 872 +amber 74 +itary 372 +▁angry 871 +▁looks 871 +ator 870 +▁sake 870 +▁holding 870 +▁evidently 870 +▁inj 869 +▁jul 869 +▁laws 869 +▁proud 869 +▁colonel 869 +▁shr 868 +▁sin 867 +▁courage 867 +anted 866 +▁apparent 297 +nes 865 +▁comb 865 +▁mort 865 +▁mod 864 +▁harm 864 +▁picture 863 +ring 862 +gged 861 +▁troub 861 +▁spoken 861 +ision 860 +▁burst 860 +▁watched 860 +esty 251 +ption 859 +▁shop 858 +▁anxious 858 +oose 353 +▁scr 857 +▁finished 857 +under 377 +▁east 856 +▁ideas 856 +▁inside 856 +▁fifteen 856 +otion 374 +▁agre 202 +▁dism 855 +▁civil 855 +▁george 855 +▁henry 854 +▁fol 853 +▁touch 853 +▁space 849 +▁attack 849 +▁fallen 849 +▁formed 849 +▁surf 88 +▁thin 848 +rer 847 +▁goes 847 +iently 847 +ium 845 +▁bal 845 +▁baby 845 +▁social 845 +▁cover 844 +▁private 844 +▁gentlemen 844 +▁appre 843 +▁carriage 842 +▁advent 357 +▁waited 841 +▁ourselves 841 +uded 840 +▁speed 839 +cise 838 +itive 838 +▁servant 837 +▁wine 836 +▁noise 836 +▁pecul 5 +▁vessel 835 +▁william 835 +▁grou 217 +▁tong 173 +▁windows 834 +▁lot 833 +▁commun 833 +▁obliged 833 +▁ice 832 +erial 108 +▁favor 832 +▁impro 832 +▁jim 831 +▁meth 109 +▁vain 831 +▁peculiar 831 +▁grad 287 +▁walking 830 +▁carefully 830 +▁noticed 829 +▁everybody 829 +▁pot 828 +▁forgotten 828 +▁thro 320 +▁elect 827 +▁shoulder 826 +▁stranger 826 +▁higher 825 +▁promised 825 +▁bod 260 +▁lose 824 +▁glance 824 +▁property 824 +het 823 +▁fier 243 +▁hero 823 +▁main 823 +▁tremb 317 +▁cup 822 +▁france 822 +▁earnest 822 +▁quietly 822 +af 820 +▁american 820 +▁generally 820 +▁serious 819 +ral 818 +iction 818 +▁offered 818 +▁ine 817 +ables 817 +▁animals 817 +▁handsome 815 +▁shoulders 815 +▁sla 314 +▁dust 814 +▁symp 144 +▁built 814 +▁personal 814 +▁interrupt 267 +▁cow 813 +osite 34 +▁poet 812 +▁distinct 812 +▁affection 812 +ague 811 +iant 811 +stem 17 +▁mam 282 +▁edge 811 +▁grown 811 +▁wel 243 +▁heat 810 +▁contro 268 +acing 809 +▁desce 809 +uly 255 +cher 808 +▁unk 153 +▁murm 807 +itated 807 +▁fancy 807 +▁skin 806 +▁spent 806 +▁touched 806 +▁pier 805 +▁wasn 805 +▁faces 805 +▁pieces 805 +▁jump 804 +osity 232 +andon 230 +arily 802 +nday 230 +▁develop 801 +▁san 800 +▁whispered 800 +ingu 55 +itch 799 +▁fond 799 +enced 798 +▁butter 798 +▁surprised 798 +oted 797 +▁coat 797 +▁chamber 797 +▁intellect 797 +▁mountains 797 +wood 796 +isters 254 +▁race 795 +▁false 795 +▁german 795 +ob 794 +hip 794 +▁system 794 +▁eld 308 +onsie 87 +▁bare 793 +▁educ 243 +how 792 +▁fan 792 +▁rag 792 +▁defe 792 +▁wealth 792 +aked 791 +▁lin 791 +▁fingers 791 +key 790 +over 249 +acher 790 +▁opportunity 789 +▁pi 788 +▁arch 788 +▁dare 788 +▁surr 788 +athered 75 +▁brave 786 +▁matters 786 +▁fer 784 +▁hol 784 +▁alar 784 +▁weather 784 +▁companion 784 +▁mach 783 +ipp 782 +▁lovely 782 +▁collect 782 +▁meaning 782 +▁crown 781 +▁store 781 +▁prepared 779 +nown 123 +▁opposite 778 +aff 777 +▁gray 777 +▁cousin 777 +▁greatly 777 +▁ris 776 +▁finding 776 +▁avo 286 +▁dor 775 +▁inh 160 +▁unh 156 +▁offer 775 +▁degree 775 +▁honour 775 +▁introdu 775 +▁lake 774 +▁rail 774 +▁por 773 +▁attract 773 +▁reading 773 +raph 271 +▁frank 772 +▁innoc 275 +▁paris 772 +▁writing 772 +leep 5 +▁grat 771 +▁breast 771 +▁alto 3 +erve 769 +fortun 159 +ounced 769 +ee 770 +cing 768 +ilar 101 +▁charge 768 +night 767 +▁blow 767 +▁quarter 767 +▁feelings 767 +▁altogether 767 +ners 766 +▁fate 766 +▁hurt 766 +▁asleep 766 +▁caused 766 +▁situation 766 +▁dav 289 +▁hide 765 +▁alive 765 +cil 764 +▁mix 764 +▁band 764 +▁stat 764 +pected 763 +ding 762 +ites 762 +▁cook 762 +▁equal 761 +▁memory 761 +▁thrown 761 +▁meeting 761 +▁bent 760 +▁exec 760 +▁kingdom 760 +▁surface 760 +ron 759 +▁grey 759 +▁loss 759 +▁constit 759 +ointed 758 +▁cruel 758 +▁rough 758 +▁ess 757 +▁accompan 757 +▁duke 756 +▁fill 756 +▁liked 756 +▁swift 756 +▁curious 756 +▁opening 756 +pet 755 +chen 41 +▁seek 755 +▁servants 755 +arest 754 +oura 753 +▁due 753 +▁sides 753 +ibility 753 +▁united 753 +▁gh 247 +▁hath 752 +▁reve 752 +▁height 752 +▁questions 752 +▁satisfied 751 +▁pity 750 +▁dollars 750 +▁ange 748 +ara 747 +▁empty 747 +▁lifted 747 +▁square 747 +▁suspic 747 +▁mart 746 +▁happen 746 +▁ordered 746 +▁yours 745 +▁settled 745 +▁perm 744 +▁freed 135 +▁distingu 744 +▁laughing 744 +▁supposed 744 +osop 17 +▁rode 743 +▁hurry 743 +▁value 743 +▁talked 743 +▁listened 743 +▁papa 742 +bur 741 +▁destroy 741 +▁rejo 740 +▁attend 740 +▁hurried 740 +▁vast 739 +▁rising 739 +▁pack 738 +▁study 738 +ille 268 +▁nose 737 +▁rise 737 +▁taste 737 +▁forced 737 +▁telling 737 +▁language 737 +▁ow 736 +▁rul 736 +▁capt 736 +▁serve 736 +▁sufficient 736 +▁roof 735 +▁pin 733 +▁woods 733 +▁bas 732 +▁egg 258 +▁pan 732 +▁diam 93 +▁building 730 +▁frightened 730 +▁sam 729 +▁tit 729 +▁distur 729 +▁valley 729 +▁regular 729 +eer 224 +▁special 728 +▁station 728 +hest 223 +▁shape 727 +ocr 726 +▁sacri 726 +▁libert 81 +▁justice 726 +▁directly 726 +▁philosop 726 +con 725 +like 725 +▁fault 725 +▁sixty 725 +▁stairs 725 +▁smiling 725 +yr 724 +▁amb 724 +▁accompl 724 +▁material 724 +▁watching 724 +erved 723 +▁ordinary 723 +▁gloom 722 +▁method 722 +▁passage 722 +▁usually 722 +▁christian 722 +▁eur 24 +▁orders 721 +▁season 721 +▁seized 721 +▁weeks 720 +▁honest 720 +ify 719 +▁stock 719 +▁pride 718 +▁growing 718 +ested 717 +▁doesn 717 +▁suffer 717 +▁soldier 717 +▁bold 716 +▁ignor 716 +▁lines 716 +▁unf 715 +▁flower 715 +▁threat 715 +chief 714 +▁ende 46 +▁exer 714 +▁kitchen 714 +▁supper 713 +▁remarked 713 +ades 712 +▁wed 712 +▁discuss 712 +▁gathered 712 +▁mistress 711 +▁nearer 710 +▁affairs 710 +▁streets 710 +uses 709 +▁moving 709 +▁despair 709 +▁deb 708 +eping 708 +▁depend 708 +▁putting 708 +▁yo 707 +onsieur 0 +▁monsieur 706 +ik 705 +uce 705 +rance 188 +▁york 705 +▁catch 705 +▁contrary 705 +ude 704 +hens 704 +▁ble 704 +▁hole 703 +▁search 703 +rem 702 +▁cro 702 +eds 701 +▁bir 110 +▁cit 129 +imate 701 +icated 701 +▁imagine 701 +lin 700 +▁receive 700 +min 698 +▁march 698 +▁twice 698 +▁europe 697 +yn 696 +aly 696 +▁jer 696 +▁reck 216 +antic 693 +▁hills 693 +▁working 693 +▁movement 693 +▁arth 4 +▁favour 692 +▁pulled 692 +ole 691 +▁lod 691 +▁dign 203 +▁prove 690 +ocked 689 +usted 689 +lished 689 +▁arthur 688 +▁proved 688 +▁engaged 688 +▁possession 688 +▁aut 686 +▁haven 686 +▁observ 686 +▁uns 685 +▁aside 685 +▁stones 685 +▁process 685 +▁hence 684 +▁instance 684 +▁advantage 684 +▁authority 684 +▁cler 683 +▁farm 683 +▁brothers 683 +▁mas 682 +ception 682 +▁flew 681 +ails 680 +uls 679 +▁fart 70 +▁difference 679 +▁solemn 677 +▁weight 677 +▁rate 676 +set 675 +▁doors 675 +▁fortun 675 +▁familiar 675 +▁address 674 +▁spect 673 +▁played 673 +▁di 671 +▁holy 671 +▁aware 671 +ky 670 +▁test 670 +▁sympath 165 +lls 669 +▁dogs 669 +▁shame 669 +▁distant 669 +▁thorough 186 +▁explained 669 +▁interesting 669 +▁wore 668 +▁endeav 668 +▁ol 667 +▁bull 667 +▁busy 667 +▁named 667 +▁similar 667 +▁ale 145 +▁amount 666 +▁color 665 +▁liter 665 +▁moments 665 +▁instantly 665 +ker 664 +theless 2 +▁seated 664 +▁worked 664 +▁pard 60 +▁emper 37 +agn 121 +nds 662 +bers 662 +unch 662 +▁brit 662 +▁measure 662 +▁nevertheless 662 +▁myster 172 +▁tongue 661 +▁hearing 661 +▁throw 660 +▁excellent 660 +▁naturally 660 +illa 659 +ulous 659 +▁takes 659 +▁officer 659 +▁wil 658 +▁indian 658 +▁confidence 658 +▁suf 86 +enance 87 +▁imper 657 +▁disappeared 657 +rid 656 +itable 656 +▁splendid 656 +▁necessity 656 +▁unw 655 +▁vir 69 +▁earl 655 +▁drive 655 +▁names 655 +▁parent 172 +▁unknown 655 +▁president 655 +den 654 +▁wid 654 +▁gods 654 +▁represent 654 +▁aud 653 +▁franc 653 +▁agreed 653 +▁univers 653 +▁keeping 652 +▁smoke 650 +▁produced 650 +lv 101 +▁terror 648 +▁artic 647 +▁magic 647 +▁span 646 +▁witness 646 +▁aug 92 +▁yester 7 +▁liberty 645 +▁support 645 +▁dull 644 +▁discover 643 +asc 642 +▁coast 642 +ulation 642 +▁prisoner 642 +▁companions 642 +zy 641 +▁amaz 641 +▁drove 641 +▁wicked 641 +▁spirits 641 +por 640 +ford 640 +▁rare 640 +▁works 640 +▁mighty 640 +▁powers 640 +▁falling 640 +▁evidence 640 +vey 639 +cent 639 +▁stars 639 +▁worthy 639 +▁yesterday 638 +▁particularly 638 +▁diamond 637 +▁pointed 637 +▁possibly 637 +▁cool 636 +▁claim 636 +ora 635 +▁design 635 +▁sought 635 +▁content 635 +▁wants 634 +▁otherwise 634 +▁persu 633 +▁terms 633 +▁yield 632 +▁eliz 144 +▁knees 630 +▁rat 629 +▁officers 629 +▁trium 4 +▁forms 627 +▁finger 627 +▁occurred 627 +▁paused 626 +▁travel 626 +▁emperor 626 +▁carrying 626 +▁bob 625 +alous 114 +▁gree 625 +▁ceased 625 +▁ali 624 +▁teeth 624 +ruction 624 +▁waters 624 +▁triumph 624 +umn 623 +▁gle 623 +▁lies 623 +▁labor 623 +▁required 623 +▁ener 622 +▁failed 622 +▁pair 621 +▁shown 621 +esh 62 +soci 19 +▁double 620 +▁hom 619 +▁unhapp 131 +▁example 619 +▁entertain 619 +▁principle 619 +iles 618 +▁pill 618 +▁organ 618 +▁upper 618 +▁report 618 +▁america 618 +ental 617 +▁fruit 617 +▁group 617 +▁softly 617 +▁charles 617 +▁buy 616 +▁affair 616 +▁foreign 616 +▁singing 616 +▁possessed 616 +isc 615 +race 615 +▁huge 615 +▁milk 615 +▁gives 615 +▁inhab 615 +▁shouted 615 +▁bask 105 +▁james 614 +▁der 613 +▁cases 612 +▁rocks 612 +▁rooms 612 +asm 611 +▁excit 15 +▁proof 611 +▁intended 611 +▁fet 610 +▁leading 610 +▁unfortun 610 +▁succeeded 610 +▁satisfaction 610 +▁ber 609 +▁dance 609 +▁priest 609 +▁farther 609 +▁freedom 609 +rose 92 +▁drop 608 +▁paint 608 +▁arrange 608 +▁majesty 608 +father 102 +▁moral 607 +acle 606 +▁tele 606 +lessly 606 +▁gently 605 +▁hearts 605 +▁mentioned 605 +▁religion 604 +▁pardon 603 +▁neighbour 603 +▁rule 602 +▁invent 602 +▁demanded 602 +angers 601 +▁associ 601 +▁regret 601 +▁curi 29 +▁plat 600 +▁clearly 600 +▁progress 600 +▁seventeen 600 +▁ros 599 +▁record 599 +▁presented 599 +using 598 +▁bore 598 +▁fairy 598 +▁joined 598 +▁seventy 598 +▁key 597 +▁unto 597 +▁cabin 597 +▁sheep 597 +▁fourth 597 +▁gather 597 +▁instinct 597 +▁suffering 597 +ival 596 +▁vig 596 +▁diss 596 +▁fully 596 +attered 596 +▁events 596 +▁hidden 596 +▁advanced 596 +▁dreadful 596 +▁excitement 596 +room 595 +▁awful 595 +▁pounds 595 +▁card 594 +▁prefer 594 +▁voices 594 +▁calling 594 +road 593 +▁sen 593 +iance 593 +▁stuff 593 +▁advice 593 +▁papers 593 +▁powerful 593 +▁resolved 593 +sie 31 +▁afford 592 +▁explain 592 +▁interested 592 +▁pos 591 +iments 591 +▁birth 591 +▁grief 591 +▁impat 591 +hy 590 +▁bor 590 +▁shel 590 +bly 588 +▁brill 588 +▁shoot 588 +▁clever 588 +ese 587 +▁hither 587 +▁interv 587 +pap 52 +▁obey 586 +▁virg 115 +▁chest 586 +▁steam 586 +▁edward 586 +▁police 586 +▁excited 586 +▁shining 586 +▁undert 585 +▁considerable 585 +▁ing 584 +▁cart 584 +▁sprang 584 +▁wag 583 +▁cord 583 +▁fields 583 +▁ep 582 +▁nodd 582 +▁pictures 582 +▁conditions 582 +▁fox 581 +acious 581 +▁desired 581 +▁dur 580 +▁cred 580 +▁ride 580 +▁conqu 580 +▁fierce 580 +▁served 580 +dle 579 +▁adam 579 +▁ships 579 +▁information 579 +ighed 578 +istic 578 +▁paul 578 +▁yard 578 +▁needed 578 +▁indians 578 +▁sooner 577 +▁perceived 577 +▁importance 577 +mond 576 +ture 576 +▁crit 576 +▁hosp 576 +ternal 576 +rant 575 +▁memb 31 +▁share 575 +▁anybody 574 +cer 573 +illy 573 +▁iss 573 +▁kat 573 +▁fold 573 +▁salt 573 +▁type 573 +ortion 573 +▁merch 573 +▁couple 573 +▁absolutely 573 +asant 572 +▁citiz 572 +▁devil 572 +▁abandon 572 +▁willing 572 +▁imagination 572 +▁task 571 +▁stead 571 +▁sunday 571 +▁suffered 571 +▁curiosity 571 +ha 570 +com 570 +hold 570 +llen 570 +▁env 570 +▁rad 570 +▁appl 570 +▁occupied 570 +▁countenance 570 +▁dawn 569 +▁lock 569 +▁apparently 569 +abel 568 +▁slave 568 +▁actually 568 +▁completely 568 +iest 567 +▁betr 567 +▁mult 567 +▁tight 567 +▁welcome 567 +▁regarded 567 +ett 566 +▁dw 566 +asped 566 +▁myst 566 +▁ain 565 +▁daily 565 +▁dying 565 +▁smith 565 +▁refused 565 +▁extraord 53 +▁suggested 565 +pir 564 +▁bay 564 +▁alas 564 +▁crime 564 +▁bodies 564 +▁accident 564 +▁modern 563 +▁sigh 562 +hus 36 +iot 561 +ette 561 +asons 561 +▁brief 561 +▁elsie 561 +▁whenever 561 +with 560 +▁oce 35 +▁feat 560 +▁hate 560 +▁mile 560 +▁rapidly 560 +▁superior 560 +▁odd 559 +▁horror 559 +▁thanks 559 +ulf 558 +▁brain 558 +▁flesh 558 +▁cottage 558 +▁stepped 558 +▁professor 558 +▁jes 557 +▁club 557 +▁flight 557 +▁bon 556 +▁dun 556 +▁drag 556 +▁buck 555 +▁ninet 555 +ization 555 +▁driven 555 +▁crossed 555 +ips 554 +▁barb 554 +▁maint 554 +▁midst 554 +▁worst 554 +▁points 554 +▁nay 553 +▁size 553 +▁lands 553 +▁truly 553 +▁august 553 +▁science 553 +gers 552 +gment 67 +▁cult 552 +▁sour 552 +▁wet 551 +▁instr 551 +▁trade 551 +▁inquir 551 +▁picked 551 +▁equally 551 +▁relations 551 +vert 550 +ishop 550 +▁tempt 550 +ability 550 +▁interp 550 +▁rushed 550 +▁taught 550 +▁education 550 +▁expressed 550 +▁political 550 +now 43 +▁anger 549 +▁displ 549 +▁resist 549 +▁sty 548 +▁sylv 548 +▁dozen 548 +ids 547 +▁crim 547 +▁foolish 547 +▁hunting 547 +▁dangerous 547 +▁interrupted 547 +ially 546 +▁deck 546 +▁worn 546 +▁remarkable 546 +▁meat 545 +rop 544 +acks 544 +cked 544 +▁fis 544 +▁asking 544 +▁belief 544 +▁severe 544 +▁members 544 +▁mistake 544 +▁delighted 544 +mas 543 +▁cop 543 +ading 543 +▁obst 543 +▁behold 543 +▁gradually 543 +resp 542 +▁ken 542 +▁control 542 +▁sisters 542 +▁sixteen 542 +agnan 0 +▁assert 541 +▁dreams 541 +▁murder 541 +artagnan 541 +▁moreover 541 +▁pu 540 +▁oak 540 +aries 540 +▁nerv 540 +▁depth 540 +▁enemies 540 +▁nap 539 +asing 539 +▁lone 539 +▁hotel 539 +▁kindly 539 +▁careful 539 +▁inquired 539 +▁wings 538 +▁eleven 538 +▁relief 538 +▁wounded 538 +ification 538 +acity 537 +▁stup 59 +▁invol 536 +▁origin 536 +▁species 536 +▁exped 535 +▁physical 535 +▁creatures 535 +reg 534 +▁nur 35 +▁sex 534 +▁plenty 534 +▁newspap 534 +▁saf 51 +▁keen 532 +▁saved 532 +▁recollect 532 +▁flour 531 +▁independ 531 +aunt 530 +aling 530 +while 13 +▁begun 530 +▁glory 530 +▁hoped 530 +▁lover 530 +▁shoes 530 +kin 529 +▁et 529 +▁aim 529 +▁mamma 529 +▁frequently 529 +▁sang 528 +▁buried 528 +▁famous 528 +▁princip 528 +▁approached 528 +▁smooth 527 +▁useful 527 +kins 526 +▁tot 526 +▁proceeded 526 +husi 0 +▁ocean 525 +▁enthusi 525 +▁tar 524 +stone 524 +▁grim 524 +▁harry 524 +▁message 524 +▁fac 523 +▁slept 523 +▁stern 523 +▁charming 523 +▁hyp 522 +▁park 522 +▁distr 522 +▁loose 522 +▁desper 522 +▁consciousness 522 +aming 521 +▁alex 521 +▁lamp 521 +▁meas 521 +ication 521 +▁feather 521 +▁accepted 521 +▁colle 520 +hem 519 +▁indif 519 +▁minds 519 +▁playing 519 +▁fighting 519 +▁precious 519 +▁knife 518 +▁union 518 +▁evident 518 +oes 517 +iana 517 +▁hale 517 +▁detect 517 +▁exceed 517 +▁capital 517 +▁century 517 +▁entrance 517 +▁meanwhile 517 +▁bew 516 +▁dig 516 +▁arose 516 +▁magnific 516 +▁wondered 516 +ailed 515 +stand 515 +▁stage 515 +▁thrust 515 +▁absence 515 +▁burning 515 +▁forgive 515 +erous 514 +uments 514 +▁facts 514 +ustomed 514 +▁capable 514 +▁hut 513 +▁lest 513 +▁mour 513 +▁closely 513 +▁objects 513 +▁removed 513 +▁constant 513 +▁neighbor 513 +▁friendship 513 +lla 512 +▁horn 512 +▁unex 512 +▁virtue 512 +▁extraordinary 512 +▁sha 511 +▁alice 511 +▁level 511 +▁anyone 511 +▁explan 511 +▁indign 511 +▁jealous 511 +▁ug 510 +▁ack 4 +▁correct 510 +▁somebody 510 +itor 509 +▁forb 509 +▁admit 509 +▁basket 509 +▁recall 509 +iable 508 +▁thom 508 +pelled 508 +▁helen 508 +▁plant 508 +▁kissed 508 +▁assured 508 +▁increased 508 +▁stretched 508 +▁everywhere 508 +▁nob 507 +▁ahead 507 +▁forgot 507 +▁strike 507 +▁throat 507 +▁popular 507 +▁fare 506 +▁acknow 506 +▁assemb 506 +▁eighty 506 +▁wishes 506 +▁retreat 506 +▁kindness 506 +▁trembling 506 +▁ly 505 +▁emp 505 +▁qual 505 +▁ghost 505 +▁clouds 505 +▁larger 505 +▁contempt 505 +▁sympathy 505 +▁grandfather 505 +oe 504 +oused 504 +▁hind 504 +▁queer 504 +▁choose 504 +▁highest 504 +▁accordingly 504 +izing 503 +▁russ 503 +▁grant 503 +▁awa 502 +▁frag 502 +▁wheel 502 +for 501 +▁fro 501 +ograph 501 +▁begged 501 +▁pressed 501 +▁miserable 501 +▁notes 500 +▁choice 500 +▁slaves 500 +▁governor 500 +oms 499 +ivered 499 +▁nurse 499 +▁remind 499 +▁military 499 +▁listening 499 +▁impression 499 +iber 498 +▁ara 498 +oops 497 +anches 497 +ration 497 +ushing 497 +▁ended 497 +▁innocent 497 +▁treasure 497 +igned 496 +▁waves 496 +▁arrival 496 +▁becomes 496 +bb 495 +▁divine 495 +▁experi 495 +▁flying 495 +▁informed 495 +▁aid 494 +hamed 1 +▁sell 494 +▁bought 494 +▁excuse 494 +▁manage 494 +▁marked 494 +▁reward 494 +mend 493 +▁hadn 493 +▁crying 493 +▁ashamed 493 +▁visible 493 +▁fred 492 +▁join 492 +▁weary 492 +▁hungry 492 +▁indust 492 +▁lights 492 +▁acquaintance 492 +estic 491 +stairs 491 +▁restr 491 +▁stories 491 +abeth 3 +astic 490 +▁baron 490 +▁sounds 490 +▁throughout 490 +aker 489 +▁dish 489 +▁sout 489 +▁avoid 489 +▁bowed 489 +▁scream 489 +▁delicate 489 +▁somewhere 489 +▁mysterious 489 +ocks 488 +aches 488 +▁amid 488 +orious 488 +▁blank 488 +▁uneas 488 +▁unhappy 488 +▁bos 487 +▁caut 487 +▁impl 487 +▁dignity 487 +▁elizabeth 487 +▁obt 486 +▁pow 486 +▁wra 486 +▁comr 486 +▁stiff 486 +▁advance 486 +▁immense 486 +ilst 485 +▁gay 485 +▁tis 485 +▁elder 485 +▁goods 485 +▁hollow 485 +▁prompt 485 +▁temple 485 +▁unable 485 +▁judgment 485 +pher 484 +rows 484 +▁adventure 484 +▁understanding 484 +eral 483 +ldom 6 +▁bridge 483 +onscious 483 +▁lighted 483 +▁parents 483 +▁project 483 +▁remains 483 +▁thoroughly 483 +ffe 482 +▁cher 482 +▁weap 482 +▁safety 482 +▁admitted 482 +▁infinite 482 +▁comfortable 482 +▁ox 481 +▁rome 481 +▁price 481 +▁settle 481 +▁emotion 481 +▁escaped 481 +▁returning 481 +anges 480 +▁hers 480 +▁hopes 480 +▁convin 480 +▁pushed 480 +▁appears 480 +▁plainly 480 +▁intelligence 480 +▁coo 479 +▁vic 479 +▁unus 479 +ensive 479 +▁wound 479 +▁bearing 479 +▁thunder 479 +▁sup 478 +▁meal 478 +inging 478 +▁contem 478 +▁helped 478 +▁reckon 478 +elt 477 +▁horri 477 +▁seldom 477 +▁stupid 477 +▁intention 477 +▁ash 476 +▁hint 476 +ulated 476 +▁david 476 +▁becoming 476 +▁fem 475 +▁rope 475 +lection 475 +▁thinks 475 +▁fellows 475 +▁eggs 474 +▁redu 474 +▁secure 474 +rit 473 +gypt 1 +▁exhib 473 +aud 472 +mother 472 +▁egypt 472 +▁skill 472 +otted 471 +▁tied 471 +▁virgin 471 +▁whence 471 +▁friendly 471 +▁prospect 471 +▁religious 471 +▁extent 470 +▁original 470 +▁dan 469 +▁fed 469 +ville 469 +▁cell 469 +▁mood 469 +▁billy 469 +▁cheeks 469 +▁ 1602 +e 20074 +t 33684 +a 24496 +o 16407 +n 13320 +i 12934 +h 9663 +s 116387 +r 14172 +d 13987 +l 19501 +u 11337 +m 18529 +c 12492 +w 9890 +f 9940 +g 12024 +y 30534 +p 12630 +b 15175 +v 7854 +k 9852 +' 93573 +x 5912 +j 2879 +q 103 +z 4782