fix logic in redis error checking; updates to use newer python base i…

…mage (#22)

pubsub/README.md

@@ -218,7 +218,7 @@ Then, set the Twitter authentication information to the values you noted when se
 
 Edit `bigquery-controller.yaml`.  Set your `PUBSUB_TOPIC`, and set your `PROJECT_ID`, `BQ_DATASET`, and `BQ_TABLE` information.  
 
-(If you optionally built your own docker image as described in the Appendix, also replace the image string `gcr.io/google-samples/pubsub-bq-pipe:v4` with the name of the container image that you have built and pushed.)
+(If you optionally built your own docker image as described in the Appendix, also replace the image string `gcr.io/google-samples/pubsub-bq-pipe:v5` with the name of the container image that you have built and pushed.)
 
 ### Deploy your app
 
@@ -410,7 +410,7 @@ If you don't see evidence that your container has ever started sucessfully, doub
 This step is optional.
 
 The example app uses a [Docker](https://www.docker.com/) image that runs the app's python scripts. 
-If you like, you can just use the prebuilt docker image, `gcr.io/google-samples/pubsub-bq-pipe:v4`, for your app. This is the image used by default in the `bigquery-controller.yaml` and `twitter-stream.yaml` files.
+If you like, you can just use the prebuilt docker image, `gcr.io/google-samples/pubsub-bq-pipe:v5`, for your app. This is the image used by default in the `bigquery-controller.yaml` and `twitter-stream.yaml` files.
 
 Follow the instructions below if you'd like to add customization and use your own image instead.
 For this, you'll either need [Docker](https://docs.docker.com/installation/#installation) installed locally, or you can run Docker in the Cloud Shell. (You could also alternately use [Google Cloud Container Builder](https://cloud.google.com/container-builder/docs/).)

pubsub/bigquery-controller.yaml

@@ -13,7 +13,7 @@ spec:
     spec:
       containers:
       - name: bigquery
-        image: gcr.io/google-samples/pubsub-bq-pipe:v4
+        image: gcr.io/google-samples/pubsub-bq-pipe:v5
         env:
         - name: PROCESSINGSCRIPT
           value: pubsub-to-bigquery

pubsub/pubsub-pipe-image/Dockerfile

@@ -1,4 +1,4 @@
-FROM google/python
+FROM python:2
 
 RUN pip install --upgrade pip
 RUN pip install tweepy

pubsub/twitter-stream.yaml

@@ -13,7 +13,7 @@ spec:
     spec:
       containers:
       - name: twitter-to-pubsub
-        image: gcr.io/google-samples/pubsub-bq-pipe:v4
+        image: gcr.io/google-samples/pubsub-bq-pipe:v5
         env:
         - name: PROCESSINGSCRIPT
           value: twitter-to-pubsub

redis/README.md

@@ -214,7 +214,7 @@ Set the Twitter authentication information to the values you noted when setting
 Edit `bigquery-controller.yaml`.  Set your `PROJECT_ID`, `BQ_DATASET`, and `BQ_TABLE` information.
 
 (If you optionally built your own docker image as described in the Appendix, also replace the image string
-`gcr.io/google-samples/redis-bq-pipe:v4` with the name of the container image that you have built and pushed.)
+`gcr.io/google-samples/redis-bq-pipe:v5` with the name of the container image that you have built and pushed.)
 
 ### Deploy your app
 
@@ -424,7 +424,7 @@ If you don't see evidence that your container has ever started sucessfully, doub
 This step is optional.
 
 The example app uses a [Docker](https://www.docker.com/) image that runs the app's python scripts. 
-If you like, you can just use the prebuilt docker image, `gcr.io/google-samples/redis-bq-pipe:v4`, for your app. This is the image used by default in the `bigquery-controller.yaml` and `twitter-stream.yaml` files.
+If you like, you can just use the prebuilt docker image, `gcr.io/google-samples/redis-bq-pipe:v5`, for your app. This is the image used by default in the `bigquery-controller.yaml` and `twitter-stream.yaml` files.
 
 Follow the instructions below if you'd like to add customization and use your own image instead.
 For this, you'll either need [Docker](https://docs.docker.com/installation/#installation) installed locally, or you can run Docker in the Cloud Shell. (You could also alternately use [Google Cloud Container Builder](https://cloud.google.com/container-builder/docs/).)
@@ -451,7 +451,7 @@ gcloud docker -- push gcr.io/<your-project-name>/redis-bq-pipe:v1
 ```
 
 Finally, edit `twitter-stream.yaml` and `bigquery-controller.yaml`, and
-replace `gcr.io/google-samples/redis-bq-pipe:v4` with the name of your image.
+replace `gcr.io/google-samples/redis-bq-pipe:v5` with the name of your image.
 
 
 

redis/bigquery-controller.yaml

@@ -13,7 +13,7 @@ spec:
     spec:
       containers:
       - name: bigquery
-        image: gcr.io/google-samples/redis-bq-pipe:v4
+        image: gcr.io/google-samples/redis-bq-pipe:v5
         env:
         - name: PROCESSINGSCRIPT
           value: redis-to-bigquery

redis/redis-pipe-image/Dockerfile

@@ -1,4 +1,4 @@
-FROM google/python
+FROM python:2
 
 RUN pip install --upgrade pip
 RUN pip install tweepy

redis/redis-pipe-image/redis-to-bigquery.py

@@ -65,8 +65,9 @@ def write_to_bq(bigquery):
                 tweet = json.loads(res[1])
             except Exception, e:
                 print e
+                redis_errors += 1
                 if redis_errors > allowed_redis_errors:
-                    print "Too many redis errors: exiting."
+                    print "Too many redis-related errors: exiting."
                     return
                 continue
             # First do some massaging of the raw data

redis/twitter-stream.yaml

@@ -13,7 +13,7 @@ spec:
     spec:
       containers:
       - name: twitter-to-redis
-        image: gcr.io/google-samples/redis-bq-pipe:v4
+        image: gcr.io/google-samples/redis-bq-pipe:v5
         env:
         - name: PROCESSINGSCRIPT
           value: twitter-to-redis