logstash-input-twitter 2.2 NoMethod error with my tags but use_samples => true returns tweets and doesn't fill the log

[mobcdi]

Using the latest versions of ELK and the Logstash twitter input plugin I get the errors listed in the log file and my index isn't created in elastic but if I include use_samples=> true in my logstash config my index is created for the 1st time and tweets populated without filling the logstash.log at all.

I also edited my config file to exclude my twitter api consumer key but still included use_samples => true and that returned expected authentication error messages indicating to me that when the consumer key was listed the use_samples was able to authenticate to twitter using the key,secret,token and token secret I provide indicating its not an issue with my twitter app details and that valid twitter api keys are needed for the plugin to ingest the sample stream

I initially posted the issue to discuss but when I was able to confirm the use_samples worked and needed my twitter api to authenticate I decided to open this issue

Logstash log file after letting the service run for a few seconds.
logstashlog.txt
My logstash config file with API values removed
logstashconf.txt
Logstash.err file
logstasherr.txt

:exception=>Twitter::Error::Unauthorized, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/twitter-5.15.0/lib/twitter/streaming/response.rb:21:in `on_headers_complete'",

[mobcdi]

There are also entries in the log about undefined method filer for nil.NilClass.
Given my selected tags and accounts aren't massively popular at the moment could this be a case of the error handling missing Nil, none or no values being returned from the stream?

{:timestamp=>"2015-12-10T21:39:21.426000+0000", :message=>"undefined methodfilter' for nil:NilClass", :exception=>#<NoMethodError: undefined method filter' for nil:NilClass>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-twitter-2.2.0/lib/logstash/inputs/twitter.rb:128:in run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.0-java/lib/logstash/pipeline.rb:205:ininputworker'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.0-java/lib/logstash/pipeline.rb:198:in start_input'"], :options=>nil, :level=>:warn}`

[aneveu]

I'm facing the same issue and I know for sure it's not coming from my credentials as I'm using them in two other configuration files on the same environnement and it works perfectly.

The only differences with the last one is that I'm using follows instead of keywords as I want to retrieve the twitter feed from an account and not only from tags.

Here is the input part of my configuration file:
input { twitter { consumer_key => "xxx" consumer_secret => "xxx" oauth_token => "xxx" oauth_token_secret => "xxx" follows => [ "@elastic" ] full_tweet => true } }

And here is my stacktrace:

{:exception=>Twitter::Error::Unauthorized, :backtrace=>["/Users/audrey/Dev/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/twitter-5.15.0/lib/twitter/streaming/response.rb:21:inon_headers_complete'", "org/ruby_http_parser/RubyHttpParser.java:370:in <<'", "/Users/audrey/Dev/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/twitter-5.15.0/lib/twitter/streaming/response.rb:16:in<<'", "/Users/audrey/Dev/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-input-twitter-2.2.0/lib/logstash/inputs/twitter/patches.rb:31:in stream'", "/Users/audrey/Dev/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-input-twitter-2.2.0/lib/logstash/inputs/twitter/patches.rb:58:inrequest'", "/Users/audrey/Dev/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/twitter-5.15.0/lib/twitter/streaming/client.rb:37:in filter'", "/Users/audrey/Dev/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-input-twitter-2.2.0/lib/logstash/inputs/twitter.rb:128:inrun'", "/Users/audrey/Dev/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/pipeline.rb:206:in inputworker'", "/Users/audrey/Dev/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/pipeline.rb:199:instart_input'"], :options=>nil, :level=>:warn}

Perhaps I'm not using the follows attribute the right way? I've tried with or without the @ before the id but it makes no difference.

I've also tried to add use_samples => true as mentionned by @mobcdi and it works but indexed tweets are not relevant.

Thanks for helping :)

[awislowski]

I have same problem with logstasg 2.3.0 version.
only follows parameter for the plugin is not working.

[wathmal]

when you are using "follows" parameter you need to include twitter user's ID instead of user name or screen name.

you can use this site to grab relevant twitter id to username or twitter API.

follows => ["372841707"]
will do the fix for you.

[HawkieNorway]

What is the right syntax for a comma separated list?
follows => ["372841707","123456743"]
or
follows => ["372841707,123456743"]

[embercoat]

Hi,

I just ran into this problem aswell and while just throwing things at the wall I discovered that adding language to the configuration fixes the Unathorized error

languages => ["en"]

Devs might be able to figure out why but in the mean time, it should help a few.