Add delete support

[untergeek]

Add support to use @metadata to pass a delete through the event (to delete from Elasticsearch, ostensibly).

[jordansissel]

Twitter's normal-user keyword/user stream doesn't have a "delete".

I strongly suspect the firehose will issue deletes based on observations of deleted tweets propogating that deletion out to brokers I know who have firehose access, but I don't have access to it so I cannot confirm.

[jordansissel]

Hmm, I may be wrong there.

From https://dev.twitter.com/streaming/userstreams

Honor deletion messages. If a following deletes a tweet, you must remove the tweet from display and any application/server storage.

From https://dev.twitter.com/streaming/sitestreams

Do not expose or otherwise draw attention to status deletions or unfollows to users, other than to update displays.

I've had Logstash watching https://dev.twitter.com/streaming/reference/post/statuses/filter (keyword streams) for a few months and my Elasticsearch mapping has no mentions of deleted tweets.

+1 on supporting deletes.

[untergeek]

It's in the river here: https://github.com/elasticsearch/elasticsearch-river-twitter/blob/master/src/main/java/org/elasticsearch/river/twitter/TwitterRiver.java#L765-L769