Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Because you said you were interested #46

Open
wants to merge 53 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
553acfb
Try to find out what's happening
mdpye Jul 15, 2014
4401f08
Update spec to be compatible with redis 2.8
mdpye Oct 16, 2014
99d9b8a
Fix reconnect for exceptions
mdpye Oct 16, 2014
fb06210
More correct error detection
mdpye Oct 16, 2014
2f10baf
Avoid an unnecessary closure
mdpye Oct 16, 2014
515492f
Merge pull request #2 from pusher/reconnect_exceptions
mdpye Oct 16, 2014
b7671ae
Dirt simple pre-processing for lua scripts
mdpye Dec 2, 2014
d23793d
Readme notes on lua includes
mdpye Dec 3, 2014
746da43
Consider connected only after selecting database
pl Dec 10, 2014
39c6609
Fix specs broken by last commit
pl Dec 11, 2014
ca74978
Add a spec for connecting while redis is busy
pl Dec 11, 2014
988a5ab
Improve recently updated live protocol specs
pl Dec 12, 2014
fa39355
Merge branch 'lua_includes'
pl Dec 15, 2014
6febde6
Merge branch 'ensure_db_selected'
pl Dec 15, 2014
46ab9da
Expose LUA script text and sha
mdpye Feb 3, 2015
6c5ee84
Add a method to ensure a script is loaded
mdpye Feb 3, 2015
aa19203
Merge pull request #5 from pusher/expose_script_and_sha
mdpye Feb 4, 2015
ccda64c
Take the optimism out of the tests
mdpye Feb 17, 2015
f29f191
Update tests to match actual behaviour and interface
mdpye Feb 17, 2015
b7c31f0
Enable travis
mdpye Feb 17, 2015
2e41c5d
Notify hipchat on travis build result
mdpye Feb 17, 2015
49304a6
Streamline travis notifications
mdpye Feb 17, 2015
3828688
Rewrite to improve connection handling
mdpye Feb 11, 2015
4c9d779
Clean up logic
mdpye Mar 12, 2015
6517da2
Prevent the leaking of internal collections via implicit return
mdpye Mar 17, 2015
65182b4
Resubscribe in chunks of 5000 channels
mdpye Mar 18, 2015
412b86d
Don't check if callback list is nil
mdpye Mar 23, 2015
ed2c18a
Stop the `Hash.new` magic
danielwaterworth Mar 25, 2015
3d1c2f1
Merge branch 'bugfix_subscriptions'
danielwaterworth Mar 25, 2015
abd1e12
Reset reconnection attempt when explicitly asked to reconnect
mdpye Jul 29, 2015
c0bd0c6
Only unsubscribe a proc if there is an existing channel subscription
vivangkumar Oct 1, 2015
beffde3
Spec - unsubscribing proc for an unexisting channel subscription
vivangkumar Oct 1, 2015
d364c43
Merge pull request #9 from pusher/unsubscribe-proc-only-when-subscrip…
mdpye Oct 2, 2015
0ca739b
Change resubscribe batch size to 1000
vivangkumar Oct 7, 2015
55aa17e
Merge pull request #10 from pusher/reduce-batch-resubscribe-size
mdpye Oct 7, 2015
b8f18f8
Don't shadow variables
mdpye Feb 17, 2016
b8bbffd
make reconnect_attempts reconfigurable
May 16, 2019
ab3f97d
add ping on establishing connection v1
Jul 10, 2019
db2d1a7
ping for pubsub
Aug 7, 2019
74650fb
implement timeout to the df function
Aug 7, 2019
295e34f
add timeout var
Aug 8, 2019
807a8dc
fix tests
Aug 12, 2019
27038eb
add default_response_timeout
Aug 13, 2019
2bfff59
Merge pull request #12 from pusher/ping-init
Aug 21, 2019
0df64ac
Pin rake to a version which is supported with RSpec 2.x
mdpye May 19, 2020
58d520a
Fix optional args using named args
mdpye May 19, 2020
78af89a
Fixnum is deprecated in Ruby 2.4+
mdpye May 19, 2020
447011b
Test with recent Ruby versions
mdpye May 19, 2020
c7aa5a5
Upgrade hiredis
mdpye May 19, 2020
254335a
Remove hipchat notification from travis config
mdpye Jun 8, 2020
2a4acc3
Provide a redis instance to test again in travis
mdpye Jun 8, 2020
738d39d
Allow 200ms leeway testing lock timeout
mdpye Jun 8, 2020
9bdf3de
Merge pull request #13 from pusher/ruby2.6
mdpye Jun 8, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ Gemfile.lock
pkg/*
.DS_Store
*.swp
vendor
1 change: 1 addition & 0 deletions .ruby-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
2.6
7 changes: 7 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
language: ruby
rvm:
- 2.4
- 2.5
- 2.6
services:
- redis-server
32 changes: 31 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,35 @@
# Changelog

## 1.0.0 (2015-02-25)

[BUGFIX] Replies becoming out of sync after voluntary reconnect.

[CHANGE] Clients are now configured through the use of URIs only (not individual host, port, db params)

The previous interface was inconsistently applied, for example clients could be constructed using
individual params, but only re-configured using a uri.

[CHANGE] Client's public interface simplified considerably wrt connect / reconnect / reconfigure.

Use `connect` to connect, `reconnect` to force reconnection and `reconnect(uri)` to force reconnection
to a different server.

[CHANGE] Pubsub interface no longer returns deferrables for subscribe methods.

Rationale: Redis pubsub subscriptions can only ever be provided on a "best effort" basis where
reconnection-handling is provided - while reconnection takes place, messages will no be received.
If it is important to be aware of these periods, binding to a combination of :disconnected and
:subscribe on the pubsub client will allow one to deduce when the subscription is active.

The deferrable interface was also awkward in terms of issuing subscribe commands for multiple
channels at once.

[NEW] Inactivity timeouts:

Trigger activity on idle connections and force reconnect if no response is found.
Particularly beneficial for pubsub connections where there may be no activity from the server for
extended periods and this is otherwise indistinguishable from a hung TCP connection.

## 0.2.1 (2013-04-22)

[NEW] Support for connecting to redis on a unix socket.
Expand All @@ -15,7 +45,7 @@
* Clients now emit the following events: connected, reconnected, disconnected, reconnect_failed (passes the number of consecutive failures)
* Client is considered failed after 4 consecutive failures
* Fails all queued commands when client failed
* Can now reconfiure and reconnect an exising client
* Can now reconfigure and reconnect an exising client
* Reconnect timeout can be configured (defaults to 0.5s)

[NEW] Added `EM::Hiredis::Lock` and `EM::Hiredis::PersistentLock`
Expand Down
24 changes: 19 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,11 +79,6 @@ This configures a `PING` command to be sent if 5 seconds elapse without receivin

This configuration is per client, you may choose different value for clients with different expected traffic patterns, or activate it on some and not at all on others.

### PING and Pubsub

Because the Redis Pubsub protocol limits the set of valid commands on a connection once it is in "Pubsub" mode, `PING` is not supported in this case (though it may be in future, see https://github.com/antirez/redis/issues/420). In order to create some valid request-response traffic on the connection, a Pubsub connection will issue `SUBSCRIBE "__em-hiredis-ping"`, followed by a corresponding `UNSUBSCRIBE` immediately on success of the subscribe.
While less than ideal, this is the case where an application layer inactivity check is most valuable, and so the trade off is reasonable until `PING` is supported correctly on Pubsub connections.

### Close to the metal

Basically just bind to `:message` and `:pmessage` events:
Expand Down Expand Up @@ -135,6 +130,25 @@ If you pass a block to `subscribe` or `psubscribe`, the passed block will be cal

It's possible to subscribe to the same channel multiple time and just unsubscribe a single callback using `unsubscribe_proc` or `punsubscribe_proc`.

## Lua scripting

When loading scripts from a directory with `EventMachine::Hiredis::Client.load_scripts_from`, the scripts loaded undergo very simple preprocessing, replacing any occurrence of an "include directive" literally with the contents of the referenced file before sending the script to redis.

```
-- #include file/name.lua
```

The filename is expressed relative to the directory of scripts being loaded.

### Recommendations for library code

The implementation is extremely simple, so some sensible recommendations are:

- Put library code in a subdirectory, or using an extension other than `.lua`, to prevent library scripts being loaded as their own commands.
- Declare only "classes" into the top level scope in library code, and preferably only one per script with the classname matching the script name, to minimise possible naming collisions
- Attach free function implementations to members of a declared class as a namespace.
- Remember while developing that line numbers reported back from redis will be offset

## Developing

Hacking on em-hiredis is pretty simple, make sure you have Bundler installed:
Expand Down
8 changes: 4 additions & 4 deletions em-hiredis.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@ Gem::Specification.new do |s|
s.name = "em-hiredis"
s.version = EventMachine::Hiredis::VERSION
s.platform = Gem::Platform::RUBY
s.authors = ["Martyn Loughran"]
s.email = ["[email protected]"]
s.authors = ["Martyn Loughran", "Mike Pye"]
s.email = ["[email protected]", "[email protected]"]
s.homepage = "http://github.com/mloughran/em-hiredis"
s.summary = %q{Eventmachine redis client}
s.description = %q{Eventmachine redis client using hiredis native parser}

s.add_dependency 'hiredis', '~> 0.4.0'
s.add_dependency 'hiredis', '~> 0.6.0'

s.add_development_dependency 'em-spec', '~> 0.2.5'
s.add_development_dependency 'rspec', '~> 2.6.0'
s.add_development_dependency 'rake'
s.add_development_dependency 'rake', '~> 10'

s.rubyforge_project = "em-hiredis"

Expand Down
25 changes: 14 additions & 11 deletions lib/em-hiredis.rb
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,9 @@ class << self
end
self.reconnect_timeout = 0.5

def self.setup(uri = nil)
def self.setup(uri = nil, activity_timeout = nil, response_timeout = nil)
uri = uri || ENV["REDIS_URL"] || "redis://127.0.0.1:6379/0"
client = Client.new
client.configure(uri)
client
Client.new(uri, activity_timeout, response_timeout)
end

# Connects to redis and returns a client instance
Expand All @@ -34,10 +32,9 @@ def self.setup(uri = nil)
# Unix socket uris are supported, e.g. unix:///tmp/redis.sock, however
# it's not possible to set the db or password - use initialize instead in
# this case
def self.connect(uri = nil)
client = setup(uri)
def self.connect(uri = nil, activity_timeout = nil, response_timeout = nil)
client = setup(uri, activity_timeout, response_timeout)
client.connect
client
end

def self.logger=(logger)
Expand All @@ -58,8 +55,14 @@ def self.logger
end
end

require 'em-hiredis/event_emitter'
require 'em-hiredis/connection'
require 'em-hiredis/base_client'
require 'em-hiredis/client'
require 'digest/sha1'
require 'hiredis/reader'
require 'em-hiredis/support/event_emitter'
require 'em-hiredis/support/cancellable_deferrable'
require 'em-hiredis/support/inactivity_checker'
require 'em-hiredis/support/state_machine'
require 'em-hiredis/connection_manager'
require 'em-hiredis/redis_connection'
require 'em-hiredis/pubsub_connection'
require 'em-hiredis/redis_client'
require 'em-hiredis/pubsub_client'
Loading