Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test migration to actionmodel7 + rails7 + couchbase-ruby-client #33

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
11af7fc
test migration to actionmodel7 + rails7 + couchbase-ruby-client
CedricCouton Jun 17, 2022
ba029bf
Merge branch 'github-actions' into activemodel7-couchbase-ruby-client
simkim Jun 22, 2022
5758961
Fix base_spec.rb
simkim Jun 22, 2022
6267414
remove puts
simkim Jun 22, 2022
7c745da
Fix pesistennce_spec
simkim Jun 22, 2022
a724cd5
Fix n1ql_spec.rb
simkim Jun 23, 2022
b4f6638
Fix index_spec.rb
simkim Jun 23, 2022
c849592
Missing files
simkim Jun 23, 2022
4d0100d
Fix associations_spec.rb
simkim Jun 23, 2022
74f40b1
remove rails folder
simkim Jun 23, 2022
1f07d68
Fix has_many_spec.rb
simkim Jun 23, 2022
797c018
Add coverage report
simkim Jun 24, 2022
9308902
Try to update to ubuntu 20.04 and couchbase 7.1
simkim Jun 24, 2022
73c320e
without old ssl
simkim Jun 24, 2022
8575350
regroup apt operations
simkim Jun 24, 2022
076c04d
Remove breaking refs to MTlibcouchbase
simkim Jun 24, 2022
6d44b85
handle ci auth
simkim Jun 24, 2022
6627060
Fix flaky
simkim Jun 24, 2022
cf5c88c
Use default bucket
simkim Jun 24, 2022
5bfb1b0
Ignore more errors on cleanup
simkim Jun 24, 2022
a82197e
more rescue
simkim Jun 24, 2022
75b1e2a
Fix for ruby 2.6
simkim Jun 24, 2022
6c42d4d
Remove puts except in railtie
simkim Jun 24, 2022
8175236
Fix typo
simkim Jun 24, 2022
60e2866
Handle empty find
simkim Jun 24, 2022
ef3ef32
100% coverage on utilities/enum
simkim Jun 24, 2022
8add1c0
More coverage on collection proxy
simkim Jun 27, 2022
fa065bd
remove MTLibcouchbase link + add temp env var for couchbase connection
CedricCouton Jul 1, 2022
2f254eb
CI Fix, use env vars to setup test base and tests
simkim Jul 7, 2022
2c52505
Add env vars in CI
simkim Jul 7, 2022
58a2dd9
preserve env in sudo
simkim Jul 7, 2022
a3e10f9
matrix couchbase version
simkim Jul 7, 2022
735b293
reduce number of jobs
simkim Jul 7, 2022
3f7de4c
fix couchbase version
simkim Jul 7, 2022
6f9c767
Add escaping
simkim Jul 7, 2022
b0cc946
remove puts
DamienVoreiter Aug 18, 2022
c7f87d8
reinsert query usage and rename it to query_fn
CedricCouton Aug 18, 2022
3788688
query_fn instead of query
CedricCouton Aug 19, 2022
ff80f78
Update lib/couchbase-orm/railtie.rb
DamienVoreiter Sep 8, 2022
696e332
Merge branch 'Mapotempo:master' into activemodel7-couchbase-ruby-client
DamienVoreiter Sep 8, 2022
de0ef2e
Update lib/couchbase-orm/proxies/bucket_proxy.rb
DamienVoreiter Sep 8, 2022
eed7f40
unhardcode host config & use yml file
DamienVoreiter Sep 8, 2022
922dbb8
raise CouchbaseOrm::Error
DamienVoreiter Sep 8, 2022
b5701f6
try fix ci
CedricCouton Sep 8, 2022
684f1f9
use ip not localhost
CedricCouton Sep 8, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 18 additions & 12 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,30 +10,36 @@ jobs:
test:
strategy:
matrix:
gemfile: [ '7.0.0', '5.1.7' ]
ruby: [ '3.0', '2.7', '2.6']
exclude:
include:
- ruby: '3.0'
gemfile: '5.1.7'
gemfile: '7.0.0'
couchbase: '6.6.5'
- ruby: '3.0'
gemfile: '7.0.0'
couchbase: '7.1.0'
- ruby: '2.7'
gemfile: '5.1.7'
- ruby: '2.6'
gemfile: '7.0.0'
couchbase: '7.1.0'
- ruby: '2.6'
gemfile: '5.1.7'
couchbase: '7.1.0'
fail-fast: false
runs-on: ubuntu-18.04
name: ${{ matrix.ruby }} ${{ matrix.database }} rails-${{ matrix.gemfile }}
runs-on: ubuntu-20.04
name: ${{ matrix.ruby }} rails-${{ matrix.gemfile }} couchbase-${{ matrix.couchbase }}
steps:
- uses: actions/checkout@v2
- run: sudo apt-get update && sudo apt-get install libevent-dev
- run: sudo apt-get update && sudo apt-get install libevent-dev libev-dev python-httplib2
- uses: ruby/setup-ruby@v1
with:
ruby-version: ${{ matrix.ruby }}
bundler-cache: true
- run: sudo ./ci/run_couchbase.sh
- run: sudo ./ci/run_couchbase.sh $COUCHBASE_VERSION $COUCHBASE_BUCKET $COUCHBASE_USER $COUCHBASE_PASSWORD
- run: bundle exec rspec
env:
ACTIVE_MODEL_VERSION: ${{ matrix.gemfile }}
BUNDLE_JOBS: 4
BUNDLE_PATH: vendor/bundle
TRAVIS_TEST: true
RAILS_ENV: test
COUCHBASE_BUCKET: default
COUCHBASE_USER: tester
COUCHBASE_PASSWORD: password123
COUCHBASE_VERSION: ${{ matrix.couchbase }}
24 changes: 19 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,9 @@ To generate config you can use `rails generate couchbase_orm:config`:

It will generate this `config/couchbase.yml` for you:

```yaml
common: &common
hosts: localhost
connection_string: couchbase://localhost
username: dev_user
password: dev_password

Expand All @@ -26,10 +27,23 @@ It will generate this `config/couchbase.yml` for you:

# set these environment variables on your production server
production:
hosts: <%= ENV['COUCHBASE_HOST'] || ENV['COUCHBASE_HOSTS'] %>
bucket: <%= ENV['COUCHBASE_BUCKET'] %>
username: <%= ENV['COUCHBASE_USER'] %>
password: <%= ENV['COUCHBASE_PASSWORD'] %>
connection_string: <%= ENV['COUCHBASE_CONNECTION_STRING'] %>
bucket: <%= ENV['COUCHBASE_BUCKET'] %>
username: <%= ENV['COUCHBASE_USER'] %>
password: <%= ENV['COUCHBASE_PASSWORD'] %>
```

## Setup without Rails
If you are not using Rails, you can configure couchbase-orm with an initializer:
```ruby
# config/initializers/couchbase_orm.rb
CouchbaseOrm::Connection.config = {
connection_string: "couchbase://localhost"
username: "dev_user"
password: "dev_password"
bucket: "dev_bucket"
}
```

Views are generated on application load if they don't exist or mismatch.
This works fine in production however by default in development models are lazy loaded.
Expand Down
18 changes: 12 additions & 6 deletions ci/run_couchbase.sh
Original file line number Diff line number Diff line change
@@ -1,14 +1,20 @@
set -x
set -e
apt-get install libev-dev python-httplib2 libssl1.0.0
wget https://packages.couchbase.com/releases/5.1.0/couchbase-server-enterprise_5.1.0-ubuntu14.04_amd64.deb
dpkg -i couchbase-server-enterprise_5.1.0-ubuntu14.04_amd64.deb

VERSION=$1
BUCKET=$2
USER=$3
PASSWORD=$4


wget https://packages.couchbase.com/releases/$VERSION/couchbase-server-enterprise_$VERSION-ubuntu20.04_amd64.deb
dpkg -i couchbase-server-enterprise_$VERSION-ubuntu20.04_amd64.deb
sleep 8
sudo service couchbase-server status
/opt/couchbase/bin/couchbase-cli cluster-init -c 127.0.0.1:8091 --cluster-username=admin --cluster-password=password --cluster-ramsize=320 --cluster-index-ramsize=256 --cluster-fts-ramsize=256 --services=data,index,query,fts
sleep 5
/opt/couchbase/bin/couchbase-cli server-info -c 127.0.0.1:8091 -u admin -p password
/opt/couchbase/bin/couchbase-cli bucket-create -c 127.0.0.1:8091 -u admin -p password --bucket=default --bucket-type=couchbase --bucket-ramsize=160 --bucket-replica=0 --wait
/opt/couchbase/bin/couchbase-cli bucket-create -c 127.0.0.1:8091 -u admin -p password --bucket=$BUCKET --bucket-type=couchbase --bucket-ramsize=160 --bucket-replica=0 --wait
sleep 1
/opt/couchbase/bin/couchbase-cli user-manage -c 127.0.0.1:8091 -u admin -p password --set --rbac-username tester --rbac-password password123 --rbac-name "Auto Tester" --roles admin --auth-domain local
curl http://admin:password@localhost:8093/query/service -d 'statement=CREATE INDEX `default_type` ON `default`(`type`)'
/opt/couchbase/bin/couchbase-cli user-manage -c 127.0.0.1:8091 -u admin -p password --set --rbac-username $USER --rbac-password $PASSWORD --rbac-name "Auto Tester" --roles admin --auth-domain local
curl http://admin:password@localhost:8093/query/service -d "statement=CREATE INDEX \`default_type\` ON \`$BUCKET\`(\`type\`)"
5 changes: 3 additions & 2 deletions couchbase-orm.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,15 @@ Gem::Specification.new do |gem|
gem.required_ruby_version = '>= 2.1.0'
gem.require_paths = ["lib"]

gem.add_runtime_dependency 'mt-libcouchbase', '~> 1.2'
gem.add_runtime_dependency 'activemodel', ENV["ACTIVE_MODEL_VERSION"] || '>= 5.0'
gem.add_runtime_dependency 'couchbase'
gem.add_runtime_dependency 'radix', '~> 2.2' # converting numbers to and from any base

gem.add_development_dependency 'rake', '~> 12.2'
gem.add_development_dependency 'rspec', '~> 3.7'
gem.add_development_dependency 'yard', '~> 0.9'
gem.add_development_dependency 'minitest', '~> 5.10'
gem.add_development_dependency 'pry'
gem.add_development_dependency 'simplecov'

gem.files = `git ls-files`.split("\n")
gem.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
Expand Down
53 changes: 29 additions & 24 deletions lib/couchbase-orm.rb
Original file line number Diff line number Diff line change
@@ -1,46 +1,51 @@
# frozen_string_literal: true, encoding: ASCII-8BIT

require 'mt-libcouchbase'
MTLibcouchbase.autoload(:QueryN1QL, 'ext/query_n1ql')

module CouchbaseOrm
autoload :Error, 'couchbase-orm/error'
autoload :Connection, 'couchbase-orm/connection'
autoload :IdGenerator, 'couchbase-orm/id_generator'
autoload :Base, 'couchbase-orm/base'
autoload :HasMany, 'couchbase-orm/utilities/has_many'

def self.logger
@@logger ||= defined?(Rails) ? Rails.logger : Logger.new(STDOUT)
end

def self.logger=(logger)
@@logger = logger
end

def self.try_load(id)
result = nil
was_array = id.is_a?(Array)
if was_array && id.length == 1
id = id.first
end
result = id.respond_to?(:cas) ? id : CouchbaseOrm::Base.bucket.get(id, quiet: true, extended: true)
if was_array
result = Array.wrap(result)
end
if result && result.is_a?(Array)
return result.map { |r| self.try_load(r) }.compact
query_id = id.first
else
query_id = id
end

if result && result.value.is_a?(Hash) && result.value[:type]
ddoc = result.value[:type]
::CouchbaseOrm::Base.descendants.each do |model|
if model.design_document == ddoc
return model.new(result)
end
end
result = query_id.is_a?(Array) ? CouchbaseOrm::Base.bucket.default_collection.get_multi(query_id) : CouchbaseOrm::Base.bucket.default_collection.get(query_id)

result = Array.wrap(result) if was_array

if result&.is_a?(Array)
return result.zip(id).map { |r, id| try_load_create_model(r, id) }.compact
end
nil
end

def self.logger
@@logger ||= defined?(Rails) ? Rails.logger : Logger.new(STDOUT)
return try_load_create_model(result, id)
end

def self.logger=(logger)
@@logger = logger
private

def self.try_load_create_model(result, id)
ddoc = result&.content["type"]
return nil unless ddoc
::CouchbaseOrm::Base.descendants.each do |model|
if model.design_document == ddoc
return model.new(result, id: id)
end
end
nil
end
end

Expand Down
11 changes: 7 additions & 4 deletions lib/couchbase-orm/associations.rb
Original file line number Diff line number Diff line change
Expand Up @@ -75,12 +75,15 @@ def has_and_belongs_to_many(name, **options)
# Define reader
define_method(name) do
return instance_variable_get(instance_var) if instance_variable_defined?(instance_var)
ref_value = self.send(ref)
ref_value = nil if ref_value.respond_to?(:empty?) && ref_value.empty?

val = if options[:polymorphic]
::CouchbaseOrm.try_load(self.send(ref))
::CouchbaseOrm.try_load(ref_value) if ref_value
else
assoc.constantize.find(self.send(ref), quiet: true)
assoc.constantize.find(ref_value) if ref_value
end
val = Array.wrap(val)
val = Array.wrap(val || [])
instance_variable_set(instance_var, val)
val
end
Expand Down Expand Up @@ -181,7 +184,7 @@ def destroy_associations!
when :destroy, :delete
if model.respond_to?(:stream)
model.stream { |mod| mod.__send__(dependent) }
elsif model.is_a?(Array)
elsif model.is_a?(Array) || model.is_a?(CouchbaseOrm::ResultsProxy)
model.each { |m| m.__send__(dependent) }
else
model.__send__(dependent)
Expand Down
53 changes: 30 additions & 23 deletions lib/couchbase-orm/base.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,14 @@

require 'active_model'
require 'active_support/hash_with_indifferent_access'
require 'couchbase'
require 'couchbase-orm/error'
require 'couchbase-orm/views'
require 'couchbase-orm/n1ql'
require 'couchbase-orm/persistence'
require 'couchbase-orm/associations'
require 'couchbase-orm/proxies/bucket_proxy'
require 'couchbase-orm/proxies/collection_proxy'
require 'couchbase-orm/utilities/join'
require 'couchbase-orm/utilities/enum'
require 'couchbase-orm/utilities/index'
Expand Down Expand Up @@ -55,6 +57,14 @@ def bucket
@bucket ||= BucketProxy.new(Connection.bucket)
end

def cluster
Connection.cluster
end

def collection
CollectionProxy.new(bucket.default_collection)
end

def uuid_generator
@uuid_generator ||= IdGenerator
end
Expand Down Expand Up @@ -90,27 +100,20 @@ def attributes
@attributes ||= {}
end

def find(*ids, **options)
options[:extended] = true
options[:quiet] ||= false
def find(*ids, quiet: false)
CouchbaseOrm.logger.debug { "Base.find(l##{ids.length}) #{ids}" }

ids = ids.flatten.select { |id| id.present? }
if ids.empty?
return nil if options[:quiet]
raise MTLibcouchbase::Error::EmptyKey, 'no id(s) provided'
raise CouchbaseOrm::Error::EmptyNotAllowed, 'no id(s) provided'
end

CouchbaseOrm.logger.debug "Data - Get #{ids}"
record = bucket.get(*ids, **options)
records = record.is_a?(Array) ? record : [record]
records.map! { |record|
if record
self.new(record)
else
false
end
records = quiet ? collection.get_multi(ids) : collection.get_multi!(ids)
CouchbaseOrm.logger.debug { "Base.find found(#{records})" }
records = records.zip(ids).map { |record, id|
self.new(record, id: id) if record
}
records.select! { |rec| rec }
records.compact!
ids.length > 1 ? records : records[0]
end

Expand All @@ -121,12 +124,13 @@ def find_by_id(*ids, **options)
alias_method :[], :find_by_id

def exists?(id)
CouchbaseOrm.logger.debug "Data - Get #{id}"
!bucket.get(id, quiet: true).nil?
CouchbaseOrm.logger.debug "Data - Exists? #{id}"
collection.exists(id).exists
end
alias_method :has_key?, :exists?
end

class MismatchTypeError < RuntimeError; end

# Add support for libcouchbase response objects
def initialize(model = nil, ignore_doc_type: false, **attributes)
Expand All @@ -145,25 +149,28 @@ def initialize(model = nil, ignore_doc_type: false, **attributes)

if model
case model
when ::MTLibcouchbase::Response
doc = model.value || raise('empty response provided')
type = doc.delete(:type)
when Couchbase::Collection::GetResult
CouchbaseOrm.logger.debug "Initialize with Couchbase::Collection::GetResult"
doc = model.content || raise('empty response provided')
type = doc.delete('type')
doc.delete(:id)

if type && !ignore_doc_type && type.to_s != self.class.design_document
raise "document type mismatch, #{type} != #{self.class.design_document}"
raise CouchbaseOrm::Error::TypeMismatchError.new("document type mismatch, #{type} != #{self.class.design_document}", self)
end

@__metadata__.key = model.key
@__metadata__.key = attributes[:id]
@__metadata__.cas = model.cas

# This ensures that defaults are applied
@__attributes__.merge! doc
clear_changes_information
when CouchbaseOrm::Base
CouchbaseOrm.logger.debug "Initialize with CouchbaseOrm::Base"

clear_changes_information
attributes = model.attributes
attributes.delete(:id)
attributes.delete('type')
super(attributes)
else
clear_changes_information
Expand Down
36 changes: 28 additions & 8 deletions lib/couchbase-orm/connection.rb
Original file line number Diff line number Diff line change
@@ -1,16 +1,36 @@
# frozen_string_literal: true, encoding: ASCII-8BIT

require 'mt-libcouchbase'
require 'couchbase'

module CouchbaseOrm
class Connection
@options = {}
class << self
attr_accessor :options
@@config = nil
def self.config
@@config || {
:connection_string => "couchbase://#{ENV['COUCHBASE_HOST'] || '127.0.0.1'}",
:username => ENV['COUCHBASE_USER'],
:password => ENV['COUCHBASE_PASSWORD'],
:bucket => ENV['COUCHBASE_BUCKET']
}
end

def self.config=(config)
@@config = config
CedricCouton marked this conversation as resolved.
Show resolved Hide resolved
end

def self.cluster
@cluster ||= begin
giallon marked this conversation as resolved.
Show resolved Hide resolved
cb_config = Couchbase::Configuration.new
cb_config.connection_string = config[:connection_string] || raise(CouchbaseOrm::Error, 'Missing CouchbaseOrm connection string')
cb_config.username = config[:username] || raise(CouchbaseOrm::Error, 'Missing CouchbaseOrm username')
cb_config.password = config[:password] || raise(CouchbaseOrm::Error, 'Missing CouchbaseOrm password')
Couchbase::Cluster.connect(cb_config)
end
end

def self.bucket
@bucket ||= ::MTLibcouchbase::Bucket.new(**@options)
@bucket ||= begin
bucket_name = config[:bucket] || raise(CouchbaseOrm::Error, 'Missing CouchbaseOrm bucket name')
cluster.bucket(bucket_name)
end
end
end
end
end
Loading