Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test migration to actionmodel7 + rails7 + couchbase-ruby-client #33

Merged
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
11af7fc
test migration to actionmodel7 + rails7 + couchbase-ruby-client
CedricCouton Jun 17, 2022
ba029bf
Merge branch 'github-actions' into activemodel7-couchbase-ruby-client
simkim Jun 22, 2022
5758961
Fix base_spec.rb
simkim Jun 22, 2022
6267414
remove puts
simkim Jun 22, 2022
7c745da
Fix pesistennce_spec
simkim Jun 22, 2022
a724cd5
Fix n1ql_spec.rb
simkim Jun 23, 2022
b4f6638
Fix index_spec.rb
simkim Jun 23, 2022
c849592
Missing files
simkim Jun 23, 2022
4d0100d
Fix associations_spec.rb
simkim Jun 23, 2022
74f40b1
remove rails folder
simkim Jun 23, 2022
1f07d68
Fix has_many_spec.rb
simkim Jun 23, 2022
797c018
Add coverage report
simkim Jun 24, 2022
9308902
Try to update to ubuntu 20.04 and couchbase 7.1
simkim Jun 24, 2022
73c320e
without old ssl
simkim Jun 24, 2022
8575350
regroup apt operations
simkim Jun 24, 2022
076c04d
Remove breaking refs to MTlibcouchbase
simkim Jun 24, 2022
6d44b85
handle ci auth
simkim Jun 24, 2022
6627060
Fix flaky
simkim Jun 24, 2022
cf5c88c
Use default bucket
simkim Jun 24, 2022
5bfb1b0
Ignore more errors on cleanup
simkim Jun 24, 2022
a82197e
more rescue
simkim Jun 24, 2022
75b1e2a
Fix for ruby 2.6
simkim Jun 24, 2022
6c42d4d
Remove puts except in railtie
simkim Jun 24, 2022
8175236
Fix typo
simkim Jun 24, 2022
60e2866
Handle empty find
simkim Jun 24, 2022
ef3ef32
100% coverage on utilities/enum
simkim Jun 24, 2022
8add1c0
More coverage on collection proxy
simkim Jun 27, 2022
fa065bd
remove MTLibcouchbase link + add temp env var for couchbase connection
CedricCouton Jul 1, 2022
2f254eb
CI Fix, use env vars to setup test base and tests
simkim Jul 7, 2022
2c52505
Add env vars in CI
simkim Jul 7, 2022
58a2dd9
preserve env in sudo
simkim Jul 7, 2022
a3e10f9
matrix couchbase version
simkim Jul 7, 2022
735b293
reduce number of jobs
simkim Jul 7, 2022
3f7de4c
fix couchbase version
simkim Jul 7, 2022
6f9c767
Add escaping
simkim Jul 7, 2022
b0cc946
remove puts
DamienVoreiter Aug 18, 2022
c7f87d8
reinsert query usage and rename it to query_fn
CedricCouton Aug 18, 2022
3788688
query_fn instead of query
CedricCouton Aug 19, 2022
ff80f78
Update lib/couchbase-orm/railtie.rb
DamienVoreiter Sep 8, 2022
696e332
Merge branch 'Mapotempo:master' into activemodel7-couchbase-ruby-client
DamienVoreiter Sep 8, 2022
de0ef2e
Update lib/couchbase-orm/proxies/bucket_proxy.rb
DamienVoreiter Sep 8, 2022
eed7f40
unhardcode host config & use yml file
DamienVoreiter Sep 8, 2022
922dbb8
raise CouchbaseOrm::Error
DamienVoreiter Sep 8, 2022
b5701f6
try fix ci
CedricCouton Sep 8, 2022
684f1f9
use ip not localhost
CedricCouton Sep 8, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ jobs:
- ruby: '2.6'
gemfile: '7.0.0'
fail-fast: false
runs-on: ubuntu-18.04
runs-on: ubuntu-20.04
name: ${{ matrix.ruby }} ${{ matrix.database }} rails-${{ matrix.gemfile }}
steps:
- uses: actions/checkout@v2
- run: sudo apt-get update && sudo apt-get install libevent-dev
- run: sudo apt-get update && sudo apt-get install libevent-dev libev-dev python-httplib2
- uses: ruby/setup-ruby@v1
with:
ruby-version: ${{ matrix.ruby }}
Expand Down
5 changes: 2 additions & 3 deletions ci/run_couchbase.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
set -x
set -e
apt-get install libev-dev python-httplib2 libssl1.0.0
wget https://packages.couchbase.com/releases/5.1.0/couchbase-server-enterprise_5.1.0-ubuntu14.04_amd64.deb
dpkg -i couchbase-server-enterprise_5.1.0-ubuntu14.04_amd64.deb
wget https://packages.couchbase.com/releases/7.1.0/couchbase-server-enterprise_7.1.0-ubuntu20.04_amd64.deb
dpkg -i couchbase-server-enterprise_7.1.0-ubuntu20.04_amd64.deb
sleep 8
sudo service couchbase-server status
/opt/couchbase/bin/couchbase-cli cluster-init -c 127.0.0.1:8091 --cluster-username=admin --cluster-password=password --cluster-ramsize=320 --cluster-index-ramsize=256 --cluster-fts-ramsize=256 --services=data,index,query,fts
Expand Down
5 changes: 3 additions & 2 deletions couchbase-orm.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,15 @@ Gem::Specification.new do |gem|
gem.required_ruby_version = '>= 2.1.0'
gem.require_paths = ["lib"]

gem.add_runtime_dependency 'mt-libcouchbase', '~> 1.2'
gem.add_runtime_dependency 'activemodel', ENV["ACTIVE_MODEL_VERSION"] || '>= 5.0'
gem.add_runtime_dependency 'couchbase'
gem.add_runtime_dependency 'radix', '~> 2.2' # converting numbers to and from any base

gem.add_development_dependency 'rake', '~> 12.2'
gem.add_development_dependency 'rspec', '~> 3.7'
gem.add_development_dependency 'yard', '~> 0.9'
gem.add_development_dependency 'minitest', '~> 5.10'
gem.add_development_dependency 'pry'
gem.add_development_dependency 'simplecov'

gem.files = `git ls-files`.split("\n")
gem.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
Expand Down
53 changes: 29 additions & 24 deletions lib/couchbase-orm.rb
Original file line number Diff line number Diff line change
@@ -1,46 +1,51 @@
# frozen_string_literal: true, encoding: ASCII-8BIT

require 'mt-libcouchbase'
MTLibcouchbase.autoload(:QueryN1QL, 'ext/query_n1ql')

module CouchbaseOrm
autoload :Error, 'couchbase-orm/error'
autoload :Connection, 'couchbase-orm/connection'
autoload :IdGenerator, 'couchbase-orm/id_generator'
autoload :Base, 'couchbase-orm/base'
autoload :HasMany, 'couchbase-orm/utilities/has_many'

def self.logger
@@logger ||= defined?(Rails) ? Rails.logger : Logger.new(STDOUT)
end

def self.logger=(logger)
@@logger = logger
end

def self.try_load(id)
result = nil
was_array = id.is_a?(Array)
if was_array && id.length == 1
id = id.first
end
result = id.respond_to?(:cas) ? id : CouchbaseOrm::Base.bucket.get(id, quiet: true, extended: true)
if was_array
result = Array.wrap(result)
end
if result && result.is_a?(Array)
return result.map { |r| self.try_load(r) }.compact
query_id = id.first
else
query_id = id
end

if result && result.value.is_a?(Hash) && result.value[:type]
ddoc = result.value[:type]
::CouchbaseOrm::Base.descendants.each do |model|
if model.design_document == ddoc
return model.new(result)
end
end
result = query_id.is_a?(Array) ? CouchbaseOrm::Base.bucket.default_collection.get_multi(query_id) : CouchbaseOrm::Base.bucket.default_collection.get(query_id)

result = Array.wrap(result) if was_array

if result&.is_a?(Array)
return result.zip(id).map { |r, id| try_load_create_model(r, id) }.compact
end
nil
end

def self.logger
@@logger ||= defined?(Rails) ? Rails.logger : Logger.new(STDOUT)
return try_load_create_model(result, id)
end

def self.logger=(logger)
@@logger = logger
private

def self.try_load_create_model(result, id)
ddoc = result&.content["type"]
return nil unless ddoc
::CouchbaseOrm::Base.descendants.each do |model|
if model.design_document == ddoc
return model.new(result, id: id)
end
end
nil
end
end

Expand Down
11 changes: 7 additions & 4 deletions lib/couchbase-orm/associations.rb
Original file line number Diff line number Diff line change
Expand Up @@ -75,12 +75,15 @@ def has_and_belongs_to_many(name, **options)
# Define reader
define_method(name) do
return instance_variable_get(instance_var) if instance_variable_defined?(instance_var)
ref_value = self.send(ref)
ref_value = nil if ref_value.respond_to?(:empty?) && ref_value.empty?

val = if options[:polymorphic]
::CouchbaseOrm.try_load(self.send(ref))
::CouchbaseOrm.try_load(ref_value) if ref_value
else
assoc.constantize.find(self.send(ref), quiet: true)
assoc.constantize.find(ref_value) if ref_value
end
val = Array.wrap(val)
val = Array.wrap(val || [])
instance_variable_set(instance_var, val)
val
end
Expand Down Expand Up @@ -181,7 +184,7 @@ def destroy_associations!
when :destroy, :delete
if model.respond_to?(:stream)
model.stream { |mod| mod.__send__(dependent) }
elsif model.is_a?(Array)
elsif model.is_a?(Array) || model.is_a?(CouchbaseOrm::ResultsProxy)
model.each { |m| m.__send__(dependent) }
else
model.__send__(dependent)
Expand Down
53 changes: 30 additions & 23 deletions lib/couchbase-orm/base.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,14 @@

require 'active_model'
require 'active_support/hash_with_indifferent_access'
require 'couchbase'
require 'couchbase-orm/error'
require 'couchbase-orm/views'
require 'couchbase-orm/n1ql'
require 'couchbase-orm/persistence'
require 'couchbase-orm/associations'
require 'couchbase-orm/proxies/bucket_proxy'
require 'couchbase-orm/proxies/collection_proxy'
require 'couchbase-orm/utilities/join'
require 'couchbase-orm/utilities/enum'
require 'couchbase-orm/utilities/index'
Expand Down Expand Up @@ -55,6 +57,14 @@ def bucket
@bucket ||= BucketProxy.new(Connection.bucket)
end

def cluster
Connection.cluster
end

def collection
CollectionProxy.new(bucket.default_collection)
end

def uuid_generator
@uuid_generator ||= IdGenerator
end
Expand Down Expand Up @@ -90,27 +100,20 @@ def attributes
@attributes ||= {}
end

def find(*ids, **options)
options[:extended] = true
options[:quiet] ||= false
def find(*ids, quiet: false)
CouchbaseOrm.logger.debug { "Base.find(l##{ids.length}) #{ids}" }

ids = ids.flatten.select { |id| id.present? }
if ids.empty?
return nil if options[:quiet]
raise MTLibcouchbase::Error::EmptyKey, 'no id(s) provided'
raise CouchbaseOrm::Error::EmptyNotAllowed, 'no id(s) provided'
end

CouchbaseOrm.logger.debug "Data - Get #{ids}"
record = bucket.get(*ids, **options)
records = record.is_a?(Array) ? record : [record]
records.map! { |record|
if record
self.new(record)
else
false
end
records = quiet ? collection.get_multi(ids) : collection.get_multi!(ids)
CouchbaseOrm.logger.debug { "Base.find found(#{records})" }
records = records.zip(ids).map { |record, id|
self.new(record, id: id) if record
}
records.select! { |rec| rec }
records.compact!
ids.length > 1 ? records : records[0]
end

Expand All @@ -121,12 +124,13 @@ def find_by_id(*ids, **options)
alias_method :[], :find_by_id

def exists?(id)
CouchbaseOrm.logger.debug "Data - Get #{id}"
!bucket.get(id, quiet: true).nil?
CouchbaseOrm.logger.debug "Data - Exists? #{id}"
collection.exists(id).exists
end
alias_method :has_key?, :exists?
end

class MismatchTypeError < RuntimeError; end

# Add support for libcouchbase response objects
def initialize(model = nil, ignore_doc_type: false, **attributes)
Expand All @@ -145,25 +149,28 @@ def initialize(model = nil, ignore_doc_type: false, **attributes)

if model
case model
when ::MTLibcouchbase::Response
doc = model.value || raise('empty response provided')
type = doc.delete(:type)
when Couchbase::Collection::GetResult
CouchbaseOrm.logger.debug "Initialize with Couchbase::Collection::GetResult"
doc = model.content || raise('empty response provided')
type = doc.delete('type')
doc.delete(:id)

if type && !ignore_doc_type && type.to_s != self.class.design_document
raise "document type mismatch, #{type} != #{self.class.design_document}"
raise CouchbaseOrm::Error::TypeMismatchError.new("document type mismatch, #{type} != #{self.class.design_document}", self)
end

@__metadata__.key = model.key
@__metadata__.key = attributes[:id]
@__metadata__.cas = model.cas

# This ensures that defaults are applied
@__attributes__.merge! doc
clear_changes_information
when CouchbaseOrm::Base
CouchbaseOrm.logger.debug "Initialize with CouchbaseOrm::Base"

clear_changes_information
attributes = model.attributes
attributes.delete(:id)
attributes.delete('type')
super(attributes)
else
clear_changes_information
Expand Down
18 changes: 15 additions & 3 deletions lib/couchbase-orm/connection.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# frozen_string_literal: true, encoding: ASCII-8BIT

require 'mt-libcouchbase'
require 'couchbase'

module CouchbaseOrm
class Connection
Expand All @@ -9,8 +8,21 @@ class << self
attr_accessor :options
end

def self.cluster
@cluster ||= begin
giallon marked this conversation as resolved.
Show resolved Hide resolved
options = Couchbase::Cluster::ClusterOptions.new
if ENV["TRAVIS_TEST"]
options.authenticate("tester", "password123")
else
options.authenticate("cb_admin", "cb_admin_pwd")
end

cluster = Couchbase::Cluster.connect('couchbase://127.0.0.1', options)
giallon marked this conversation as resolved.
Show resolved Hide resolved
end
end

def self.bucket
@bucket ||= ::MTLibcouchbase::Bucket.new(**@options)
@bucket ||= cluster.bucket("default")
end
end
end
2 changes: 2 additions & 0 deletions lib/couchbase-orm/error.rb
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ def initialize(message = nil, record = nil)
super(message, record)
end
end
class TypeMismatchError < Error; end
class RecordExists < Error; end
class CouchbaseOrm::Error::EmptyNotAllowed < Error; end
end
end
45 changes: 20 additions & 25 deletions lib/couchbase-orm/n1ql.rb
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,11 @@ module ClassMethods
# n1ql :by_rating, emit_key: :rating
# end
#
# Post.by_rating.stream do |response|
# Post.by_rating do |response|
# # ...
# end
# TODO: add range keys [:startkey, :endkey]
def n1ql(name, query: nil, emit_key: [], **options)
def n1ql(name, select: nil, emit_key: [], **options)
CedricCouton marked this conversation as resolved.
Show resolved Hide resolved
emit_key = Array.wrap(emit_key)
emit_key.each do |key|
raise "unknown emit_key attribute for n1ql :#{name}, emit_key: :#{key}" if key && @attributes[key].nil?
Expand All @@ -48,10 +48,9 @@ def n1ql(name, query: nil, emit_key: [], **options)
@indexes[name] = method_opts

singleton_class.__send__(:define_method, name) do |**opts, &result_modifier|
opts = options.merge(opts)

values = convert_values(opts[:key])
current_query = build_query(method_opts[:emit_key], values, query, **opts)
opts = options.merge(opts).reverse_merge(scan_consistency: :request_plus)
values = convert_values(opts.delete(:key))
current_query = run_query(method_opts[:emit_key], values, select: select, **opts.except(:include_docs))

if result_modifier
opts[:include_docs] = true
Expand Down Expand Up @@ -110,25 +109,21 @@ def build_order(keys, descending)
"#{keys.dup.push("meta().id").map { |k| "#{k} #{descending ? "desc" : "asc" }" }.join(",")}"
end

def build_query(keys, values, query, descending: false, limit: nil, **options)
if query
query.call(bucket, values)
else
bucket_name = bucket.bucket
where = build_where(keys, values)
order = build_order(keys, descending)
query = bucket.n1ql
.select("raw meta().id")
.from("`#{bucket_name}`")
.where(where)
if order
query = query.order_by(order)
end
if limit
query = query.limit(limit)
end
query
end
def build_limit(limit)
limit ? "limit #{limit}" : ""
end

def run_query(keys, values, select: nil, descending: false, limit: nil, **options)
bucket_name = bucket.name
where = build_where(keys, values)
order = build_order(keys, descending)
limit = build_limit(limit)
select ||= "raw meta().id"
raise "select must be a string" unless select.is_a?(String)
n1ql_query = "select #{select} from `#{bucket_name}` where #{where} order by #{order} #{limit}"
result = cluster.query(n1ql_query, Couchbase::Options::Query.new(**options))
CouchbaseOrm.logger.debug "N1QL query: #{n1ql_query} return #{result.rows.to_a.length} rows"
N1qlProxy.new(result)
end
end
end
Expand Down
Loading