Skip to content

Ruby on Rails plugin to add PostgreSQL cursor support to Active Record PostgreSQL plugin for processing large result sets.

License

Notifications You must be signed in to change notification settings

cluesque/postgresql-cursor

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PostgreSQLCursor

PostgreSQL Cursor is an extension to the ActiveRecord PostgreSQLAdapter for very large result sets. It provides a cursor open/fetch/close interface to access data without loading all rows into memory, and instead loads the result rows in “chunks” (default of 10_000 rows), buffers them, and returns the rows one at a time.

For web pages, an application would not want to process a large amount of data, usually employing a Pagination scheme to present it to the users. Background processes sometimes need to generate a large

Previous solutions employ pagination to fetch each block, then re-running the query for the next “page”. This plugin avoids re-executing the query by using the PostgreSQL cursors.

Like the #find_by_sql method, #find_cursor returns each row as a hash instead of an instantiated model class. The rationale for this is performance, though an option to return instances is available. Julian’s benchmarks showed returning instances was a factor of 4 slower than return the hash.

A Rails/ActiveRecord plugin for the PostgreSQL database adapter that will add cursors to a find_cursor() method to process very large result sets.

the find_cursor method uses cursors to pull in one data block (of x records) at a time, and return each record as a Hash to a procedural block. When each data block is exhausted, it will fetch the next one.

find_cursor_by_sql takes a custom SQL statement and returns each row.

Example

Account.find_cursor(:account_cursor, true, :all, :conditions=>["status = ?", 'good']) do |row|
  puts row.to_json
end

Account.find_cursor_by_sql(:account_cursor, "select ...", *args) do |row|
  puts row.to_json
end

Authors

Allen Fair, [email protected], github.com/afair

Thank you to:

Copyright © 2010 Allen M. Fair, released under the MIT license

About

Ruby on Rails plugin to add PostgreSQL cursor support to Active Record PostgreSQL plugin for processing large result sets.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Ruby 100.0%