PostgreSQLCursor¶ ↑

PostgreSQL Cursor is an extension to the ActiveRecord PostgreSQLAdapter for very large result sets. It provides a cursor open/fetch/close interface to access data without loading all rows into memory, and instead loads the result rows in “chunks” (default of 10_000 rows), buffers them, and returns the rows one at a time.

For web pages, an application would not want to process a large amount of data, usually employing a Pagination scheme to present it to the users. Background processes sometimes need to generate a large

Previous solutions employ pagination to fetch each block, then re-running the query for the next “page”. This plugin avoids re-executing the query by using the PostgreSQL cursors.

Like the #find_by_sql method, #find_cursor returns each row as a hash instead of an instantiated model class. The rationale for this is performance, though an option to return instances is available. Julian’s benchmarks showed returning instances was a factor of 4 slower than return the hash.

A Rails/ActiveRecord plugin for the PostgreSQL database adapter that will add cursors to a find_cursor() method to process very large result sets.

the find_cursor method uses cursors to pull in one data block (of x records) at a time, and return each record as a Hash to a procedural block. When each data block is exhausted, it will fetch the next one.

find_cursor_by_sql takes a custom SQL statement and returns each row.

Example¶ ↑

Account.find_cursor(:account_cursor, true, :all, :conditions=>["status = ?", 'good']) do |row|
  puts row.to_json
end

Account.find_cursor_by_sql(:account_cursor, "select ...", *args) do |row|
  puts row.to_json
end

Authors¶ ↑

Allen Fair, [email protected], github.com/afair

Thank you to:

• Iulian Dogariu, github.com/iulianu (Fixes)

• Julian Mehnle, www.mehnle.net (Suggestions)

Copyright © 2010 Allen M. Fair, released under the MIT license