Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iter: merge package example of iterator with complex datasource #71787

Closed
haraldrudell opened this issue Feb 17, 2025 · 3 comments · May be fixed by #71788
Closed

iter: merge package example of iterator with complex datasource #71787

haraldrudell opened this issue Feb 17, 2025 · 3 comments · May be fixed by #71788
Labels
Documentation Issues describing a change to documentation.

Comments

@haraldrudell
Copy link

haraldrudell commented Feb 17, 2025

Proposal Details

An example is added to the iter package comment featuring:

  • A stack-allocated iterator based on struct
  • Demonstrates an iterator for reading lines from a file
  • handling of state, error and panic
  • separating consuming the iterator from data retrieval
  • the code design used for every other problem solved by Go

Text added to package comment:

Iterators with complex datasource

Iterators encapsulating complex datasources offer value by separating
the iterator consumer from data-retrieval concerns in terms of databases,
networking and file systems. Intra-thread iteration over a function means
additional freedom in designing the iterator for concurrency, synchronization
and threading.

There are four needs on such iterators, referring to the below example:

  1. Receive and maintain internal state: filename, errp, osFile
  2. Provide iteration values and determine end of iteration: [LineReader.Lines]
  3. Release resources upon end of iteration or panic: [LineReader.cleanup]
  4. Propagate error conditions outside the for statement: errp

The below construct ensures faster stack allocation, as opposed to on the heap,
and features potentially reusable iterator-state encapsulated in struct.

The code example (formatted for package comment):

func Example() {

	// errorHandler prints error message and exits 1 on error
	var err error
	defer errorHandler(&err)

	// create test file
	var filename = filepath.Join(os.TempDir(), "test.txt")
	if err = os.WriteFile(filename, []byte("one\ntwo\n"), 0o600); err != nil {
		return
	}

	// iterate over lines from test.txt
	//	- first argument shows iterator allocated on the stack
	//	- second argument shows providing data to iterator
	//	- third argument shows receiving error from iterator
	//	- —
	//   - stack allocation is faster than heap allocation
	//   - LineReader is on stack even if NewLineReader is in another module
	//   - LineReader pointer receiver is more performant
	for line := range NewLineReader(&LineReader{}, filename, &err).Lines {
		fmt.Println("iterator line:", line)
	}
	// return here, err may be non-nil

	// Output:
	// iterator line: one
	// iterator line: two
}

// LineReader provides an iterator reading a file line-by-line
type LineReader struct {
	// the file lines are being read from
	filename string
	// a pointer to store occurring errors
	errp *error
	// the open file
	osFile *os.File
}

// NewLineReader returns an iterator over the lines of a file
//   - [LineReader.Lines] is iterator function
//   - new-function provides LineReader encapsulation
func NewLineReader(fieldp *LineReader, filename string, errp *error) (lineReader *LineReader) {
	if fieldp != nil {
		lineReader = fieldp
		lineReader.osFile = nil
	} else {
		lineReader = &LineReader{}
	}
	lineReader.filename = filename
	lineReader.errp = errp

	return
}

// Lines is the iterator providing text-lines from the file filename
//   - defer cleanup ensures cleanup is executed on panic
//     in Lines method or for block
//   - cleanup updates *LineReader.errp
func (r *LineReader) Lines(yield func(line string) (keepGoing bool)) {
	var err error
	defer r.cleanup(&err)

	if r.osFile, err = os.Open(r.filename); err != nil {
		return // i/o error
	}
	var scanner = bufio.NewScanner(r.osFile)
	for scanner.Scan() {
		if !yield(scanner.Text()) {
			return // iteration canceled by break or such
		}
	}
	err = scanner.Err()
	// reached end of file or error
}

// LineReader.Lines is iter.Seq string
var _ iter.Seq[string] = (&LineReader{}).Lines

// cleanup is invoked on iteration end or any panic
//   - errp: possible error from Lines
func (r *LineReader) cleanup(errp *error) {
	var err error
	if r.osFile != nil {
		err = r.osFile.Close()
	}
	if err != nil || *errp != nil {
		// aggregate errors in order of occurrence
		*r.errp = errors.Join(*r.errp, *errp, err)
	}
}

// errorHandler prints error message and exits 1 on error
//   - deferrable
func errorHandler(errp *error) {
	var err = *errp
	if err == nil {
		return
	}
	fmt.Fprintln(os.Stderr, err)
	os.Exit(1)
}
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/649420 mentions this issue: iter: provide iterator example handling errors and resources

@gabyhelp
Copy link

Related Discussions

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

@seankhliao
Copy link
Member

seankhliao commented Feb 17, 2025

not a proposal as there's no API changes.
but I think this looks generally out of scope for a package example? it lacks clarity in demonstrating the core concepts of iter.

@seankhliao seankhliao closed this as not planned Won't fix, can't repro, duplicate, stale Feb 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Documentation Issues describing a change to documentation.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants