Write specs for new Ruby 3.0 features and changes

[eregon]

ruby/spec already contains some specs for 3.0, but we should aim to cover all new features and important changes.
This will improve the test coverage of these features (and maybe discover a few bugs along the way), allow other Ruby implementations to implement the changes faster with more confidence and document clearly the new behavior.

The new specs should be within a version guard block:

ruby_version_is "3.0" do
  # New specs
end

NOTE: https://rubyreferences.github.io/rubychanges/3.0.html gives more details for many features and changes.

From https://github.com/ruby/ruby/blob/master/doc/NEWS-3.0.0.md:

NEWS for Ruby 3.0.0

This document is a list of user visible feature changes
since the 2.7.0 release, except for bug fixes.

Note that each entry is kept to a minimum, see links for details.

Language changes

• Keyword arguments are now separated from positional arguments.
  Code that resulted in deprecation warnings in Ruby 2.7 will now
  result in ArgumentError or different behavior. [Feature #14183]

• Procs accepting a single rest argument and keywords are no longer
  subject to autosplatting. This now matches the behavior of Procs
  accepting a single rest argument and no keywords.
  [Feature #16166]

  pr = proc{|*a, **kw| [a, kw]}
  
  pr.call([1])
  # 2.7 => [[1], {}]
  # 3.0 => [[[1]], {}]
  
  pr.call([1, {a: 1}])
  # 2.7 => [[1], {:a=>1}] # and deprecation warning
  # 3.0 => [[[1, {:a=>1}]], {}]

• Arguments forwarding (...) now supports leading arguments.
  [Feature #16378]

  def method_missing(meth, ...)
    send(:"do_#{meth}", ...)
  end

• Pattern matching (case/in) is no longer experimental. [Feature #17260]

• One-line pattern matching is redesigned. [EXPERIMENTAL]

  • => is added. It can be used like a rightward assignment.
    [Feature #17260]

    0 => a
    p a #=> 0
    
    {b: 0, c: 1} => {b:}
    p b #=> 0

  • in is changed to return true or false. [Feature #17371]

    # version 3.0
    0 in 1 #=> false
    
    # version 2.7
    0 in 1 #=> raise NoMatchingPatternError

• Find-pattern is added. [EXPERIMENTAL]
  [Feature #16828]

  case ["a", 1, "b", "c", 2, "d", "e", "f", 3]
  in [*pre, String => x, String => y, *post]
    p pre  #=> ["a", 1]
    p x    #=> "b"
    p y    #=> "c"
    p post #=> [2, "d", "e", "f", 3]
  end

• Endless method definition is added. [EXPERIMENTAL]
  [Feature #16746]

  def square(x) = x * x

• Interpolated String literals are no longer frozen when
  # frozen-string-literal: true is used. [Feature #17104]

• Magic comment shareable_constant_value added to freeze constants.
  See {Magic Comments}[rdoc-ref:doc/syntax/comments.rdoc@Magic+Comments] for more details.
  [Feature #17273]

• Deprecation warnings are no longer shown by default (since Ruby 2.7.2).
  Turn them on with -W:deprecated (or with -w to show other warnings too).
  [Feature #16345]

• $SAFE and $KCODE are now normal global variables with no special behavior.
  C-API methods related to $SAFE have been removed.
  [Feature #16131] [Feature #17136]

• yield in singleton class definitions in methods is now a SyntaxError
  instead of a warning. yield in a class definition outside of a method
  is now a SyntaxError instead of a LocalJumpError. [Feature #15575]

• When a class variable is overtaken by the same definition in an
  ancestor class/module, a RuntimeError is now raised (previously,
  it only issued a warning in verbose mode). Additionally, accessing a
  class variable from the toplevel scope is now a RuntimeError.
  [Bug #14541]

• (to be sync'd from truffleruby) Assigning to a numbered parameter is now a SyntaxError instead of
  a warning.

Command line options

--help option

• When the environment variable RUBY_PAGER or PAGER is present and has
  a non-empty value, and the standard input and output are tty, the --help
  option shows the help message via the pager designated by the value.
  [Feature #16754]

--backtrace-limit option

• (to be sync'd from truffleruby) The --backtrace-limit option limits the maximum length of a backtrace.
  [Feature #8661]

Core classes updates

Outstanding ones only.

Array

• The following methods now return Array instances instead of
  subclass instances when called on subclass instances:
  [Bug #6087]

  • Array#drop
  • Array#drop_while
  • Array#flatten
  • Array#slice!
  • Array#slice / Array#[]
  • Array#take
  • Array#take_while
  • Array#uniq
  • Array#*

• Can be sliced with Enumerator::ArithmeticSequence

  dirty_data = ['--', 'data1', '--', 'data2', '--', 'data3']
  dirty_data[(1..).step(2)] # take each second element
  # => ["data1", "data2", "data3"]

Binding

• Binding#eval when called with one argument will use "(eval)"
  for __FILE__ and 1 for __LINE__ in the evaluated code.
  [Bug #4352] [Bug #17419]

ConditionVariable

• @aardvark179 ConditionVariable#wait may now invoke the block/unblock scheduler
  hooks in a non-blocking context. [Feature #16786]

Dir

• Dir.glob and Dir.[] now sort the results by default, and
  accept the sort: keyword option. [Feature #8709]

ENV

• ENV.except has been added, which returns a hash excluding the
  given keys and their values. [Feature #15822]

• Windows: Read ENV names and values as UTF-8 encoded Strings
  [Feature #12650]

Encoding

• Added new encoding IBM720. [Feature #16233]

• Changed default for Encoding.default_external to UTF-8 on Windows
  [Feature #16604]

Fiber

• Fiber.new(blocking: true/false) allows you to create non-blocking
  execution contexts. [Feature #16786]

• Fiber#blocking? tells whether the fiber is non-blocking. [Feature #16786]

• Fiber#backtrace and Fiber#backtrace_locations provide per-fiber backtrace.
  [Feature #16815]

• The limitation of Fiber#transfer is relaxed. [Bug #17221]

GC

• GC.auto_compact= and GC.auto_compact have been added to control
  when compaction runs. Setting auto_compact= to true will cause
  compaction to occur during major collections. At the moment,
  compaction adds significant overhead to major collections, so please
  test first! [Feature #17176]

Hash

• Hash#transform_keys and Hash#transform_keys! now accept a hash that maps
  keys to new keys. [Feature #16274]

• Hash#except has been added, which returns a hash excluding the
  given keys and their values. [Feature #15822]

IO

• IO#nonblock? now defaults to true. [Feature #16786]

• @aardvark179 IO#wait_readable, IO#wait_writable, IO#read, IO#write and other
  related methods (e.g. IO#puts, IO#gets) may invoke the scheduler hook
  #io_wait(io, events, timeout) in a non-blocking execution context.
  [Feature #16786]

Kernel

• Kernel#clone when called with the freeze: false keyword will call
  #initialize_clone with the freeze: false keyword.
  [Bug #14266]

• Kernel#clone when called with the freeze: true keyword will call
  #initialize_clone with the freeze: true keyword, and will
  return a frozen copy even if the receiver is unfrozen.
  [Feature #16175]

• Kernel#eval when called with two arguments will use "(eval)"
  for __FILE__ and 1 for __LINE__ in the evaluated code.
  [Bug #4352]

• (added in truffleruby) Kernel#lambda now warns if called without a literal block.
  [Feature #15973]

• @aardvark179 Kernel.sleep invokes the scheduler hook #kernel_sleep(...) in a
  non-blocking execution context. [Feature #16786]

Module

• Module#include and Module#prepend now affect classes and modules
  that have already included or prepended the receiver, mirroring the
  behavior if the arguments were included in the receiver before
  the other modules and classes included or prepended the receiver.
  [Feature #9573]

  class C; end
  module M1; end
  module M2; end
  C.include M1
  M1.include M2
  p C.ancestors #=> [C, M1, M2, Object, Kernel, BasicObject]

• Module#public, Module#protected, Module#private, Module#public_class_method,
  Module#private_class_method, toplevel "private" and "public" methods
  now accept single array argument with a list of method names. [Feature #17314]

• Module#attr_accessor, Module#attr_reader, Module#attr_writer and Module#attr
  methods now return an array of defined method names as symbols.
  [Feature #17314]

• Module#alias_method now returns the defined alias as a symbol.
  [Feature #17314]

Mutex

• Mutex is now acquired per-Fiber instead of per-Thread. This change
  should be compatible for essentially all usages and avoids blocking when
  using a scheduler. [Feature #16792]

Proc

• Proc#== and Proc#eql? are now defined and will return true for
  separate Proc instances if the procs were created from the same block.
  [Feature #14267]

Queue / SizedQueue

• @aardvark179 Queue#pop, SizedQueue#push and related methods may now invoke the
  block/unblock scheduler hooks in a non-blocking context.
  [Feature #16786]

Ractor

• New class added to enable parallel execution. See rdoc-ref:ractor.md for
  more details.

Random

• Random::DEFAULT now refers to the Random class instead of being a Random instance,
  so it can work with Ractor.
  [Feature #17322]

• Random::DEFAULT is deprecated since its value is now confusing and it is no longer global,
  use Kernel.rand/Random.rand directly, or create a Random instance with Random.new instead.
  [Feature #17351]

String

• The following methods now return or yield String instances
  instead of subclass instances when called on subclass instances:
  [Bug #10845]

  • String#*
  • String#capitalize
  • String#center
  • String#chomp
  • String#chop
  • String#delete
  • String#delete_prefix
  • String#delete_suffix
  • String#downcase
  • String#dump
  • String#each_char
  • String#each_grapheme_cluster
  • String#each_line
  • String#gsub
  • String#ljust
  • String#lstrip
  • String#partition
  • String#reverse
  • String#rjust
  • String#rpartition
  • String#rstrip
  • String#scrub
  • String#slice!
  • String#slice / String#[]
  • String#split
  • String#squeeze
  • String#strip
  • String#sub
  • String#succ / String#next
  • String#swapcase
  • String#tr
  • String#tr_s
  • String#upcase

Symbol

• Symbol#to_proc now returns a lambda Proc. [Feature #16260]

• Symbol#name has been added, which returns the name of the symbol
  if it is named. The returned string is frozen. [Feature #16150]

Fiber

• @aardvark179 Introduce Fiber.set_scheduler for intercepting blocking operations and
  Fiber.scheduler for accessing the current scheduler. See
  rdoc-ref:fiber.md for more details about what operations are supported and
  how to implement the scheduler hooks. [Feature #16786]

• Fiber.blocking? tells whether the current execution context is
  blocking. [Feature #16786]

• @aardvark179 Thread#join invokes the scheduler hooks block/unblock in a
  non-blocking execution context. [Feature #16786]

Thread

• Thread.ignore_deadlock accessor has been added for disabling the
  default deadlock detection, allowing the use of signal handlers to
  break deadlock. [Bug #13768]

Warning

• Warning#warn now supports a category keyword argument.
  [Feature #17122]

Stdlib updates

Set

• SortedSet has been removed for dependency and performance reasons.

• Set#join is added as a shorthand for .to_a.join.

• Set#<=> is added.

Socket

• Add :connect_timeout to TCPSocket.new [Feature #17187]

Compatibility issues

Excluding feature bug fixes.

• Regexp literals and all Range objects are frozen. [Feature #8948] [Feature #16377] [Feature #15504]

  /foo/.frozen? #=> true
  (42...).frozen? # => true

• EXPERIMENTAL: Hash#each consistently yields a 2-element array. [Bug #12706]

  • Now { a: 1 }.each(&->(k, v) { }) raises an ArgumentError
    due to lambda's arity check.

• When writing to STDOUT redirected to a closed pipe, no broken pipe
  error message will be shown now. [Feature #14413]

• TRUE/FALSE/NIL constants are no longer defined.

• Integer#zero? overrides Numeric#zero? for optimization. [Misc #16961]

• Enumerable#grep and Enumerable#grep_v when passed a Regexp and no block no longer modify
  Regexp.last_match. [Bug #17030]

• Requiring 'open-uri' no longer redefines Kernel#open.
  Call URI.open directly or use URI#open instead. [Misc #15893]

• SortedSet has been removed for dependency and performance reasons.

Stdlib compatibility issues

• Default gems

  • The following libraries are promoted to default gems from stdlib.

    • English
    • abbrev
    • base64
    • drb
    • debug
    • erb
    • find
    • net-ftp
    • net-http
    • net-imap
    • net-protocol
    • open-uri
    • optparse
    • pp
    • prettyprint
    • resolv-replace
    • resolv
    • rinda
    • set
    • securerandom
    • shellwords
    • tempfile
    • tmpdir
    • time
    • tsort
    • un
    • weakref

  • The following extensions are promoted to default gems from stdlib.

    • digest
    • io-nonblock
    • io-wait
    • nkf
    • pathname
    • syslog
    • win32ole

• Bundled gems

  • net-telnet and xmlrpc have been removed from the bundled gems.
    If you are interested in maintaining them, please comment on
    your plan to https://github.com/ruby/xmlrpc
    or https://github.com/ruby/net-telnet.

• SDBM has been removed from the Ruby standard library. [Bug #8446]

  • The issues of sdbm will be handled at https://github.com/ruby/sdbm

• WEBrick has been removed from the Ruby standard library. [Feature #17303]

  • The issues of WEBrick will be handled at https://github.com/ruby/webrick

C API updates

• C API functions related to $SAFE have been removed.
  [Feature #16131]

• C API header file ruby/ruby.h was split. [GH-2991]

  This should have no impact on extension libraries,
  but users might experience slow compilations.

• Memory view interface [EXPERIMENTAL]

  • The memory view interface is a C-API set to exchange a raw memory area,
    such as a numeric array or a bitmap image, between extension libraries.
    The extension libraries can share also the metadata of the memory area
    that consists of the shape, the element format, and so on.
    Using these kinds of metadata, the extension libraries can share even
    a multidimensional array appropriately.
    This feature is designed by referring to Python's buffer protocol.
    [Feature #13767] [Feature #14722]

• Ractor related C APIs are introduced (experimental) in "include/ruby/ractor.h".

Implementation improvements

• New method cache mechanism for Ractor. [Feature #16614]

  • Inline method caches pointed from ISeq can be accessed by multiple Ractors
    in parallel and synchronization is needed even for method caches. However,
    such synchronization can be overhead so introducing new inline method cache
    mechanisms, (1) Disposable inline method cache (2) per-Class method cache
    and (3) new invalidation mechanism. (1) can avoid per-method call
    synchronization because it only uses atomic operations.
    See the ticket for more details.

• The number of hashes allocated when using a keyword splat in
  a method call has been reduced to a maximum of 1, and passing
  a keyword splat to a method that accepts specific keywords
  does not allocate a hash.

• super is optimized when the same type of method is called in the previous call
  if it's not refinements or an attr reader or writer.

Miscellaneous changes

• Methods using ruby2_keywords will no longer keep empty keyword
  splats, those are now removed just as they are for methods not
  using ruby2_keywords.

• (seems untestable as 2.7 would only order reversed if TTY) When an exception is caught in the default handler, the error
  message and backtrace are printed in order from the innermost.
  [Feature #8661]

• (to be sync'd from truffleruby) Accessing an uninitialized instance variable no longer emits a
  warning in verbose mode. [Feature #17055]

[lxxxvi]

I think this is already covered and can be marked completed:

Symbol#to_proc now returns a lambda Proc

spec/core/symbol/to_proc_spec.rb

Lines 32 to 37 in 7526ccb

	ruby_version_is "3.0" do
	it "returns a Proc with #lambda? true" do
	pr = :to_s.to_proc
	pr.should.lambda?
	end

[andrykonchin]

Revised the "Language changes" and "Core classes updates" sections and marked items that are already tested.

[eregon]

Thanks!

[moofkit]

Pattern matching (case/in) is no longer experimental. [Feature #17260]

Hi! It looks like already done by this commit 1093159

[eregon]

@moofkit Yes, but that change doesn't check there is no warning in 3.0+ for regular pattern matching (1-line pattern matching is still experimental in 3.0). Could you add a spec in that file that check -> { some pattern matching }.should not_complain for 3.0+?

[baweaver]

I'll take on a few pattern matching items:

• One-line match (=>) [Feature #17260]
• One-line match (in) [Feature #17371]
• Find pattern [Feature #16828]

[baweaver]

Will wait to go after Feature #16828 until I understand conventions for contributing to this repo.

[baweaver]

Feature#16378 (Argument forwarding with leading arg) had been completed by the following commit:

b089bf7

[JuanCrg90]

@eregon

Hash#except has been added, which returns a hash excluding the
given keys and their values. [Feature #15822]

Is already implemented on : #786

[eregon]

@JuanCrg90 Thanks! I noted those as done.

[JuanCrg90]

@eregon

Warning
Warning#warn now supports a category keyword argument.
[Feature #17122]

Was already implemented on: 05ff9d434c

[eregon]

@marcandre I wonder, do you have specs for Ractor?
I know you wrote https://github.com/marcandre/backports/tree/master/lib/backports/ractor.
Or maybe you used CRuby tests to check it?

[marcandre]

@eregon, oops, missed that notification, sorry.

Yes, I used CRuby tests to check my backport (and I was glad to have them all pass, except those that were checking that MovedError or similar).

[herwinw]

The "Core classes updates" has two sections named "Fiber". The second one is in between "Symbol" and "Thread".
The last item of the second section is "Thread#join invokes the scheduler hooks block/unblock in a non-blocking execution context", this should probably be moved to "Thread" (similar to "ConditionVariable" and "Queue / SizedQueue")