Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Desired size 13039606 exceeds maximum table vector size #1411

Open
cgay opened this issue Dec 25, 2021 · 2 comments
Open

Desired size 13039606 exceeds maximum table vector size #1411

cgay opened this issue Dec 25, 2021 · 2 comments

Comments

@cgay
Copy link
Member

cgay commented Dec 25, 2021

It's 2022, we should be able to put 10_000_000 elements in a hash table.

define function main
    (name :: <string>, arguments :: <vector>)
  let n = string-to-integer(arguments[0]);
  let t = make(<table>);
  for (k from 1 to n)
    t[k] := k;
  end;
end function main;

main(application-name(), application-arguments());
$ _build/bin/table-size 10000000
GC Warning: Repeated allocation of very large block (appr. size 7061504):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 69545984):
        May lead to memory leak and poor performance
Desired size 13039606 exceeds maximum table vector size.
Backtrace:
  0
  default-handler:dylan:dylan##1 + 0x12
  0x79643a6e616c7964
  default-last-handler:common-dylan-internals:common-dylan##0 + 0x2f6
  error:dylan:dylan##0 + 0x23
  error:dylan:dylan##1 + 0x76
  compute-entry-count:internal:dylan + 0x110
  make-table-vector:internal:dylan + 0x56
  rehash-table:dylan-extensions:dylan + 0x21a
  puthash:internal:dylan + 0x5b
  main:table-size:table-size + 0x8e

Looking at dylan/table.dylan briefly, it seems like

define constant $entry-counts =
could easily be the problem. Probably designed with 32-bit limits in mind.

(The GC warning for a 7MB allocation seems kind of quaint also at this point, but that's obviously a separate issue.)

@housel
Copy link
Member

housel commented Aug 3, 2022

We can generate a new sequence of primes, but what do you think would be a reasonable limit? 5.0e8?

@cgay
Copy link
Member Author

cgay commented Aug 3, 2022

"5.0e8 should be enough for anyone." --me, 2022

As usual I wish I had put more context when I filed this bug. I suspect it was something someone else was doing in a different language and I tried it in Dylan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants