groupby2<> crash on StdVisitor

[hamaney]

Hi,

Thank you for this fantastic library!!

I ran into a bit of an issue while working with group-by2. Whenever I attempt to calculate the standard deviation on the column, it unexpectedly crashes on me. I've compared the visitor syntax with the test functions in the documentation, but I can't quite figure out what's causing the crash. If you could offer any assistance, I'd appreciate it.

I am on the latest release on vcpkg.

    std::vector<unsigned long> idx = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17};
    std::vector<unsigned long> xidx = idx;
    std::vector<int> d = {1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
    std::vector<int> s = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1};
    std::vector<int> c = {0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5};
    std::vector<std::string> bc = { "A", "B", "C", "A", "B", "C", "A", "B", "C", "A", "B", "C", "A", "B", "C", "A", "B", "C"};
    std::vector<int> count = {1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3};

    int size_ = idx.size();
    assert(d.size() == size_);
    assert(s.size() == size_);
    assert(c.size() == size_);
    assert(bc.size() == size_);
    assert(count.size() == size_);

    StdDataFrame<unsigned long> df;
    df.load_data(
        std::move(idx),
        std::make_pair("d_col", d),
        std::make_pair("s_col", s),
        std::make_pair("c_col", c),
        std::make_pair("bc_col", bc),
        std::make_pair("count_col", count),
        std::make_pair("ul_col", xidx));

   // works well
    auto df1= df.groupby1<std::string>(
        "bc_col",
        FirstVisitor<unsigned long, unsigned long>(),
        std::make_tuple("count_col", "count_mean_col", MeanVisitor<int>()),
        std::make_tuple("count_col", "count_stddev_col", StdVisitor<int>()));
    
   df1.write<std::ostream, std::string, int, unsigned long>(
            std::cout, io_format::csv2);

    auto df2= df.groupby2<int, std::string>(
        "d_col",
        "bc_col",
        FirstVisitor<unsigned long, unsigned long>(),
        std::make_tuple("count_col", "count_mean_col", MeanVisitor<int>()),
        std::make_tuple("count_col", "count_stddev_col", StdVisitor<int>()));    <------- adding this causes a crash

[hosseinmoein]

If you look at file dataframe_tester_2.cc and function test_groupby_2(), you see the following comment

// Due to a bug in MS VC++ compiler being used by appveyor, this cannot be
// compiled by them

I don't remember what the issue were. I wrote the tests a while back.

But I would suggest using a different compiler version or a different compiler. Aslo, stdev requires a division. it might be a division by zero problem in your case. You have to see how many elements you get as a result of the groupby.

[hosseinmoein]

I just ran your code. Your problem is division by zero. In one of the groupby's the column count results in one element.
I should have returned a NaN in that case. I will fix the code in master

[hosseinmoein]

This has been fixed in master

[hamaney]

Nice. Thanks for the quick response.

[hamaney]

I was wondering if you have any insight on the expected timeframe for these changes to be incorporated into vcpkg and made available in a release.

[hosseinmoein]

I usually release every few month if there are enough additions/changes accumulated. Right now, I don't have enough changes and I just released.

BTW, I have never released to vcpkg -- I don't even know how. One of my users do (I don't know who). Also I just checked,
the vcpkg release is 2 releases behind the latest release. You can't just use the master for now?

[hamaney]

I'm finding that vcpkg really streamlines the releasing process—it's quite impressive. Speaking of the latest release, it appears that vcpkg is in sync with the merged PR at microsoft/vcpkg#32091.

I'm inclined to let vcpkg handle all my dependencies. Currently, I'm in the process of exploring the library, so putting together a quick hack for rapid prototyping should suffice for now.