PDL Advent calendar 2024 tracker

[mohawk2]

From PDLPorters/pdl#34, better to have separately:

@duffee writes: I'm starting to think that a blog post using Test::PDL to test whether I truely understand how a $matrix * $vector operation works. (Advent calendar warning)

As a starter for an Advent posting, the newly-incorporated Test::PDL, which I'm using to replace all tapprox and nearly all instances of these constructs in the codebase:

• approx
• ok all($x == $y)
• is_deeply [$p->unpdl], [...]
• ok sum(abs($out-$in)) < 1e-4
• is $out == $in, 1
• ok tapprox(...)
• ok all approx(...)
• ok all($x == $y)
• is( int(at(sum($c-$ans))), 0, "conv2d()" )
• ok +($ans[0] == 50) && ($ans[1] == 1) && ($ans[2] == 2)

... is really good. It checks all types (unless you say not to) and dims are identical, whether NaNs/badvalues/infinities are all the same, and when something fails, you get a message like:

#   Failed test 'inplace sorting with bad values'
#   at Basic/t/ufunc.t line 56.
#     1/10 values do not match
#          got: Double   D [10]       (PB   ) [0 1 2 4 5 6 7 8 BAD BAD]
#     expected: Double   D [10]       (PB   ) [0 1 2 4 5 6 7 8 9 BAD]
# First <=5 values differ at: 
# [
#  [8]
# ]
# Those 'got' values: [BAD]
# Those 'expected' values: [9]

That one was from finding a bug in qsort that had lain hidden, because all(pdl('5 3 BAD') == pdl('5 BAD 2')) returns 1; the BAD does not make all return false because there are any true values.

[mohawk2]

Link in to a collab/takeover of PWC, see manwar/perlweeklychallenge-club#11123

[duffee]

Current order and statuses:

1. An Introduction - duffee - LIVE
2. cartography - mohawk2 - LIVE
3. Installing SciPDL - duffee - LIVE
4. func/interpolation - mohawk2 - LIVE
5. k-means clustering of EV cars - duffee - LIVE
6. PDL::Finance::TA - vikasnkumar - LIVE
7. PDL with Javascript - vikasnkumar - LIVE
8. simplex - mohawk2 - LIVE
9. Exploring Data with D3.js and Mojolicious - duffee - LIVE
10. Playing Nice with Bad Values - duffee - LIVE
11. gsl_rng - mohawk2 - LIVE
12. EEG visualisation - shugo - LIVE
13. The Sound of Perl - HaraldJoerg - LIVE
14. From simple statistics to ANOVA - duffee - LIVE
15. Pitch in Praat - jjatria - LIVE
16. using Test::PDL - duffee - - LIVE
17. Simple string processing, as below - mohawk2 - LIVE
18. Mandelbrot - vikasnkumar - LIVE
19. Ray tracing with matrices - duffee - LIVE
20. PDL internals - mohawk2 - LIVE
21. Image processing - duffee assisted by craig de forest - LIVE
22. Hough transform for image feature extraction - sivoais - LIVE
23. generalised random number generation - chrisarg - LIVE
24. PDL community, developing @nrdvana neat no-dependency use of PDL in Math::3Space - mohawk2 - LIVE
25. Spectral processing - karl g - LIVE

Specific ideas:

• adding a new key to PP (which will include a little description of the clever design of PP, and a pretty GraphViz picture of its current structure)
• make a dataflowing (using the newish flowing idiom) version of the scoring formula in https://blogs.perl.org/users/enkidu/2020/03/enter-the-matrix-with-pdl.html
• an explanation of broadcasting (no longer "threading") and dummy dims being adding from C_ij = 1/(x_i - y_j) to being C_ij = 1/(x_ij - y_ij) with additional indices being non-bolded
• unpacking the "MathGraph" from the demo 3d, because it's event-driven/dataflow programming
• applying watermarks to images and videos (inspired by an old Perl Journal article)
• plotting a Roche lobe in 3D/TriD mesh plotting (use the Earth plotting in demo 3d)
• writing up Ingo's curl on a vector field question
• something about https://metacpan.org/pod/Data::Frame
• Perl Jupyter notebooks, using https://metacpan.org/pod/Devel::IPerl - Allen Downey is putting most of his new books into Jupyter on github (https://github.com/AllenDowney/ThinkDSP)
• @duffee referred to a finance post to help with the Time Series analysis book, also PDL::Stats::TS
• "PDL for Julia users"

[vikasnkumar]

@duffee @mohawk2 if you are falling short of posts, I can write another post that shows how to plot PDL data using Javascript in the browser, Template Toolkit and the Highcharts library. Probably can be used as a second post for using PDL with financial data on a backend webserver like Mojolicious or Dancer or something else.

[vikasnkumar]

I created a second post for you guys and opened a new issue in the pdl.github.io repo. thanks for the encouragement.

[duffee]

@mohawk2 No real preference which demos, other than please include the plotting demos for the visual appeal. I've already written my own take on Bad Values, but if you feel the need to include the Bad Values demo, put it after mine because the demo looks more complete/longer. I'm not even precious about the order they go in, but I think an Introduction should be Dec 1 and that the Finance::TA posts should be closer together and on weekdays for the exposure. There are at least one pair which is order-dependant. I was using the Test::PDL to lead into the Vector Transformations with Matrices.

Looking at my old post, we should also be looking at including an Advent Window image for each post and a short blurb on the author. I'll throw up a PR for my first post today to get the ball rolling (if you haven't already).

[vikasnkumar]

@mohawk2 OK , I did the PR.

[mohawk2]

There's now a post on PerlMonks mentioning the calendar: https://www.perlmonks.org/?node_id=11162961

[mohawk2]

@mohawk2 this page https://pdl.perl.org/advent/blog/ definitely has some CSS issues. maybe some CSS file is not included correctly. Happens on both mobile and desktop browser.

This was due to the Styleshout CSS not having room for a "tags" line, and has been fixed.

@mohawk2 @duffee RSS/Atom feeds are not being generated in the Advent blog. I posted the advent calendar link on Mastodon and folks have asked for the RSS feed for it. i remember seeing it in the YAML file but in View Source it does not get generated. probably a setting to be enabled

The feeds were in fact being generated - thanks @vikasnkumar for adding links to the RSS one with #38!

[duffee]

The 5th is up for review and I've put Shugo's post up for comments.

@mohawk2 if you think it flows better to have vik's second post on the following day,then let's do that. Before we were catering to the "binge" crowd, I'd thought that Monday the 9th would be better exposure.

String processing was a suggestion by you on Perlmonks: simple string comparison for efficiency using bytes and badvalues to compare DNA strands.

I've emailed Len at Advent Planet. Will be added tonight.

[mohawk2]

Tips on how to get a local version of the Statocles site running so you can iterate are on #38 - please do this so you can see what works (verbatim needs 4-space indent instead of ```, for instance)

There are now various Advent PRs to copy for style; the site.yml has the author bios, as mentioned on the IRC channel, so if you don't have an entry there, please include that in your PR or just put a bio you want in a comment on the PR.

[duffee]

Did you want vik's PDL with JavaScript for the 7th (your suggestion from 2 days ago) or my D3.js and Mojolicious (the preliminary post)?

[jjatria]

@duffee I'm writing an article on signal processing / speech analysis. Not too in-depth, I'm aiming for a bit of an introductory tone, but largely a PDL-translation of https://pinguinorodriguez.cl/blog/pitch-in-praat