ccs_accum_nbad type mismatch

[moocow-the-bovine]

see #5 (comment)

• ccs_accum_nbad should get out_type => 'indx' rather than out_type => 'int+'
• tests for ccs_accum_nbad should ensure $missing->type == $nzvalsIn->type
• CCS fix should wait until PDL segfaults with current behavior have been dealt with

[mohawk2]

Thank you for your extremely thorough attention to this fascinating, subtle problem!

For the high-level PDL::CCS::Nd wrappers, I think it's always the case that NNzOut==NnzIn, so it doesn't make sense there. I can imagine cases with structured data where it does make sense though, and it could be wasteful (especially for ufuncs) to force NNzOut==NnzIn there, so I'm inclined to leave those dimensions distinct.

My recommendations are based on what I've learned from investigating this, and also from my recent experience of using a lot of RedoDimsCode in PDL::LinearAlgebra so I could just pass in null instead of the agonising nonsense with zeroes. They are (I removed the ones you've indicated above since you obviously know your library better than I!):

• use PDL_MAYBE_SIZE from https://github.com/PDLPorters/pdl-linearalgebra/blob/f789c4100d04ba9d1b50f8c18249bdef29338496/Real/real.pd#L63-L75, for example at https://github.com/PDLPorters/pdl-linearalgebra/blob/f789c4100d04ba9d1b50f8c18249bdef29338496/Real/real.pd#L204-L208 - this only sets the dimsize if the passed ndarray is a null
• this would allow you to pass in null as the output params but would still allow you to not do so (though, srsly, why)
• set the badflag as desired in the Code with $PDLSTATESETBAD(ndarrayname), outside an inserted broadcastloop - see PDLPorters/pdl@76f353e for examples
• make the CopyBadStatusCode just call the PDL.propagate_badflag with the set values (i.e. PDL->propagate_badflag(ndarrayname, !!(ndarrayname->state & PDL_BADVAL));), i.e. to do the same as the new API does
• just make the missing Par be the last value of your valsIn, use RedoDimsCode to give that a dim of NnzInPlus1, and in loop (NnzInPlus1) %{ %} use an if (NnzInPlus1 == $SIZE(NnzInPlus1)-1) break; at the top to not accidentally process the missing value

I believe with those points, you could heavily reduce the PMCode (though not eliminate thanks to the slicing requirement).

[moocow-the-bovine]

• "easy" (= symptomatic) fixes applied in 4b653fe , uploaded to CPAN as PDL::CCS v1.23.20
• badflag-handling for ufunc counter-functions (nbad, ngood, nnz) adjusted too (but probably not sufficiently)
• broadcastloop and $PDLSETSTATE*(ndarrayname) in Code appear not to work with my (probably outdated) PDL-2.019 (-> yes, I should upgrade, but am not going to do that right now)
• signature-altering changes, moving dim-checks to RedoDimsCode (in a way which is compatible with older PDLs), and minimizing/eliminating PMCode are all out of my (TUIT) price range at the moment

... leaving this issue open for now, although I may copy+paste your suggestions (for which many thanks, in particular for the linked examples!) into separate issues later.

[mohawk2]

Switching the minimum PDL 2.081, as discussed on #13, will allow using indx where appropriate, which will solve the type mismatch issue.

[moocow-the-bovine]

... leaving this issue open for now, although I may copy+paste your suggestions (for which many thanks, in particular for the linked examples!) into separate issues later.

@mohawk2
I've made #16 for the outstanding overhaul issues. I think that with your recent #15 and 2ad95d3 , this issue can be closed. Feel free to re-open if you disagree.