clean paridecl.pxd

[videlec]

Remove declarations from paridecl.pxd that already appear in auto_paridecl.pxd. This was just achieved by running the following script

#! /usr/bin/env python
import re   
    
fnc = re.compile("    (int|long|void|GEN) +([A-Za-z][0-9A-Za-z_]*)\(.*\)")
    
with open('auto_paridecl.pxd') as f: 
    functions = set()
    for line in f.read().split('\n'):
        match = fnc.match(line)
        if match:
            out_type, fname = match.groups() 
            functions.add(fname)
    
f_in = open('paridecl.pxd') 
f_out = open('paridecl.pxd.rewritten', 'w')
for line in f_in.read().split('\n'):
    match = fnc.match(line)
    if match:
        out_type, fname = match.groups()
        if fname in functions:
            print('undeclare {}'.format(fname))
            continue
    f_out.write(line)
    f_out.write("\n")

f_in.close()
f_out.close()

[jdemeyer]

Given that you have a script to clean things up, wouldn't it be better to run that script instead at installation time as part of the autogen process? This way, we get the best of both worlds: we don't need to mess with paridecl.pxd in the sources (Jeroen happy) and we don't have duplicate declarations in the installed paridecl.pxd file (Vincent happy).

Then we could also decide to install just one file, i.e. take

paridecl.pxd + auto_paridecl.pxd - duplicates

and put that in one file.

[videlec]

I like the idea. Though the autogen process is operating before the build and not at installation (if a distinction needs to be done). We certainly want to keep the name paridecl.pxd for the file containing all relevant pari declarations. And I do not want to modify a file in the sources. For doing it concretely I can for example move the current paridecl.pxd as a template in autogen/?
Furthermore, it would be good to say more explicitely how the paridecl.pxd was generated (I believe you did it based on PARI sources of a certain version).

[jdemeyer]

Furthermore, it would be good to say more explicitely how the paridecl.pxd was generated

Partially automatic and partially by hand. It's copying the paridecl.h file from PARI, take the parts that we need and change from C to Cython syntax.

But I don't often update the complete file. Sometimes I just update a small part (say for example, everything related to L-functions), sometimes I add 1 function that I need. So it's a bit an organic process which is not easy to document exactly.

[videlec]

The cleaning code is now integrated in the autogen/.

[jdemeyer]

Can you explain me why you need to change the logic in generator.py so much? Why do you need to loop twice over the PARI functions? And why does the signature of handle_pari_function have to change?

Maybe it works, but it looks overcomplicated to me.

[videlec]

The problem is that there is a discrepancy between the list of functions in pari.desc and what could be generated in the Cython header paridecl.pxd. In order to do this I decided to first filter the functions from pari.desc that we will be able to treat. In this context, a function is treatable if it passes can_handle_function and parse_prototype.

[jdemeyer]

Wouldn't it be much simpler to just return a boolean value from handle_pari_function() to indicate whether the function was treatable? In other words, start handle_pari_function() with

        try:
            args, ret = parse_prototype(prototype, help)
        except NotImplementedError:
            return False  # Skip unsupported prototype codes

and then check the return value of handle_pari_function.

[videlec]

Would also work. I wanted to avoid calling twice parse_prototype.

[videlec]

done.

Though, in the future we might want to distinguish between functions for which we can actually write the definition in paridecl.pxd automatically and the one for which we are able to write a method of a Gen.

[jdemeyer]

done.

That's not really what I meant (I said to change handle_pari_function; that way you wouldn't need to call parse_prototype twice), but that's OK.

This should be return False though:

return  # Skip unsupported prototype codes

But you're still looping twice over all functions... the whole point of changing handle_pari_function (or can_handle_function as you did) is that this is not needed.

[videlec]

Then I don't understand what you suggest.

[videlec]

Ho now I see.

[jdemeyer]

By the way, since this changes significantly the installed .pxd files, I would like to merge this in cypari2-2.0.0 and not in cypari2-1.x.y. OK for you?

[videlec]

Actually, I don't see how to avoid going through the list of functions twice. The code is doing

1. write the template declaration file in paridecl.pxd without the handled functions (as a prerequisite we need the list of handled functions)

2. write the auto declarated function after the template part

[jdemeyer]

Maybe we could simplify the generator by keeping the files in memory instead of immediately writing them out.

[jdemeyer]

The largest file is a bit over 1MB in size, so it shouldn't be a problem to manage it in memory.