Re: [Cython] [Pyrex] newbie list processing question

On May 5, 2008, at 11:46 AM, Daniel Ashbrook wrote:
> Robert Bradshaw wrote:
>>> def addOne(l):
>>>    return [i+1 for i in l]
> >> ...
>> This last implementation of addOne should work as is in Cython,  
>> and will be nearly optimal (assuming your CPU has reasonable  
>> branch prediction). However, if you are manipulating word-sized  
>> integers, using C arrays will give you a manyfold over python  
>> arithmetic.
>
> So in the real code, I'm actually doing float math. And I'll be  
> wanting to return my results in a list object. There will be many  
> thousands of float results; what's the best way to deal with that?  
> Use a C-specific data structure to store the results then turn it  
> into a list somehow?

I would do all my computations with C doubles, and convert to a list  
only at the very end. I would also seriously consider using NumPy  
arrays http://numpy.scipy.org/ . There is a fair amount of support  
for NumPy/Cython integration, and it's only going to get better this  
summer (two SoC projects).

>> Did you do
>> cdef int i
>> for i from 0 <= i < len(L):
>>     ...
>
> Ah, I missed the "cdef int i" part of it.

Yeah. One thing you can do is use the -a option, which will spit out  
an "annotated" html file that will catch stuff like this (bright  
yellow lines means there's a lot of unnecessary conversion going on,  
usually indicating that a variable wasn't cdef'd.

>> In Cython, if one writes
>> L[i]
>> where i is a cdef int, then it checks to see at runtime if L is a  
>> list and accesses its elements via a macro. Otherwise one can use  
>> PyList_SetItem and friends, but as you have noticed that is  
>> cumbersome (as well ahs being hard to read).
>
> Oh ho! That works very nicely; I'll include code to help others in  
> the future:
>
>   cdef int i
>   for i from 0 <= i < len(l):
>     l[i] = l[i] + 1
>
>
> Thanks for the help!

No problem.

- Robert