My notes on the morning session of the Scipy 2007 Tutorials -- Day 2

 Eric Jones and Bill Spotz

•  Interface with legacy code
• Speed up existing code
• Write everything in python.  Then go back and optimize the "hot spots"
• Good for finite difference problems

•  Start with stand alone F90 application -- legacy system
• driven by text files
• Python becomes your main function
• Can remove the file interpreters and text processing from Fortran code base.  Let Python do it.
• Can just import FFT and signal processing stuff from scipy or numpy.
  
• General Issue with FORTRAN
• use large number of globals
• causes "states" with wrapped modules
• makes threading difficult
• Not very modular
• One or two very long functions -- not easy to break up
• doesn't handle memory allocation well
• good for us since we can then manage the memory from python
• FORTRAN 90 does memory management but doesn't easily map to C (base for Python).
• Problems with global variables
• If wrapped functions can be called in different order.  Result can differ depending on the state of the global variables
• You are keeping info the user doesn't know about
• Fixes
• Remove the ability of calling functions out of order by wrapping functions at the highest level.
• This can be a problem because perhaps calling the lower level functions separately is useful.
• Can still have threading problems.
  
• Get rid of global variables and include them in the argument list for functions
• This is best option but may be a lot of work.
• Problems with non-modularity
• There is no easy solution
• Generally want to only wrap FORTRAN simulation code.  Leave the pre and post processing exercises to Python.
• Problems with long argument lists
• long lists encouraged by FORTRAN
• need to determine what each argument refers to
• Switching to Python gets rid of need for many arguments used for return arguments or array indicies
• Use classes to group arguments
• class to define environment
• class to define sources
• class to define algorithm
• Look for the nouns in the problem definitions to identify potential classes
• When designing your OO interface is the API understandable by someone who is only interested in the science

 C/C++ Integration

• SWIG -- www.swig.org
  
• ctypes -- python standard library
  
• Pyrex -- nz.cosc.canterburz.ac.nz/greg/python/Pyrex
  
• boost -- www.boost.org/libs/python/doc/index.html
  
• weave -- www.scipy.org
  

FORTRAN Integration

• f2py -- part of numpy
  

• You will hand write a C function with a certain format that Python can then recognize and then call.
• This special function will call the C function you wish to have wrapped.
• Your python arguments are converted for use by the C function
• Things to worry about
• Reference counting
• Ways to build
• Build by hand
• Using setup.py -- the Python equivalent to a 'makefile'
  

• weave.blitz
•  import weave
• weave.blitz("expression") # Where expression is something like "a = b + c"
• Can get factors of 2 or 3 speedups
• Doesn't handle array broadcasting
  
• weave.inline("return_val=3;")
• Allows you to write c code in your Python program.
• Only pay the compile time costs once
• Requires that the end user have a compiler on their system
  
• ext_tools
• Can build functions when code is initially built.  You now have a standalone modules that you can distribute.
• This will be part of your setup.py file.
  

•  Scipy Tutorial Web Page
• SWIG Examples and code available - SigNumPyTutorial
  
• Simple wrapper and interface generator
• Parses c/C++ prototypes
• Generates wrapper code
• For python the wrapper code conforms to python C API
• A python prozy file is also generated
• Can take header files as direct input
• Interface files are more common (swig directives)
• %include
• %etc
• SWIG is a code generator
• does not compile code like f2py
• Will need to use distutils to compile swig-generated extension modules
• convenient
• portable
• Good C++ support coverage
  
• templates, smart pointers, etc...
• SWIG Type MAPS
• allows SWIG to work across multiple target languages
• %typemap(_) -> some functionality

• Author: Pearu Peterson at Center for Nonlinear Studies Tallinn, Estonia
• Automagically "wraps" Fortran 77/90/95 libraries for use in Python
  
•  f2py is specifically built to wrap Fortran functions using NumPy arrays.
• f2py -c -m fcopy fcopy.f -compiler=mingw32
• -c : compile code and build and extension module
• -m: name the extension module
• Simplest Usage Result
• Looks exactly like Fortran but it is now callable by Python.
• Give f2py some hints as to what these variables are wuse for and how they may be related in Python.
• f2py directives
• inset Fortran comments directly into the source file
  
• help f2py interpret the source
• resulting interface is more Pythonic
• multidimensional array issues
• Python and Numeric use C conventions for array storage (row major order).  Fortran uses column major ordering
• Numeric - last dimension varies the fastest
• Fortran - first dimension varies the fastest
• f2py handles the conversion back and forth between the representations if you mix them in your code.
• You will likely end up with a copy
• However, you can now have numpy allocate the arrays in Fortran ordering
• Your code will be faster if you can avoid mixing the representations
• Distribution of f2py based code (numpy.distutils)
  
• recipient must have f2py and scipy_distutils installed
• create setup.py file
• Distribute *.f files with setup.py file
• Optionally distribute *.pyf file if you've spruced up the interface in a separate interface file.
• 
  

 My notes on the Scipy 2007 Day 2 Tutorial afternoon session.

 Travis Oliphant

•  Text
• scipy.io
• Binary
• numpy.fromfile
• read raw binary data
  

•  numpy.fft

•  scipy.interpolate
  
• scipy.ndimage.map_coordinates
• A way to re-grid data
  
• scipy.sandbox.delaunay
• create triangulation
• linearly interpolate from triangulation

•  Edge-detection (edge.py)
• audio filtering (part2.py)
• general filtering (scipy.ndimage.convolve)
• 1-d linear filtering (scipy.signal.lfilter)

•  True deconvolution is an inverse problem
• You have lost information and want it back
• How do you get the information back?
• Make assumptions and add additional information
• Use probability theory

•  scipy has a host of tools
• some need to be refactored
• more tools are needed

Tool lists

• ndimage
• measure
• interpolate
• morphology
• filters

• from ipython prompt> scipy?