Which framework should I use to develop a REST application in Python?

Recently, I started looking into REST solutions in Python. Although the possibilities are many, it was hard to find many satisfactory products to be used to develop REST applications in a simple, clean way.
To feet my needs, the final product had to satisfy the following criteria:

• it has to support Python 3 cleanly;
• it should enforce as little design constraints as possible on the resulting application;
• as a consequence of the previous point, it should make it easy to swap the chosen solution with another at will;
• optional, but good to have: it should make it possible and not too awkward to use a DI library on top of it.

Here is a list of the frameworks I checked out.

Django REST framework

This one seems to be one of the most adopted, probably because of the popularity of Django. Django REST Framework is a Django plugin that adds some functionality to handle REST more easily. It's very well documented and it's not too hard to customize the logic of specific endpoints, or plug in additional code on the resources. 

Like with anything in Django, most of the basics work out of the box, and everything is easy and cool as long as you follow the Django (in this case, Django REST framework) way to do things. The design of your applications must follow the rules strictly, and obviously you need to use the entirety of the Django framework, or write some very awkward code.

This makes it almost impossible not only to change the REST layer, but everything else in your application will be entangled in Django. 

Flask RESTful

Flask RESTful is more lightweight than Django REST framework, and seems to be well documented and supported. I like the OO approach and the extendibility. The problem with it is Flask: in the documentation, they basically suggest against using Python 3. This makes me very uncomfortable using the framework in general, let alone an extension on top of it.

Another thing I don't like is the fact that you need to extend Resource to create a resource, and you need to pass the class arguments as dictionaries using resource_class_kwargs and resource_class_args: it makes it awkward to inject services and to use a DI library.

Werkzeug

Werkzeug is the WSGI library Flask is based upon. It's a very thin layer, so it doesn't force your design too much. The only complain may be that the layer is almost too thin: it doesn't do much to hide WSGI. Another big point against it is that, according to their website, support for python 3 is still highly experimental.

Falcon

Falcon ended up being my framework of choice. I love its minimalistic approach, and the fact that it uses duck typing when creating resources, and that it takes initialized classes when configuring the resulting application: it makes it very easy to use DI with it, encouraging the developers to follow the single responsibility principle. It's also advertised as being extremely fast compared to other frameworks, although I'm not really concerned with performance at this level when coding with Python. And it supports Python 3! 

I'll publish a post with my approach to REST development with Falcon in the near future.

Conclusion

Obviously, this post is not an evaluation on the quality of these frameworks per se, but more a collection of thoughts on which library fits my needs the most. I'd like to get your feedback and feel free to suggest more frameworks.

Python: performance comparison of itertools dropwhile and takewhile against simple generators

While wandering around on the internet, i stumbled upon this thread, in which is discussed whether or not dropwhile and takewhile should be deprecated, and later removed, from the itertools library. As for Hettinger, who wrote itertools, the use of dropwhile and takewhile lead to less readable code, as everything they are used for can be implemented using generators. While reading, I thought 'ok, maybe they are less readable, but shouldn't be the purpose of itertools to provide a toolbox to perform loops in an efficient way using mainly pure c code? Even if generators are more readable, which is debatable, there's no way that generators can be faster than pure c code'. As in the discussion efficiency wasn't mentioned, i decided to profile generators against dropwhile and takewhile myself.

In the discussion mentioned above, the following use case is used to show the use of dropwhile/takewhile: iterate over text delimited by start and end markers. Here is the timing code:

The result was quite surprising: generators, in this case, are faster than dropwhile and takewhile by something around 30-40%.  I tested the code with both python 2.7.3 and python 3.3.0 with similar results (python 3.3.0 being slower for both functions).

import timeit
import dis 
import random
from itertools import dropwhile,takewhile
FILE_PATH = "test_data/data/text_with_start_end_markers.txt"
FILE = [line for line in open(FILE_PATH)]
START_MARKER = 'start_marker'
END_MARKER = 'end_marker'

def iter_block_generator(lines, start_marker, end_marker):
  lines = iter(lines)
  for line in lines:
    if line.startswith(start_marker):
      yield line
      break
  for line in lines:
    if line.startswith(end_marker):
      return
    yield line

def iter_block_itertools(lines, start_marker, end_marker):
  return takewhile(lambda x: not x.startswith(end_marker),
                   dropwhile(lambda x: not x.startswith(start_marker),
                             lines)
                  )


print("check that both solutions return the same result:")
join_using_itertools = \
        "".join(iter_block_itertools(FILE,
                                     START_MARKER,
                                     END_MARKER
                                    )
                )
join_using_generator = \
        "".join(iter_block_generator(FILE,
                                     START_MARKER,
                                     END_MARKER
                                     )
                )
assert join_using_itertools == join_using_generator
iter_block_generator_func = \
        "''.join(iter_block_generator(FILE," + \
                                     "START_MARKER," + \
                                     "END_MARKER))"
  
iter_block_itertools_func = \
        "''.join(iter_block_itertools(FILE," + \
                                     "START_MARKER," + \
                                     "END_MARKER))"


for function in (iter_block_generator_func,iter_block_itertools_func):
 print(function)
 print(timeit.repeat(
                function,
                repeat=1,
                number=5,
                setup="from __main__ import " + \
                      "iter_block_generator," + \
                      "iter_block_itertools," + \
                      "FILE,START_MARKER,END_MARKER"
                )
       )
       

I'm probably missing something, so let me know if you find a good reason for this.

Gaussian filter python implementation

This post is, hopefully, a part of a bigger tutorial about edge detection. My final goal is to implement a Canny edge detector in python, it's just an exc
ercise to get a better understanding about the matter.

The first step in Canny algorithm is to apply a gaussian filter to the image, in order to get rid of some noise that will make edge detection harder.

I used this guide as a reference.

One-dimensional window

Here is the algorithm that applies the gaussian filter to a one dimentional list. The first step is to calculate wiindow weights, than, for every element in the list, we'll place the window over it, multiply the elements by their corresponding weight and then sum them up.

def get_window_weights(N):
    support_points = [(float(3 * i)/float(N))**2.0 for i in range(-N,N + 1)]
    gii_factors = [exp(-(i/2.0)) for i in support_points]
    ki = float(sum(gii_factors))
    return [giin/ki for giin in gii_factors]

def apply_filter(index,array,window):
    N = (len(window)-1)/2
    #fix out of range exception
    array_l = [array[0] for i in range(N)] + array + [array[-1] for i in range(N)]
    return sum(
            (float(array_l[N + index + i]) * window[N+i]
            for i in range(-N,N+1)
            )
            )

def gaussian_filter(data,window_weights,filter_func = apply_filter):
    ret = []
    for i in range(len(data)):
        ret.append(filter_func(i,data,window_weights))
    return ret

In order to apply the filter to images, we need a function that can work on pixels. The basic idea is to execute the same operations on every component of the color.

def sum_filtered_pixels(pix1,pix2):
    return tuple([pix1[i]+pix2[i] for i in range(len(pix1))])

def apply_filter_to_pixel(index,array,window):
    N = (len(window)-1)/2
    #fix out of range exception
    array_l = [array[0] for i in range(N)] + array + [array[-1] for i in range(N)]
    return reduce(sum_filtered_pixels,
            ( tuple([float(v) * window[N+i] for v in array_l[N + index + i]])
            for i in range(-N,N+1)
            )
            )

Bidimensional window

Obviously, we need to apply the filter to an image, so we need it to work with a bidimensional window. As explained in the guide, we can use a divide-et-impera approach and use the one dimensional algorithm. All we have to do is to run 1d gauss filter over all the pixel lines, and than again over all the columns.

def gaussian_filter_2d(matrix,window_weights,filter_func = apply_filter):
    new_matrix = []
    for i in range(len(matrix)):
        new_matrix.append(gaussian_filter(matrix[i],window_weights,filter_func))   
    #apply 1d gaussian filter line by line
    for i in range(len(matrix[0])):
        temp_list = gaussian_filter([new_matrix[t][i] for t in range(len(matrix))],
                                    window_weights,filter_func)
        for t in range(len(matrix)):
            new_matrix[t][i] = temp_list[t]
    return new_matrix

def gaussian_blur(img_in,img_out,window_size):
    img = Image.open(img_in)
    width,height = img.size
    window_weights = get_window_weights(window_size)
    pixmap = gaussian_filter_2d([
                                 [img.getpixel((w,h)) for w in range(width)]
                                 for h in range(height)
                                ],
                                window_weights,
                                apply_filter_to_pixel)

    new_image = Image.new("RGB",(width,height))
    for h in range(height):
        for w in range(width):
            new_image.putpixel(
                          (w,h),(int(pixmap[h][w][0]),
                                int(pixmap[h][w][1]),
                                 int(pixmap[h][w][2]))
                        )
    new_image.save(img_out)

if __name__ == '__main__':
    gaussian_blur('wombat.jpg',"wombat_blurred.jpg",5)

Here is the result:

Digest http authorization on SOAP services with Suds

It took me some time to figure out how to access a SOAP service protected by http digest authentication with urllib2 and suds. Here's what i came out with.

import urllib2

URL = 'http://example.com/service/' 

ah = urllib2.HTTPDigestAuthHandler()
password = "mypass"
ah.add_password(None,'http://www.example.com/','username',password)
urllib2.install_opener(urllib2.build_opener(ah))

from suds.client import Client
url = "http://www.example.com/wsdl/"
client = Client(url)

client.options.transport.urlopener = urllib2.build_opener(ah)