What is Hack?


Using Python Suds in Multi Threaded and Multi Process Environments

Working on a team that primarily builds integrations between systems, you will inevitably end up writing integrations against SOAP based APIs. If, like me, your team uses python, you will likely end up using Suds. As you can see from this stackoverflow post, there aren’t a great deal of options for SOAP in python; and none are phenomenal.

While Suds is not actively maintained, it has wide adoption, and is fairly stable and mature. Suds is actually a pretty good abstraction around SOAP, and it encourages idiomatic python. I have my fair share of frustrations with Suds, but for the most part, it “just works”. It does, however, have two major flaws that make it unusable out of the box for my use-case.

Suds is not Thread safe

The Problem

For me, this manifested only in production (wsgi is designed such that most development environments, including mine, are single threaded) with a SAXParseException:

File "/venv_path/lib/python2.6/site-packages/suds/client.py", line 542, in __call__
  return client.invoke(args, kwargs)
File "/venv_path/lib/python2.6/site-packages/suds/client.py", line 602, in invoke
  result = self.send(soapenv)
File "/venv_path/lib/python2.6/site-packages/suds/client.py", line 649, in send
  result = self.failed(binding, e)
File "/venv_path/lib/python2.6/site-packages/suds/client.py", line 702, in failed
  r, p = binding.get_fault(reply)
File "/venv_path/lib/python2.6/site-packages/suds/bindings/binding.py", line 258, in get_fault
  faultroot = sax.parse(string=reply)
File "/venv_path/lib/python2.6/site-packages/suds/sax/parser.py", line 136, in parse
File "/usr/lib64/python2.6/xml/sax/expatreader.py", line 107, in parse
  xmlreader.IncrementalParser.parse(self, source)
File "/usr/lib64/python2.6/xml/sax/xmlreader.py", line 123, in parse
File "/usr/lib64/python2.6/xml/sax/expatreader.py", line 211, in feed
File "/usr/lib64/python2.6/xml/sax/handler.py", line 38, in fatalError
  raise exception
SAXParseException: <unknown>:15:2: mismatched tag

The Solution

Ideally this could be fixed in suds properly, but since the library is no-longer maintained, we opted for a simple workaround. Basically we use a thread safe queue, to add a lock around suds usage:

To Set Up:

from Queue import Queue
from contextlib import contextmanager

suds_service_queue = Queue(MAX_THREADS)
for n in range(MAX_THREADS):

def get_suds_client():
    client = suds_service_queue.get() # blocks until client available
        yield client

To Use:

from setup import get_suds_client
with get_suds_client() as client:

This technique obviously requires additional resources compared to using a single suds Client instance, but for us that trade-off is worth it. If you have flexible real-time requirements, or limited parallelism, you can make MAX_THREADS = 1 to achieve thread safety without any overhead.

Suds is not Friendly in Multi-Process Environments

The Problem

After we solved our thread safety problems, we ran into another exception coming from scripts running on the same server as our wsgi application. The OSError exceptions were of the form:

File "/venv_path/venv/lib/python2.6/site-packages/suds/client.py", line 109, in __init__
  options.cache = ObjectCache(days=1)
File "/venv_path/venv/lib/python2.6/site-packages/suds/cache.py", line 145, in __init__
File "/venv_path/venv/lib/python2.6/site-packages/suds/cache.py", line 277, in checkversion
File "/venv_path/venv/lib/python2.6/site-packages/suds/cache.py", line 251, in clear
  os.remove(os.path.join(self.location, fn))
OSError: [Errno 13] Permission denied: '/tmp/suds/suds-6882323804701353659-document.px'

There is an open ticket related to this bug in suds, but we didn’t want to fork suds to apply the patch provided. Instead, we found an alternative solution that works well for us.

The Solution

To understand our solution to this problem, we need to look more closely at how to reproduce it. First we must have a system in which there are at least two users - in our case apache and app-prod. The first of these users (say, apache) to use a suds client will create a directory at /tmp/suds which it will store the suds document cache. Assuming the umask of this user is configured in a standard way, and that app-prod is not in the apache group the process run by app-prod will suffer the dreaded Permission denied OSError

Suds allows us to provide an ObjectCache object to the client constructor, and in turn the ObjectCache constructor allows us to specify the path of the document cache.

We initially considered adding the pid to the path to ensure permission safety, but were concerned with the side effect of creating many folders as the lifetime of the machine goes on (every script that uses suds would create a new folder, and our default system only cleans up /tmp on reboot. Instead we decided to use the uid of the process, which would protect us from permission issues, while limiting the number of folders to the number of active users.

Check out the resulting class.

import os
import suds
from suds.cache import ObjectCache

class SudsApi(object):
    _suds_client = None

    def __init(self, wsdl_url):
        self.wsdl_url = wsdl_url

    def client(self):
        if self._suds_client is None:
            cache_path = "/tmp/{0}-suds".format(os.getuid())
            cache = ObjectCache(cache_path, days=1)
            self._suds_client = suds.client.Client(self.wsdl_url,
        return self._suds_client


Hopefully these tricks will help you sucessfully use suds in your production environment.

Live Demo of the New Geo Queries in Mongo 2.4

MongoDB version 2.4 will bring with it a host of exciting new features, and chief among them are the additional support for geo data. In addition to the 2d geospatial indexing and geospatial querying that mongo has supported for a while, the 2.3.2 development release introduces the “2dsphere” index and support for the GeoJSON format.

I was lucky enough to have the opportunity to help out during the QA process for this feature, and wanted to share a simple little demo I put together as a sanity check. What I really love is that I was able to put together this demo in a few short hours knowing only a little bit about the google maps JS api, and knowing nothing about mongo geo features. This is a huge boon to rapidly prototyping location aware systems!


You can toggle between the three new query types by selecting the name of the operation on the top bar.

  • $within: Click to add points points to the map. Once you have added three points, an initial polygon will be drawn, and the server will fetch up to 500 points or lines from the database, and they will be rendered on the map. You can click on points to see what landmark they represent. You can continue adding points to your polygon, but if it self-intersects at all, you will get no results. Self intersecting polygons are invalid.

  • $geoIntersects: Click to add points as with $within. Once you have added two points, a line will be drawn between them. The server will fetch some lines that intersect with that line. You can continue adding points, and you will generate a line string. The number of results is limited to 50.

  • $near: Click to add a point. Mongo will fetch the nearest 10 points/lines, and draw them on the map. It’s pretty fun to do this in the areas with water in them!

At any time you can clear any current interactions using the Clear button on the bottom left.

You can get the source code here


  • This was a very quick project built to internally demo and test these features - there are many many things I would change if this were to serve production customers.
  • Something funny happens with line colouring in this demo.  I’m not sure if I am to blame, or Google.

Discuss on Hacker News