Defensive programming type-checking

Question

I have issues with dynamically typed languages, and I tend to worry about type a lot.

Numpy has different behaviour depending on if something is a matrix or a plain ndarray, or a list. I didn't originally have all these asserts, but I inserted them while trying to debug a type error.

from numpy import * sigmoid = vectorize(lambda(x): 1.0/(1.0+exp(-x))) def prob_hOn_givenV (v,b,w): assert isinstance(v,matrix) assert isinstance(w,matrix) assert isinstance(b,matrix) assert shape(v)[1]==shape(w)[0] #print("|v|="+str(shape(v)) +"|w|="+str(shape(w)) ) return sigmoid(b+v*w) #sum up rows (v is a column vector) def prob_vOn_givenH (h,a,w): assert isinstance(h,matrix) assert isinstance(a,matrix) assert isinstance(w,matrix) assert shape(h)[1]==shape(w.T)[0] return sigmoid(a+h*w.T) #sum up columns. (h is a row vector)

Is there a better way? Perhaps a defensive programming toolkit? There is also duplication in the code, but I can't see a nice way of removing it while maintaining meaning. People familiar with Restricted Boltzman Machines will recognise the conditional formula used for Gibbs sampling for contrastive divergence.

The key point is that a compiler's static type checking is a kind of test, but an incomplete one (otherwise statically typed languages would have no bugs). In a dynamically typed languages, unit tests take the place of the compiler's type checks, and add some degree of assurance that the code is correct (something the compiler can never do for you, whether statically or dynamically typed). — Wayne Conrad
– Wayne Conrad, Commented Jan 13, 2014 at 14:23
Aside: from numpy import * -- don't do this. This will replace some common Python builtins, like any and all, with numpy versions which can silently return exactly the opposite values. — DSM
– DSM, Commented Jan 13, 2014 at 15:48

Robert Kern · Accepted Answer · 2014-01-13 11:17:48Z

First, I recommend avoiding the matrix type altogether. The problems it causes outweighs the minor syntactical convenience it provides, in my (and quite a few others') opinion.

Second, the idiomatic way to do this kind of thing in numpy is not to assert that the inputs are particular types, but to coerce them to an ndarray. np.asarray() does this efficiently; if the input is an ndarray, the input passes through untouched; if it is an ndarray subclass, then a new ndarray will be created but only as a view on the original data so no data will be copied.

import numpy as np def sigmoid(x): # No need to use vectorize() for this. It's already vectorized. return 1.0 / (1.0 + np.exp(-x)) def prob_hOn_givenV(v, b, w): v = np.asarray(v) b = np.asarray(b) w = np.asarray(w) # Don't bother checking the shape. `np.dot()` will do that for us. # Strictly speaking, the `asarray(v)` and `asarray(w)` aren't necessary # since `np.dot(v, w)` already does that internally. But it does no harm. return sigmoid(b + np.dot(v, w))

my issue wit ndarray was that it isn't nice for checking shape. Because ndarray doesn't differentiate between shape of (1,N),(N,1) and (N). and that gives me headaches trying to ensure I have the formula right (esp when w is square) — Frames Catherine White
– Frames Catherine White, Commented Jan 13, 2014 at 14:15

Marcin · Accepted Answer · 2014-01-13 18:51:07Z

These are the best links to pythonic ways of doing various checks:

Pre- and Post-conditions and Type Enforcement

You can use particular sets of asserts (or any other piece of code for more complicated checks, for example checking if an list of ints truly contains all ints) as a function, and then use that function as a parameter to the decorator, making a very clean description of what you're checking.

The accept/returns decorator is a quick and clean way of checking your inputs. The pre- and post-condition checking decorator is more suited for larger or more complicated sets of checks, whether it's type enforcement, or parameter range, or any other relationship between the data, like if an array's elements are sorted.

I used to include all these checks in the function alone, but with some functions it makes the code rather unreadable, as the real functionality gets lost in all the checks. This is a fairly simple way to organize your checks, and make your code easier to read.

+1 for not trusting duck typing ;)

Stack Exchange Network

Defensive programming type-checking

2 Answers 2

You must log in to answer this question.

Hot Network Questions

Defensive programming type-checking

2 Answers 2

You must log in to answer this question.

Related

Hot Network Questions