Here's the solution I came up with:
- Run Blender's built in "select interior face" algorithm
- Note which faces are selected
- Loop over every face, and cast a ray from the "camera" position to the face.
- If the ray hits (the center of) the face, we know it's visible to the camera, so don't delete it.
- If the ray doesn't hit the face (hits another face) it means it's probably interior.
This is a bit confusing, so I made two images to show more detail:


Commented code is below. Please note this is taken from a large file so I might have forgot to declare some variable references. Happy to clarify if anyone has questions in comments.
import bpy import bmesh from mathutils import Vector cameraOrigin = Vector( ( 0, 0, 10 ) ) ops = bpy.ops scene = bpy.context.scene mesh = bpy.ops.mesh current_mesh = scene.objects[ 0 ] current_mesh_data = current_mesh.data scene.objects.active = current_mesh # Blender's "select interior faces" doesn't actually select interior faces, it # selects faces that share edges with 2 or more other faces. In my case this # erroneously selects some faces that are visible but connect to multiple other # faces. The solution is to select interior faces, then ignore any faces that # the camera (positioned at 0,0,10) can see, and delete the remaining ones # See http//blender.stackexchange.com/questions/57540/automated-way-to-make-select-interior-faces-ignore-select-faces-that-are-visib def removeInteriorFaces( mesh_data ): # First do the built in selection mesh.select_interior_faces() # And store all faces inside that selection indices = [] bm = bmesh.from_edit_mesh( mesh_data ) for index, face in enumerate( bm.faces ): if face.select: indices.append( ( index, face.calc_center_median_weighted() ) ) # Deselect everything... mesh.select_all() # Switch to object mode to do scene raycasting (doesn't work in edit mode # I don't think, got error "has no mesh data to be used for ray casting" ops.object.mode_set( mode = 'OBJECT' ) outside = [] for index_data in indices: index = index_data[ 0 ] center = index_data[ 1 ] direction = center - cameraOrigin; direction.normalize() # Cast a ray from the "camera" position to the face we think is interior result, location, normal, faceIndex, object, matrix = scene.ray_cast( cameraOrigin, direction ) # If the ray actually hit the face, as in the face index from the # selection matches the face index from the raycast, then this face # *is* visible to the camera, so don't remove it! if faceIndex == index: outside.append( faceIndex ) # Build a list of the "true" interior face indices, which is the original # indices from Blender's built in "select interior faces", but without the # faces we know the camera can see invisible_interior_faces = [ data[ 0 ] for data in indices if data[ 0 ] not in outside ] print( 'Removing ',len( invisible_interior_faces ),'invisible faces' ) # Select the faces (in object mode this is easy, strangely)... if len( invisible_interior_faces ) > 0: for index in invisible_interior_faces: mesh_data.polygons[ index ].select = True ops.object.mode_set( mode = 'EDIT' ) # Then delete them if len( invisible_interior_faces ) > 0: mesh.delete( type = 'FACE' ) ops.object.mode_set( mode = 'EDIT' ) removeInteriorFaces( current_mesh_data )