I would use Delete[] for the purpose of deleting selected rows:
Delete[Array[C, {4, 4}], {{2}, {3}}] (* {{C[1, 1], C[1, 2], C[1, 3], C[1, 4]}, {C[4, 1], C[4, 2], C[4, 3], C[4, 4]}} *) To delete columns, you could do a preliminary transposition first...