0
$\begingroup$

I am searching for an approach for solving the following problem:

Given I have a large amount of printed and scanned documents. I am already able to detect text and the corresponding bounding boxes in that document. I have the texts and the coordinates of the bounding boxes. Now, if I want to extract specific data like "insurance contract number" or "recipient" and stuff like that, I would have to manually program that (to some degree).

I was wondering if there is an approach to show the computer what is the correct data extraction. Is it possible to train a model against a large amount of documents and the correctly extracted data as labels. So the features would be the extracted text plus the coordinates and the model should learn which data to extract depending on the training examples with correctly extracted text.

All algorithms I came across so far only "allow" one label. From my understanding this apporach would need like 20 labels for every training example. And also the prediction is also only "correct", if all labels are matching.

Anyone has an idea how to approach that?

Improve this question
$\endgroup$

3 Answers 3

Reset to default
1
+50
$\begingroup$

Both of the answers from FelixGK and Wargream have merit. With the information you have posted though, I am not sure you are approaching the problem with the right objective, so the expected solution may seem further away than it probably is.

As FelixGK suggests, you should break the models down into a set of one-vs-the-rest binary classifiers, one for each label.

I am not sure what you mean when you state

From my understanding this apporach would need like 20 labels for every training example. And also the prediction is also only "correct", if all labels are matching.

Do you expect one model to collect multiple data elements within a document and assign the correct label to the element? This is probably difficult in a single model, but could be easier using some form of ensemble approach, like a set of binary classifiers. If you want to try it all in one model, I would look at neural networks and specifically, a network that implements a SoftMax activation function in the final layer. This will allow you to produce a multi-class output with probabilities assigned to each class. When using softmax, you generally want to accept the result with the highest probability.

Improve this answer
$\endgroup$
0
$\begingroup$

One thing that comes to mind would be a Restricted Boltzmann Machine or alternatively, if you need a more powerfull model, a Deep Believe Net. Both learn to recognize familiar examples. You train them without labels, you just give them a set of training data. They will then adjust their weights, so that the global energy over the sample is minimal. If you have a trained RBM or DBN you can put an example in, and look at the energy. The lower the energy, the closer the example to the sample which was used to train your model.

You could train one model per target class, and then put your examples through all models, and return those labels for which the model-energy is below a threshold.

Generally, you could take any binary classifier and train one per class to classify membership.

Improve this answer
$\endgroup$
0
$\begingroup$

We can relate this problem to object detection (given an image, create a bounding box on the objects present and identify the detected objects). Take a look at this neural network based object detection algorithm.

Your training data would be the images with bounding boxes as labels and the classification of each boxes. Just manually label some and then switch the placement of the objects to create more training data.

Finally, use OCR to translate the detected objects into text data.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.