Automating bank cheque analysis by using Microsoft Cognitive Services

Just looking for code? https://github.com/albertherd/ChequeAnalyser

Microsoft Cognitive Services is a rich set of AI services, such as Computer Vision, Speech Recognition, Decision making and NLP. The great thing about these tools is that you don’t really have to be an AI expert to make use of these tools, as these models come pre-trained and production ready. You’ll just feed it your information and let the framework work for you.

We’ll be looking at one area of Microsoft’s Cognitive Services – Computer vision. More specifically, we’ll be looking at the handwriting API – you’ll provide the handwriting and the system will provide you with the actual text. We’ve already worked with the Computer Vision API from Microsoft Cognitive services – we used this API to tag our photo album.

Let’s look at today’s scenario – we’re a fictitious bank which processes bank cheques. These cheques come hand-written from our clients, which contain instructions on how to transfer money from one account to the holder’s account.

A cheque typically has the following information:

  • Issue Date
  • Payee
  • Amount (in digits)
  • Amount (in words, for cross reference)
  • Payer’s account number
  • (Other information, which was omitted for this proof of concept

This is how our fictitious cheque looks like.

Fictitious Cheque
Ficticious cheque – credit: https://www.dreamstime.com/royalty-free-stock-image-blank-check-false-numbers-image7426056

This is how our fictitious cheque looks when we’re looking at the regions we’re interested in, represented in bounding boxes.

0_analysis_template
Cheque Template with bounding boxes representing areas of interest

Let’s consider these three handwritten cheques.

The attached application does the following analysis:

  • Import these cheques as images.
  • Send the images over to Microsoft Cognitive Services
  • Extract all the handwriting / text found in the image
  • Consider only those text which we’re interested in (as represented with bounding boxes previously)
  • Forward this extracted information to whatever system needed. In our case, we’re just printing them to screen.

The below is the resultant information derived from the sample cheques.

ChequeAnalyserResult
Results of Cheque Analysis

Most of the heavy lifting is done by the Microsoft Cognitive Services, making these AI tools available to the masses. Of course, with a bit more business logic, the information that can be extracted from these tools can be greatly improved, making them production ready.

As with the previous example, this example uses the TPL Dataflow library, which is an excellent tool for Actor-Based multithreaded applications.

If you want to try this yourself, you’ll need:

Until the next one!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s