Card Scanner on Android Using Firebase’s ML Kit and CameraX

Machine learning has changed the way users interact with mobile applications. It offers brand new experience to users, making apps capable of leveraging various features such as providing accurate, location-based recommendations; detecting and manipulating text from images; instantaneously detecting micro diseases, and much more!

ML can make mobile apps more user-friendly and easy-to-use…exactly the kinds of user experiences that we as developers want to create. 😉

So in this blog, we’ll be learning about text recognition, which is the process of recognizing textual information in images, videos, documents, and other sources. Using this, you can make apps that perform different tasks like translating various languages (Google Translate) and converting text to a pdf or any other format (CamScanner, Google Keep), just to name a couple.

Here, we’ll be making an Android app that scans business cards and recognizes the most important text, like contact numbers, email addresses, and names of the person/organization from the scanned image. Here’s the full project source code:

Thanks to ML Kit’s Text Recognition API, this task is actually super simple. This pre-trained model can recognize text from the image in any Latin-based language (and more, with cloud-based text recognition).

On top of this, we’ll use the CameraX library to scan a given card, and then integrate it with the ML Kit’s API. CameraX’s analysis step makes it easy to feed images to the image detector, hence reducing the overhead code of any native or additional camera library.

1. Setup Firebase in your project and add the Vision dependency

  • This is a simple step to set up Firebase in your project. You can find a good tutorial here.
  • Add the dependencies for the ML Kit Android libraries to your module’s (app-level) Gradle file (usually app/build.gradle):

Now, like with ML Kit’s other APIs, Text Recognition has 2 different types:

  1. The first one is the On-Device API, which runs text recognition on the device itself. It’s free and recognizes more than 50 languages.
  2. The second one is the Cloud API, which runs on Google Cloud and is more accurate and recognizes and identifies a wider range of languages and special characters. It’s free for the first 1000 requests, which is good enough if you just want to play around.

2. Implement camera functionality in your app

The Vision API needs an image to extract the data from, so either create an app that lets you upload images from the gallery or create an app that uses the Camera API to click a picture and use it instead.

As discussed earlier, we’ll be using CameraX to implement the camera functionality in the app. Using it provides us with an additional advantage of reducing code, as the Firebase Vision API has in-built methods to support CameraX API, it easier for us to call different methods with less overhead

3. Recognize text in images

After selecting an input image for processing, we run the text recognizer to extract useful information from it.

a) Run the text recognizer

  • To create a FirebaseVisionImage object from a media.Image object, such as when capturing an image from a device’s camera, pass the media.Image object and the image’s rotation to FirebaseVisionImage.fromMediaImage(). As we’re using CameraX, the OnImageCapturedListener and ImageAnalysis.Analyzer classes calculate the rotation value for you, so you just need to convert the rotation to one of ML Kit’s ROTATION_ constants before calling FirebaseVisionImage.fromMediaImage():
private class YourImageAnalyzer : ImageAnalysis.Analyzer {
    private fun degreesToFirebaseRotation(degrees: Int): Int = when(degrees) {
        0 -> FirebaseVisionImageMetadata.ROTATION_0
        90 -> FirebaseVisionImageMetadata.ROTATION_90
        180 -> FirebaseVisionImageMetadata.ROTATION_180
        270 -> FirebaseVisionImageMetadata.ROTATION_270
        else -> throw Exception("Rotation must be 0, 90, 180, or 270.")

    override fun analyze(imageProxy: ImageProxy?, degrees: Int) {
        val mediaImage = imageProxy?.image
        val imageRotation = degreesToFirebaseRotation(degrees)
        if (mediaImage != null) {
            val image = FirebaseVisionImage.fromMediaImage(mediaImage, imageRotation)
            // Pass image to an ML Kit Vision API
            // ...
  • Then, pass the media.Image object and the rotation value to FirebaseVisionImage.fromMediaImage():

b) Get an instance of FirebaseVisionTextRecognizer

The code snippet below shows how to use the on-device model:

c) Pass the image to the processImage() method:

Pass the image to the FirebaseVisionTextRecognizer object created above.

val result = detector.processImage(image)
        .addOnSuccessListener { firebaseVisionText ->
            // Task completed successfully
            // ...
        .addOnFailureListener { e ->
            // Task failed with an exception
            // ...

4. Extract text from blocks of recognized text

If the text recognition operation succeeds, a FirebaseVisionText object will be passed to the success listener.

  • It contains the full text recognized in the image and zero or more TextBlock objects.
  • Each TextBlock represents a rectangular block of text, which contains zero or more Line objects.
  • Each Line object contains zero or more Element objects, which represent words and word-like entities (dates, numbers, and so on).

For each TextBlock, Line, and Element object, you can get the text recognized in the region and the bounding coordinates of that region.

The received text is organized into pages, blocks, paragraphs, words, and symbols. For each unit of organization, you can get information such as its dimensions and the languages it contains.

Given that we want to extract specific details from the card like name, phone number, email address, etc., we can follow a specific hierarchy to get to the elements within the line (block->line->element->element-text). Once you get to the text, you can apply your logic to get the details in a proper format (as different cards have different formats).

Below is an example that shows how to extract a phone number of the form “011–22246488” having a ‘-’ as a separator. You can check for various other test cases (i.e. when the phone number is in a different format) by applying a switch case or any other logic according to your needs. Similar logic can be applied for extracting the email address or name of the person/organization, etc.

//detector is an instane of FirebaseVisionTextRecognizer 
   .addOnSuccessListener { firebaseVisionText ->
                    // Task completed successfully

                    for (block in firebaseVisionText.textBlocks) {
                        val blockText = block.text
                        val blockFrame = block.boundingBox
                        for (line in block.lines) {
                            val lineText = line.text
                        //once you get the text elements, you can easily extract the text, email address, name mentioned 
                        //on the visiting card or any other image containing text. 
                        //Apply your logic accordingly.
                        //One sample is shown below for extracting the number of the form:- 011-22246388
                            for (element in line.elements) {
                                var elementText = element.text

                                  val split=  elementText.split('-')
                                    var part1=""
                                    var part2=""
                                   split.let {
                                        part1 = it[0]
                                        part2 = it[1]

                                    var numeric = true
                                    var num: Double = parseDouble("0")
                                    try {
                                        num = parseDouble(elementText)
                                    } catch (e: NumberFormatException) {
                                        numeric = false
                                    if (numeric)
                                        print("Phone number detected is $num")
                                        print(" Phone number is not detected on the card.")

Thanks for reading! If you enjoyed this story, please click the 👏 button and share it to help others find it!

Have feedback? Let’s connect on Twitter.


Our team has been at the forefront of Artificial Intelligence and Machine Learning research for more than 15 years and we're using our collective intelligence to help others learn, understand and grow using these new technologies in ethical and sustainable ways.

Comments 0 Responses

Leave a Reply

Your email address will not be published. Required fields are marked *

wix banner square