Face Recognition With ML Kit in Jetpack Compose

Introduction

The incorporation of machine learning (ML) capabilities has increased the speed and intelligence of applications created for current Android devices. User experiences are improved and several chores are automated by this integration. Face recognition, a technology that allows apps to recognize faces within photos, is one excellent example. In this article, we'll explore the field of facial recognition using the Jetpack Compose framework and the ML Kit.

Understanding ML Kit

When we face tough jobs like changing words from one language to another, finding faces, or recognizing voices on Android devices, we can make things simpler by using special computer programs called machine learning models. These models are a subset of a bigger thing called AI.

To bring this machine-learning magic to Mobile devices, we use something called the ML Kit. Think of it as a special toolkit for phones. It helps apps talk to the machine-learning models and brings Google's smart machine-learning skills to both Android and iOS apps. It's like a bridge connecting the app and the clever models that understand things like languages and faces.

Adding ML Kit to a mobile device is quite simple. ML Kit offers a handy way to tackle challenging tasks like spotting objects, understanding gestures, reading text, classifying sounds, recognizing speech, suggesting words, giving clever responses, and much more. We can even use third-party ML Kits like TensorFlow Lite or build our very own machine-learning models. But for now, let's stick with Google's ML Kit and create an app that recognizes text.

Face Recognition with ML Kit

Extracting faces from photographs is a component of face recognition that involves the automatic identification and verification of individuals based on their unique facial features. This technology has rapidly evolved and found its way into various applications.

Integrating ML Kit Face Recognition with Jetpack Compose

To integrate ML Kit's Face recognition capabilities with Jetpack Compose, follow these steps:

Step 1. Add Dependencies

Add the necessary dependencies to your app's build.gradle file.

// Face features
implementation 'com.google.mlkit:face-detection:16.1.5'
implementation("io.coil-kt:coil-compose:2.3.0")
implementation "androidx.lifecycle:lifecycle-viewmodel-compose:2.6.1"

Step 2. Set Up the Photo Picker

In order to simplify this application, we'll establish a photo picker. Of course, you have the option to use the camera to select the photo.

For now, let's configure the photo picker using the following code-

var imageUri: Any? by remember { mutableStateOf(R.drawable.img) }

val photoPicker = rememberLauncherForActivityResult(
    contract = ActivityResultContracts.PickVisualMedia()
) {
    if (it != null) {
        Log.d("PhotoPicker", "Selected URI: $it")
        imageUri = it
    } else {
        Log.d("PhotoPicker", "No media selected")
    }
}


Column(
    modifier = Modifier.fillMaxSize(),
    horizontalAlignment = Alignment.CenterHorizontally,
    verticalArrangement = Arrangement.Center
) {
    AsyncImage(
        modifier = Modifier
            .size(250.dp)
            .clickable {
                photoPicker.launch(
                    PickVisualMediaRequest(
                        ActivityResultContracts.PickVisualMedia.ImageOnly
                    )
                )
            },
        model = ImageRequest.Builder(LocalContext.current).data(imageUri)
            .crossfade(enable = true).build(),
        contentDescription = "Avatar Image",
        contentScale = ContentScale.Crop,
    )

    Spacer(modifier = Modifier.height(24.dp))

    // Coming Up Next
}

If you're looking for more details on how we're configuring this setup, I've already written an article on the process of selecting images from the gallery without needing special permissions. Feel free to check out the article by clicking here.

Step 3. Perform Face Recognition

To monitor the data, we'll generate a data class called MainScreenState. This class will be stored within our view model to endure configuration changes. The structure of the class will resemble the following-

data class MainScreenState(
    val extractedText :String = "Not detected yet..",
    val isButtonEnabled:Boolean = true
)

Next, we'll generate a view model responsible for housing all the face recognition logic. The methods within this view model will be triggered upon clicking the image composable.

fun detectFace(
        text: Any?, context: Context
    ) {
        var image: InputImage? = null
        try {
            image = InputImage.fromFilePath(context, text as Uri)
        } catch (e: IOException) {
            e.printStackTrace()
        }

        val highAccuracyOpts = FaceDetectorOptions.Builder()
            .setPerformanceMode(FaceDetectorOptions.PERFORMANCE_MODE_ACCURATE)
            .setLandmarkMode(FaceDetectorOptions.LANDMARK_MODE_ALL)
            .setClassificationMode(FaceDetectorOptions.CLASSIFICATION_MODE_ALL).build()


        val detector = FaceDetection.getClient(highAccuracyOpts)

        val result = image?.let {
            detector.process(it).addOnSuccessListener { faces ->

                onTextToBeTranslatedChange("${faces.size} faces detected")

            }.addOnFailureListener { e ->
                // Task failed with an exception
                // ...
                Toast.makeText(
                    context, "Some error caught", Toast.LENGTH_SHORT
                ).show()
            }
        }

    }

The FaceDetection takes InputImage as a parameter; for this, we will convert our image URI to InputImage and add this inside a try-catch block. Now we process FaceDetection and add a Success and Failure listener.

Step 4. Display Detected Faces

To display the detected faces, we will create a function that will trigger the state of the app, like the one below.

fun onImageRecorganized(text: String) {
    _state.value = state.value.copy(
        extractedText = text
    )
}

To set this to screen, we will use the state of the ViewModel and display it in a row composable like here.

@Composable
fun ImagePickerScreen(
    viewModel: MainViewModel = androidx.lifecycle.viewmodel.compose.viewModel()
) {
    val state = viewModel.state.value
    val context = LocalContext.current

        val photoPicker = rememberLauncherForActivityResult(
        contract = ActivityResultContracts.PickVisualMedia()
    ) {
        if (it != null) {
            Log.d("PhotoPicker", "Selected URI: $it")
            imageUri = it
            viewModel.detectFace(imageUri, context = context)
        } else {
            Log.d("PhotoPicker", "No media selected")
        }
    }


    Column(
        modifier = Modifier.fillMaxSize(),
        horizontalAlignment = Alignment.CenterHorizontally,
        verticalArrangement = Arrangement.Center
    ) {
        //existing code

        val scrollState = rememberScrollState()
        Row(modifier = Modifier.fillMaxWidth().verticalScroll(scrollState)) {
            Text(
                text = state.extractedText,
                textAlign = TextAlign.Center,
                modifier = Modifier.fillMaxWidth()
            )
        }
    }
}

To check out the application download the zip file link in the article.

Output

Conclusion

The functionality and user experience of your Android app can be greatly improved by integrating face recognition features utilizing ML Kit within Jetpack Compose applications. Developers can easily construct applications that extract faces from photos using the potent combination of ML Kit's pre-trained models and Jetpack Compose's declarative UI approach, opening up opportunities for creative and intelligent solutions across multiple domains. Keep in mind that for more detailed instructions and best practices, consult the official documentation of ML Kit and Jetpack Compose.