Image segmentation (also knowns as semantic segmentation) refers to the process of linking each pixel in an image to a class label. These labels could include a person, car, flower, piece of furniture, etc., just to mention a few. We have covered image segmentation before here and here. In this guide, we’ll look at how this can be done in Android mobile applications.
The first step is to create an Android project. Once you do, note the application ID. Next, log into your Fritz account and register the application. This will set up communication between your application and Fritz AI.
Click on the next button and name your application. While you are there, don’t forget to enter your application ID. You can find the ID in your app’s
With that out of the way, you can now install the Fritz AI SDK. In your root-level Gradle file
(build.gradle) include the Maven repository for Fritz.
Now, add the dependencies for the SDK in
app/build.gradle. We add the Fritz Core, Image Segmentation, and Vision dependencies. Including the Image Segmentation model in your application will make your application larger in size. Now that you have changed the Gradle files, ensure that you sync that with your project. That will download all the necessary dependencies.
Before you close that file add
renderscript support to improve image processing performance. Also, specify
aaptOptions to prevent compression of TFLite models.
Now register the
FritzCustomModelService in the AndroidManifest.
The next step is to initialize the SDK by calling Fritz.configure() with your API Key.
With that in place, click next to verify that your application is able to communicate with Fritz.
Use Fritz Pre-trained Models
The model we will use will create a mask on the people that have been detected. Apart from the people segmentation, Fritz AI also allows us to do pet, sky, living room, outdoor, and hair segmentation.
The App Elements
The application is made up of a button that will choose an image and the image view for displaying the result of the segmentation.
Obtaining the User Permissions
We can obtain the image by overriding the
onClick method of the button and attaching a click listener. We also obtain permission from the user in order to select an image.
Once we obtain the necessary permission, we get the image via the
pickImage method. The method uses an
Intent to get the image.
Next, override the method used to request permission.
Obtaining the Image
At this point, we can now obtain the image and create a Bitmap. The object detection model requires a Bitmap image.
Create a FritzVisionImage from an image
Now use the Bitmap to create a
Get a Segmentation Predictor
At this point, we can now load the on-device model.
Create a Segmentation Predictor
Since the model is loaded on the device we can get the segmentation predictor immediately.
Run Prediction on the FritzVisionImage
The next step is to simply run the predictions using the predictor we just obtained.