How to draw boundingBox around detected object?

[Mason Buchanan]

I am currently working on an object detection application (written in Java) on Android that uses Google ML Kit to detect objects within a live streamed camera preview. My issue today lies with WHY exactly the code needed to display a boundingBox around the detected object isn't appearing within my display.

The application successfully launches and I see camera preview stream is visible, the only issue is that it is unable to display boundingBox Rectangle around the detected objects, despite code being put in place to implement this. The application itself is able to detect objects, it's merely the visual display that is missing. My code displayed below states the box, and I understand that I must draw the boundingBox myself, and my question is specifically aimed at how I would go about trying to draw it. It is a single activity application:

public class MainActivity extends AppCompatActivity  {

    private ListenableFuture<ProcessCameraProvider> cameraProviderFuture;
    private class YourAnalyzer implements ImageAnalysis.Analyzer {

        @Override
        @ExperimentalGetImage
        public void analyze(ImageProxy imageProxy) {
            Image mediaImage = imageProxy.getImage();
            if (mediaImage != null) {
                //Log.d("TAG", "mediaImage is throwing null");
                InputImage image =
                        InputImage.fromMediaImage(mediaImage, imageProxy.getImageInfo().getRotationDegrees());
                //Pass image to an ML Kit Vision API
                //...

                ObjectDetectorOptions options =
                        new ObjectDetectorOptions.Builder()
                                .setDetectorMode(ObjectDetectorOptions.STREAM_MODE)
                                .enableClassification()  // Optional
                                .build();

                ObjectDetector objectDetector = ObjectDetection.getClient(options);

                objectDetector.process(image)
                        .addOnSuccessListener(detectedObjects -> {
                            Log.d("TAG", "onSuccess" + detectedObjects.size());
                            for (DetectedObject detectedObject : detectedObjects) {
                                Rect boundingBox = detectedObject.getBoundingBox();
                                Integer trackingId = detectedObject.getTrackingId();
                                for (DetectedObject.Label label : detectedObject.getLabels()) {
                                    String text = label.getText();
                                    int index = label.getIndex();
                                    float confidence = label.getConfidence();
                                }
                            }
                        })
                        .addOnFailureListener(e -> Log.e("TAG", e.getLocalizedMessage()))
                        .addOnCompleteListener(result -> imageProxy.close());
            }
        }
    }


    @Override
    protected void onCreate(@Nullable Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        cameraProviderFuture = ProcessCameraProvider.getInstance(this);

        PreviewView previewView = findViewById(R.id.previewView);

        cameraProviderFuture.addListener(() -> {
            try {
                ProcessCameraProvider cameraProvider = cameraProviderFuture.get();
                bindPreview(cameraProvider);
            } catch (ExecutionException | InterruptedException e) {
                // No errors need to be handled for this Future.
                // This should never be reached.
            }
        }, ContextCompat.getMainExecutor(this));
    }

    void bindPreview(@NonNull ProcessCameraProvider cameraProvider) {

        PreviewView previewView = findViewById(R.id.previewView);
        Preview preview = new Preview.Builder()
                .build();

        CameraSelector cameraSelector = new CameraSelector.Builder()
                .requireLensFacing(CameraSelector.LENS_FACING_BACK)
                .build();

        preview.setSurfaceProvider(previewView.getSurfaceProvider());

        ImageAnalysis imageAnalysis =
                new ImageAnalysis.Builder()
                        .setTargetResolution(new Size(1280,720))
                        .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
                        .build();
        imageAnalysis.setAnalyzer(ContextCompat.getMainExecutor(this), new YourAnalyzer());

        Camera camera = cameraProvider.bindToLifecycle((LifecycleOwner)this, cameraSelector, preview, imageAnalysis);
    }
}

Any information needed to supplement this question will be provided upon request.

[Mason Buchanan]

Here is an updated attempt; I have used this example from Google Codelabs that demonstrates how one would use a facial feature detection dependency from Google ML to draw a rectangular box around the detected face. This is the part of code that was of interest to my problem;

private void processFaceContourDetectionResult(List<Face> faces) {
   // Task completed successfully
   if (faces.size() == 0) {
       showToast("No face found");
       return;
   }
   mGraphicOverlay.clear();
   for (int i = 0; i < faces.size(); ++i) {
       Face face = faces.get(i);
       FaceContourGraphic faceGraphic = new FaceContourGraphic(mGraphicOverlay);
       mGraphicOverlay.add(faceGraphic);
       faceGraphic.updateFace(face);
   }
}

I had attempted to use it as reference to draw a box around detected objects in the same way only to no avail. However, I have learned that I must draw the graphic overlay in order for the results to be displayed on the application. Perhaps I should have reworded this question to "How to draw overlay using Google ML?"

[Mason Buchanan]

If I was using java.awt.Image, I'd create a createGraphics() image from Graphics2D object and then draw on that. However, this is not the case. I'm using a android.media.Image, and so I'm unaware on how to draw a graphic overlay on my input image. If anybody could provide any assistance, it would be much appreciated.

[Mason Buchanan]

Another update; I have added the following code within the onSuccess method using this StackOverflow answer in order to attempt to draw an overlay within my application display, and wonder why this attempt has not been successful?

.addOnSuccessListener(detectedObjects -> {
                            Log.d("TAG", "onSuccess" + detectedObjects.size());
                            for (DetectedObject detectedObject : detectedObjects) {
                                Rect boundingBox = detectedObject.getBoundingBox();
                                int left = boundingBox.left;
                                int top = boundingBox.top;
                                int right = boundingBox.right;
                                int bottom = boundingBox.bottom;

                                ShapeDrawable drawable = new ShapeDrawable();
                                drawable.getPaint().setColor(Color.RED);
                                drawable.getPaint().setStyle(Paint.Style.STROKE);
                                drawable.getPaint().setStrokeWidth(5f);
                                
                                Integer trackingId = detectedObject.getTrackingId();
                                for (DetectedObject.Label label : detectedObject.getLabels()) {
                                    String text = label.getText();
                                    int index = label.getIndex();
                                    float confidence = label.getConfidence();
                                }
                            }
                        })