Swift has been around for a while, and over time, it has introduced features found in contemporary object-oriented programming languages. These include optionals, generics, tuples, structs that can have methods, extensions and protocols, and many others. However, if your application depends on a C++ library, you won’t be able to rely solely on Swift. Fortunately, Objective-C++ can help us bridge this gap.

Since its inception, Swift has seamlessly interacted with Objective-C, and we’ll utilize Objective-C as a bridge between Swift and C++. Essentially, Objective-C++ is Objective-C with the ability to link with C++ code. Using this, we’ll develop a basic application that uses OpenCV to identify the Toptal logo within an image. Upon detecting the logo, the application will open the Toptal homepage.

As stated on the OpenCV web page:

“OpenCV was designed for computational efficiency and with a strong focus on real-time applications. Written in optimized C/C++, the library can take advantage of multi-core processing.”

This makes it an excellent choice for building fast and dependable computer vision applications.

Building the OpenCV Objective-C++ Wrapper

This tutorial will guide you through designing an application that identifies the Toptal logo within an image and opens the Toptal web page. Start by creating a new Xcode project and set up CocoaPods by running pod init. Add OpenCV to your Podfile using pod 'OpenCV' and then run pod install in Terminal. Remember to uncomment the use_frameworks! line in your Podfile.

Now that OpenCV is integrated into your Xcode project, the next step is to connect it with Swift. Here’s a concise outline of the process:

Step 1: Create a new Objective-C class named OpenCVWrapper. When Xcode prompts you with “Would you like to configure an Objective-C bridging header?”, select “Create bridging header.” This bridging header acts as a link, allowing you to import Objective-C classes and make them accessible within your Swift code.

Step 2: To enable C++ usage within Objective-C, change the file extension from OpenCVWrapper.m to OpenCVWrapper.mm. You can do this directly within Xcode’s project navigator. The .mm extension signifies a change from Objective-C to Objective-C++.

Step 3: Import OpenCV into OpenCVWrapper.mm using the import statement provided below. It’s crucial to place this import above #import "OpenCVWrapper.h" to avoid a known BOOL conflict. OpenCV includes an enum with a value NO, which clashes with Objective-C’s BOOL NO value. As long as you don’t require classes that utilize this enum, this is the most straightforward solution. If you do, you’ll need to undefine BOOL before importing OpenCV.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

#ifdef __cplusplus 

#import <opencv2/opencv.hpp> 

#import <opencv2/imgcodecs/ios.h> 

#import <opencv2/videoio/cap_ios.h> 

#endif

Step 4: Add #import "OpenCVWrapper.h" to your bridging header.

Open OpenCVWrapper.mm and define a private interface to declare private properties:

1
2
3
4
5
6
7
8

@interface OpenCVWrapper() <CvVideoCameraDelegate>

@property (strong, nonatomic) CvVideoCamera *videoCamera;

@property (assign, nonatomic) cv::Mat logoSample;

@end

To instantiate CvVideoCamera, you need to pass a UIImageView to it. This will be done through a designated initializer.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

- (instancetype)initWithParentView:(UIImageView *)parentView delegate:(id<OpenCVWrapperDelegate>)delegate {

    

    if (self = [super init]) {

        self.delegate = delegate;

        

        parentView.contentMode = UIViewContentModeScaleAspectFill;

        

        self.videoCamera = [[CvVideoCamera alloc] initWithParentView:parentView];

        self.videoCamera.defaultAVCaptureDevicePosition = AVCaptureDevicePositionBack;

        self.videoCamera.defaultAVCaptureSessionPreset = AVCaptureSessionPresetHigh;

        self.videoCamera.defaultAVCaptureVideoOrientation = AVCaptureVideoOrientationPortrait;

        self.videoCamera.defaultFPS = 30;

        self.videoCamera.grayscaleMode = [NSNumber numberWithInt:0].boolValue;

        self.videoCamera.delegate = self;

        

        // Convert UIImage to Mat and store greyscale version

        UIImage *templateImage = [UIImage imageNamed:@"toptal"];

        cv::Mat templateMat;

        UIImageToMat(templateImage, templateMat);

        cv::Mat grayscaleMat;

        cv::cvtColor(templateMat, grayscaleMat, CV_RGB2GRAY);

        self.logoSample = grayscaleMat;

        

        [self.videoCamera start];

    }

    return self;

}

Within the initializer, configure CvVideoCamera to render video inside the provided parentView and utilize a delegate to receive the cv::Mat image for analysis.

The processImage: method originates from the CvVideoCameraDelegate protocol and will house the template matching logic.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

- (void)processImage:(cv::Mat&)image {

    

    cv::Mat gimg;

    

    // Convert incoming img to greyscale to match template

    cv::cvtColor(image, gimg, CV_BGR2GRAY);

    

    // Get matching

    cv::Mat res(image.rows-self.logoSample.rows+1, self.logoSample.cols-self.logoSample.cols+1, CV_32FC1);

    cv::matchTemplate(gimg, self.logoSample, res, CV_TM_CCOEFF_NORMED);

    cv::threshold(res, res, 0.5, 1., CV_THRESH_TOZERO);

    

    double minval, maxval, threshold = 0.9;

    cv::Point minloc, maxloc;

    cv::minMaxLoc(res, &minval, &maxval, &minloc, &maxloc);

    

    // Call delegate if match is good enough

    if (maxval >= threshold)

    {

        // Draw a rectangle for confirmation

        cv::rectangle(image, maxloc, cv::Point(maxloc.x + self.logoSample.cols, maxloc.y + self.logoSample.rows), CV_RGB(0,255,0), 2);

        cv::floodFill(res, maxloc, cv::Scalar(0), 0, cv::Scalar(.1), cv::Scalar(1.));

        

        [self.delegate openCVWrapperDidMatchImage:self];

    }

}

The provided image is first converted to grayscale to match the format of the Toptal logo template image, which was converted to grayscale during initialization. Next, we check for matches using a specific threshold to accommodate variations in scaling and angles. Considering angles is essential as the logo might be captured from different perspectives. When the set threshold is met, a delegate is triggered to open the Toptal webpage.

Ensure to include NSCameraUsageDescription in your Info.plist file to prevent the application from crashing when [self.videoCamera start]; is called.

Integrating with Swift

Up to this point, the focus has been on Objective-C++ as it handles the core logic. The Swift code is quite simple:

1. In ViewController.swift, create a UIImageView for CvVideoCamera to render its content.
2. Instantiate OpenCVWrapper, passing in the UIImageView instance.
3. Implement the OpenCVWrapperDelegate protocol to open Toptal’s webpage upon logo detection.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

class ViewController: UIViewController {

    

    var wrapper: OpenCVWrapper!

    

    @IBOutlet var imageView: UIImageView!

    override func viewDidLoad() {

        super.viewDidLoad()

        // Do any additional setup after loading the view, typically from a nib.

    

        wrapper = OpenCVWrapper.init(parentView: imageView, delegate: self)

    }

}

extension ViewController: OpenCVWrapperDelegate {

    //MARK: - OpenCVWrapperDelegate

    

    func openCVWrapperDidMatchImage(_ wrapper: OpenCVWrapper) {

        UIApplication.shared.open(URL.init(string: "https://toptal.com")!, options: [:], completionHandler: nil)

    }

}

OpenCV Swift in Action

This article illustrated how to integrate C++ code with Swift by creating wrapper classes that bridge the gap between them. Now, when the Toptal Logo is detected through the camera, the application will open the Toptal homepage.

For future enhancements, consider executing the CV template matching in a background thread. This approach prevents the main thread from being blocked, ensuring a responsive UI. Keep in mind that this is a basic OpenCV example, and template matching might not always yield perfect results. The primary goal was to demonstrate how to start utilizing OpenCV.

If you’d like to delve deeper into OpenCV and its advanced applications within iOS, check out Real-time Object Detection Using MSER in iOS. This resource provides a comprehensive guide on image detection using the iPhone’s rear camera.