Setting up OpenCV for Android without an IDE
Getting OpenCV running on Android without an IDE like Android Studion or Eclipse is actually very simple. It is just underdocumented. So here it is.
Installing the SDKs
If you haven’t already done so, install the Android SDK and make sure the ‘/tools’ directory in the SDK directory is on the system path. Now download the OpenCV Android SDK and unpack it in your workspace.
$ unzip OpenCV-2.4.x-android-sdk.zip
Another tool you need to install in order to be able to build from the command
line is ant
.
The android
tool
The android
script, which can be found in <android-sdk-dir>/tools
directory,
is the primary tool we’ll use to manage our Android project from the command
line. You can read more about it here.
To use this and other SDK tools, it is convenient to add the
<android-sdk-dir>/tools
directory on the system path.
Creating a new Android app project
To create a new project using the android
tool, do
$ android create project --target 1 --name myproject\
--activity MainActivity --path myproject --package com.myproject
Note here the value taken by the --target
option. It specifies the API level
to build your project against. To see a list of available targets on your
system, do
$ android list
And pick a target number. You can always install new target runtime APIs and other supporting libraries using the SDK manager, which you can invoke like so:
$ android
Just to verify that everything is working fine, connect an Android device to your dev machine, and compile and install the project, like so:
$ ant debug && adb install -r bin/myproject-debug.apk
The app (called “MainActivity”) should now have been installed on your device.
Setting up the OpenCV for Android library project
To properly initialize the OpenCV for Android SDK to use your target runtimes,
go to the SDK library, which in my case at this point is
../OpenCV-2.4.x-android-sdk/sdk/java
and do
$ android update lib-project --target 1 --path .
Referencing the OpenCV for Android library
Returning to our workspace, we should now have an Android app stub in
./myproject
. Now we need to make this project reference the OpenCV Android
SDK. Do this like so
$ cd myproject
$ android update project --path . --library ../OpenCV-2.4.x-android-sdk/sdk/java
Requesting the permission to use the hardware camera
Change your AndroidManifest.xml
to look like this:
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.myproject"
android:versionCode="1"
android:versionName="1.0">
<application android:label="@string/app_name" android:icon="@drawable/ic_launcher">
<activity android:name="MainActivity"
android:label="@string/app_name"
android:theme="@android:style/Theme.NoTitleBar.Fullscreen"
android:screenOrientation="landscape"
android:configChanges="keyboardHidden|orientation">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
</application>
<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera"
android:required="false"/>
<uses-feature android:name="android.hardware.camera.autofocus"
android:required="false"/>
<uses-feature android:name="android.hardware.camera.front"
android:required="false"/>
<uses-feature android:name="android.hardware.camera.front.autofocus"
android:required="false"/>
</manifest>
The only change here is the addition of the permission declarations near the
end and two attributes in the <activity>
tag that are generally used in
most camera applications.
Adding the camera view to the layout
Change your res/layout/main.xml
to look like this:
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout
xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools"
xmlns:opencv="http://schemas.android.com/apk/res-auto"
android:layout_width="match_parent"
android:layout_height="match_parent" >
<org.opencv.android.JavaCameraView
android:layout_width="fill_parent"
android:layout_height="fill_parent"
android:visibility="gone"
android:id="@+id/cameraview"
opencv:show_fps="true"
opencv:camera_id="any" />
</LinearLayout>
This uses a SurfaceView
provided by the OpenCV SDK, called the
JavaCameraView
. This class handles the drawing of the preview frames and
delivering them to our application for realtime processing.
Camera initialization
OpenCV is a native library and has Java wrappers for Android. Because of this,
we need to explicitly load the library before we attempt to use it. Note that
failing to do so will cause your app to crash with an UnsatisfiedLinkError
.
Here’s the code for src/com/myproject/MainActivity.java
. We’ll walk through
it in a while.
package com.myproject;
import org.opencv.android.*;
import org.opencv.core.*;
import org.opencv.imgproc.*;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewListener2;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewFrame;
import android.app.Activity;
import android.os.Bundle;
import android.view.SurfaceView;
import android.util.Log;
public class MainActivity extends Activity implements CvCameraViewListener2
{
private CameraBridgeViewBase cameraView = null;
private static final String tag = "myproject";
// This is called when OpenCV is loaded or when there is an error
// while loading OpenCV.
private BaseLoaderCallback opencvLoaderCallback = new BaseLoaderCallback(this) {
@Override
public void onManagerConnected(int status) {
switch (status) {
case LoaderCallbackInterface.SUCCESS:
{
Log.d(tag, "Loaded OpenCV successfully");
// Init camera and start preview.
cameraView.enableView();
} break;
default:
super.onManagerConnected(status);
}
}
};
@Override
public void onCreate(Bundle savedInstanceState)
{
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
cameraView = (CameraBridgeViewBase) findViewById(R.id.cameraview);
cameraView.setVisibility(SurfaceView.VISIBLE);
cameraView.setCvCameraViewListener(this);
}
@Override
public void onResume() {
super.onResume();
// Load the OpenCV library asynchronously. The callback object
// opencvLoaderCallback will be notified when this is complete.
OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_3, this, opencvLoaderCallback);
}
@Override
public void onPause() {
super.onPause();
// Release the camera.
if (cameraView != null) {
cameraView.disableView();
cameraView = null;
}
}
// Begin implementation of CvCameraViewListener2
// This method is called for every preview frame. It should return a
// Mat object representing the frame to be showed to the user.
@Override
public Mat onCameraFrame(CvCameraViewFrame frame) {
return frame.gray();
}
@Override
public void onCameraViewStarted(int width, int height) {
}
@Override
public void onCameraViewStopped() {
}
// End implementation of CvCameraViewListener2
}
Compile and install this app like so
$ ant debug && adb install -r bin/myproject-debug.apk
When you open the app first time, you may see a dialog that asks your permission to install the OpenCV manager app on the phone. Install that app and run MainActivity. You should now see a grayscale preview with an FPS count.
Code walkthrough
Most of the code is really easy to understand. Here’s the typical flow of events when the app is started.
onCreate()
is called, which sets up the layout and initializes local variables.onResume()
is called, which requests for an async loading of OpenCV for Android. This is accomplished via theOpenCVLoader.initAsync()
call, which, among other arguments, takes aBaseLoaderCallback
object which is notified when loading is complete. In our implementation of the callback, we just enable the preview.onPause()
is called whenever the app is backgrounded by the user. We use this to release the camera. This is important because if we do not release the camera properly, no other app, including our own app can subsequently open the camera.onCameraFrame()
, a part of theCvCameraViewListener2
interface, is called on every preview frame. It is the job of this method to process each frame and return a frame as aMat
object which will then be displayed to the user as preview. We are just returning a grayscale version of each frame now, which is available viaCvCameraViewFrame.gray()
. There is alsoCvCameraView.rgba()
, which returns aMat
object in the RGBA format. For more information, refer to the Javadocs for OpenCV for Android.
Canny edge detection
As a last demonstration of how processed frames are drawn on the preview, we’ll
modify onCameraFrame()
to detect edges in every frame.
@Override
public Mat onCameraFrame(CvCameraViewFrame frame) {
Mat gray = frame.gray();
Imgproc.Canny(gray, gray, 80, 100, 3, true);
return gray;
}
Here, the org.opencv.imgproc.Imgproc.Canny()
routine is used to detect
edges and the resultant image with just edge information is returned, which
is then drawn as preview.
Final words
Some people can’t imagine writing Android apps without an IDE. I can’t imagine struggling with bloat just to try something out quickly. There are some excellent plugins for both vim and emacs that make writing Java bearable without the bloated, slow and bug ridden IDEs.