Tuesday, July 10, 2012

YUV420 to bitmap conversion in Android

There are a couple methods to retrive image data from camera.  You could start built-in camera app from your activity, or you could call the camera API to take a picture and return it to your activity as a JPG.  However, sometimes you don't need a full-resolution image. In that case, you can just grab the preview image from the camera preview screen.  You can do that by calling Camera.setOneShotPreviewCallback(f) where f is a callback function Camera.PreviewCallback().  Soon after the function is executed, your callback functions is called and passed to it the image data.



mCameraDevice.setOneShotPreviewCallback(
    new Camera.PreviewCallback() {
        @Override
        public void onPreviewFrame(byte[] data, Camera camera) {
            // do something is image data
        }
});

However, the image data is in YUV420 format.  The API actually lets you specify other formats, but most phones only support YUV420.

Support of YUV420 in Android is limited.  You will have to convert it to bitmap first before you can do something useful with the image.  The class YuvImage lets you convert YUV420 data to JPG, but not much else, but you can convert JPG to bitmap like this.

android.hardware.Camera.Parameters param;
int format = param.getPreviewFormat();
YuvImage image = new YuvImage(data, format, width, height, null);
ByteArrayOutputStream out = new ByteArrayOutputStream();
image.compressToJpeg(rect, 100, out);
Bitmap bmp = BitmapFactory.decodeByteArray(out.toByteArray(), 
                                           0, bs.length);
Obviously, this is not the best way to go around.  It is reasonably fast (the conversion code is JNI-wrapped), but the image quality suffers.  I found the resulted images lost contrast and color is dimmed.   You can see the difference in the image below (left is the result obtained by converting YUV420 -> JPG -> bitmap, right is when decoding YUV420 directly to bitmap, more about this later)
Another method is convert YUV420 to bitmap directly.  There are code out here, like one in this post at stackoverflow, which works well.

public int[] decodeYUV420SP( byte[] yuv420sp, int width, int height) {
       final int frameSize = width * height;
       int rgb[]=new int[width*height];
       for (int j = 0, yp = 0; j < height; j++) {
           int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
           for (int i = 0; i < width; i++, yp++) {
               int y = (0xff & ((int) yuv420sp[yp])) - 16;
               if (y < 0) y = 0;
               if ((i & 1) == 0) {
                   v = (0xff & yuv420sp[uvp++]) - 128;
                   u = (0xff & yuv420sp[uvp++]) - 128;
               }
               int y1192 = 1192 * y;
               int r = (y1192 + 1634 * v);
               int g = (y1192 - 833 * v - 400 * u);
               int b = (y1192 + 2066 * u);
               if (r < 0) r = 0; else if (r > 262143) r = 262143;
               if (g < 0) g = 0; else if (g > 262143) g = 262143;
               if (b < 0) b = 0; else if (b > 262143) b = 262143;
               rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000)
                       | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
           }
       }
       return rgb;   }
This funtion returns an array of RGB image data that you can construct the bitmap out of it like this Bitmap bmpx = Bitmap.createBitmap(rgb, width, height, Bitmap.Config.ARGB_8888); However, this function is quite slow.  Given that my program only need a small portion at the center of image, it is wasteful to decode the whole image only to crop most of it out later.  So I modified the function to accept the rectangle bounding the area I want decoded.  This is the modified function.

public int[] decodeYUV420SP(byte[] yuv420sp, int width, 
                            int height, Rect rect) {
     final int frameSize = width * height;
     int ow = rect.right - rect.left;
     int oh = rect.bottom - rect.top;
     int rgb[] = new int[ow * oh];
     for (int j = 0, yp = 0, ic = 0; j < height; j++) {
         if (j <= rect.top) {
             yp += width;
             continue;
         }
         if (j > rect.bottom)
             break;
         int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
         for (int i = 0; i < width; i++, yp++) {
             int y = (0xff & ((int) yuv420sp[yp])) - 16;
             if (y < 0)
                 y = 0;
             if ((i & 1) == 0) {
                 v = (0xff & yuv420sp[uvp++]) - 128;
                 u = (0xff & yuv420sp[uvp++]) - 128;
             }
             if (i <= rect.left || i > rect.right)
                 continue;
             int y1192 = 1192 * y;
             int r = (y1192 + 1634 * v);
             int g = (y1192 - 833 * v - 400 * u);
             int b = (y1192 + 2066 * u);

             if (r < 0) r = 0; else if (r > 262143) r = 262143;
             if (g < 0) g = 0; else if (g > 262143) g = 262143;
             if (b < 0) b = 0; else if (b > 262143) b = 262143;

             int rx = 0xff000000 | ((r << 6) & 0xff0000)
                    | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
             rgb[ic] = rx;
             ic++;
         }
     }
     return rgb; } 
 With this code,  my app (English->Thai OCR dictionary) can decode the YUV420 in reasonable time and still have better image quality.

No comments:

Post a Comment