用JavaVC替换JMF连接摄像头
浏览数：256 /
时间：2015年06月08日
JMF太老了，各种问题得不到解决，Oracle也没再升级过，如果能找到新东西，最好能把它扔掉。
最近OpenCV比较火，还有人用Java封装了OpenCV，成立了JavaCV项目，通过改造VideoInput这个基于C语言的项目，能够用Java来调用摄像头，JMF可以扔掉了。
如果想测试，非常简单，把那些编译好的jar文件放入Build Path即可，如果是在Windows X86环境下，则只需要把带Window和x86的包，以及不带有任何平台信息包放到Build Path即可。测试的程序可以用项目主页上面那个Demo。代码如下：
不过这个代码包含了人脸识别，而人脸识别需要下载一个xml文件来定义识别的模式，代码中那个xml链接下载下来的xml文件是会解析出错的，可能是那个版本比较新，JavaCV还没跟上，所以解析不了。经过搜索，发现有人提供了一个老版本的xml文件，能够顺利解析，会在视频中把人脸用绿框框住。
所以，把代码中的
运行后发现很多线条和框框，画面也是斜的，如下图所示：
这是用来演示JavaCV对图像的处理，如果对这些没有兴趣，可以简单地用以下代码来显示摄像头的图像：
import org.bytedeco.javacpp.opencv_core.IplImage;
import org.bytedeco.javacv.CanvasFrame;
import org.bytedeco.javacv.FrameGrabber;
public class Demo3 {
public static void main(String[] args) throws Exception {
// The available FrameGrabber classes include OpenCVFrameGrabber (opencv_highgui),
// DC1394FrameGrabber, FlyCaptureFrameGrabber, OpenKinectFrameGrabber,
// PS3EyeFrameGrabber, VideoInputFrameGrabber, and FFmpegFrameGrabber.
FrameGrabber grabber = FrameGrabber.createDefault(1);
grabber.start();
// FAQ about IplImage:
// - For custom raw processing of data, createBuffer() returns an NIO direct
// buffer wrapped around the memory pointed by imageData, and under Android we can
// also use that Buffer with Bitmap.copyPixelsFromBuffer() and copyPixelsToBuffer().
// - To get a BufferedImage from an IplImage, we may call getBufferedImage().
// - The createFrom() factory method can construct an IplImage from a BufferedImage.
// - There are also a few copy*() methods for BufferedImage<->IplImage data transfers.
IplImage grabbedImage = grabber.grab();
CanvasFrame frame = new CanvasFrame("Some Title", CanvasFrame.getDefaultGamma()/grabber.getGamma());
while (frame.isVisible() && (grabbedImage = grabber.grab()) != null) {
frame.showImage(grabbedImage);
}
frame.dispose();
grabber.stop();
}
}
如下图所示：
当然，这样只是把摄像头的图像显示出来，没有任何用处，所以如果你想要处理其中的图像，就需要增加代码。例如，如果你需要截取摄像头的图像做进一步处理，就可以通过IplImage的getBufferedImage()方法来得到BufferedImage，然后再处理这个BufferedImage，例如解析二维码。
不过，JavaCV默认是把图像显示在一个CanvasFrame对象中，这个对象继承了JFrame，所以只能用作顶层窗口，不能嵌入到其他窗口中。为了能够把它嵌入到其他窗口，可以修改一下代码，继承一个Canvas对象：
这个ImageCanvas使用了双缓冲机制来显示视频（实际上是不间断地显示图像），避免图像闪烁。在此之前，尝试了网上找到的双缓冲代码，效果都不理想，只有这个能够不闪烁。
不过用了这个控件来显示视频，就不能全屏了。
加入控件后，显示图像只需要调用setImg(BufferedImage img)函数即可。也就是把
canvas.setImg(grabbedImage.getBufferedImage());
最近OpenCV比较火，还有人用Java封装了OpenCV，成立了JavaCV项目，通过改造VideoInput这个基于C语言的项目，能够用Java来调用摄像头，JMF可以扔掉了。
如果想测试，非常简单，把那些编译好的jar文件放入Build Path即可，如果是在Windows X86环境下，则只需要把带Window和x86的包，以及不带有任何平台信息包放到Build Path即可。测试的程序可以用项目主页上面那个Demo。代码如下：
import java.io.File;
import java.net.URL;
import org.bytedeco.javacv.*;
import org.bytedeco.javacpp.*;
import org.bytedeco.javacpp.indexer.*;
import static org.bytedeco.javacpp.opencv_core.*;
import static org.bytedeco.javacpp.opencv_imgproc.*;
import static org.bytedeco.javacpp.opencv_calib3d.*;
import static org.bytedeco.javacpp.opencv_objdetect.*;
public class Demo {
public static void main(String[] args) throws Exception {
String classifierName = null;
if (args.length > 0) {
classifierName = args[0];
} else {
URL url = new URL("https://raw.github.com/Itseez/opencv/2.4/data/haarcascades/haarcascade_frontalface_alt.xml");
File file = Loader.extractResource(url, null, "classifier", ".xml");
file.deleteOnExit();
classifierName = file.getAbsolutePath();
}
// Preload the opencv_objdetect module to work around a known bug.
Loader.load(opencv_objdetect.class);
// We can "cast" Pointer objects by instantiating a new object of the desired class.
CvHaarClassifierCascade classifier = new CvHaarClassifierCascade(cvLoad(classifierName));
if (classifier.isNull()) {
System.err.println("Error loading classifier file \"" + classifierName + "\".");
System.exit(1);
}
// The available FrameGrabber classes include OpenCVFrameGrabber (opencv_highgui),
// DC1394FrameGrabber, FlyCaptureFrameGrabber, OpenKinectFrameGrabber,
// PS3EyeFrameGrabber, VideoInputFrameGrabber, and FFmpegFrameGrabber.
FrameGrabber grabber = FrameGrabber.createDefault(0);
grabber.start();
// FAQ about IplImage:
// - For custom raw processing of data, createBuffer() returns an NIO direct
// buffer wrapped around the memory pointed by imageData, and under Android we can
// also use that Buffer with Bitmap.copyPixelsFromBuffer() and copyPixelsToBuffer().
// - To get a BufferedImage from an IplImage, we may call getBufferedImage().
// - The createFrom() factory method can construct an IplImage from a BufferedImage.
// - There are also a few copy*() methods for BufferedImage<->IplImage data transfers.
IplImage grabbedImage = grabber.grab();
int width = grabbedImage.width();
int height = grabbedImage.height();
IplImage grayImage = IplImage.create(width, height, IPL_DEPTH_8U, 1);
IplImage rotatedImage = grabbedImage.clone();
// Objects allocated with a create*() or clone() factory method are automatically released
// by the garbage collector, but may still be explicitly released by calling release().
// You shall NOT call cvReleaseImage(), cvReleaseMemStorage(), etc. on objects allocated this way.
CvMemStorage storage = CvMemStorage.create();
// The OpenCVFrameRecorder class simply uses the CvVideoWriter of opencv_highgui,
// but FFmpegFrameRecorder also exists as a more versatile alternative.
FrameRecorder recorder = FrameRecorder.createDefault("output.avi", width, height);
recorder.start();
// CanvasFrame is a JFrame containing a Canvas component, which is hardware accelerated.
// It can also switch into full-screen mode when called with a screenNumber.
// We should also specify the relative monitor/camera response for proper gamma correction.
CanvasFrame frame = new CanvasFrame("Some Title", CanvasFrame.getDefaultGamma()/grabber.getGamma());
// Let‘s create some random 3D rotation
CvMat randomR = CvMat.create(3, 3), randomAxis = CvMat.create(3, 1);
// We can easily and efficiently access the elements of matrices and images
// through an Indexer object with the set of get() and put() methods.
DoubleIndexer Ridx = randomR.createIndexer(), axisIdx = randomAxis.createIndexer();
axisIdx.put(0, (Math.random()-0.5)/4, (Math.random()-0.5)/4, (Math.random()-0.5)/4);
cvRodrigues2(randomAxis, randomR, null);
double f = (width + height)/2.0; Ridx.put(0, 2, Ridx.get(0, 2)*f);
Ridx.put(1, 2, Ridx.get(1, 2)*f);
Ridx.put(2, 0, Ridx.get(2, 0)/f); Ridx.put(2, 1, Ridx.get(2, 1)/f);
System.out.println(Ridx);
// We can allocate native arrays using constructors taking an integer as argument.
CvPoint hatPoints = new CvPoint(3);
while (frame.isVisible() && (grabbedImage = grabber.grab()) != null) {
cvClearMemStorage(storage);
// Let‘s try to detect some faces! but we need a grayscale image
cvCvtColor(grabbedImage, grayImage, CV_BGR2GRAY);
CvSeq faces = cvHaarDetectObjects(grayImage, classifier, storage,
1.1, 3, CV_HAAR_DO_CANNY_PRUNING);
int total = faces.total();
for (int i = 0; i < total; i++) {
CvRect r = new CvRect(cvGetSeqElem(faces, i));
int x = r.x(), y = r.y(), w = r.width(), h = r.height();
cvRectangle(grabbedImage, cvPoint(x, y), cvPoint(x+w, y+h), CvScalar.RED, 1, CV_AA, 0);
// To access or pass as argument the elements of a native array, call position() before.
hatPoints.position(0).x(x-w/10) .y(y-h/10);
hatPoints.position(1).x(x+w*11/10).y(y-h/10);
hatPoints.position(2).x(x+w/2) .y(y-h/2);
cvFillConvexPoly(grabbedImage, hatPoints.position(0), 3, CvScalar.GREEN, CV_AA, 0);
}
// Let‘s find some contours! but first some thresholding
cvThreshold(grayImage, grayImage, 64, 255, CV_THRESH_BINARY);
// To check if an output argument is null we may call either isNull() or equals(null).
CvSeq contour = new CvSeq(null);
cvFindContours(grayImage, storage, contour, Loader.sizeof(CvContour.class),
CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE);
while (contour != null && !contour.isNull()) {
if (contour.elem_size() > 0) {
CvSeq points = cvApproxPoly(contour, Loader.sizeof(CvContour.class),
storage, CV_POLY_APPROX_DP, cvContourPerimeter(contour)*0.02, 0);
cvDrawContours(grabbedImage, points, CvScalar.BLUE, CvScalar.BLUE, -1, 1, CV_AA);
}
contour = contour.h_next();
}
cvWarpPerspective(grabbedImage, rotatedImage, randomR);
frame.showImage(rotatedImage);
recorder.record(rotatedImage);
}
frame.dispose();
recorder.stop();
grabber.stop();
}
}
import java.net.URL;
import org.bytedeco.javacv.*;
import org.bytedeco.javacpp.*;
import org.bytedeco.javacpp.indexer.*;
import static org.bytedeco.javacpp.opencv_core.*;
import static org.bytedeco.javacpp.opencv_imgproc.*;
import static org.bytedeco.javacpp.opencv_calib3d.*;
import static org.bytedeco.javacpp.opencv_objdetect.*;
public class Demo {
public static void main(String[] args) throws Exception {
String classifierName = null;
if (args.length > 0) {
classifierName = args[0];
} else {
URL url = new URL("https://raw.github.com/Itseez/opencv/2.4/data/haarcascades/haarcascade_frontalface_alt.xml");
File file = Loader.extractResource(url, null, "classifier", ".xml");
file.deleteOnExit();
classifierName = file.getAbsolutePath();
}
// Preload the opencv_objdetect module to work around a known bug.
Loader.load(opencv_objdetect.class);
// We can "cast" Pointer objects by instantiating a new object of the desired class.
CvHaarClassifierCascade classifier = new CvHaarClassifierCascade(cvLoad(classifierName));
if (classifier.isNull()) {
System.err.println("Error loading classifier file \"" + classifierName + "\".");
System.exit(1);
}
// The available FrameGrabber classes include OpenCVFrameGrabber (opencv_highgui),
// DC1394FrameGrabber, FlyCaptureFrameGrabber, OpenKinectFrameGrabber,
// PS3EyeFrameGrabber, VideoInputFrameGrabber, and FFmpegFrameGrabber.
FrameGrabber grabber = FrameGrabber.createDefault(0);
grabber.start();
// FAQ about IplImage:
// - For custom raw processing of data, createBuffer() returns an NIO direct
// buffer wrapped around the memory pointed by imageData, and under Android we can
// also use that Buffer with Bitmap.copyPixelsFromBuffer() and copyPixelsToBuffer().
// - To get a BufferedImage from an IplImage, we may call getBufferedImage().
// - The createFrom() factory method can construct an IplImage from a BufferedImage.
// - There are also a few copy*() methods for BufferedImage<->IplImage data transfers.
IplImage grabbedImage = grabber.grab();
int width = grabbedImage.width();
int height = grabbedImage.height();
IplImage grayImage = IplImage.create(width, height, IPL_DEPTH_8U, 1);
IplImage rotatedImage = grabbedImage.clone();
// Objects allocated with a create*() or clone() factory method are automatically released
// by the garbage collector, but may still be explicitly released by calling release().
// You shall NOT call cvReleaseImage(), cvReleaseMemStorage(), etc. on objects allocated this way.
CvMemStorage storage = CvMemStorage.create();
// The OpenCVFrameRecorder class simply uses the CvVideoWriter of opencv_highgui,
// but FFmpegFrameRecorder also exists as a more versatile alternative.
FrameRecorder recorder = FrameRecorder.createDefault("output.avi", width, height);
recorder.start();
// CanvasFrame is a JFrame containing a Canvas component, which is hardware accelerated.
// It can also switch into full-screen mode when called with a screenNumber.
// We should also specify the relative monitor/camera response for proper gamma correction.
CanvasFrame frame = new CanvasFrame("Some Title", CanvasFrame.getDefaultGamma()/grabber.getGamma());
// Let‘s create some random 3D rotation
CvMat randomR = CvMat.create(3, 3), randomAxis = CvMat.create(3, 1);
// We can easily and efficiently access the elements of matrices and images
// through an Indexer object with the set of get() and put() methods.
DoubleIndexer Ridx = randomR.createIndexer(), axisIdx = randomAxis.createIndexer();
axisIdx.put(0, (Math.random()-0.5)/4, (Math.random()-0.5)/4, (Math.random()-0.5)/4);
cvRodrigues2(randomAxis, randomR, null);
double f = (width + height)/2.0; Ridx.put(0, 2, Ridx.get(0, 2)*f);
Ridx.put(1, 2, Ridx.get(1, 2)*f);
Ridx.put(2, 0, Ridx.get(2, 0)/f); Ridx.put(2, 1, Ridx.get(2, 1)/f);
System.out.println(Ridx);
// We can allocate native arrays using constructors taking an integer as argument.
CvPoint hatPoints = new CvPoint(3);
while (frame.isVisible() && (grabbedImage = grabber.grab()) != null) {
cvClearMemStorage(storage);
// Let‘s try to detect some faces! but we need a grayscale image
cvCvtColor(grabbedImage, grayImage, CV_BGR2GRAY);
CvSeq faces = cvHaarDetectObjects(grayImage, classifier, storage,
1.1, 3, CV_HAAR_DO_CANNY_PRUNING);
int total = faces.total();
for (int i = 0; i < total; i++) {
CvRect r = new CvRect(cvGetSeqElem(faces, i));
int x = r.x(), y = r.y(), w = r.width(), h = r.height();
cvRectangle(grabbedImage, cvPoint(x, y), cvPoint(x+w, y+h), CvScalar.RED, 1, CV_AA, 0);
// To access or pass as argument the elements of a native array, call position() before.
hatPoints.position(0).x(x-w/10) .y(y-h/10);
hatPoints.position(1).x(x+w*11/10).y(y-h/10);
hatPoints.position(2).x(x+w/2) .y(y-h/2);
cvFillConvexPoly(grabbedImage, hatPoints.position(0), 3, CvScalar.GREEN, CV_AA, 0);
}
// Let‘s find some contours! but first some thresholding
cvThreshold(grayImage, grayImage, 64, 255, CV_THRESH_BINARY);
// To check if an output argument is null we may call either isNull() or equals(null).
CvSeq contour = new CvSeq(null);
cvFindContours(grayImage, storage, contour, Loader.sizeof(CvContour.class),
CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE);
while (contour != null && !contour.isNull()) {
if (contour.elem_size() > 0) {
CvSeq points = cvApproxPoly(contour, Loader.sizeof(CvContour.class),
storage, CV_POLY_APPROX_DP, cvContourPerimeter(contour)*0.02, 0);
cvDrawContours(grabbedImage, points, CvScalar.BLUE, CvScalar.BLUE, -1, 1, CV_AA);
}
contour = contour.h_next();
}
cvWarpPerspective(grabbedImage, rotatedImage, randomR);
frame.showImage(rotatedImage);
recorder.record(rotatedImage);
}
frame.dispose();
recorder.stop();
grabber.stop();
}
}
不过这个代码包含了人脸识别，而人脸识别需要下载一个xml文件来定义识别的模式，代码中那个xml链接下载下来的xml文件是会解析出错的，可能是那个版本比较新，JavaCV还没跟上，所以解析不了。经过搜索，发现有人提供了一个老版本的xml文件，能够顺利解析，会在视频中把人脸用绿框框住。
所以，把代码中的
URL url = new URL("https://raw.github.com/Itseez/opencv/2.4/data/haarcascades/haarcascade_frontalface_alt.xml");
改为URL url = new URL("https://raw.githubusercontent.com/Danukeru/FOUCAM/master/haarcascade_frontalface.xml");
即可顺利运行。运行后发现很多线条和框框，画面也是斜的，如下图所示：
这是用来演示JavaCV对图像的处理，如果对这些没有兴趣，可以简单地用以下代码来显示摄像头的图像：
import org.bytedeco.javacpp.opencv_core.IplImage;
import org.bytedeco.javacv.CanvasFrame;
import org.bytedeco.javacv.FrameGrabber;
public class Demo3 {
public static void main(String[] args) throws Exception {
// The available FrameGrabber classes include OpenCVFrameGrabber (opencv_highgui),
// DC1394FrameGrabber, FlyCaptureFrameGrabber, OpenKinectFrameGrabber,
// PS3EyeFrameGrabber, VideoInputFrameGrabber, and FFmpegFrameGrabber.
FrameGrabber grabber = FrameGrabber.createDefault(1);
grabber.start();
// FAQ about IplImage:
// - For custom raw processing of data, createBuffer() returns an NIO direct
// buffer wrapped around the memory pointed by imageData, and under Android we can
// also use that Buffer with Bitmap.copyPixelsFromBuffer() and copyPixelsToBuffer().
// - To get a BufferedImage from an IplImage, we may call getBufferedImage().
// - The createFrom() factory method can construct an IplImage from a BufferedImage.
// - There are also a few copy*() methods for BufferedImage<->IplImage data transfers.
IplImage grabbedImage = grabber.grab();
CanvasFrame frame = new CanvasFrame("Some Title", CanvasFrame.getDefaultGamma()/grabber.getGamma());
while (frame.isVisible() && (grabbedImage = grabber.grab()) != null) {
frame.showImage(grabbedImage);
}
frame.dispose();
grabber.stop();
}
}
如下图所示：
当然，这样只是把摄像头的图像显示出来，没有任何用处，所以如果你想要处理其中的图像，就需要增加代码。例如，如果你需要截取摄像头的图像做进一步处理，就可以通过IplImage的getBufferedImage()方法来得到BufferedImage，然后再处理这个BufferedImage，例如解析二维码。
不过，JavaCV默认是把图像显示在一个CanvasFrame对象中，这个对象继承了JFrame，所以只能用作顶层窗口，不能嵌入到其他窗口中。为了能够把它嵌入到其他窗口，可以修改一下代码，继承一个Canvas对象：
public class ImageCanvas extends Canvas {
private BufferedImage img;
@Override public void update(Graphics g) {
paint(g);
}
@Override public void paint(Graphics g) {
// Calling BufferStrategy.show() here sometimes throws
// NullPointerException or IllegalStateException,
// but otherwise seems to work fine.
try {
if (getWidth() <= 0 || getHeight() <= 0) {
return;
}
BufferStrategy strategy = getBufferStrategy();
do {
do {
g = strategy.getDrawGraphics();
if (img != null) {
g.drawImage(img, 0, 0, getWidth(), getHeight(), null);
}
g.dispose();
} while (strategy.contentsRestored());
strategy.show();
} while (strategy.contentsLost());
} catch (NullPointerException e) {
} catch (IllegalStateException e) { }
}
public BufferedImage getImg() {
return img;
}
public void setImg(BufferedImage img) {
this.img = img;
repaint();
}
public void init(){
new Thread(){
public void run(){
boolean error = true;
while(error){
try{
error = false;
createBufferStrategy(2);
}catch(IllegalStateException e){
error = true;
try {
Thread.sleep(100);
} catch (InterruptedException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
}
}
}
}.start();
}
使用的时候就把这个ImageCanvas当作普通的控件加入到窗口中，然后调用init()函数即可。init函数的作用是在控件显示出来后才调用createBufferStrategy(2)语句，否则会抛出IllegalStateException异常。private BufferedImage img;
@Override public void update(Graphics g) {
paint(g);
}
@Override public void paint(Graphics g) {
// Calling BufferStrategy.show() here sometimes throws
// NullPointerException or IllegalStateException,
// but otherwise seems to work fine.
try {
if (getWidth() <= 0 || getHeight() <= 0) {
return;
}
BufferStrategy strategy = getBufferStrategy();
do {
do {
g = strategy.getDrawGraphics();
if (img != null) {
g.drawImage(img, 0, 0, getWidth(), getHeight(), null);
}
g.dispose();
} while (strategy.contentsRestored());
strategy.show();
} while (strategy.contentsLost());
} catch (NullPointerException e) {
} catch (IllegalStateException e) { }
}
public BufferedImage getImg() {
return img;
}
public void setImg(BufferedImage img) {
this.img = img;
repaint();
}
public void init(){
new Thread(){
public void run(){
boolean error = true;
while(error){
try{
error = false;
createBufferStrategy(2);
}catch(IllegalStateException e){
error = true;
try {
Thread.sleep(100);
} catch (InterruptedException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
}
}
}
}.start();
}
这个ImageCanvas使用了双缓冲机制来显示视频（实际上是不间断地显示图像），避免图像闪烁。在此之前，尝试了网上找到的双缓冲代码，效果都不理想，只有这个能够不闪烁。
不过用了这个控件来显示视频，就不能全屏了。
加入控件后，显示图像只需要调用setImg(BufferedImage img)函数即可。也就是把
frame.showImage(grabbedImage);
改为canvas.setImg(grabbedImage.getBufferedImage());
郑重声明：本站内容如果来自互联网及其他传播媒体，其版权均属原媒体及文章作者所有。转载目的在于传递更多信息及用于网络分享，并不代表本站赞同其观点和对其真实性负责，也不构成任何其他建议。
