今年一进公司就开始搞相机开发,一直是在搞相机应用层,不过本身的学习心一直没停,闲暇之余就研究一下相机的framework、HAL层的东西,平时工做中碰到的和本身接触到的,相机中最复杂的就是预览了,有了一些体会也不想放下,因此决定写一系列关于Android相机的博客,把本身学习到的东西记录下来。java
提及Android相机的东西,从应用层的角度来看,基本就是四个重要的节点了:openCamera、createCaptureSession、preview、capture,最复杂的就是preview了,要理解preview,那么就要求你们对Android的View显示系统有必定的理解,才能更好的理解相机的预览。相机的预览其实就是使用预览区的SurfaceView对应的surface建立一条预览流,而后framework从预览surface当中获取到显示buffer,这里用于显示的buffer会根据数量来获取,华为手机的相机framework+HAL两部分通常总共须要7个buffer,每一个buffer都对应预览区的一屏的大小,它就是HAL、算法各层填充完毕后,要交给SurfaceFlinger用于显示的预览区大小的全部像素点的byte数组,这7个buffer每次在CameraServer进程获取一个,而后经过HIDL下发给CameraDaemon进程,交给算法、HAL层进行着色渲染,完成后再经过CameraServer进程交给SurfaceFlinger,最后显示在屏幕上,这样不断的轮转,咱们就看到了预览区会不断的变更,这里的buffer轮转也就是相机最核心的部分了。咱们后期的博客具体在讲关于buffer轮转的知识。android
这节咱们先来讲说从应用层调用openCamera以后的执行逻辑。openCamera的方法实现是在frameworks\base\core\java\android\hardware\camera2\CameraManager.java类中完成的,CameraManager和咱们应用中常用的WindowManager、PackageManager同样,均可以经过context上下文获取到,它也是在frameworks\base\core\java\android\app\SystemServiceRegistry.java类中经过Context.CAMERA_SERVICE键值注册的,源码以下:c++
咱们拿到CameraManager类的对象以后,就能够调用它的openCamera方法来打开相机了,CameraManager类的openCamera方法的源码以下:算法
@RequiresPermission(android.Manifest.permission.CAMERA)
public void openCamera(@NonNull String cameraId,
@NonNull final CameraDevice.StateCallback callback, @Nullable Handler handler)
throws CameraAccessException {
openCameraForUid(cameraId, callback, handler, USE_CALLING_UID);
}
这个方法的实现很简单,就是调用openCameraForUid来进一步处理,咱们先来看看调用该方法须要传递的参数,第一个表示要打开的目标Camera的id,华为手机上该值通常有两个:0和1,0表示后摄,固然也是主摄,1表示前摄,咱们怎么知道该值的取值呢?能够经过调用CameraManager类的getCameraIdList()方法来获取,该方法会将当前已经注册成功的camera硬件对应的id列表返回给咱们应用层,硬件注册都是驱动层的东西了,那一步离咱们如今的阶段还很远。咱们再来看一下第二个参数CameraDevice.StateCallback,它是定义在frameworks\base\core\java\android\hardware\camera2\CameraDevice.java类中的一个内部类,StateCallback类的定义源码以下:api
public static abstract class StateCallback {
/**
* An error code that can be reported by {@link #onError}
* indicating that the camera device is in use already.
*
* <p>
* This error can be produced when opening the camera fails due to the camera
* being used by a higher-priority camera API client.
* </p>
*
* @see #onError
*/
public static final int ERROR_CAMERA_IN_USE = 1;
/**
* An error code that can be reported by {@link #onError}
* indicating that the camera device could not be opened
* because there are too many other open camera devices.
*
* <p>
* The system-wide limit for number of open cameras has been reached,
* and more camera devices cannot be opened until previous instances are
* closed.
* </p>
*
* <p>
* This error can be produced when opening the camera fails.
* </p>
*
* @see #onError
*/
public static final int ERROR_MAX_CAMERAS_IN_USE = 2;
/**
* An error code that can be reported by {@link #onError}
* indicating that the camera device could not be opened due to a device
* policy.
*
* @see android.app.admin.DevicePolicyManager#setCameraDisabled(android.content.ComponentName, boolean)
* @see #onError
*/
public static final int ERROR_CAMERA_DISABLED = 3;
/**
* An error code that can be reported by {@link #onError}
* indicating that the camera device has encountered a fatal error.
*
* <p>The camera device needs to be re-opened to be used again.</p>
*
* @see #onError
*/
public static final int ERROR_CAMERA_DEVICE = 4;
/**
* An error code that can be reported by {@link #onError}
* indicating that the camera service has encountered a fatal error.
*
* <p>The Android device may need to be shut down and restarted to restore
* camera function, or there may be a persistent hardware problem.</p>
*
* <p>An attempt at recovery <i>may</i> be possible by closing the
* CameraDevice and the CameraManager, and trying to acquire all resources
* again from scratch.</p>
*
* @see #onError
*/
public static final int ERROR_CAMERA_SERVICE = 5;
/** @hide */
@Retention(RetentionPolicy.SOURCE)
@IntDef(prefix = {"ERROR_"}, value =
{ERROR_CAMERA_IN_USE,
ERROR_MAX_CAMERAS_IN_USE,
ERROR_CAMERA_DISABLED,
ERROR_CAMERA_DEVICE,
ERROR_CAMERA_SERVICE })
public @interface ErrorCode {};
/**
* The method called when a camera device has finished opening.
*
* <p>At this point, the camera device is ready to use, and
* {@link CameraDevice#createCaptureSession} can be called to set up the first capture
* session.</p>
*
* @param camera the camera device that has become opened
*/
public abstract void onOpened(@NonNull CameraDevice camera); // Must implement
/**
* The method called when a camera device has been closed with
* {@link CameraDevice#close}.
*
* <p>Any attempt to call methods on this CameraDevice in the
* future will throw a {@link IllegalStateException}.</p>
*
* <p>The default implementation of this method does nothing.</p>
*
* @param camera the camera device that has become closed
*/
public void onClosed(@NonNull CameraDevice camera) {
// Default empty implementation
}
/**
* The method called when a camera device is no longer available for
* use.
*
* <p>This callback may be called instead of {@link #onOpened}
* if opening the camera fails.</p>
*
* <p>Any attempt to call methods on this CameraDevice will throw a
* {@link CameraAccessException}. The disconnection could be due to a
* change in security policy or permissions; the physical disconnection
* of a removable camera device; or the camera being needed for a
* higher-priority camera API client.</p>
*
* <p>There may still be capture callbacks that are invoked
* after this method is called, or new image buffers that are delivered
* to active outputs.</p>
*
* <p>The default implementation logs a notice to the system log
* about the disconnection.</p>
*
* <p>You should clean up the camera with {@link CameraDevice#close} after
* this happens, as it is not recoverable until the camera can be opened
* again. For most use cases, this will be when the camera again becomes
* {@link CameraManager.AvailabilityCallback#onCameraAvailable available}.
* </p>
*
* @param camera the device that has been disconnected
*/
public abstract void onDisconnected(@NonNull CameraDevice camera); // Must implement
/**
* The method called when a camera device has encountered a serious error.
*
* <p>This callback may be called instead of {@link #onOpened}
* if opening the camera fails.</p>
*
* <p>This indicates a failure of the camera device or camera service in
* some way. Any attempt to call methods on this CameraDevice in the
* future will throw a {@link CameraAccessException} with the
* {@link CameraAccessException#CAMERA_ERROR CAMERA_ERROR} reason.
* </p>
*
* <p>There may still be capture completion or camera stream callbacks
* that will be called after this error is received.</p>
*
* <p>You should clean up the camera with {@link CameraDevice#close} after
* this happens. Further attempts at recovery are error-code specific.</p>
*
* @param camera The device reporting the error
* @param error The error code.
*
* @see #ERROR_CAMERA_IN_USE
* @see #ERROR_MAX_CAMERAS_IN_USE
* @see #ERROR_CAMERA_DISABLED
* @see #ERROR_CAMERA_DEVICE
* @see #ERROR_CAMERA_SERVICE
*/
public abstract void onError(@NonNull CameraDevice camera,
@ErrorCode int error); // Must implement
}
咱们从这个类所定义的方法就可以很是清楚的看到,它这几个回调的意图了:onOpened就是成功打开camera以后的回调,并且它会返回一个CameraDevice camera对象给咱们应用层,基本上操做相机全部重要的工做都是由它来中转实现的,因此应用层拿到这个对象以后,就可使用它做不少其余的工做了,在接下来的分析过程当中,咱们也会看到,这个对象在framework中是怎么构建好,而后又是怎么回传给咱们应用层的;onClosed方法就是当相机关闭时的回调了;onDisconnected方法就是相机断开链接时的回调;onError方法就是相机出错时的回调了。
咱们再来看看最后一个参数Handler handler,为何要传一个Handler进来呢?它的目的就是为了保证线程不切换,假如咱们在应用层在工做线程B中执行openCamera的方法,同时将线程B对应的Handler对象传进来,那么打开成功以后,framework为了保证线程同步,也会使用该handler对象,将消息发送到线程B的Looper循环上,这就是传入一个Handler对象的缘由了,咱们在后面的分析过程当中会看到它的使用。数组
好,参数分析完了,咱们继续来看一下openCameraForUid方法的实现,源码以下:session
public void openCameraForUid(@NonNull String cameraId,
@NonNull final CameraDevice.StateCallback callback, @Nullable Handler handler,
int clientUid)
throws CameraAccessException {
if (cameraId == null) {
throw new IllegalArgumentException("cameraId was null");
} else if (callback == null) {
throw new IllegalArgumentException("callback was null");
} else if (handler == null) {
if (Looper.myLooper() != null) {
handler = new Handler();
} else {
throw new IllegalArgumentException(
"Handler argument is null, but no looper exists in the calling thread");
}
}
openCameraDeviceUserAsync(cameraId, callback, handler, clientUid);
}
首先仍是参数判断,咱们调用方法时传入的参数必须合法,不然直接抛出异常,参数合法以后,再调用openCameraDeviceUserAsync来进一步执行camera的打开工做,源码以下:app
private CameraDevice openCameraDeviceUserAsync(String cameraId,
CameraDevice.StateCallback callback, Handler handler, final int uid)
throws CameraAccessException {
CameraCharacteristics characteristics = getCameraCharacteristics(cameraId);
CameraDevice device = null;
synchronized (mLock) {
ICameraDeviceUser cameraUser = null;
android.hardware.camera2.impl.CameraDeviceImpl deviceImpl =
new android.hardware.camera2.impl.CameraDeviceImpl(
cameraId,
callback,
handler,
characteristics,
mContext.getApplicationInfo().targetSdkVersion);
ICameraDeviceCallbacks callbacks = deviceImpl.getCallbacks();
try {
if (supportsCamera2ApiLocked(cameraId)) {
// Use cameraservice's cameradeviceclient implementation for HAL3.2+ devices
ICameraService cameraService = CameraManagerGlobal.get().getCameraService();
if (cameraService == null) {
throw new ServiceSpecificException(
ICameraService.ERROR_DISCONNECTED,
"Camera service is currently unavailable");
}
cameraUser = cameraService.connectDevice(callbacks, cameraId,
mContext.getOpPackageName(), uid);
} else {
// Use legacy camera implementation for HAL1 devices
int id;
try {
id = Integer.parseInt(cameraId);
} catch (NumberFormatException e) {
throw new IllegalArgumentException("Expected cameraId to be numeric, but it was: "
+ cameraId);
}
Log.i(TAG, "Using legacy camera HAL.");
cameraUser = CameraDeviceUserShim.connectBinderShim(callbacks, id);
}
} catch (ServiceSpecificException e) {
if (e.errorCode == ICameraService.ERROR_DEPRECATED_HAL) {
throw new AssertionError("Should've gone down the shim path");
} else if (e.errorCode == ICameraService.ERROR_CAMERA_IN_USE ||
e.errorCode == ICameraService.ERROR_MAX_CAMERAS_IN_USE ||
e.errorCode == ICameraService.ERROR_DISABLED ||
e.errorCode == ICameraService.ERROR_DISCONNECTED ||
e.errorCode == ICameraService.ERROR_INVALID_OPERATION) {
// Received one of the known connection errors
// The remote camera device cannot be connected to, so
// set the local camera to the startup error state
deviceImpl.setRemoteFailure(e);
if (e.errorCode == ICameraService.ERROR_DISABLED ||
e.errorCode == ICameraService.ERROR_DISCONNECTED ||
e.errorCode == ICameraService.ERROR_CAMERA_IN_USE) {
// Per API docs, these failures call onError and throw
throwAsPublicException(e);
}
} else {
// Unexpected failure - rethrow
throwAsPublicException(e);
}
} catch (RemoteException e) {
// Camera service died - act as if it's a CAMERA_DISCONNECTED case
ServiceSpecificException sse = new ServiceSpecificException(
ICameraService.ERROR_DISCONNECTED,
"Camera service is currently unavailable");
deviceImpl.setRemoteFailure(sse);
throwAsPublicException(sse);
}
// TODO: factor out callback to be non-nested, then move setter to constructor
// For now, calling setRemoteDevice will fire initial
// onOpened/onUnconfigured callbacks.
// This function call may post onDisconnected and throw CAMERA_DISCONNECTED if
// cameraUser dies during setup.
deviceImpl.setRemoteDevice(cameraUser);
device = deviceImpl;
}
return device;
}
该方法中首先调用getCameraCharacteristics(cameraId)获取到目标camera的特性参数,这个方法的实现就是在咱们前面提到的CameraServer进程当中,由于手机开机时,camera驱动的注册就开始了,注册完成以后,全部camera硬件设备的特性参数都已经保存在CameraServer进程当中了,咱们这里只须要获取就能够了,你们若是想了解,能够继续往下追查。接下来使用咱们传入的参数构造一个CameraDeviceImpl对象,它其实也就是最终要经过StateCallback回调接口的onOpened方法返回给咱们应用层的那个参数对象了,只不过这里刚构建好,它里边最核心的一个成员变量尚未赋值,就和Surface同样,咱们虽然能够new Surface创造一个Surface对象,可是它在native层还要执行不少的初始化逻辑,还有它所对应的buffer空间和native层的指针没有初始化,那么这个Surface只是个壳子,没有什么实际的用处。咱们来看一下frameworks\base\core\java\android\hardware\camera2\impl\CameraDeviceImpl.java类的构造方法,源码以下:composer
public CameraDeviceImpl(String cameraId, StateCallback callback, Handler handler,
CameraCharacteristics characteristics, int appTargetSdkVersion) {
if (cameraId == null || callback == null || handler == null || characteristics == null) {
throw new IllegalArgumentException("Null argument given");
}
mCameraId = cameraId;
mDeviceCallback = callback;
mDeviceHandler = handler;
mCharacteristics = characteristics;
mAppTargetSdkVersion = appTargetSdkVersion;
final int MAX_TAG_LEN = 23;
String tag = String.format("CameraDevice-JV-%s", mCameraId);
if (tag.length() > MAX_TAG_LEN) {
tag = tag.substring(0, MAX_TAG_LEN);
}
TAG = tag;
Integer partialCount =
mCharacteristics.get(CameraCharacteristics.REQUEST_PARTIAL_RESULT_COUNT);
if (partialCount == null) {
// 1 means partial result is not supported.
mTotalPartialCount = 1;
} else {
mTotalPartialCount = partialCount;
}
}
这里比较重要的信息就是关于tag的赋值了,它是执行String.format("CameraDevice-JV-%s", mCameraId)逻辑来赋值的,咱们也能够在打开相机时搜索到这个关键字,它对咱们实际的处理问题能够说是一个点,咱们能够从这里判断确定有应用层想要打开相机设备,并且能够根据id看到想要打开哪一个目标设备。回到openCameraDeviceUserAsync方法中,接下来调用刚才建立好的CameraDeviceImpl对象的getCallbacks()方法获取一个ICameraDeviceCallbacks对象,它是经过ICameraDeviceCallbacks.aidl定义的,因此它就是一个binder对象,而它的做用就是把这个对象传递到CameraServer进程当中,来配合回调各个节点方法的,咱们立刻就会看到。接下来判断if (supportsCamera2ApiLocked(cameraId))条件,它的意思就是当前是否支持camera2.0的协议,这是Android对于相机的优化,咱们假设这里支持,那么继续调用cameraUser = cameraService.connectDevice(callbacks, cameraId, mContext.getOpPackageName(), uid)逻辑来给局部变量cameraUser赋值,这句逻辑往下就是咱们最核心的地方了,而这个返回值也就是CameraServer进程派给咱们的操做表明,咱们应用的各项工做都是由它进行中转来实现了。
咱们继续往下看完这个方法的实现,而后再回头来分析cameraService.connectDevice的逻辑实现。接下来最后的两句就是调用deviceImpl.setRemoteDevice(cameraUser)方法将CameraServer返回给咱们的对象保存下来,这句逻辑执行完成后,这个deviceImpl才算真正具有功能了,最后就是给局部变量device赋值。咱们继续看一下frameworks\base\core\java\android\hardware\camera2\impl\CameraDeviceImpl.java类的setRemoteDevice方法,源码以下:dom
public void setRemoteDevice(ICameraDeviceUser remoteDevice) throws CameraAccessException {
synchronized(mInterfaceLock) {
// TODO: Move from decorator to direct binder-mediated exceptions
// If setRemoteFailure already called, do nothing
if (mInError) return;
mRemoteDevice = new ICameraDeviceUserWrapper(remoteDevice);
IBinder remoteDeviceBinder = remoteDevice.asBinder();
// For legacy camera device, remoteDevice is in the same process, and
// asBinder returns NULL.
if (remoteDeviceBinder != null) {
try {
remoteDeviceBinder.linkToDeath(this, /*flag*/ 0);
} catch (RemoteException e) {
CameraDeviceImpl.this.mDeviceHandler.post(mCallOnDisconnected);
throw new CameraAccessException(CameraAccessException.CAMERA_DISCONNECTED,
"The camera device has encountered a serious error");
}
}
mDeviceHandler.post(mCallOnOpened);
mDeviceHandler.post(mCallOnUnconfigured);
}
}
这里会使用CameraServer返回给咱们的remoteDevice构建一个ICameraDeviceUserWrapper对象,就是将它再包装一层,而后赋值给成员变量mRemoteDevice,咱们能够看到,最终咱们拿到的那个CameraDevice对象其实离真正给咱们干活的对象已经很远了,中间都通过了好几层包装;而后再调用mDeviceHandler.post(mCallOnOpened)经过应用层,打开相机的工做已经完成了,mDeviceHandler就是咱们一开始调用openCamera方法时传入的第三个参数了,因此使用它来post发送一个消息,那么回调也会在当时执行openCamera方法的线程上,这样就能够保证线程不会切换了。mCallOnOpened是CameraDeviceImpl类的一个成员变量,定义源码以下:
private final Runnable mCallOnOpened = new Runnable() {
@Override
public void run() {
StateCallbackKK sessionCallback = null;
synchronized(mInterfaceLock) {
if (mRemoteDevice == null) return; // Camera already closed
sessionCallback = mSessionStateCallback;
}
if (sessionCallback != null) {
sessionCallback.onOpened(CameraDeviceImpl.this);
}
mDeviceCallback.onOpened(CameraDeviceImpl.this);
}
};
这里的mDeviceCallback成员变量也就是前面构造CameraDeviceImpl对象时传入的,也就是咱们调用openCamera方法的第二个参数了,这时的CameraDeviceImpl对象已经准备好了,因而经过回调接口将CameraDeviceImpl对象返回给应用层,应用层就能够经过它来执行各类逻辑真正的控制camera了。
好,回调应用层的逻辑分析完,
如下部分涉及到c++
咱们回过头来继续看一下cameraService.connectDevice方法是如何打开camera的。
cameraService是经过binder进程间通讯,执行CameraManagerGlobal.get().getCameraService()获取到的Camera服务端的一个binder代理对象,它实际的实现是native层的CameraService。CameraServer进程的启动函数定义在frameworks\av\camera\cameraserver\main_cameraserver.cpp文件中,它的main函数源码以下:
using namespace android;
int main(int argc __unused, char** argv __unused)
{
signal(SIGPIPE, SIG_IGN);
// Set 3 threads for HIDL calls
hardware::configureRpcThreadpool(3, /*willjoin*/ false);
sp<ProcessState> proc(ProcessState::self());
sp<IServiceManager> sm = defaultServiceManager();
ALOGI("ServiceManager: %p", sm.get());
CameraService::instantiate();
ProcessState::self()->startThreadPool();
IPCThreadState::self()->joinThreadPool();
}
这里会调用CameraService::instantiate()方法,而CameraService是继承了BinderService,因此instantiate()方法实际也是执行的BinderService对象的instantiate()方法,该方法定义在frameworks\native\libs\binder\include\binder\BinderService.h文件中,源码以下:
template<typename SERVICE>
class BinderService
{
public:
static status_t publish(bool allowIsolated = false) {
sp<IServiceManager> sm(defaultServiceManager());
return sm->addService(
String16(SERVICE::getServiceName()),
new SERVICE(), allowIsolated);
}
static void publishAndJoinThreadPool(bool allowIsolated = false) {
publish(allowIsolated);
joinThreadPool();
}
static void instantiate() { publish(); }
static status_t shutdown() { return NO_ERROR; }
private:
static void joinThreadPool() {
sp<ProcessState> ps(ProcessState::self());
ps->startThreadPool();
ps->giveThreadPoolName();
IPCThreadState::self()->joinThreadPool();
}
};
在这里执行publish方法时,以CameraService类的getServiceName()方法返回的char*为键值(media.camera)将CameraService对象添加到service_manager当中的。因此接下来咱们就来看一下CameraService类的connectDevice方法的实现,源码以下:
Status CameraService::connectDevice(
const sp<hardware::camera2::ICameraDeviceCallbacks>& cameraCb,
const String16& cameraId,
const String16& clientPackageName,
int clientUid,
/*out*/
sp<hardware::camera2::ICameraDeviceUser>* device) {
ATRACE_CALL();
Status ret = Status::ok();
String8 id = String8(cameraId);
sp<CameraDeviceClient> client = nullptr;
ret = connectHelper<hardware::camera2::ICameraDeviceCallbacks,CameraDeviceClient>(cameraCb, id,
CAMERA_HAL_API_VERSION_UNSPECIFIED, clientPackageName,
clientUid, USE_CALLING_PID, API_2,
/*legacyMode*/ false, /*shimUpdateOnly*/ false,
/*out*/client);
if(!ret.isOk()) {
logRejected(id, getCallingPid(), String8(clientPackageName),
ret.toString8());
return ret;
}
*device = client;
return ret;
}
这里的几个参数咱们要看一下,第一个cameraCb就是咱们在framework中获得的那个binder对象,之后的部分逻辑都会经过它来进行中转;cameraId表示咱们要打开的目标设备;clientPackageName表示请求执行打开camera设备的应用进程包名;clientUid表示应用进程的uid;最后的device就是返回给framework的对象了,注释已经写的很是清楚了,它是一个输出参数。接下来执行sp<CameraDeviceClient> client = nullptr声明一个CameraDeviceClient对象,在给它赋值成功以后,将它再赋值给输出参数device,因此说咱们在framework中看到的那个cameraUser实际上就是这里的CameraDeviceClient对象了。
接下来咱们继续分析connectHelper方法的实现,源码以下:
template<class CALLBACK, class CLIENT>
Status CameraService::connectHelper(const sp<CALLBACK>& cameraCb, const String8& cameraId,
int halVersion, const String16& clientPackageName, int clientUid, int clientPid,
apiLevel effectiveApiLevel, bool legacyMode, bool shimUpdateOnly,
/*out*/sp<CLIENT>& device) {
binder::Status ret = binder::Status::ok();
String8 clientName8(clientPackageName);
int originalClientPid = 0;
ALOGI("CameraService::connect call (PID %d \"%s\", camera ID %s) for HAL version %s and "
"Camera API version %d", clientPid, clientName8.string(), cameraId.string(),
(halVersion == -1) ? "default" : std::to_string(halVersion).c_str(),
static_cast<int>(effectiveApiLevel));
sp<CLIENT> client = nullptr;
{
// Acquire mServiceLock and prevent other clients from connecting
std::unique_ptr<AutoConditionLock> lock =
AutoConditionLock::waitAndAcquire(mServiceLockWrapper, DEFAULT_CONNECT_TIMEOUT_NS);
if (lock == nullptr) {
ALOGE("CameraService::connect (PID %d) rejected (too many other clients connecting)."
, clientPid);
return STATUS_ERROR_FMT(ERROR_MAX_CAMERAS_IN_USE,
"Cannot open camera %s for \"%s\" (PID %d): Too many other clients connecting",
cameraId.string(), clientName8.string(), clientPid);
}
// Enforce client permissions and do basic sanity checks
if(!(ret = validateConnectLocked(cameraId, clientName8,
/*inout*/clientUid, /*inout*/clientPid, /*out*/originalClientPid)).isOk()) {
return ret;
}
// Check the shim parameters after acquiring lock, if they have already been updated and
// we were doing a shim update, return immediately
if (shimUpdateOnly) {
auto cameraState = getCameraState(cameraId);
if (cameraState != nullptr) {
if (!cameraState->getShimParams().isEmpty()) return ret;
}
}
status_t err;
sp<BasicClient> clientTmp = nullptr;
std::shared_ptr<resource_policy::ClientDescriptor<String8, sp<BasicClient>>> partial;
if ((err = handleEvictionsLocked(cameraId, originalClientPid, effectiveApiLevel,
IInterface::asBinder(cameraCb), clientName8, /*out*/&clientTmp,
/*out*/&partial)) != NO_ERROR) {
switch (err) {
case -ENODEV:
return STATUS_ERROR_FMT(ERROR_DISCONNECTED,
"No camera device with ID \"%s\" currently available",
cameraId.string());
case -EBUSY:
return STATUS_ERROR_FMT(ERROR_CAMERA_IN_USE,
"Higher-priority client using camera, ID \"%s\" currently unavailable",
cameraId.string());
default:
return STATUS_ERROR_FMT(ERROR_INVALID_OPERATION,
"Unexpected error %s (%d) opening camera \"%s\"",
strerror(-err), err, cameraId.string());
}
}
if (clientTmp.get() != nullptr) {
// Handle special case for API1 MediaRecorder where the existing client is returned
device = static_cast<CLIENT*>(clientTmp.get());
return ret;
}
// give flashlight a chance to close devices if necessary.
mFlashlight->prepareDeviceOpen(cameraId);
int facing = -1;
int deviceVersion = getDeviceVersion(cameraId, /*out*/&facing);
if (facing == -1) {
ALOGE("%s: Unable to get camera device \"%s\" facing", __FUNCTION__, cameraId.string());
return STATUS_ERROR_FMT(ERROR_INVALID_OPERATION,
"Unable to get camera device \"%s\" facing", cameraId.string());
}
sp<BasicClient> tmp = nullptr;
if(!(ret = makeClient(this, cameraCb, clientPackageName, cameraId, facing, clientPid,
clientUid, getpid(), legacyMode, halVersion, deviceVersion, effectiveApiLevel,
/*out*/&tmp)).isOk()) {
return ret;
}
client = static_cast<CLIENT*>(tmp.get());
LOG_ALWAYS_FATAL_IF(client.get() == nullptr, "%s: CameraService in invalid state",
__FUNCTION__);
err = client->initialize(mCameraProviderManager);
if (err != OK) {
ALOGE("%s: Could not initialize client from HAL.", __FUNCTION__);
// Errors could be from the HAL module open call or from AppOpsManager
switch(err) {
case BAD_VALUE:
return STATUS_ERROR_FMT(ERROR_ILLEGAL_ARGUMENT,
"Illegal argument to HAL module for camera \"%s\"", cameraId.string());
case -EBUSY:
return STATUS_ERROR_FMT(ERROR_CAMERA_IN_USE,
"Camera \"%s\" is already open", cameraId.string());
case -EUSERS:
return STATUS_ERROR_FMT(ERROR_MAX_CAMERAS_IN_USE,
"Too many cameras already open, cannot open camera \"%s\"",
cameraId.string());
case PERMISSION_DENIED:
return STATUS_ERROR_FMT(ERROR_PERMISSION_DENIED,
"No permission to open camera \"%s\"", cameraId.string());
case -EACCES:
return STATUS_ERROR_FMT(ERROR_DISABLED,
"Camera \"%s\" disabled by policy", cameraId.string());
case -ENODEV:
default:
return STATUS_ERROR_FMT(ERROR_INVALID_OPERATION,
"Failed to initialize camera \"%s\": %s (%d)", cameraId.string(),
strerror(-err), err);
}
}
// Update shim paremeters for legacy clients
if (effectiveApiLevel == API_1) {
// Assume we have always received a Client subclass for API1
sp<Client> shimClient = reinterpret_cast<Client*>(client.get());
String8 rawParams = shimClient->getParameters();
CameraParameters params(rawParams);
auto cameraState = getCameraState(cameraId);
if (cameraState != nullptr) {
cameraState->setShimParams(params);
} else {
ALOGE("%s: Cannot update shim parameters for camera %s, no such device exists.",
__FUNCTION__, cameraId.string());
}
}
if (shimUpdateOnly) {
// If only updating legacy shim parameters, immediately disconnect client
mServiceLock.unlock();
client->disconnect();
mServiceLock.lock();
} else {
// Otherwise, add client to active clients list
finishConnectLocked(client, partial);
}
} // lock is destroyed, allow further connect calls
// Important: release the mutex here so the client can call back into the service from its
// destructor (can be at the end of the call)
device = client;
return ret;
}
该方法是一个模板方法,这里的handleEvictionsLocked、client->initialize方法执行出错以后的日志打印对于咱们分析问题也会有很是大的帮助,咱们平时工做中常常碰到相机打开失败的问题,若是有这些日志,那就说明问题确定是出在CameraServer进程往下哪里的逻辑中了。
往下咱们就主要来看一下makeClient、client->initialize这两句方法的实现。makeClient方法的源码以下:
Status CameraService::makeClient(const sp<CameraService>& cameraService,
const sp<IInterface>& cameraCb, const String16& packageName, const String8& cameraId,
int facing, int clientPid, uid_t clientUid, int servicePid, bool legacyMode,
int halVersion, int deviceVersion, apiLevel effectiveApiLevel,
/*out*/sp<BasicClient>* client) {
if (halVersion < 0 || halVersion == deviceVersion) {
// Default path: HAL version is unspecified by caller, create CameraClient
// based on device version reported by the HAL.
switch(deviceVersion) {
case CAMERA_DEVICE_API_VERSION_1_0:
if (effectiveApiLevel == API_1) { // Camera1 API route
sp<ICameraClient> tmp = static_cast<ICameraClient*>(cameraCb.get());
*client = new CameraClient(cameraService, tmp, packageName, cameraIdToInt(cameraId),
facing, clientPid, clientUid, getpid(), legacyMode);
} else { // Camera2 API route
ALOGW("Camera using old HAL version: %d", deviceVersion);
return STATUS_ERROR_FMT(ERROR_DEPRECATED_HAL,
"Camera device \"%s\" HAL version %d does not support camera2 API",
cameraId.string(), deviceVersion);
}
break;
case CAMERA_DEVICE_API_VERSION_3_0:
case CAMERA_DEVICE_API_VERSION_3_1:
case CAMERA_DEVICE_API_VERSION_3_2:
case CAMERA_DEVICE_API_VERSION_3_3:
case CAMERA_DEVICE_API_VERSION_3_4:
if (effectiveApiLevel == API_1) { // Camera1 API route
sp<ICameraClient> tmp = static_cast<ICameraClient*>(cameraCb.get());
*client = new Camera2Client(cameraService, tmp, packageName, cameraIdToInt(cameraId),
facing, clientPid, clientUid, servicePid, legacyMode);
} else { // Camera2 API route
sp<hardware::camera2::ICameraDeviceCallbacks> tmp =
static_cast<hardware::camera2::ICameraDeviceCallbacks*>(cameraCb.get());
*client = new CameraDeviceClient(cameraService, tmp, packageName, cameraId,
facing, clientPid, clientUid, servicePid);
}
break;
default:
// Should not be reachable
ALOGE("Unknown camera device HAL version: %d", deviceVersion);
return STATUS_ERROR_FMT(ERROR_INVALID_OPERATION,
"Camera device \"%s\" has unknown HAL version %d",
cameraId.string(), deviceVersion);
}
} else {
// A particular HAL version is requested by caller. Create CameraClient
// based on the requested HAL version.
if (deviceVersion > CAMERA_DEVICE_API_VERSION_1_0 &&
halVersion == CAMERA_DEVICE_API_VERSION_1_0) {
// Only support higher HAL version device opened as HAL1.0 device.
sp<ICameraClient> tmp = static_cast<ICameraClient*>(cameraCb.get());
*client = new CameraClient(cameraService, tmp, packageName, cameraIdToInt(cameraId),
facing, clientPid, clientUid, servicePid, legacyMode);
} else {
// Other combinations (e.g. HAL3.x open as HAL2.x) are not supported yet.
ALOGE("Invalid camera HAL version %x: HAL %x device can only be"
" opened as HAL %x device", halVersion, deviceVersion,
CAMERA_DEVICE_API_VERSION_1_0);
return STATUS_ERROR_FMT(ERROR_ILLEGAL_ARGUMENT,
"Camera device \"%s\" (HAL version %d) cannot be opened as HAL version %d",
cameraId.string(), deviceVersion, halVersion);
}
}
return Status::ok();
}
通常驱动版本halVersion和设备版本deviceVersion是相同的,因此进入第一个if分支,effectiveApiLevel参数是在调用connectHelper方法时传入的,值为API_2,因此进入else分支,直接使用咱们上边传进来的参数构造一个CameraDeviceClient对象,而该对象也就是咱们应用进程和CameraServer进程通讯的使者了,全部的工做都是由它来进行中转的。咱们继续看一下它的构造方法的实现。frameworks\av\services\camera\libcameraservice\api2\CameraDeviceClient.cpp类的构造方法源码以下:
CameraDeviceClient::CameraDeviceClient(const sp<CameraService>& cameraService,
const sp<hardware::camera2::ICameraDeviceCallbacks>& remoteCallback,
const String16& clientPackageName,
const String8& cameraId,
int cameraFacing,
int clientPid,
uid_t clientUid,
int servicePid) :
Camera2ClientBase(cameraService, remoteCallback, clientPackageName,
cameraId, cameraFacing, clientPid, clientUid, servicePid),
mInputStream(),
mStreamingRequestId(REQUEST_ID_NONE),
mRequestIdCounter(0) {
ATRACE_CALL();
ALOGI("CameraDeviceClient %s: Opened", cameraId.string());
}
由于它是继承Camera2ClientBase的,因此也会执行Camera2ClientBase类的构造方法。frameworks\av\services\camera\libcameraservice\common\Camera2ClientBase.cpp类的构造方法的源码以下:
template <typename TClientBase>
Camera2ClientBase<TClientBase>::Camera2ClientBase(
const sp<CameraService>& cameraService,
const sp<TCamCallbacks>& remoteCallback,
const String16& clientPackageName,
const String8& cameraId,
int cameraFacing,
int clientPid,
uid_t clientUid,
int servicePid):
TClientBase(cameraService, remoteCallback, clientPackageName,
cameraId, cameraFacing, clientPid, clientUid, servicePid),
mSharedCameraCallbacks(remoteCallback),
mDeviceVersion(cameraService->getDeviceVersion(TClientBase::mCameraIdStr)),
mDeviceActive(false)
{
ALOGI("Camera %s: Opened. Client: %s (PID %d, UID %d)", cameraId.string(),
String8(clientPackageName).string(), clientPid, clientUid);
mInitialClientPid = clientPid;
mDevice = new Camera3Device(cameraId);
LOG_ALWAYS_FATAL_IF(mDevice == 0, "Device should never be NULL here.");
}
在这里又出现了一个很是重要的对象Camera3Device,后边咱们就会看到,咱们操做相机的全部工做在CameraServer进程都是由它中转来和CameraDaemon进程进行通讯的。frameworks\av\services\camera\libcameraservice\device3\Camera3Device.cpp的构造方法的源码以下:
Camera3Device::Camera3Device(const String8 &id):
mId(id),
mOperatingMode(NO_MODE),
mIsConstrainedHighSpeedConfiguration(false),
mStatus(STATUS_UNINITIALIZED),
mStatusWaiters(0),
mUsePartialResult(false),
mNumPartialResults(1),
mTimestampOffset(0),
mNextResultFrameNumber(0),
mNextReprocessResultFrameNumber(0),
mNextShutterFrameNumber(0),
mNextReprocessShutterFrameNumber(0),
mListener(NULL),
mVendorTagId(CAMERA_METADATA_INVALID_VENDOR_ID)
{
ATRACE_CALL();
camera3_callback_ops::notify = &sNotify;
camera3_callback_ops::process_capture_result = &sProcessCaptureResult;
ALOGV("%s: Created device for camera %s", __FUNCTION__, mId.string());
}
这里的sProcessCaptureResult是一个函数指针,从它的命名也很容易判断出来,它就是处理结果回调用,那是哪一个结果回调呢?固然是CameraDaemon进程对一帧图片处理完成以后的结果回调了,注意这里的capture不单指拍照,预览的回调也是经过该接口传回来进行处理的。
该构造的对象都建立好了,再回到CameraService类的connectHelper方法中,继续来看一下client->initialize(mCameraProviderManager)逻辑的实现,这里的client就是CameraDeviceClient了,frameworks\av\services\camera\libcameraservice\api2\CameraDeviceClient.cpp类的initialize方法的源码以下:
status_t CameraDeviceClient::initialize(sp<CameraProviderManager> manager) {
return initializeImpl(manager);
}
template<typename TProviderPtr>
status_t CameraDeviceClient::initializeImpl(TProviderPtr providerPtr) {
ATRACE_CALL();
status_t res;
res = Camera2ClientBase::initialize(providerPtr);
if (res != OK) {
return res;
}
String8 threadName;
mFrameProcessor = new FrameProcessorBase(mDevice);
threadName = String8::format("CDU-%s-FrameProc", mCameraIdStr.string());
mFrameProcessor->run(threadName.string());
mFrameProcessor->registerListener(FRAME_PROCESSOR_LISTENER_MIN_ID,
FRAME_PROCESSOR_LISTENER_MAX_ID,
/*listener*/this,
/*sendPartials*/true);
return OK;
}
它也是转调initializeImpl方法来实现的,首先调用Camera2ClientBase::initialize(providerPtr)来执行父类的初始化,而后调用mFrameProcessor = new FrameProcessorBase(mDevice)给成员变量mFrameProcessor赋值,从该成员的命名上就能够看出来,它就是用来处理预览帧的,它是继承了Thread类的,咱们看到相机的预览实际就是它在一个无限循环当中不断的处理request来完成的。frameworks\av\services\camera\libcameraservice\common\FrameProcessorBase.h类的定义源码以下:
class FrameProcessorBase: public Thread {
public:
explicit FrameProcessorBase(wp<CameraDeviceBase> device);
virtual ~FrameProcessorBase();
struct FilteredListener: virtual public RefBase {
virtual void onResultAvailable(const CaptureResult &result) = 0;
};
// Register a listener for a range of IDs [minId, maxId). Multiple listeners
// can be listening to the same range. Registering the same listener with
// the same range of IDs has no effect.
// sendPartials controls whether partial results will be sent.
status_t registerListener(int32_t minId, int32_t maxId,
const wp<FilteredListener>& listener,
bool sendPartials = true);
status_t removeListener(int32_t minId, int32_t maxId,
const wp<FilteredListener>& listener);
void dump(int fd, const Vector<String16>& args);
protected:
static const nsecs_t kWaitDuration = 10000000; // 10 ms
wp<CameraDeviceBase> mDevice;
virtual bool threadLoop();
Mutex mInputMutex;
Mutex mLastFrameMutex;
struct RangeListener {
int32_t minId;
int32_t maxId;
wp<FilteredListener> listener;
bool sendPartials;
};
List<RangeListener> mRangeListeners;
// Number of partial result the HAL will potentially send.
int32_t mNumPartialResults;
void processNewFrames(const sp<CameraDeviceBase> &device);
virtual bool processSingleFrame(CaptureResult &result,
const sp<CameraDeviceBase> &device);
status_t processListeners(const CaptureResult &result,
const sp<CameraDeviceBase> &device);
CameraMetadata mLastFrame;
};
mFrameProcessor构造好以后,调用mFrameProcessor->run(threadName.string())来启动它,run方法中的参数就是对当前线程的命名,若是咱们碰到相机ANR的问题,也能够经过命名来分析这条线程的逻辑。
最后调用mFrameProcessor->registerListener来注册回调。第三个参数的定义类型为FilteredListener,实际参数就是当前的CameraDeviceClient对象,FilteredListener也是定义在FrameProcessorBase.h头文件中,它只有一个方法,onResultAvailable,很明显就是一帧处理好了,须要回调给应用层时,就会经过该方法来完成。
咱们接下来在看一下刚才那句Camera2ClientBase::initialize(providerPtr)逻辑,由于它还有不少父类初始化的工做。该方法的源码以下:
template <typename TClientBase>
status_t Camera2ClientBase<TClientBase>::initialize(sp<CameraProviderManager> manager) {
return initializeImpl(manager);
}
template <typename TClientBase>
template <typename TProviderPtr>
status_t Camera2ClientBase<TClientBase>::initializeImpl(TProviderPtr providerPtr) {
ATRACE_CALL();
ALOGV("%s: Initializing client for camera %s", __FUNCTION__,
TClientBase::mCameraIdStr.string());
status_t res;
// Verify ops permissions
res = TClientBase::startCameraOps();
if (res != OK) {
return res;
}
if (mDevice == NULL) {
ALOGE("%s: Camera %s: No device connected",
__FUNCTION__, TClientBase::mCameraIdStr.string());
return NO_INIT;
}
res = mDevice->initialize(providerPtr);
if (res != OK) {
ALOGE("%s: Camera %s: unable to initialize device: %s (%d)",
__FUNCTION__, TClientBase::mCameraIdStr.string(), strerror(-res), res);
return res;
}
wp<CameraDeviceBase::NotificationListener> weakThis(this);
res = mDevice->setNotifyCallback(weakThis);
return OK;
}
它也是一个模板方法,范型TProviderPtr就是咱们前面从CameraService类中调用过来时传入的的参数mCameraProviderManager,它的类型为CameraProviderManager。咱们一开始说咱们能够经过调用CameraManager类的getCameraIdList()方法来获取到当前已成功注册的相机设备的id,那到底是怎么获取到的呢?就是经过CameraProviderManager类中提供的数据来获取的,由于每个相机设备的注册信息都会保存在该类中,因此它知道全部的相机设备信息,能够直接返回给咱们。接下来咱们继续分析mDevice->initialize(providerPtr)这句逻辑的实现。这里的mDevice就是咱们前面初始化好的Camera3Device了,frameworks\av\services\camera\libcameraservice\device3\Camera3Device.cpp类的initialize方法的源码以下:
status_t Camera3Device::initialize(sp<CameraProviderManager> manager) {
ATRACE_CALL();
Mutex::Autolock il(mInterfaceLock);
Mutex::Autolock l(mLock);
ALOGV("%s: Initializing HIDL device for camera %s", __FUNCTION__, mId.string());
if (mStatus != STATUS_UNINITIALIZED) {
CLOGE("Already initialized!");
return INVALID_OPERATION;
}
if (manager == nullptr) return INVALID_OPERATION;
sp<ICameraDeviceSession> session;
ATRACE_BEGIN("CameraHal::openSession");
status_t res = manager->openSession(mId.string(), this,
/*out*/ &session);
ATRACE_END();
if (res != OK) {
SET_ERR_L("Could not open camera session: %s (%d)", strerror(-res), res);
return res;
}
res = manager->getCameraCharacteristics(mId.string(), &mDeviceInfo);
if (res != OK) {
SET_ERR_L("Could not retrive camera characteristics: %s (%d)", strerror(-res), res);
session->close();
return res;
}
std::shared_ptr<RequestMetadataQueue> queue;
auto requestQueueRet = session->getCaptureRequestMetadataQueue(
[&queue](const auto& descriptor) {
queue = std::make_shared<RequestMetadataQueue>(descriptor);
if (!queue->isValid() || queue->availableToWrite() <= 0) {
ALOGE("HAL returns empty request metadata fmq, not use it");
queue = nullptr;
// don't use the queue onwards.
}
});
if (!requestQueueRet.isOk()) {
ALOGE("Transaction error when getting request metadata fmq: %s, not use it",
requestQueueRet.description().c_str());
return DEAD_OBJECT;
}
auto resultQueueRet = session->getCaptureResultMetadataQueue(
[&queue = mResultMetadataQueue](const auto& descriptor) {
queue = std::make_unique<ResultMetadataQueue>(descriptor);
if (!queue->isValid() || queue->availableToWrite() <= 0) {
ALOGE("HAL returns empty result metadata fmq, not use it");
queue = nullptr;
// Don't use the queue onwards.
}
});
if (!resultQueueRet.isOk()) {
ALOGE("Transaction error when getting result metadata queue from camera session: %s",
resultQueueRet.description().c_str());
return DEAD_OBJECT;
}
mInterface = std::make_unique<HalInterface>(session, queue);
std::string providerType;
mVendorTagId = manager->getProviderTagIdLocked(mId.string());
return initializeCommonLocked();
}
这里有一句很是重要的逻辑status_t res = manager->openSession(mId.string(), this, /*out*/ &session),它会在CameraDaemon进程中执行真正的camera的open过程,同时会返回一个session对象给咱们,前面咱们说过,相机核心的事务其中就包括createCaptureSession,固然两个session不是一个意思,可是这里openSession的逻辑拿到的东西却很是重要,这个咱们放到之后再说。最后继续调用initializeCommonLocked完成初始化。initializeCommonLocked方法的源码以下:
status_t Camera3Device::initializeCommonLocked() { /** Start up status tracker thread */ mStatusTracker = new StatusTracker(this); status_t res = mStatusTracker->run(String8::format("C3Dev-%s-Status", mId.string()).string()); if (res != OK) { SET_ERR_L("Unable to start status tracking thread: %s (%d)", strerror(-res), res); mInterface->close(); mStatusTracker.clear(); return res; } /** Register in-flight map to the status tracker */ mInFlightStatusId = mStatusTracker->addComponent(); /** Create buffer manager */ mBufferManager = new Camera3BufferManager(); mTagMonitor.initialize(mVendorTagId); /** Start up request queue thread */ mRequestThread = new RequestThread(this, mStatusTracker, mInterface.get()); res = mRequestThread->run(String8::format("C3Dev-%s-ReqQueue", mId.string()).string()); if (res != OK) { SET_ERR_L("Unable to start request queue thread: %s (%d)", strerror(-res), res); mInterface->close(); mRequestThread.clear(); return res; } mPreparerThread = new PreparerThread(); internalUpdateStatusLocked(STATUS_UNCONFIGURED); mNextStreamId = 0; mDummyStreamId = NO_STREAM; mNeedConfig = true; mPauseStateNotify = false; // Measure the clock domain offset between camera and video/hw_composer camera_metadata_entry timestampSource = mDeviceInfo.find(ANDROID_SENSOR_INFO_TIMESTAMP_SOURCE); if (timestampSource.count > 0 && timestampSource.data.u8[0] == ANDROID_SENSOR_INFO_TIMESTAMP_SOURCE_REALTIME) { mTimestampOffset = getMonoToBoottimeOffset(); } // Will the HAL be sending in early partial result metadata? camera_metadata_entry partialResultsCount = mDeviceInfo.find(ANDROID_REQUEST_PARTIAL_RESULT_COUNT); if (partialResultsCount.count > 0) { mNumPartialResults = partialResultsCount.data.i32[0]; mUsePartialResult = (mNumPartialResults > 1); } camera_metadata_entry configs = mDeviceInfo.find(ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS); for (uint32_t i = 0; i < configs.count; i += 4) { if (configs.data.i32[i] == HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED && configs.data.i32[i + 3] == ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS_INPUT) { mSupportedOpaqueInputSizes.add(Size(configs.data.i32[i + 1], configs.data.i32[i + 2])); } } return OK; } 这里又出现了几个对象:Camera3BufferManager、RequestThread、PreparerThread。Camera3BufferManager是用来管理buffer的,RequestThread是用来处理预览的request的,PreparerThread是用来做一些准备工做的,你们知道,由于相机预览是一帧接一帧,不断的在建立、处理请求,频率很是高,通常是30fps,固然,比Android的Vsync信号同步的16ms仍是要慢,因此这里开了多条线程用来同时处理不一样的工做,以保证相机的效率。 好,到这里,咱们这篇博客也就介绍完了,如今回头一想,是否是很奇怪,咱们要讲的是openCamera的逻辑,可是从头至尾就只看到了一些对象的建立,没有真正的打开相机设备啊?这也就是咱们上面说的status_t res = manager->openSession(mId.string(), this, /*out*/ &session)逻辑的重要性了,真正打开相机的逻辑就是从这里进去处理的,这个咱们放在下一次继续讲,openCamera后续还有很大的一段逻辑要咱们分析,完成以后,咱们才能大致明白相机究竟是怎么打开的。