Android Pie Feature: Restrictions on use of non-SDK interfaces

时间 2019-11-24

标签 android pie feature restrictions use non sdk interfaces 栏目 Android 繁體版

原文原文链接

今年的6月12号 David Brazdil 和 Nicolas Geoffray 在 Android Developers Blog 上 post 了一篇题名 An Update on non-SDK restrictions in Android P 的文章。大体内容是他们发现不少开发者使用 non-SDK interfaces，这致使用户的崩溃上涨以及开发者不断采起应急措施，Google但愿从这个版本开始着手解决此问题。2月份推送的 Developer Preview 和 Beta 1 版本上会经过 Toast 或 logcat 指示使用 non-SDK interfaces，另提供 StrictMode 检测自家应用 restrictions 的行为。html

啰嗦一下，David Brazdil 是一名 Android ART compiler developer，hiddenapi这个Feature大部分commiter是他，review 这些能够帮助快速理解hiddenapi具体作了哪些。若是你感兴趣，能够从下图蓝色星星的提交开始阅读向上撸。说明下，non-SDK interfaces是口头语，hiddenapi才是代码层面的术语。java

android-review.googlesource.com/q/owner:dbr…linux

镇定一下，正式分析开始。android

首先，虚拟机加载的是dex文件（oat牵扯东西太多，因而没有它的戏份），app developer经常使用的Api好比Activity.java / View.java是通过aosp源代码编译生成classes.dex进而打包到framework.jar中，还好比okHttp.jar。api

简单说明下dex format，它是一种8位字节的二进制流文件，各个数据按顺序紧密的排列，无间隙且整个应用中全部的Java源文件都放在一个dex中架构

上图中的文件头部分，记录了dex文件的信息，全部字段一个分部；索引区部分，主要包含字符串、类型、方法原型、域、方法的索引；索引区最终又被存储在数据区，其中连接数据区，主要存储动态连接库，so库的信息。app

下图以./leakcanary-sample/build/intermediates/transforms/dexBuilder/debug/11/com/example/leakcanary/MainActivity.dex为例。恰好此MainActivity.java包含了private / protected / public三个modifier的Filed / Method，public这里指的是<init>，它表明实例的初始化方法，还有一个<cinit>表明在jvm第一次加载class文件时调用，包括静态变量初始化语句和静态块的执行。jvm

dex文件中的register字段: Dalvik 最初目标是运行在以ARM 作CPU 的机器上的，ARM 芯片的一个主要特色是寄存器多。寄存器多的话有好处，就是能够把操做数放在寄存器里，而不是像传统VM 同样放在栈中。天然，操做寄存器是比操做内存（栈嘛，其实就是一块内存区域）快。registers 变量表示该方法运行过程当中会使用多少个寄存器。ide

注意到access字段了吗？它表明访问标志，定义在如下两个结构体中，hiddenapi思路即是从access_flag下手。出于的考虑是不增长dex size，也没有代码侵入dex的gen过程，透过./host/linux-x86/bin/hiddenapi从新encode一次dex文件，修改access_flag的动做 David Brazdil 称为encode。函数

// A decoded version of the field of a class_data_item
  struct ClassDataField {
    uint32_t field_idx_delta_;  // delta of index into the field_ids array for FieldId
    uint32_t access_flags_;  // access flags for the field
    ClassDataField() :  field_idx_delta_(0), access_flags_(0) {}

   private:
    DISALLOW_COPY_AND_ASSIGN(ClassDataField);
  };
  ClassDataField field_;

  // Read and decode a field from a class_data_item stream into field
  void ReadClassDataField();

  // A decoded version of the method of a class_data_item
  struct ClassDataMethod {
    uint32_t method_idx_delta_;  // delta of index into the method_ids array for MethodId
    uint32_t access_flags_;
    uint32_t code_off_;
    ClassDataMethod() : method_idx_delta_(0), access_flags_(0), code_off_(0) {}

   private:
    DISALLOW_COPY_AND_ASSIGN(ClassDataMethod);
  };
复制代码

下面看一下hiddenapi如何encode过程， 举例aosp编译okhttp.jar,

aosp自7.x开始逐步迁移到Go语言构建build system，为了兼容mk增长一层映射，看到 BUILD_JAVA_LIBRARY 很熟悉了吧。

编译的末尾环节调用hiddenapi加工dex，执行hiddenapi --dex=xx/classes1.dex --light-greylist=? --dark-greylist=? --blacklist=?

# ./build/core/java.mk
ifneq ($(filter $(LOCAL_MODULE),$(PRODUCT_BOOT_JARS)),) # is_boot_jar
  $(eval $(call hiddenapi-copy-dex-files,$(built_dex_intermediate),$(built_dex_hiddenapi)))
  built_dex_copy_from := $(built_dex_hiddenapi)
else # !is_boot_jar
  built_dex_copy_from := $(built_dex_intermediate)
endif # is_boot_jar

# ./build/core/definitions.mk
define hiddenapi-copy-dex-files
$(2): $(1) $(HIDDENAPI) $(INTERNAL_PLATFORM_HIDDENAPI_LIGHT_GREYLIST) \
      $(INTERNAL_PLATFORM_HIDDENAPI_DARK_GREYLIST) $(INTERNAL_PLATFORM_HIDDENAPI_BLACKLIST)
	@rm -rf $(dir $(2))
	@mkdir -p $(dir $(2))
	find $(dir $(1)) -maxdepth 1 -name "classes*.dex" | sort | \
		xargs -I{} cp -f {} $(dir $(2))
	find $(dir $(2)) -name "classes*.dex" | sort | sed 's/^/--dex=/' | \
		xargs $(HIDDENAPI) --light-greylist=$(INTERNAL_PLATFORM_HIDDENAPI_LIGHT_GREYLIST) \
		                   --dark-greylist=$(INTERNAL_PLATFORM_HIDDENAPI_DARK_GREYLIST) \
		                   --blacklist=$(INTERNAL_PLATFORM_HIDDENAPI_BLACKLIST)
endef
复制代码

light-greylist / dark-greylist / blacklist 三个文件由external/doclava解析生成（包括检查@hiden修饰，public / private / proteced / 包级别修饰等），这三个文件有默认，在以下路径。咱们在最开始logcat中看到的Notification#isGroupChild，它在private list中，格式嘛：Methods are encoded as: class_descriptor->method_name(parameter_types)return_type，Fields are encoded as: class_descriptor->field_name:field_type

回到encode的逻辑，它是一个binary文件，代码在./art/tools/hiddenapi/hiddenapi.cc，处理过程很是简单，解析参数、修改access_flag。

OpenApiFile函数作的事儿:读取上面三个文件，OpenDexFiles函数经过ArtDexFileLoader加载dex文件并建立dexFile结构体，它对应dex format描述。

//./art/tools/hiddenapi/hiddenapi.cc
  // Paths to text files which contain the lists of API members.
  std::string light_greylist_path_;
  std::string dark_greylist_path_;
  std::string blacklist_path_;
复制代码

重要逻辑在CategorizeAllClasses方法中，而后盯着SetHidden，修改每个dexclass最后更新Checksums。阅读代码时注意下access_flag在规范中指定LEB128编码，它是一中可变长度编码，表示任意有符号或无符号整数的。在 .dex 文件中，LEB128 仅用于对 32 位数字进行编码。实在不懂百度一下LEB编解码过程，很基础的东西。

至于为何要编码，请阅读《信息论》通讯专业必修课。

void CategorizeAllClasses(const DexFile& dex_file) {
    for (uint32_t class_idx = 0; class_idx < dex_file.NumClassDefs(); ++class_idx) {
      DexClass klass(dex_file, class_idx);
      const uint8_t* klass_data = klass.GetData();
      if (klass_data == nullptr) {
        continue;
      }

      for (ClassDataItemIterator it(klass.GetDexFile(), klass_data); it.HasNext(); it.Next()) {
        DexMember member(klass, it);

        // Catagorize member and overwrite its access flags.
        // Note that if a member appears on multiple API lists, it will be categorized
        // as the strictest.
        bool is_hidden = true;
        if (member.IsOnApiList(blacklist_)) {
          member.SetHidden(HiddenApiAccessFlags::kBlacklist);
        } else if (member.IsOnApiList(dark_greylist_)) {
          member.SetHidden(HiddenApiAccessFlags::kDarkGreylist);
        } else if (member.IsOnApiList(light_greylist_)) {
          member.SetHidden(HiddenApiAccessFlags::kLightGreylist);
        } else {
          member.SetHidden(HiddenApiAccessFlags::kWhitelist);
          is_hidden = false;
        }

        if (print_hidden_api_ && is_hidden) {
          std::cout << member.GetApiEntry() << std::endl;
        }
      }
    }
  }
  
  // Sets hidden bits in access flags and writes them back into the DEX in memory.
  // Note that this will not update the cached data of ClassDataItemIterator
  // until it iterates over this item again and therefore will fail a CHECK if
  // it is called multiple times on the same DexMember.
  void SetHidden(HiddenApiAccessFlags::ApiList value) {
    const uint32_t old_flags = it_.GetRawMemberAccessFlags();
    const uint32_t new_flags = HiddenApiAccessFlags::EncodeForDex(old_flags, value);
    CHECK_EQ(UnsignedLeb128Size(new_flags), UnsignedLeb128Size(old_flags));

    // Locate the LEB128-encoded access flags in class data.
    // `ptr` initially points to the next ClassData item. We iterate backwards
    // until we hit the terminating byte of the previous Leb128 value.
    const uint8_t* ptr = it_.DataPointer();
    if (it_.IsAtMethod()) {
      ptr = ReverseSearchUnsignedLeb128(ptr);
      DCHECK_EQ(DecodeUnsignedLeb128WithoutMovingCursor(ptr), it_.GetMethodCodeItemOffset());
    }
    ptr = ReverseSearchUnsignedLeb128(ptr);
    DCHECK_EQ(DecodeUnsignedLeb128WithoutMovingCursor(ptr), old_flags);

    // Overwrite the access flags.
    UpdateUnsignedLeb128(const_cast<uint8_t*>(ptr), new_flags);
  }
复制代码

CategorizeAllClasses完事儿后，dex中每个class的成员（field / method）根据三个名单打上kBlacklist / kDarkGreylist / kLightGreylist / kWhitelist 标签。OK，接下来是runtime。

Runtime和Art虚拟机实例指得一回事儿，天然在./art/runtime/jni_internal.cc中。找方法和找变量见FindMethodID与FindFieldID，关键路径上关注ShouldBlockAccessToMember方法。

static jmethodID FindMethodID(ScopedObjectAccess& soa, jclass jni_class, const char* name, const char* sig, bool is_static) REQUIRES_SHARED(Locks::mutator_lock_) {
  ObjPtr<mirror::Class> c = EnsureInitialized(soa.Self(), soa.Decode<mirror::Class>(jni_class));
  if (c == nullptr) {
    return nullptr;
  }
  ArtMethod* method = nullptr;
  auto pointer_size = Runtime::Current()->GetClassLinker()->GetImagePointerSize();
  if (c->IsInterface()) {
    method = c->FindInterfaceMethod(name, sig, pointer_size);
  } else {
    method = c->FindClassMethod(name, sig, pointer_size);
  }
  if (method != nullptr && ShouldBlockAccessToMember(method, soa.Self())) {
    method = nullptr;
  }
  if (method == nullptr || method->IsStatic() != is_static) {
    ThrowNoSuchMethodError(soa, c, name, sig, is_static ? "static" : "non-static");
    return nullptr;
  }
  return jni::EncodeArtMethod(method);
}

static jfieldID FindFieldID(const ScopedObjectAccess& soa, jclass jni_class, const char* name, const char* sig, bool is_static) REQUIRES_SHARED(Locks::mutator_lock_) {
  StackHandleScope<2> hs(soa.Self());
  Handle<mirror::Class> c(
      hs.NewHandle(EnsureInitialized(soa.Self(), soa.Decode<mirror::Class>(jni_class))));
  if (c == nullptr) {
    return nullptr;
  }
  ArtField* field = nullptr;
  mirror::Class* field_type;
  ClassLinker* class_linker = Runtime::Current()->GetClassLinker();
  if (sig[1] != '\0') {
    Handle<mirror::ClassLoader> class_loader(hs.NewHandle(c->GetClassLoader()));
    field_type = class_linker->FindClass(soa.Self(), sig, class_loader);
  } else {
    field_type = class_linker->FindPrimitiveClass(*sig);
  }
  if (field_type == nullptr) {
    // Failed to find type from the signature of the field.
    DCHECK(soa.Self()->IsExceptionPending());
    StackHandleScope<1> hs2(soa.Self());
    Handle<mirror::Throwable> cause(hs2.NewHandle(soa.Self()->GetException()));
    soa.Self()->ClearException();
    std::string temp;
    soa.Self()->ThrowNewExceptionF("Ljava/lang/NoSuchFieldError;",
                                   "no type \"%s\" found and so no field \"%s\" "
                                   "could be found in class \"%s\" or its superclasses", sig, name,
                                   c->GetDescriptor(&temp));
    soa.Self()->GetException()->SetCause(cause.Get());
    return nullptr;
  }
  std::string temp;
  if (is_static) {
    field = mirror::Class::FindStaticField(
        soa.Self(), c.Get(), name, field_type->GetDescriptor(&temp));
  } else {
    field = c->FindInstanceField(name, field_type->GetDescriptor(&temp));
  }
  if (field != nullptr && ShouldBlockAccessToMember(field, soa.Self())) {
    field = nullptr;
  }
  if (field == nullptr) {
    soa.Self()->ThrowNewExceptionF("Ljava/lang/NoSuchFieldError;",
                                   "no \"%s\" field \"%s\" in class \"%s\" or its superclasses",
                                   sig, name, c->GetDescriptor(&temp));
    return nullptr;
  }
  return jni::EncodeArtField(field);
}
复制代码

ShouldBlockAccessToMember这方法用来check来自Reflection / JNI / Link的restricting invoke，link还记得发生在什么阶段吗？类的加载过程。 oat的方法调用，虚拟机加载类文件后经过蹦床（trampoline）函数找到方法入口地址。

enum AccessMethod {
  kNone,  // internal test that does not correspond to an actual access by app
  kReflection,
  kJNI,
  kLinking,
};

template<typename T>
ALWAYS_INLINE static bool ShouldBlockAccessToMember(T* member, Thread* self)
    REQUIRES_SHARED(Locks::mutator_lock_) {
  hiddenapi::Action action = hiddenapi::GetMemberAction(
      member, self, IsCallerTrusted, hiddenapi::kJNI);
  if (action != hiddenapi::kAllow) {
    hiddenapi::NotifyHiddenApiListener(member);
  }

  return action == hiddenapi::kDeny;
}

template<typename T>
inline Action GetMemberAction(T* member,
                              Thread* self,
                              std::function<bool(Thread*)> fn_caller_is_trusted,
                              AccessMethod access_method)
    REQUIRES_SHARED(Locks::mutator_lock_) {
  DCHECK(member != nullptr);

  // Decode hidden API access flags.
  // NB Multiple threads might try to access (and overwrite) these simultaneously,
  // causing a race. We only do that if access has not been denied, so the race
  // cannot change Java semantics. We should, however, decode the access flags
  // once and use it throughout this function, otherwise we may get inconsistent
  // results, e.g. print whitelist warnings (b/78327881).
  HiddenApiAccessFlags::ApiList api_list = member->GetHiddenApiAccessFlags();

  Action action = GetActionFromAccessFlags(member->GetHiddenApiAccessFlags());
  if (action == kAllow) {
    // Nothing to do.
    return action;
  }

  // Member is hidden. Invoke `fn_caller_in_platform` and find the origin of the access.
  // This can be *very* expensive. Save it for last.
  if (fn_caller_is_trusted(self)) {
    // Caller is trusted. Exit.
    return kAllow;
  }

  // Member is hidden and caller is not in the platform.
  return detail::GetMemberActionImpl(member, api_list, action, access_method);
}
复制代码

GetHiddenApiAccessFlags就是前面取出encode的access flag，只是此处叫HiddenApiAccessFlags。

到这里，基本Feature的workflow讲解完毕，仍然存在不少细节没有描述，好比：

doclava如何分类名单；
虚拟机相关都没有覆盖，虚拟机解决架构上两大需求，core-java加载，GC；
non-SDK interfaces对system_server、system app、other app如何区别对待，ApplicationInfo / Zygote有作一些改动。

昨天从Japan返来，调整下，明天开工啦