没有免费的午饭定理(No Free Lunch Theorem),这个定理说明html
若学习算法
L
a
L_a
L a
L
b
L_b
L b
L
b
L_b
L b
L
a
L_a
L a web
这里说的表现好就是前面所说的泛化能力更强。而后出现了下面这个公式算法
E
o
t
e
(
L
a
∣
X
,
f
)
=
∑
h
∑
x
∈
χ
−
X
P
(
x
)
I
(
h
(
x
)
≠
f
(
x
)
)
P
(
h
∣
X
,
L
a
)
E_{ote}(L_a|X,f) = \sum_{h}\sum_{x\in\chi-X}P(x)\mathbb{I}(h(x)\neq f(x))P(h|X,L_a)
E o t e ( L a ∣ X , f ) = ∑ h ∑ x ∈ χ − X P ( x ) I ( h ( x ) ̸ = f ( x ) ) P ( h ∣ X , L a ) app
使人生畏的长公式，不过咱们来依次解读它。svg
留坑，周末更
χ
\chi
χ 函数
H
H
H 学习
L
a
L_a
L a
L
a
L_a
L a
L
b
L_b
L b 测试
P
(
h
∣
X
,
L
a
)
P(h|X,L_a)
P ( h ∣ X , L a )
L
a
L_a
L a
X
X
X
h
h
h
L
a
L_a
L a
X
X
X
h
h
h
y
=
h
(
x
)
y=h(x)
y = h ( x )
X
X
X
X
X
X
h
h
h
P
(
h
∣
X
,
L
a
)
P(h|X,L_a)
P ( h ∣ X , L a )
h
h
h spa
f
f
f .net
好，上面那个公式到了
E
o
t
e
(
L
a
∣
X
,
f
)
=
∑
h
∑
x
∈
χ
−
X
P
(
x
)
I
(
h
(
x
)
≠
f
(
x
)
)
P
(
h
∣
X
,
L
a
)
E_{ote}(L_a|X,f) = \sum_{h}\sum_{x\in\chi-X}P(x)\mathbb{I}(h(x)\neq f(x))P(h|X,L_a)
E o t e ( L a ∣ X , f ) = ∑ h ∑ x ∈ χ − X P ( x ) I ( h ( x ) ̸ = f ( x ) ) P ( h ∣ X , L a )
首先看这个
E
E
E
E
E
E
o
t
e
_{ote}
o t e
E
o
t
e
(
L
a
∣
X
,
f
)
E_{ote}(L_a|X,f)
E o t e ( L a ∣ X , f )
L
a
L_a
L a
P
(
x
)
P(x)
P ( x )
P
(
x
)
P(x)
P ( x )
I
(
h
(
x
)
≠
f
(
x
)
)
\mathbb{I}(h(x)\neq f(x))
I ( h ( x ) ̸ = f ( x ) )
P
(
h
∣
X
,
L
a
)
P(h|X,L_a)
P ( h ∣ X , L a )
L
a
L_a
L a
X
X
X
h
h
h
其实这里最开始最令我困惑的是什么呢？是两个求和符号，由于这里求和符号感受不规范啊！有木有，不事后来以为理解了意思就大概行了.
∑
h
\sum_h
∑ h
χ
{\chi}
χ
P
(
h
∣
X
,
L
a
)
P(h|X,L_a)
P ( h ∣ X , L a )
∑
x
∈
χ
−
X
\sum_{x\in{\chi-X}}
∑ x ∈ χ − X
好了，公式的每一部分都说清楚了，来总体理解一下，这个公式就是说:
L
a
L_a
L a
h
h
h
而后下面考虑二分类问题，先要说明，对于咱们想要求得的真实目标函数
f
f
f
f
f
f
{
0
,
1
}
\left \{ 0,1 \right\}
{ 0 , 1 }
χ
\chi
χ
x
x
x
f
1
(
x
)
=
0
f_1(x)=0
f 1 ( x ) = 0
f
2
(
x
)
=
1
f_2(x) = 1
f 2 ( x ) = 1
∣
χ
∣
\left | \chi \right|
∣ χ ∣
2
∣
χ
∣
2^{\left | \chi \right |}
2 ∣ χ ∣
h
(
x
)
h(x)
h ( x )
h
(
x
)
=
0
h(x) = 0
h ( x ) = 0
1
2
\frac{1}{2}
2 1
$\sum_f E_{ote}\left ( L_a|X,f\right) $
=
∑
f
∑
h
∑
x
∈
χ
−
X
P
(
x
)
I
(
h
(
x
)
≠
f
(
x
)
)
P
(
h
∣
X
,
L
a
)
= \sum_f\sum_h\sum_{x\in\chi-X}P(x)\mathbb{I}(h(x)\neq f(x))P(h|X,L_a)
= ∑ f ∑ h ∑ x ∈ χ − X P ( x ) I ( h ( x ) ̸ = f ( x ) ) P ( h ∣ X , L a )
=
∑
x
∈
χ
−
X
P
(
x
)
∑
h
P
(
h
∣
X
,
L
a
)
∑
f
I
(
h
(
x
)
≠
f
(
x
)
)
=\sum_{x\in\chi-X}P(x)\sum_hP(h|X,L_a)\sum_f\mathbb{I}(h(x)\neq f(x))
= ∑ x ∈ χ − X P ( x ) ∑ h P ( h ∣ X , L a ) ∑ f I ( h ( x ) ̸ = f ( x ) )
=
∑
x
∈
χ
−
X
P
(
x
)
∑
h
P
(
h
∣
X
,
L
a
)
1
2
2
∣
χ
∣
=\sum_{x\in\chi-X}P(x)\sum_hP(h|X,L_a)\frac{1}{2}2^{\left | \chi \right |}
= ∑ x ∈ χ − X P ( x ) ∑ h P ( h ∣ X , L a ) 2 1 2 ∣ χ ∣
=
1
2
2
∣
χ
∣
∑
x
∈
χ
−
X
P
(
x
)
∑
h
P
(
h
∣
X
,
L
a
)
=\frac{1}{2}2^{\left | \chi \right |}\sum_{x\in\chi-X}P(x)\sum_hP(h|X,L_a)
= 2 1 2 ∣ χ ∣ ∑ x ∈ χ − X P ( x ) ∑ h P ( h ∣ X , L a )
=
2
∣
χ
∣
−
1
∑
x
∈
χ
−
X
P
(
x
)
⋅
1
=2^{\left | \chi \right |-1}\sum_{x\in\chi-X}P(x)\cdot 1
= 2 ∣ χ ∣ − 1 ∑ x ∈ χ − X P ( x ) ⋅ 1
第一步是怎么推导出来的呢？这里涉及到一个求和运算
a
i
∈
{
a
1
,
a
2
,
.
.
.
,
a
m
}
a_i\in \left\{ a_1,a_2,...,a_m \right\}
a i ∈ { a 1 , a 2 , . . . , a m }
b
j
∈
{
b
1
,
b
2
,
.
.
.
,
b
n
}
b_j\in \left\{ b_1,b_2,...,b_n \right\}
b j ∈ { b 1 , b 2 , . . . , b n }
c
k
∈
{
c
1
,
c
2
,
.
.
.
,
c
o
}
c_k\in \left\{ c_1,c_2,...,c_o \right\}
c k ∈ { c 1 , c 2 , . . . , c o }
那么
∑
i
m
∑
j
n
∑
k
o
a
i
b
j
c
k
\sum_i^m\sum_j^n\sum_k^oa_ib_jc_k
∑ i m ∑ j n ∑ k o a i b j c k
=
∑
i
m
a
i
∑
j
n
b
j
∑
k
o
c
k
=\sum_i^ma_i\sum_j^nb_j\sum_k^oc_k
= ∑ i m a i ∑ j n b j ∑ k o c k
(
a
1
+
a
2
+
.
.
.
+
a
m
)
(
b
1
+
b
2
+
.
.
.
+
b
n
)
(
c
1
+
c
2
+
.
.
.
+
c
o
)
(a_1+a_2+...+a_m)(b_1+b_2+...+b_n)(c_1+c_2+...+c_o)
( a 1 + a 2 + . . . + a m ) ( b 1 + b 2 + . . . + b n ) ( c 1 + c 2 + . . . + c o )
a
1
b
1
c
1
+
a
2
b
1
c
1
+
.
.
.
a
m
b
1
c
1
+
.
.
.
+
a
m
b
n
c
o
{a_1}{b_1}{c_1}+{a_2}{b_1}{c_1}+...{a_m}{b_1}{c_1}+...+{a_m}{b_n}{c_o}
a 1 b 1 c 1 + a 2 b 1 c 1 + . . . a m b 1 c 1 + . . . + a m b n c o
第二步
∑
f
I
(
h
(
x
)
≠
f
(
x
)
)
=
1
2
2
∣
χ
∣
\sum_f\mathbb{I}(h(x)\neq f(x)) =\frac{1}{2}2^{\left | \chi \right |}
∑ f I ( h ( x ) ̸ = f ( x ) ) = 2 1 2 ∣ χ ∣
2
∣
χ
∣
2^{\left | \chi \right |}
2 ∣ χ ∣
f
f
f
f
(
x
)
=
1
f(x) = 1
f ( x ) = 1
f
(
x
)
=
0
f(x)=0
f ( x ) = 0
f
f
f
h
(
x
)
h\left( x \right)
h ( x )
h
(
x
)
=
0
h(x)=0
h ( x ) = 0
1
1
1
f
(
x
)
f(x)
f ( x )
1
2
2
∣
χ
∣
\frac{1}{2}2^{\left | \chi \right |}
2 1 2 ∣ χ ∣
第三步到第四步就更好理解了
通过这么一通推导后，发现得出指望的表达式中关于没有具体算法的，因此是算法无关的! 若是错误，望指出共交流，共学习！