一个二分类模型，寻找决策边界最宽的一个超平面将数据进行分割
距离
求点到平面的距离，只要求点到点在方向量上的映射即可
w
T
X
+
b
=
0
w^{T}X+b=0
w T X + b = 0
X
′
X
′
′
⃗
=
(
X
′
′
−
X
′
)
\vec{{X'X''}}=(X''-X')
X ′ X ′ ′
= ( X ′ ′ − X ′ )
w
T
X
′
=
−
b
,
w
T
X
′
′
=
−
b
w^{T}X' = -b,w^{T}X'' = -b
w T X ′ = − b , w T X ′ ′ = − b
W
T
W^{T}
W T
(
w
T
(
x
′
′
−
x
′
)
)
=
0
\left(w^{T}(x''-x') \right)=0
( w T ( x ′ ′ − x ′ ) ) = 0
d
=
∣
w
T
∣
w
∣
(
X
−
X
′
)
∣
=
1
∣
w
∣
∣
w
T
x
+
b
∣
d=\left|\frac {w^{T}}{|w|}(X-X') \right|=\frac {1}{|w|}|w^{T}x+b|
d = ∣ ∣ ∣ ∣ ∣ w ∣ w T ( X − X ′ ) ∣ ∣ ∣ ∣ = ∣ w ∣ 1 ∣ w T x + b ∣ 目标
y
(
x
)
=
w
T
Φ
(
x
i
)
+
b
y_(x)=w^{T}\Phi(x_i)+b
y ( x ) = w T Φ ( x i ) + b
y
(
x
i
)
>
0
<
=
>
y
i
=
+
1
y_{(x_i)>0} <=>y_{i}=+1
y ( x i ) > 0 < = > y i = + 1
y
(
x
i
)
<
0
<
=
>
y
i
=
−
1
y_{(x_i)<0} <=>y_{i}=-1
y ( x i ) < 0 < = > y i = − 1
y
i
∗
y
(
x
i
)
>
0
y_{i}*y_(x_i)>0
y i ∗ y ( x i ) > 0
y
i
∗
(
w
T
Φ
(
x
i
)
+
b
)
∣
∣
W
∣
∣
\frac{y_{i}*(w^{T}\Phi(x_i)+b)}{||W||}
∣ ∣ W ∣ ∣ y i ∗ ( w T Φ ( x i ) + b )
m
a
x
{
1
∣
∣
W
∣
∣
m
i
n
{
y
i
∗
(
w
T
Φ
(
x
i
)
+
b
)
}
}
max\lbrace\frac{1}{||W||}min\lbrace{y_{i}*(w^{T}\Phi(x_i)+b)}\rbrace\rbrace
m a x { ∣ ∣ W ∣ ∣ 1 m i n { y i ∗ ( w T Φ ( x i ) + b ) } }
y
(
i
)
∗
y
(
x
i
)
>
0
y_{(i)}*y_{(x_i)}>0
y ( i ) ∗ y ( x i ) > 0
y
(
i
)
∗
y
(
x
i
)
≥
1
y_{(i)}*y_{(x_i)}≥1
y ( i ) ∗ y ( x i ) ≥ 1
y
i
∗
(
w
T
Φ
(
x
i
)
+
b
)
≥
1
y_{i}*(w^{T}\Phi(x_i)+b)≥1
y i ∗ ( w T Φ ( x i ) + b ) ≥ 1
y
i
∗
(
w
T
Φ
(
x
i
)
+
b
)
y_{i}*(w^{T}\Phi(x_i)+b)
y i ∗ ( w T Φ ( x i ) + b )
m
a
x
1
∣
∣
W
∣
∣
max\frac{1}{||W||}
m a x ∣ ∣ W ∣ ∣ 1
y
i
∗
(
w
T
Φ
(
x
i
)
+
b
)
≥
1
y_{i}*(w^{T}\Phi(x_i)+b)≥1
y i ∗ ( w T Φ ( x i ) + b ) ≥ 1
m
i
n
w
,
b
1
2
w
2
min_{w,b}\frac{1}{2}w^2
m i n w , b 2 1 w 2
先转化
带约束的优化问题：
m
i
n
x
f
0
(
x
)
min_{x} f_{0}(x)
m i n x f 0 ( x )
f
i
(
x
)
≤
0
,
i
=
1
,
.
.
.
m
f_i(x)≤0,i=1,...m
f i ( x ) ≤ 0 , i = 1 , . . . m
h
i
(
x
)
≤
0
,
i
=
1
,
.
.
.
q
h_i(x)≤0,i=1,...q
h i ( x ) ≤ 0 , i = 1 , . . . q
m
i
n
L
(
x
,
λ
,
v
)
=
f
(
x
)
+
∑
i
=
1
m
λ
i
f
i
(
x
)
+
∑
i
=
1
q
v
i
h
i
(
x
)
min L_{(x,λ,v)}=f_{(x)}+\sum_{i=1}^{m}λ_{i}f_{i}(x)+\sum_{i=1}^{q}v_{i}h_{i}(x)
m i n L ( x , λ , v ) = f ( x ) + i = 1 ∑ m λ i f i ( x ) + i = 1 ∑ q v i h i ( x )
L
(
w
,
b
,
a
)
=
1
2
∣
∣
w
∣
∣
2
−
∑
i
=
1
n
α
i
(
y
i
(
w
T
∗
Φ
(
x
i
)
+
b
)
−
1
)
L(w,b,a)=\frac{1}{2}||w||^2-\sum_{i=1}^{n}α_i(y_i(w^T*Φ_{(x_i)+b})-1)
L ( w , b , a ) = 2 1 ∣ ∣ w ∣ ∣ 2 − i = 1 ∑ n α i ( y i ( w T ∗ Φ ( x i ) + b ) − 1 )
y
i
∗
(
w
T
Φ
(
x
i
)
+
b
)
≥
1
y_{i}*(w^{T}\Phi(x_i)+b)≥1
y i ∗ ( w T Φ ( x i ) + b ) ≥ 1
SVM求解
分别对w和b求偏导，分别得到两个条件（基于对偶性质）
m
i
n
w
,
b
m
a
x
a
L
(
w
,
b
,
a
)
=
m
a
x
a
m
i
n
w
,
b
L
(
w
,
b
,
a
)
min_{w,b}max_{a}L_{(w,b,a)}=max_{a}min_{w,b}L_{(w,b,a)}
m i n w , b m a x a L ( w , b , a ) = m a x a m i n w , b L ( w , b , a )
δ
L
δ
W
=
0
=
>
w
=
∑
i
=
1
n
α
i
y
i
Φ
(
x
n
)
\frac{δL}{δW}=0 => w=\sum_{i=1}^{n}α_iy_iΦ_{(x_n)}
δ W δ L = 0 = > w = i = 1 ∑ n α i y i Φ ( x n )
KaTeX parse error: Can't use function '$' in math mode at position 9: 带入原始公式:$̲L(w,b,a)=\frac{… \frac{1}{2}*w{T}*w-w T\sum_{i=1}{n}α_iy_iΦ_{(x_i)}-b\sum_{i=1} {n}α_iy_i+\sum_{i=1}^{n}α_iKaTeX parse error: Can't use function '$' in math mode at position 2: $̲=\sum_{i=1}{n}α… min_α \frac{1}{2}\sum_{i=1}{n}\sum_{j=1} {n}α_iα_jy_iy_jΦ_(x_i)Φ_(y_i)-\sum_{i=1}^{n}Φ_i$$
∑
i
=
1
n
α
i
y
i
=
0
\sum_{i=1}^{n}α_iy_i=0
∑ i = 1 n α i y i = 0
α
i
≥
0
α_i≥0
α i ≥ 0
要使整体很小当C大时
ξ
i
ξ_i
ξ i
需要映射到高维实际上并没有映射到高维，还是在低维计算将结果映射到高维
可以认为转化到一个无穷高的维度
k
(
x
,
y
)
=
e
x
p
−
∣
∣
x
−
y
∣
∣
2
2
σ
2
k_{(x,y)}=exp{-\frac{||x-y||^2}{2σ^2}}
k ( x , y ) = e x p − 2 σ 2 ∣ ∣ x − y ∣ ∣ 2
别人写的，很好。点着个跳转。
