[Foundation of Supervised Learning] part 1 - 2

2023. 1. 12. 22:18

๐ŸŽฏ Keyword ๐ŸŽฏ

- Generalization
- training error
- validation error
- test error
- Curse of dimension
- Regularization
- Ensemble
- cross-validation

 

Machine Learning์€ ๊ทธ ์ž์ฒด๋กœ Data์˜ ๊ฒฐํ•์œผ๋กœ ์ธํ•œ ๋ถˆํ™•์‹ค์„ฑ์„ ํฌํ•จํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.

 

๋ชจ๋“  Data ์‚ดํ•„ ์ˆ˜ ์—†๊ธฐ ๋•Œ๋ฌธ์ด์ฃ .

 

๊ทธ๋ž˜์„œ ์šฐ๋ฆฌ์—๊ฒŒ ์ค‘์š”ํ•œ ๊ฒƒ์€

 

Generalization์ž…๋‹ˆ๋‹ค.

 

๋ชจ๋ธ์ด ์ผ๋ฐ˜ํ™”๋œ ์„ฑ๋Šฅ์„ ์ธก์ •ํ•˜๊ธฐ ์œ„ํ•œ measurement๋กœ์จ Generalization error E ๋ฅผ ์ •์˜ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

 

Supervised Learning์— ์žˆ์–ด ๋‹ค์Œ๊ณผ ๊ฐ™์€ Error๋ฅผ ํ†ตํ•ด ๋ฅผ Generalization error E ์ตœ์†Œํ™” ํ•ฉ๋‹ˆ๋‹ค.

  • training error
  • validation error
  • test error

 

  • Squared error e
  • binary error e

 

์ด๋Ÿฌํ•œ Error ๋“ค์„ ๊ณ„์‚ฐํ•˜์—ฌ ์šฐ๋ฆฌ๋Š” 

 

loss function or cost function ์ด๋ผ๊ณ  ๋ถ€๋ฆ…๋‹ˆ๋‹ค.

 

 

Training Error E

=> model์„ ์ฃผ์–ด์ง„ data set์— ๋งž์ถ”์–ด ํ•™์Šตํ•˜๋Š”๋ฐ ์‚ฌ์šฉํ•˜๋Š” error.

-> ์ฆ‰, ์ฃผ์–ด์ง„ Sample์—์„œ model parameter๋ฅผ ์ตœ์ ํ™” ํ•˜๋„๋ก ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.

-> ๋”ฐ๋ผ์„œ Generalization error E๋ฅผ approximationํ•˜๋Š”๋ฐ ์ ํ•ฉํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

 

 

Test Error E

=> Training Sample๊ณผ Overlap ๋˜์ง€ ์•Š๋„๋ก data set์—์„œ ์ผ๋ถ€ Sample๋ฅผ ์ผ๋ถ€ ๋–ผ์–ด์„œ Test sample์„ ์ •์˜.

-> Real world์—์„œ์˜ error๋ฅผ ํ‘œํ˜„ํ•˜๋Š” ๊ฒƒ.

 

 

๋ชฉ์  : Test Error E๊ฐ€ 0์œผ๋กœ ๊ทผ์‚ฌํ•  ์ˆ˜ ์žˆ๋„๋ก ํ•˜๋Š” ๊ฒƒ.

 

How to achieve the goal in practice ?

  1. Test Error E๊ฐ€ Train Error E์— ๊ทผ์‚ฌํ•˜๋„๋ก ํ•˜๋Š” ๊ฒƒ.
  2. Train Error E๊ฐ€ 0์— ๊ทผ์‚ฌํ•˜๋„๋ก ํ•˜๋Š” ๊ฒƒ.
  • Test Error E๊ฐ€ Train Error E์— ๊ทผ์‚ฌํ•˜๋„๋ก ํ•˜๋Š” ๊ฒƒ.
    • ์‹คํŒจ ์‹œ Overfitting -> high variance
      • ํ•ด๊ฒฐ ๋ฐฉ์•ˆ : reqularization, more data
  • Traing Error E๊ฐ€ 0์— ๊ทผ์‚ฌํ•˜๋„๋ก ํ•˜๋Š” ๊ฒƒ,
    • ์‹คํŒจ ์‹œ Underfitting -> high bias
      • ํ•ด๊ฒฐ ๋ฐฉ์•ˆ :  optimization, more complex model

 

 

Bias - Variance trade-off ๊ด€๊ณ„

-> ์ด ๋‘ ๊ฐ€์ง€์˜ ์š”์†Œ๋ฅผ ์ ์ ˆํžˆ generalizationํ•ด์„œ ๋ชจ๋ธ์˜ ๋ณต์žก๋„๋ฅผ ๊ตฌ์„ฑํ•˜๋Š” ๊ฒƒ์ด  ์ค‘์š”ํ•ฉ๋‹ˆ๋‹ค.

 

 

ํ˜„ ์‹œ์ 
-> CV, NLP ๋“ฑ ๋ณต์žก๋„ ์ฆ๊ฐ€ ์†๋„๋Š” ๋นจ๋ผ์ง€์ง€๋งŒ Data set ์ƒ˜ํ”Œ ์ˆ˜๊ฐ€ ๊ทธ๊ฒƒ์„ ๋”ฐ๋ผ๊ฐ€์ง€ ๋ชป ํ•ฉ๋‹ˆ๋‹ค.
์ด ๊ฒƒ์—์„œ Overfitting ๋ฌธ์ œ๊ฐ€ ๋ฐœ์ƒํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

 

Avoid overfitting

-> Curse of dimension (์ฐจ์›์˜ ์ €์ฃผ ๋ฌธ์ œ)

 

=> ๋งŒ์•ฝ, ์ž…๋ ฅ Data ๋˜๋Š” ์ž…๋ ฅ feature์˜ ์ฐจ์›์ด ์ฆ๊ฐ€ํ•œ๋‹ค๋ฉด ์ง€์ˆ˜์ ์œผ๋กœ ์ƒ˜ํ”Œ์˜ ์ˆซ์ž๊ฐ€ ๋Š˜์–ด๋‚˜์•ผ ํ•˜์ง€๋งŒ ๊ทธ๋ ‡๊ฒŒ ํ•˜๊ธฐ ์–ด๋ ต์Šต๋‹ˆ๋‹ค. 

-> data๋ฅผ ๋Š˜๋ฆฌ๋ฉด ๋œ๋‹ค ! => ํ˜„์‹ค์ ์œผ๋กœ ์–ด๋ ค์›€

 

-> Data augmentation ํ•ด๊ฒฐ ๋ฐฉ์•ˆ

- Regularization

- Ensemble

 

 

K-fold cross-validation

-> training set๋ฅผ k๊ฐœ์˜ Group์œผ๋กœ ๊ตฌ๋ถ„ํ•ฉ๋‹ˆ๋‹ค.

->  K - 1๊ฐœ์˜ group์„ training, 1๊ฐœ๋ฅผ validation์œผ๋กœ ์ด์šฉํ•ฉ๋‹ˆ๋‹ค.

=> validation์€ ๋ชจ๋ธ์˜ ์ตœ์ ํ™”๋ฅผ ์œ„ํ•จ์ž…๋‹ˆ๋‹ค.

=> ๋ชจ๋ธ์˜ ์ผ๋ฐ˜ํ™”์— ๋„์›€์„ ์ค๋‹ˆ๋‹ค.

 

=> ๊ทธ๋ฆฌ๊ณ  ๋ชจ๋ธ์˜ ์ตœ์ข… ์„ฑ๋Šฅ์€ Test Data Set๋ฅผ ์ด์šฉํ•ด์„œ ์ธก์ •ํ•˜๊ฒŒ ๋ฉ๋‹ˆ๋‹ค.

 

 

'Artificial Intelligence' ์นดํ…Œ๊ณ ๋ฆฌ์˜ ๋‹ค๋ฅธ ๊ธ€

[Gradient Descent] part 2 - 3  (0) 2023.01.14
[Linear Regression] part 2 - 2  (0) 2023.01.14
[Linear Regression] part 2 - 1  (0) 2023.01.14
[Foundation of Supervised Learning] part 1 - 1  (0) 2023.01.12
Deep Learning ๊ธฐ๋ฐ˜ ์ดํ•ด  (0) 2022.02.21

BELATED ARTICLES

more