๋ณธ๋ฌธ ๋ฐ”๋กœ๊ฐ€๊ธฐ
  • What would life be If we had no courage to attemp anything?

Hadoop2

[Hadoop]MapReduce2 ๐ŸŸ๋งต๋ฆฌ๋“€์Šค ํ”„๋ ˆ์ž„์›Œํฌ(MapReduce Framework) ![img](https://velog.velcdn.com/images%2Fkimdukbae%2Fpost%2Fffe11dec-16ee-46c9-97da-fd847a068776%2Fimage.png) ๋งต ๋ฆฌ๋“€์Šค๋ฅผ ๋น„์œ ํ•œ ๊ทธ๋ฆผ์˜ ์˜ˆ์‹œ (https://velog.io/@kimdukbae/MapReduce) ๊ฐ๊ฐ์˜ record ๋˜๋Š” tuple => (KEY,VALUE)์Œ์œผ๋กœ ํ‘œํ˜„ ์ฃผ์š” ๊ตฌ์„ฑ์š”์†Œ๋Š” MapReduce & HDFS MapReduce๋Š” ์†Œํ”„ํŠธ์›จ์–ด์˜ ์ˆ˜ํ–‰์„ ๋ถ„์‚ฐ, HDFS๋Š” ๋ฐ์ดํ„ฐ๋ฅผ ๋ถ„์‚ฐ ๋ฉ”์ธ(main) ํ•จ์ˆ˜๋ฅผ ํ•œ๊ฐœ์˜ master machine์—์„œ ์ˆ˜ํ–‰ ๋จธ์‹ ์ด ๋งต ํ•จ์ˆ˜๋ฅผ ์ˆ˜ํ–‰ํ•˜๊ธฐ ์ „์— ์ „์ฒ˜๋ฆฌ๋ฅผ ํ•˜๊ฑฐ๋‚˜ ๋ฆฌ๋“€์Šค ํ•จ์ˆ˜์˜ ๊ฒฐ๊ณผ๋ฅผ ํ›„์ฒ˜๋ฆฌ ํ•˜๋Š”๋ฐ ์‚ฌ์šฉํ•  ์ˆ˜ .. 2022. 8. 30.
[Hadoop]MapReduce1 ๐Ÿคทโ€โ™€๏ธ๋น…๋ฐ์ดํ„ฐ ๋ถ„์„์„ ์œ„ํ•ด `๋ณ‘๋ ฌ ๋ถ„์‚ฐ ์•Œ๊ณ ๋ฆฌ์ฆ˜`์„ ์ด์šฉํ•˜๋Š” ์ด์œ ๋Š”? ์ ์€ ์ˆ˜์˜ ๊ฐ’ ๋น„์‹ผ ํ•˜์ด ์—”๋“œ(high-end) ๋จธ์‹ ์„ ์ด์šฉํ•˜๋Š” `์Šค์ผ€์ผ-์—…(Scale-Up)`๋ฐฉ์‹์— ๋น„ํ•ด ๋น„๊ต์  ์ €๋ ดํ•œ **๋กœ์šฐ ์—”๋“œ(low-end)๋จธ์‹ ๋“ค์„ ๋Œ€๋Ÿ‰์œผ๋กœ ๋‘๊ณ  ์‚ฌ์šฉํ•˜๋Š” `์Šค์ผ€์ผ-์•„์›ƒ(Scale-Out)` ๋ฐฉ์‹์ด ๋น„์šฉ์ ์œผ๋กœ ๋ณด๋‹ค ํšจ์œจ์ **์ด๊ธฐ ๋•Œ๋ฌธ! ๐Ÿคทโ€โ™€๏ธ๋งต๋ฆฌ๋“€์Šค ํ”„๋ ˆ์ž„์›Œํฌ๋Š” ์™œ ์“ฐ๋Š”๋ฐ? ๊ฐ’ ์‹ผ ์—ฌ๋Ÿฌ ๋Œ€์˜ ์ปดํ“จํ„ฐ๋“ค๋กœ ์ด๋ฃจ์–ด์ง„ ๊ฑฐ๋Œ€ํ•œ ํด๋Ÿฌ์Šคํ„ฐ(cluster)์—์„œ ๋น…๋ฐ์ดํ„ฐ๋ฅผ ์ฒ˜๋ฆฌํ•˜๊ธฐ ์œ„ํ•œ ์Šค์ผ€์ผ๋Ÿฌ๋ธ” ๋ณ‘๋ ฌ ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ SW ๊ฐœ๋ฐœ์„ ์‰ฝ๊ฒŒ ํ•  ์ˆ˜ ์žˆ๋„๋ก ํ•ด์ฃผ๋Š” ํ”„๋กœ๊ทธ๋ž˜๋ฐ ๋ชจ๋ธ ๐Ÿคทโ€โ™€๏ธ๊ทธ๋ ‡๋‹ค๋ฉด ๋งต๋ฆฌ๋“€์Šค ํ”„๋ ˆ์ž„์›Œํฌ๋Š” ์–ด๋–ป๊ฒŒ ์ž‘๋™ํ•ด? HDFS(Hadoop DistributedFileSystem)์ด๋ผ ๋ถˆ๋ฆฌ๋Š” ๋ถ„์‚ฐ ํŒŒ์ผ ์‹œ์Šคํ…œ์ด ๋น…๋ฐ์ดํ„ฐ.. 2022. 8. 29.

ํ‹ฐ์Šคํ† ๋ฆฌํˆด๋ฐ”