首頁 大數據百科正文

一圖簡述大數據技術生態圈

  下面是一張生態圖,主要的組件都是為了方便大家從底層的MapReduce模型中脫離出來,用高層語言來做分布式計算,下文將分別為你作簡述。

一圖簡述大數據技術生態圈

  1、HBase

  是一個高可靠性、高性能、面向列、可伸縮的分布式存儲系統,利用HBase技術可在廉價PC Server上搭建起大規模結構化數據集群。像Facebook,都拿它做大型實時應用。

  2、Hive

  Facebook領導的一個數據倉庫工具,可以將結構化的數據文件映射為一張數據庫表,并提供完整的sql查詢功能,可以將sql語句轉換為MapReduce任務進行運行。其優點是學習成本低,可以通過類SQL語句快速實現簡單的MapReduce統計。像一些data scientist 就可以直接查詢,不需要學習其他編程接口。

  3、Pig

  Yahoo開發的,并行地執行數據流處理的引擎,它包含了一種腳本語言,稱為Pig Latin,用來描述這些數據流。Pig Latin本身提供了許多傳統的數據操作,同時允許用戶自己開發一些自定義函數用來讀取、處理和寫數據。在LinkedIn也是大量使用。

  4、Cascading/Scalding

  Cascading是Twitter收購的一個公司技術,主要是提供數據管道的一些抽象接口,然后又推出了基于Cascading的Scala版本就叫Scalding。Coursera是用Scalding作為MapReduce的編程接口放在Amazon的EMR運行。

  5、Zookeeper

  一個分布式的,開放源碼的分布式應用程序協調服務,是Google的Chubby一個開源的實現。

  6、Oozie

  一個基于工作流引擎的開源框架。由Cloudera公司貢獻給Apache的,它能夠提供對Hadoop MapReduce和Pig Jobs的任務調度與協調。

  7、Azkaban

  跟上面很像,Linkedin開源的面向Hadoop的開源工作流系統,提供了類似于cron 的管理任務。

  8、Tez

  Hortonworks主推的優化MapReduce執行引擎,與MapReduce相比較,Tez在性能方面更加出色。

  注:本文由ETHINK提供,作者:不可錯過的大數據,版權著作權屬原創者所有,編輯:Fynlch(王培),數據觀微信公眾號(ID:cbdioreview),欲了解更多大數據行業相關資訊,可搜索數據觀(中國大數據產業觀察網www.cskjbg.com)進入查看。

責任編輯:王培

分享:
延伸閱讀
    速讀區塊鏈
    貴州

    貴州大數據產業政策

    貴州大數據產業動態

    貴州大數據企業

    更多
    大數據概念_大數據分析_大數據應用_大數據百科專題
    企業
    更多
    987彩票 www.424221.com | www.26878y.com | www.66332u.com | www.68689v.com | www.99677y.com | www.61497.com | www.1035u.com | www.981090.com | www.901327.com | www.458178.com | www.717699.com | www.33358c.com | www.cll168.com | www.937986.com | www.516105.com | www.66621i.com | www.99001r.com | www.126xc.com | www.rcw8844.com | www.558738.com | www.50026j.com | www.5484a.com | www.803243.com | www.98528h.com | www.3479v.com | www.560671.com | www.68689c.com | www.068wy.com | www.546411.com | www.50022c.com | www.23hf.com | www.33997t.com | www.089wy.com | www.477070.com | www.99099hh.com | www.c300.biz | www.52072c.com | www.sxyl9.com | www.375791.com | www.07163j.com | www.911750.com | www.80767m.com | www.9818r.cc | www.346377.com | www.43818s.com | www.803372.com | www.32123h.com | www.am2220.com | www.96386i.com | www.085wy.com | www.275073.com | www.500vp.com | www.155091.com | www.987jc.com | www.348918.com | www.8499p.com | www.317787.com | www.7793d.com | www.550414.com | www.5854z.cc | www.73166j.com | www.815807.com | www.67258c.com | www.576734.com | www.2373p.com | www.207882.com | www.85gcw.com | www.035168.com | www.jcai4.com | www.53900j.com | www.820737.com | www.54844.com | www.339713.com | www.065wy.com | www.bxcp3.com | www.284488.com | www.hj2111.com | www.50080y.com | www.155301.com | www.901013.com | www.38396.com | www.168232.com | www.445888a.com | www.16878d.com | www.150882.com | www.544094.com | www.58ipo.com | www.26163u.com | www.196062.com | www.858776.com | www.05881.cc | www.99677x.com | www.226105.com | www.771219.com | www.126kc.com | www.26878d.com | www.106265.com | www.596353.com | www.976931.com | www.1368j.cc | www.5091h.com | www.26878h.com | www.178379.com | www.550174.com | www.899495.com | www.5xcr.com | www.202115.com | www.50024g.com | www.50024c.com | www.011225.com | www.196063.com | www.602857.com | www.911283.com | www.2qxc.com | www.71gcw.com | www.2373d.com | www.61655x.com | www.52072h.com | www.099323.com | www.178379.com | www.401271.com | www.595601.com | www.669278.com | www.802133.com | www.am2220.com | www.605999aa.com | www.9889hc.com | www.913gc.com | www.509866.com | www.220v.cc | www.ca2932.com | www.le33.cc | www.533081.com | www.422689.com | www.xinyc8.com | www.bxcp3.com | www.33997j.com | www.65707x.com | www.26163g.com | www.69998.cc | www.848255.com | www.679957.com | www.552432.com | www.rcw321.com | www.91233c.com | www.60007j.com | www.50024q.com | www.9478i.com | www.083wy.com | www.849642.com | www.lcw997.com | www.183676.com | www.43818q.com | www.08588x.com | www.ac3177.com | www.760250.com | www.500710.com | www.80065p.com | www.66332m.com | www.061wy.com | www.910282.com | www.299850.com | www.15355c.com | www.885pz.com | www.899342.com | www.237344.com | www.77801h.com | www.126xc.com | www.810926.com | www.178739.com | www.82676.com | www.ct6676.com | www.338051.com | www.07163c.com | www.082wy.com | www.514211.com | www.15355j.com | www.fo54.com | www.77114a.com | www.371h.cc | www.542116.com | www.00840z.com | www.c2656.com | www.230757.com | www.9928b.com | www.550973.com | www.61233t.com | www.cp7777p.com | www.66376f.com | www.ywzuqiu.com | www.193722.com | www.5441q.com | www.477070.com | www.25688p.com | www.579031.com | www.50052n.com | www.562601.com | www.9149d.com | www.551412.com | www.8667d.com | www.544096.com | www.9205e.com | www.hcw332.com | www.9999y.cc | www.139620.com | www.zcwf6.com | www.68682a.com | www.918109.com | www.97655k.com | www.643388.com | www.7793b.com | www.292726.com | www.063wy.com | www.50080a.com | www.621316.com | www.2408a.com | www.023989.com | www.937160.com | www.25688f.com | www.330291.com | www.47d.cc | www.88266w.com | www.496918.com | www.5666hc.com | www.52072e.com | www.622307.com | www.009ac.com | www.52303p.com | www.700539.com | www.686pp.com | www.57578f.com | www.196081.com | www.860705.com | www.05028.com | www.68568p.com | www.569080.com | www.cai001.vip | www.8039s.com | www.54400l.com | www.98528f.com | www.858856.com | www.060wy.com | www.77114k.com | www.75538d.com | www.433737.com | www.701541.com | www.178012.com | www.080wy.com | www.7116a.com | www.00665e.com | www.71399d.com | www.223805.com | www.393890.com | www.689744.com | www.987931.com | www.44fzc.com | www.15236.cc | www.8039y.com | www.77803b.com | www.50080z.com | www.76520m.com | www.081306.com | www.192880.com | www.523780.com | www.377675.com | www.511642.com | www.544089.com | www.163cp1.cc | www.591017.com | www.607356.com | www.630933.com | www.685756.com | www.814646.com | www.760170.com | www.786778.com | www.695899.com | www.648668.com | www.662182.com | www.584211.com | www.528335.com | www.444601.com | www.330985.com | www.303482.com | www.83993i.com | www.66621h.com | www.68689l.com | www.26299z.com | www.5854o.cc | www.966013.com | www.817824.com | www.594277.com | www.308388.com | www.001562.com | www.35155k.com | www.34788z.com | www.9050.cc | www.c705.cc | www.602450.com | www.235644.com | www.50732t.com | www.50054n.com | www.bxcp3.com | www.939763.com | www.454589.com | www.77114s.com | www.23579b.com | www.cpyz1.com | www.573898.com | www.083016.com | www.54400x.com | www.072wy.com | www.675630.com | www.068625.com | www.32123q.com | www.122888a.com | www.403489.com | www.43818g.com | www.2632d.com | www.641377.com | www.52072g.com | www.910xj.com | www.602434.com | www.68568e.com | www.080wy.com | www.637012.com | www.84499n.com | www.c300.vip | www.84499t.com | www.2632b.com | www.394077.com | www.81233w.com | www.974919.com | www.009275.com | www.1035d.com | www.577983.com | www.88266d.com | www.932135.com | www.43818n.com | www.cb3288.com | www.65707l.com | www.c147.vip | www.26163n.com | www.843879.com | www.96386g.com | www.783802.com | www.71399x.com | www.775704.com | www.60123l.com | www.577369.com | www.8839i.com | www.452803.com | www.2875v.com | www.217380.com | www.sytg7.com | www.49956b.com | www.799849.com | www.50064h.com | www.369053.com | www.061wy.com | www.77114e.com | www.891079.com | www.66026.cc | www.89894l.com | www.880686.com | www.81699.cc | www.78949q.com | www.775780.com | www.0014n.com | www.50788n.com | www.wcp666888.com | www.3479u.com | www.60007g.com | www.671566.com | www.781566.com | www.60007w.com | www.577513.com | www.9lbc.com | www.39957d.com | www.131763.com | www.727076.com | www.022jx.cc | www.76636.com | www.57578d.com | www.426060.com | www.901387.com | www.33tdc.com | www.89939.cc | www.26299v.com | www.186753.com | www.646959.com | www.csgc8.com | www.9478v.com | www.50064o.com | www.xinyc1.com | www.556346.com | www.tcp969.com | www.ya958.com | www.9155g.com | www.81520r.com | www.66376e.com | www.223085.com | www.ywzuqiu.com | www.890482.com | www.ch8911.com | www.df76.com | www.55czj.com | www.3552m.com | www.12455d.com | www.50080l.com | www.66376b.com | www.26878m.com | www.29178c.com | www.50051k.com | www.52072l.com | www.89894s.com | www.87668f.com | www.99677o.com | www.81233v.com | www.89894o.com | www.88266m.com | www.66653i.com | www.674007.com | www.bxcp3.com | www.8000hc.com | www.44czt.com | www.gai50.com | www.df39.com | www.955494.com | www.810715.com | www.608799.com | www.393890.com | www.180435.com | www.80065c.com | www.12455z.com | www.50064z.com | www.3478f.cc | www.cp7777p.com | www.590808.com | www.250552.com | www.61655z.com | www.66653f.com | www.3552d.com | www.qbwc6.com | www.802865.com | www.394511.com | www.60123t.com | www.50788b.com | www.089wy.com | www.766076.com | www.282263.com | www.50026n.com | www.7782z.com | www.cp8018.cc | www.560703.com | www.80075f.com | www.25688j.com | www.hf5883.com | www.yc7701.com | www.65707v.com | www.1035q.com | www.713799.com | www.86267k.com | www.9478n.com | www.857380.com | www.055983.com | www.27989.cc | www.689169.com | www.78700f.com | www.2350n.com | www.583792.com | www.33588x.com | www.hczx4.com | www.314611.com | www.49956x.com | www.957588.com | www.66376q.com | www.33tdc.com | www.234827.com | www.6832p.com | www.544092.com | www.61233y.com | www.lcw997.com | www.35155m.com | www.979505.com | www.77802a.com | www.yic9.com | www.66332q.com | www.igcp1.com | www.081506.com | www.w983.com | www.52072s.com | www.918361.com | www.08588p.com | www.708330.com | www.50054t.com | www.514177.com | www.3479l.com | www.175969.com | www.blc8.com | www.bxcp3.com | www.632599.com | www.8667c.com | www.183587.com | www.hm3377.com | www.33588d.com | www.599380.com | www.5854z.cc | www.68568q.com | www.769117.com | www.9155g.com | www.193629.com | www.976291.com | www.50080q.com | www.186287.com | www.xinyc5.com | www.99343.cc | www.76520x.com | www.638262.com | www.913gc.com | www.12455u.com | www.508869.com | www.hm2277.com | www.50054f.com | www.053059.com | www.669772.com | www.098wy.com | www.77802y.com | www.109031.com | www.661526.com | www.lqc2.com | www.6611b.cc | www.78949s.com | www.032059.com | www.544559.com | www.xinyc5.com | www.910xj.com | www.52072n.com | www.68689g.com | www.355264.com | www.694050.com | www.lczg4.com | www.2109b.com | www.68689h.com | www.77801u.com | www.210793.com | www.550284.com | www.808913.com | www.c299.com | www.fo54.com | www.070wy.com | www.3933w.cc | www.6661f.cc | www.9818e.cc | www.66509.cc | www.21202p.com | www.57578d.com | www.99677t.com | www.35155r.com | www.54400u.com | www.77802y.com | www.99677b.com | www.009275.com | www.003851.com | www.83993y.com | www.87668h.com | www.81233e.com | www.78700n.com | www.66332c.com | www.50066c.com | www.00840u.com | www.91233z.com | www.77114g.com | www.52072t.com | www.66201.cc | www.2373a.com | www.bai355.com | www.c1391.com | www.840979.com | www.621477.com | www.376817.com | www.153918.com | www.61655h.com | www.81678l.com | www.9818m.cc | www.zfcp2.com | www.893023.com | www.599457.com | www.le33.cc | www.53900o.com | www.60123y.com | www.4388.cc | www.cp8015.cc | www.621441.com | www.191059.com | www.36166z.com | www.8039z.com | www.hx6622.com | www.589144.com | www.99677v.com | www.60108r.com | www.088wy.com | www.689744.com | www.175904.com | www.60123m.com | www.ya383.com | www.602266.com | www.83993a.com | www.12455p.com | www.ct8877.com | www.370583.com | www.39500o.com | www.022w.cc | www.609717.com | www.60108r.com | www.2408c.com | www.679610.com | www.68689r.com | www.2109l.com | www.646277.com | www.55885j.com | www.5666hc.com | www.548338.com | www.15355i.com | www.le53.com | www.224066.com | www.12455m.com | www.818827.com | www.65707d.com | www.95wcp.com | www.324977.com | www.26878d.com | www.740707.com | www.50052i.com | www.ybao6.com | www.193522.com | www.4189c.com | www.516105.com | www.50051h.com | www.655101.com | www.80065b.com | www.849642.com | www.33598x.com | www.lcw997.com | www.41518b.com | www.cp7777p.com | www.50026q.com | www.938621.com | www.50051o.com | www.914077.com | www.15355u.com | www.738070.com | www.50054g.com | www.530781.com | www.3479d.com | www.190883.com | www.fen00.com | www.66652a.com | www.tcp969.com | www.67258c.com | www.23040.com | www.cpw | www.66653e.com | www.98528f.com | www.00840t.com | www.277739.com | www.fh3399.com | www.08588z.com | www.500710.com | www.ya618.com | www.15355i.com | www.483770.com | www.vt34.com | www.66653d.com | www.262072.com | www.966083.com | www.25688c.com | www.99638b.com | www.739366.com | www.1035f.com | www.33112a.com | www.423770.com | www.cp500.biz | www.41855.cc | www.50024m.com | www.376477.com | www.cp7777b.com | www.5091c.com | www.26163l.com | www.fh7557.com | www.771219.com | www.bxcp3.com | www.50054b.com | www.77802i.com | www.371680.com | www.866546.com | www.5xch.com | www.8039c.com | www.97655p.com | www.88266r.com | www.370837.com | www.711448.com | www.cb7188.com | www.106ak.com | www.21202j.com | www.99968z.com | www.66332g.com | www.106265.com | www.djcp234.com | www.558414.com | www.725422.com | www.938537.com | www.fh7557.com | www.528qx.com | www.8999ka.com | www.9356c.com | www.34788k.com | www.71233f.com | www.07163u.com | www.50051e.com | www.55228h.com | www.68689r.com | www.71399h.com | www.78949c.com | www.77802r.com | www.78949i.com | www.80065q.com | www.84499i.com | www.77114y.com | www.66652d.com | www.68689c.com | www.65707y.com | www.50052a.com | www.36788y.com | www.95111w.com | www.66376q.com | www.16878x.com | www.5086s.com | www.664by.com | www.df79.com | www.909888.com | www.674679.com | www.482770.com | www.187396.com | www.77114m.com | www.16878u.com | www.29277w.com | www.083wy.com | www.318548.com | www.832793.com | www.539126.com | www.178352.com | www.50051h.com | www.33598d.com | www.bxcp3.com | www.916730.com | www.566897.com | www.079776.com | www.91233q.com | www.3479e.com | www.ca2932.com | www.566897.com | www.81520k.com | www.50732d.com | www.12czj.com | www.700590.com | www.178762.com | www.86267p.com | www.371p.cc | www.775703.com | www.095588.com | www.60123c.com | www.hx1183.com | www.470680.com | www.33588u.com | www.055m.cc | www.627029.com | www.66653u.com | www.3552n.com | www.689826.com | www.71233s.com | www.3479l.com | www.703617.com | www.77802z.com | www.9889hc.com | www.wcp666888.com | www.61655g.com | www.055r.cc | www.527990.com | www.bxcp3.com | www.c2809.com | www.178625.com | www.22503.cc | www.611874.com | www.12455g.com | www.cp0123.cc | www.88266v.com | www.9889hc.com | www.309135.com | www.8039s.com | www.570477.com | www.65707l.com | www.715699.com | www.15355u.com | www.913075.com | www.50989j.com | www.980221.com | www.55885n.com | www.ck5589.com | www.66621j.com | www.976500.com | www.39500l.com | www.900122.com | www.26878x.com | www.755907.com | www.71233l.com | www.xcw866.com | www.15355k.com | www.375990.com | www.781566.com | www.106036.com | www.lczg5.com | www.50026o.com | www.806875.com | www.29277l.com | www.326608.com | www.sytg8.com | www.49956s.com | www.679313.com | www.8039f.com | www.129177.com | www.989774.com | www.55228b.com | www.yc7701.com | www.fh6116.com | www.281877.com | www.401279.com | www.5xct.com | www.87668x.com | www.389044.com | www.c6129.com | www.61233h.com | www.191380.com | www.911283.com | www.8816g.com | www.77801u.com | www.wcp777888.com | www.083wy.com | www.81233x.com | www.202115.com | www.771245.com | www.918ae.com | www.09500w.com | www.209703.com | www.812680.com | www.857t.com | www.68689k.com | www.99638m.com | www.562602.com | www.cp8016.cc | www.2632e.com | www.84499n.com | www.003963.com | www.526093.com | www.895465.com | www.014ac.com | www.07163h.com | www.34788a.com | www.088185.com | www.444083.com | www.718322.com | www.cp718.vip | www.5666hc.com | www.9711c.com | www.71399l.com | www.53900n.com | www.99638f.com | www.323518.com | www.558416.com | www.701496.com | www.878743.com | www.c1968.com | www.fen09.com | www.659zf.com | www.2109d.com | www.9478f.com | www.29178f.com | www.62118f.com | www.97655p.com | www.33997m.com | www.43818e.com | www.50732l.com | www.51331m.com | www.68682k.com | www.71233o.com | www.66621d.com | www.60123z.com | www.50024t.com | www.32123w.com | www.12455p.com | www.71233y.com | www.50026f.com | www.66368.com | www.5091j.com | www.985jc.com | www.8jcw.com | www.998632.com | www.871203.com | www.629053.com | www.435050.com | www.191059.com | www.78949d.com | www.26299z.com | www.33997t.com | www.1248i.cc | www.qmkl7.com | www.876061.com | www.602721.com | www.340477.com | www.068627.com | www.50026b.com | www.52303s.com | www.686pp.com | www.942677.com | www.580896.com | www.170882.com | www.29178a.com | www.9155d.com | www.gyfc1.com | www.hcw823.com | www.yc7701.com | www.41518h.com | www.7782l.com | www.c137.vip | www.504533.com | www.91233r.com | www.54400k.com | www.062wy.com | www.guocai667.com | www.197880.com | www.91233y.com | www.055wt.cc | www.717229.com | www.096308.com | www.41518n.com | www.hy5504.com | www.498266.com | www.50052b.com | www.2350j.com | www.667537.com | www.77802b.com | www.5522k.cc | www.662182.com | www.68689j.com | www.122by.com | www.602985.com | www.41518d.com | www.99ckb.com | www.397411.com | www.81520d.com | www.870080a.com | www.196880.com | www.9928ff.com | www.690776.com | www.36166g.com | www.zcwf9.com | www.202115.com | www.9155e.com | www.670665.com | www.34788h.com | www.c5829.com | www.177584.com | www.5095h.com | www.444539.com | www.50051j.com | www.745855.com | www.12455d.com | www.890482.com | www.32123h.com | www.907118.com | www.33598r.com | www.901572.com | www.41518y.com | www.937160.com | www.34788w.com | www.858776.com | www.84499n.com | www.786819.com | www.77802g.com | www.611949.com | www.50024p.com | www.599049.com | www.15239.cc | www.303717.com | www.917gc.com | www.99677q.com | www.c6770.com | www.35918w.com | www.23040.com | www.41518r.com | www.347977.com | www.9dwj.com | www.39957b.com | www.680352.com | www.8667o.com | www.150883.com | www.ct8844.com | www.81520m.com | www.401280.com | www.06czj.com | www.35155x.com | www.589544.com | www.565hc.com | www.50074d.com | www.651012.com | www.m.55xj.vip | www.34788c.com | www.508533.com | www.loll3.com | www.71233r.com | www.202115.com | www.890992.com | www.cll168.com | www.50026b.com | www.348377.com | www.336766.com | www.5522r.cc | www.34788d.com | www.308680.com | www.826556.com | www.055g.cc | www.77802f.com | www.bxcp3.com | www.551416.com | www.54968f.com | www.1888ac.com | www.60007d.com | www.68689j.com | www.264499.com | www.cjcp600.com | www.011888aa.com | www.2632e.com | www.51331h.com | www.50732r.com | www.196065.com | www.guocai667.com | www.768299.com | www.063wy.com | www.9478p.com | www.99677t.com | www.73736f.com | www.wcp666888.com | www.555446.com | www.807683.com | www.youcai77.com | www.bxcp3.com | www.3890q.com | www.21202z.com | www.73166d.com | www.26163d.com | www.50064h.com | www.71399o.com | www.89894u.com | www.129177.com | www.yc7701.com | www.351857.com | www.456586.com | www.le33.cc | www.530877.com | www.560629.com | www.551549.com | www.586960.com | www.591029.com | www.581166.com | www.555446.com | www.542770.com | www.456639.com | www.346177.com | www.303467.com | www.129989.com | www.79520a.com |