Storm 配置图文解析

小程序：扫一扫查出行
【扫一扫了解最新限行尾号】
复制小程序

參考阅读：http://www.xiaofateng.com/?

p=959

==============================
| sample-topology |
| ------------------------ | Task 1 Task 2 Task 3
| | Worker Process 1 | | T1 T2 T3
| | +--------+ | | Spout => Bolt => Bolt
| | +------+ | +----+ | | | parallelism parallelism parallelism
| | | T3 | | | T2 | | | | hint=2 hint=2 hint=6
| | +------+ | +----+ | | |
| | | +----+ | | | combined parallelism = 2 + 2 + 6 = 10
| | +------+ | | T2 | | | |
| | | T3 | | +----+ | | | Each of the 2 worker processes will spawn 10/2=5 threads
| | +------+ +--------+ | |
| | | | T1: parallelism hint = initial executors
| | +------+ +--------+ | |
| | | T3 | | T1 | | | T2: the T2 bolt was configured to use 2 executors and four tasks.
| | +------+ +--------+ | | For this reason each executor runs two tasks for this bolt.
| ------------------------ |
| |
| ------------------------ | Config conf = new Config();
| | Worker Process 2 | |
| | +--------+ | | // run as 2 workers on 2 supervisors
| | +------+ | +----+ | | | conf.setNumWorkers(2);
| | | T3 | | | T2 | | | |
| | +------+ | +----+ | | | // T1: 2 executors for spout
| | | +----+ | | | topologyBuilder.setSpout("T1-spout", new T1Spout(), 2);
| | +------+ | | T2 | | | |
| | | T3 | | +----+ | | | // T2: 2 executors for bolt with total 4 tasks
| | +------+ +--------+ | | topologyBuilder.setBolt("T2-bolt", new T2Bolt(), 2)
| | | | .setNumTasks(4).shuffleGrouping("T1-spout");
| | +------+ +--------+ | | // T3: 6 executors for bolt (default 1 task for 1 executor)
| | | T3 | | T1 | | | topologyBuilder.setBolt("T3-bolt", new T3Bolt(), 6).shuffleGrouping("T2-bolt");
| | +------+ +--------+ | |
| ------------------------ | StormSubmitter.submitTopology("sample-topology", conf, topologyBuilder.createTopology());
==============================