开启Spark history server

配置

Word count: 50Reading time: 1 min

 2018/11/09 

配置 spark-defaults.conf

spark.eventLog.enabled             true
spark.eventLog.dir                 file:///home/spark-2.1.1-bin-hadoop2.7/history_log
spark.history.update.interval      1
spark.history.retainedApplications 50
spark.history.ui.port              18080

然后启动

1	sbin/start-history-server.sh /home/spark-2.1.1-bin-hadoop2.7/history_log

Author：ligang

Link：https://ligang945.github.io/2018/11/09/%E5%BC%80%E5%90%AFSpark%20history%20server/

Publish date：November 9th 2018, 7:51:10 pm

Update date：November 21st 2023, 7:24:46 pm

License：本文采用知识共享署名-非商业性使用 4.0 国际许可协议进行许可

Next Post

修改乱码文件名
Previous Post

批量将代码从GBK转为UTF-8

CATALOG