spark：ML和MLlib的区别

ML和MLlib的区别如下：

• ML是升级版的MLlib，最新的Spark版本优先支持ML。
• ML支持DataFrame数据结构和Pipelines，而MLlib仅支持RDD数据结构。
• ML明确区分了分类模型和回归模型，而MLlib并未在顶层做此类区分。
• ML通过DataFrame元数据来区分连续和分类变量。
• ML中的随机森林支持更多的功能：包括重要度、预测概率输出等，而MLlib不支持。

official documentation:

• The main differences between this API and the original MLlib ensembles API are:
• support for DataFrames and ML Pipelines
• separation of classification vs. regression
• use of DataFrame metadata to distinguish continuous and categorical features
• more functionality for random forests: estimates of feature importance, as well as the predicted probability of each class (a.k.a. class conditional probabilities) for classification.