kaggle初学者应该如何参加机器学习比赛.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.问题建模"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 业务理解\n",
    "理解赛题的具体含义,理解业务才能构造出与业务相关性高的特征."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 赛题数据\n",
    "拿到数据首先明确数值特征部分和类别特征部分. 然后思考能够构造出哪些特征,并且考虑哪些特征或者数据是不能够使用的."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 评价指标 AUC\n",
    "本数据题采用的是AUC作为评价指标，二分类问题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.4 是否存在线下验证\n",
    "时序验证 :一般选择最近邻的1-3天\n",
    "\n",
    "交叉验证:与k相关大, k偏小则性能不稳定. k偏大则计算量大."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 探索性分析\n",
    "一般使用可视化、统计检测完成"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 数据集大小、 字段类型\n",
    "数据集多大，每个字段是什么类型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 缺失值情况\n",
    "缺失值是否严重，是否缺失值有特殊含义"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 特征之间是否冗余\n",
    "比如身高用m和cm同时表示了两个特征就表示冗余"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 是否存在时间信息\n",
    "潜在的穿越问题，本题由于有时间特征，而且根据baseline训练集和测试集的两个数据采集时间段并不一致"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5 标签分布\n",
    "是否存在类别不平衡问题：本题存在绝对的类别不平衡问题，欺诈人数远远低于非欺诈人数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.6 训练集和测试集的分布\n",
    "是否训练集中有的字段是测试集没有的，或者二者均存在不同字段"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 特征工程"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1  数据预处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.1.1 数值特征可视化:数值特征使用散点图进行可视化绘制,去掉离群点."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.1.2 缺失值处理\n",
    "\n",
    "缺失值处理: 缺失值并非全是脏数据, 其背后可能存在具体的业务意义,这种需要根据业务意义进行填充.\n",
    "\n",
    "另外一部分是真实的缺失,那么可以考虑用统计值进行填充,或者不填充,对于树模型来说是可以处理缺失的."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.1.3错误值处理\n",
    "需要注意的是除了明显的违反逻辑常识的错误值之外, 有的数据集中可能使用\n",
    "\n",
    "某一特定的数值对缺失进行填充,这种需要注意\n",
    "\n",
    "明显错误值：体重--1000kg、暗错误值：根据业务理解，比如在某个特定的特征中，-1表示了数据缺失定义为了-1，这时候需要处理暗错误值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.1.4 假标签处理\n",
    "假标签处理: 如果在训练集中出现明显的错误标签(在业务背景下),那么直接删除该样本,或者也可以使用统计特征进行替换. \n",
    "\n",
    "对于标签和评估指标不一致的情况需要根据实际要求对标签进行数值转换."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 特征提取"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.1 类别特征"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.1.1  编码\n",
    "对于模型不能学习的字符串特征可以进行编码(自然数编码和独热编码,使用区别在于该特征是否具有大小的意义)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.1.2  计数统计(count)\n",
    "给类别特征做count运算,反应类别在整体下的一个热度情况.但是对于异常值是很敏感,比如某一个类别特别大的情况, 这个时候特征会影响模型的泛化能力."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.1.3  计数排名\n",
    "根据统计进行类别连续值排名,可以缩小异常值的权重,减少异常值的影响."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.1.4  目标编码\n",
    "根据标签来做特征,比如统计标签下的统计特征(譬如房租赛当中统计每个板块下的均价).\n",
    "\n",
    "但是这样的做法很容易导致过拟合,需要使用交叉验证来解决.\n",
    "\n",
    "将数据拆分成多份,用n-1份作为已知数据,剩下一份作为未知数据,在已知数据中构造特征,赋值到未知数据中,从而避免过拟合,提升泛化能力"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.1.5  交叉组合\n",
    "类别与类别进行交叉组合,可以让类别之间的粒度更细. 类别与数值进行组合,可以反映类别的统计特征,比如类别下的平均值统计等."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.1.6  防止过拟合\n",
    "使用交叉统计进行特征构造(比如数据拆分为5分,每使用4份构造一份,构造五次可以拼出一个完整的集合). \n",
    "\n",
    "时序特征进行构造(使用前一天和前两天的一些信息).在这种情况下可能部分类别特征没有同时存在于两个集合中\n",
    "\n",
    "(比如房租比赛中的plate62, 和JD比赛中的action type5),此时可以选择使用统计特征填充."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.2 数值特征"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.2.1 分桶——将数值特征转换为离散特征，或者分位数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.2.2 根据业务做特征交叉\n",
    "对不同的特征进行加减乘除"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.2.3 交叉组合\n",
    "对类别数值两类特征做交叉组合构建新特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.2.4 时间特征处理\n",
    "可以反映周期性和趋势线.并且时间越近效果是越好的.\n",
    "\n",
    "对于日期变量是可以做One-Hot的\n",
    "\n",
    "时序相关特征可以进行历史平移和滑窗统"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2.2.5 多值特征处理\n",
    "\n",
    "多值特征,可以完全展开one_hot,也可以词频统计,也可以wordToVector, embeding等进行降维."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 特征选择\n",
    "这一步没有哪种方法是一定最好的，需要根据具体问题具体分析，部分机器学习算法，如XGBoost、lightGBM可以做到自动帮助特征选择\n",
    "\n",
    "另外补充一点，目前有一些kaggle算法已经做到了通过一些辅助库进行调参自动化选择最优参数，貌似是利用贝叶斯法对参数选择\n",
    "\n",
    "大佬总结：在比赛中的时候需要尝试多种方式,没有哪一种一定是最好的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.3.1 过滤法\n",
    "卡方检验和互信息衡量x和y的相关性;相关系数衡量特征之间的相关性."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.3.2 封装法\n",
    "前向搜索: 选定好模型,先放入几个基本特征,然后依次往其中丢入特征,保留效果好的特征,属于启发式算法,但是可能陷入局部最优解.\n",
    "    \n",
    "后向搜索: 依次从模型中剔除特征的思路,耗费时间更长,数据量大的时候难以使用."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.3.3 嵌入法\n",
    "根据树模型返回的特征重要性来选择特征.boosting类别的模型基本都可以做特征选择，而且很多baseline采用"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.必备模型和模型融合"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 XGBoost和lightGBM\n",
    "\n",
    "对特征处理的要求低\n",
    "\n",
    "对类别特征和连续特征友好\n",
    "\n",
    "缺失值不需要填充"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 模型融合\n",
    "理论分析: 让模型做到优质而尽量不同. 保证特征差异,样本差异,模型差异的情况下, 可以使得融合能有更好的效果. \n",
    "\n",
    "实际中可以处理出多套特征,使用重合较少的样本,使用不同的理论模型来达到这一目的.\n",
    "\n",
    "训练过程融合是树模型本身带有的, 因此可以使用调参达到这个目的.\n",
    "\n",
    "结果融合: 分类(投票),平均(回归), stacking."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.竞赛总结\n",
    "好的竞赛总结比竞赛过程更总要\n",
    "\n",
    "赛后及时总结：自己的整体思路、关键代码、自己的不足、还需要做哪些尝试\n",
    "\n",
    "学习优秀的方案：不要局限于自己的思维模式，其他人是如何思考的，哪些是可以借鉴的。进行对比发现自己的不足之处\n",
    "\n",
    "初学者：一定要有耐心和毅力，既然选择了打比赛，那就需要多花时间和多学习，慢慢来，不要一口气想拿个TOP10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}