Sampling_and_Confidence_Intervals.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Sampling and Confidence Intervals.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyNpWP7s7eHKLdo7nFx3iKcT",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/mukul-mschauhan/Data-Science-Projects/blob/master/Sampling_and_Confidence_Intervals.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KQmLsXEsCWfG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "secJ3RvoVi9s",
        "colab_type": "text"
      },
      "source": [
        "# Sampling\n",
        "\n",
        "1. The Population Distribution is the probability distribution of the population data.\n",
        "\n",
        "2. Lets say that there are 5 students in statistics class and the marks obtained by them are 80, 70, 70, 72.\n",
        "\n",
        "3. The Mean of the Probability Distribution is Sum of (x*p(x)). This means that if the test is repeated n no of times, on an average a student would score 73 marks.\n",
        "\n",
        "4. Now in the above case, the mean is constant because it was the population. However it is not possible in case of Samples.\n",
        "\n",
        "5. Suppose we draw two samples from the above examples. Lets See how it goes\n",
        "\n",
        "6. Now we can see that each sample has a different mean which is clear that sample mean is not always the same as Population Mean\n",
        "\n",
        "7. Rule - If I take the average of all the sample means, the mean arrived would be the population mean.\n",
        "\n",
        "8. Sampling Error - Different Samples will have different results. The difference of Sample Statistic and Population Parameter is called Sampling Error. Thus Sampling Error = Sample Mean - Population Mean\n",
        "\n",
        "9. Rule - Mean of the Sampling Distribution of Xbar is always equal to Pop mean. Thus Mew(xbar) = mew.\n",
        "\n",
        "10. Standard Deviation of Sample Mean = PopSD/sqrt(n)\n",
        "\n",
        "Q.1. The Mean wage per hour for all the 5000 employees who work at a large company is $27.50. Let Xbar be the mean wage per hour for a random sample of certain employees selected from the co. Find the Sample Mean and Standard Error. for a sample size of 30, 75 and 200.\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IsrLW55oZj0S",
        "colab_type": "code",
        "outputId": "4b20a588-f5cc-4647-87a2-489f159b027c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "import numpy as np\n",
        "# Given - mew = 27.50, sd = 3.70\n",
        "sd = 3.70\n",
        "samplemean = 27.50\n",
        "n = 30\n",
        "se = sd/np.sqrt(30)\n",
        "\n",
        "print(\"The Sample Mean: \", samplemean)\n",
        "print(\"Standard Error: \", se)\n",
        "\n",
        "# Thus, we take all the data and prepare a sampling distribution of xbar,\n",
        "# mean and SD of that distribution would be 27.50 with a SD of 0.675."
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "The Sample Mean:  27.5\n",
            "Standard Error:  0.6755244875897048\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {"cell_type": "code",
     "# Function to Calculate Standard Error",
     "def calcstdev(m):
        "mean_vals = m.mean()"
        "stdev = np.sqrt(m.transform(lambda x: (x-x.mean())**2).sum()/(m.shape[0]-1))"
         "return(stdev/np.sqrt(m.shape[0]))"
    }
    
    
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "64tRQGAqaICw",
        "colab_type": "text"
      },
      "source": [
        "Q.2. In a recent SAT Survey, mean score of all examinees was 1020. Assume that the distribution to be normal with the mean of 1020 and SD of 153. \n",
        "\n",
        "Let X bar be the mean of SAT Score of Random Sample. Calculate Mew(xbar) and SD(Xbar) for sample of 50 and 1000 people."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y7_CTJCTZw3g",
        "colab_type": "code",
        "outputId": "7c6e2b72-1d9f-403b-dfda-a34c842c75f2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "samplemean = 1020\n",
        "sd = 153\n",
        "n = 1000\n",
        "se = sd/np.sqrt(n)\n",
        "\n",
        "print(\"The Sample Mean: \", samplemean)\n",
        "print(\"Standard Error: \", se)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "The Sample Mean:  1020\n",
            "Standard Error:  4.838284820057621\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U0KO_GR5aois",
        "colab_type": "text"
      },
      "source": [
        "Q.3. Acc, to Sallie Mae Survey and College Data, Students carry a credit card debt on an average of #3173 in 2008. Suppose the Prob Dist. is unknown but the mean is 3173 and SD os $750. Let Xbar be the mean cc debt of random sample of 400 students.\n",
        "\n",
        "* What is the Probability that the mean of credit card debt for the sample is within $70 of pop mean."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vl4FDvhtahZF",
        "colab_type": "code",
        "outputId": "6dc09d03-f61c-4ff8-c0b8-1a6e46727845",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "samplemean = 3173\n",
        "sd = 750\n",
        "n = 400\n",
        "se = sd/np.sqrt(n)\n",
        "print(\"Standard Error\", se)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Standard Error 37.5\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0bAg2BhlbKdK",
        "colab_type": "code",
        "outputId": "8f53e4bc-a2d0-4759-9e03-ab0b90cf0680",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Probability of mean within 70 dollars\n",
        "samplemean+70, samplemean-70\n",
        "\n",
        "# Thus we need to find the probability of value between 3103 and 3243"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(3243, 3103)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pIcQvnBSbYQF",
        "colab_type": "code",
        "outputId": "e22574c7-96ba-4be7-b1cd-bd76e2771e65",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Apply Z Score: For Xbar = 3103\n",
        "zscore1 = (3103 - 3173)/(se)\n",
        "zscore2 = (3243 - 3173)/(se)\n",
        "print(zscore1, zscore2)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "-1.8666666666666667 1.8666666666666667\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZfC60Vg6btv-",
        "colab_type": "code",
        "outputId": "d895eb37-9a0c-4e1e-a9f8-056818dd72b6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "from scipy import stats\n",
        "stats.norm.cdf(1.866667) - stats.norm.cdf(-1.866667)\n",
        "\n",
        "# Thus the probability that mean of the credit card debt would be within $70 would be 93.8%."
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.9380518951645551"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WLZ-Kt2tcihT",
        "colab_type": "text"
      },
      "source": [
        "## Estimation\n",
        "\n",
        "1. Estimation is a procedure through which the numerical values are assigned to Population Parameter based on information collected from a sample.\n",
        "\n",
        "2. The Value assigned to Population Parameter based on a Sample Statistic is called Estimate\n",
        "\n",
        "3. There are two types of Estimate - Point Estimate and another is interval estimate.\n",
        "\n",
        "4. In a Point Estimate we hve a single value assigned to the population parameter where as in Interval Estimate, we construct an interval around the point.\n",
        "\n",
        "5. The no that we add and substract to point estimate is called Margin of Error.\n",
        "\n",
        "6. The No that we add/substract is the SD of sample mean and the level of Confidence Attached to that interval.\n",
        "\n",
        "7. Confidence Level suggests that we are confident at a certain level that value will fall in a certain range. Thus the Interval is given by Point Estimate +/- Margin of Error.\n",
        "\n",
        "8. Rule: When the SD Is known:  When Pop SD is known and Distribution is Normal.\n",
        "Then Sample SD is given by Standard Error and Interval Estimate = xbar+/-z*se.\n",
        "\n",
        "Q. A Publishing Company has published new textbooks for schools. Before deciding the price, the company wants to know the average price of all books. Took a Sample of 25 textbooks and checked their price. The mean came out to be $145 & the SD of all the books in market was $35. What is the Point and Interval Estimate at 90% Confidence level"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CeIQ4EBdb4rg",
        "colab_type": "code",
        "outputId": "2d5fddd3-61c2-4e70-e6c0-8bf16298d3c4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "import numpy as np\n",
        "from scipy import stats\n",
        "n = 25\n",
        "sd = 35\n",
        "se = sd/np.sqrt(n)\n",
        "mean = 145\n",
        "\n",
        "point_estimate = 145\n",
        "\n",
        "cf = 0.90\n",
        "zvalue = stats.norm.ppf(0.95) # 1.644\n",
        "\n",
        "print(\"The Interval Estimate would be in range of \", point_estimate-zvalue*se, \"to \", point_estimate+zvalue*se\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "The Interval Estimate would be in range of  133.4860246113397 to  156.5139753886603\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Hdq4mFdwb-Uz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}