anti-money-laundering-detection-with-gnn.ipynb

{"cells":[{"cell_type":"markdown","metadata":{},"source":["# Anti Money Laundering Detection with GNN node classification\n","### This notenook includes GNN model training and dataset implementation with PyG library. In this example, we used HI-Small_Trans.csv as our dataset for training and testing.  "]},{"cell_type":"code","execution_count":10,"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.execute_input":"2023-10-07T05:03:44.040372Z","iopub.status.busy":"2023-10-07T05:03:44.039793Z","iopub.status.idle":"2023-10-07T05:03:58.147392Z","shell.execute_reply":"2023-10-07T05:03:58.146038Z","shell.execute_reply.started":"2023-10-07T05:03:44.040343Z"},"trusted":true},"outputs":[],"source":["import datetime\n","import os\n","from typing import Callable, Optional\n","import pandas as pd\n","from sklearn import preprocessing\n","import numpy as np\n","import torch\n","\n","from torch_geometric.data import (\n","    Data,\n","    InMemoryDataset\n",")\n","\n","pd.set_option('display.max_columns', None)\n","path = '/path/to/HI-Small_Trans.csv'\n","df = pd.read_csv(path)"]},{"cell_type":"markdown","metadata":{},"source":["# Data visualization and possible feature engineering\n","Let's look into the dataset"]},{"cell_type":"code","execution_count":11,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:33:22.849494Z","iopub.status.busy":"2023-10-07T05:33:22.849049Z","iopub.status.idle":"2023-10-07T05:33:22.867501Z","shell.execute_reply":"2023-10-07T05:33:22.866198Z","shell.execute_reply.started":"2023-10-07T05:33:22.849448Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["          Timestamp  From Bank    Account  To Bank  Account.1  \\\n","0  2022/09/01 00:20         10  8000EBD30       10  8000EBD30   \n","1  2022/09/01 00:20       3208  8000F4580        1  8000F5340   \n","2  2022/09/01 00:00       3209  8000F4670     3209  8000F4670   \n","3  2022/09/01 00:02         12  8000F5030       12  8000F5030   \n","4  2022/09/01 00:06         10  8000F5200       10  8000F5200   \n","\n","   Amount Received Receiving Currency  Amount Paid Payment Currency  \\\n","0          3697.34          US Dollar      3697.34        US Dollar   \n","1             0.01          US Dollar         0.01        US Dollar   \n","2         14675.57          US Dollar     14675.57        US Dollar   \n","3          2806.97          US Dollar      2806.97        US Dollar   \n","4         36682.97          US Dollar     36682.97        US Dollar   \n","\n","  Payment Format  Is Laundering  \n","0   Reinvestment              0  \n","1         Cheque              0  \n","2   Reinvestment              0  \n","3   Reinvestment              0  \n","4   Reinvestment              0  \n"]}],"source":["print(df.head())"]},{"cell_type":"markdown","metadata":{},"source":["After the viewing the dataframe, we suggest that we can extract all accounts from receiver and payer among all transcation for sorting the suspicious accounts. We can transform the whole dataset into node classification problem by considering accounts as nodes while transcation as edges."]},{"cell_type":"markdown","metadata":{},"source":["The object columns should be encoded into classes with sklearn LabelEncoder."]},{"cell_type":"code","execution_count":12,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:36:52.790986Z","iopub.status.busy":"2023-10-07T05:36:52.790429Z","iopub.status.idle":"2023-10-07T05:36:52.797831Z","shell.execute_reply":"2023-10-07T05:36:52.796721Z","shell.execute_reply.started":"2023-10-07T05:36:52.790952Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["Timestamp              object\n","From Bank               int64\n","Account                object\n","To Bank                 int64\n","Account.1              object\n","Amount Received       float64\n","Receiving Currency     object\n","Amount Paid           float64\n","Payment Currency       object\n","Payment Format         object\n","Is Laundering           int64\n","dtype: object\n"]}],"source":["print(df.dtypes)"]},{"cell_type":"markdown","metadata":{},"source":["Check if there are any null values"]},{"cell_type":"code","execution_count":13,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:40:11.526713Z","iopub.status.busy":"2023-10-07T05:40:11.526397Z","iopub.status.idle":"2023-10-07T05:40:12.913554Z","shell.execute_reply":"2023-10-07T05:40:12.912335Z","shell.execute_reply.started":"2023-10-07T05:40:11.526687Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["Timestamp             0\n","From Bank             0\n","Account               0\n","To Bank               0\n","Account.1             0\n","Amount Received       0\n","Receiving Currency    0\n","Amount Paid           0\n","Payment Currency      0\n","Payment Format        0\n","Is Laundering         0\n","dtype: int64\n"]}],"source":["print(df.isnull().sum())"]},{"cell_type":"markdown","metadata":{},"source":["There are two columns representing paid and received amount of each transcation, wondering if it is necessary to split the amount into two columns when they shared the same value, unless there are transcation fee/transcation between different currency. Let's find out "]},{"cell_type":"code","execution_count":14,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:45:40.568327Z","iopub.status.busy":"2023-10-07T05:45:40.567898Z","iopub.status.idle":"2023-10-07T05:45:40.594713Z","shell.execute_reply":"2023-10-07T05:45:40.593358Z","shell.execute_reply.started":"2023-10-07T05:45:40.568296Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["Amount Received equals to Amount Paid:\n","False\n","Receiving Currency equals to Payment Currency:\n","False\n"]}],"source":["print('Amount Received equals to Amount Paid:')\n","print(df['Amount Received'].equals(df['Amount Paid']))\n","print('Receiving Currency equals to Payment Currency:')\n","print(df['Receiving Currency'].equals(df['Payment Currency']))"]},{"cell_type":"markdown","metadata":{},"source":["It seens involved the transcations between different currency, let's print it out"]},{"cell_type":"code","execution_count":15,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:46:16.614934Z","iopub.status.busy":"2023-10-07T05:46:16.614531Z","iopub.status.idle":"2023-10-07T05:46:17.289425Z","shell.execute_reply":"2023-10-07T05:46:17.288314Z","shell.execute_reply.started":"2023-10-07T05:46:16.614907Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["                Timestamp  From Bank    Account  To Bank  Account.1  \\\n","1173     2022/09/01 00:22       1362  80030A870     1362  80030A870   \n","7156     2022/09/01 00:28      11318  800C51010    11318  800C51010   \n","7925     2022/09/01 00:12        795  800D98770      795  800D98770   \n","8467     2022/09/01 00:01       1047  800E92CF0     1047  800E92CF0   \n","11529    2022/09/01 00:22      11157  80135FFC0    11157  80135FFC0   \n","...                   ...        ...        ...      ...        ...   \n","5078167  2022/09/10 23:30      23537  803949A90    23537  803949A90   \n","5078234  2022/09/10 23:59      16163  803638A90    16163  803638A90   \n","5078236  2022/09/10 23:55      16163  803638A90    16163  803638A90   \n","5078316  2022/09/10 23:44     215064  808F06E11   215064  808F06E10   \n","5078318  2022/09/10 23:45     215064  808F06E11   215064  808F06E10   \n","\n","         Amount Received Receiving Currency  Amount Paid Payment Currency  \\\n","1173           52.110000               Euro        61.06        US Dollar   \n","7156           76.060000               Euro        89.12        US Dollar   \n","7925           17.690000  Australian Dollar        12.52        US Dollar   \n","8467           19.430000               Euro        22.77        US Dollar   \n","11529          98.340000               Euro       115.24        US Dollar   \n","...                  ...                ...          ...              ...   \n","5078167     26421.500000             Shekel      7823.96        US Dollar   \n","5078234     47517.490000        Saudi Riyal     12667.62        US Dollar   \n","5078236     11329.850000        Saudi Riyal      3020.41        US Dollar   \n","5078316         0.000006            Bitcoin         0.07        US Dollar   \n","5078318         0.000004            Bitcoin         0.05        US Dollar   \n","\n","        Payment Format  Is Laundering  \n","1173               ACH              0  \n","7156               ACH              0  \n","7925               ACH              0  \n","8467               ACH              0  \n","11529              ACH              0  \n","...                ...            ...  \n","5078167            ACH              0  \n","5078234            ACH              0  \n","5078236            ACH              0  \n","5078316            ACH              0  \n","5078318           Wire              0  \n","\n","[72158 rows x 11 columns]\n","---------------------------------------------------------------------------\n","                Timestamp  From Bank    Account  To Bank  Account.1  \\\n","1173     2022/09/01 00:22       1362  80030A870     1362  80030A870   \n","7156     2022/09/01 00:28      11318  800C51010    11318  800C51010   \n","7925     2022/09/01 00:12        795  800D98770      795  800D98770   \n","8467     2022/09/01 00:01       1047  800E92CF0     1047  800E92CF0   \n","11529    2022/09/01 00:22      11157  80135FFC0    11157  80135FFC0   \n","...                   ...        ...        ...      ...        ...   \n","5078167  2022/09/10 23:30      23537  803949A90    23537  803949A90   \n","5078234  2022/09/10 23:59      16163  803638A90    16163  803638A90   \n","5078236  2022/09/10 23:55      16163  803638A90    16163  803638A90   \n","5078316  2022/09/10 23:44     215064  808F06E11   215064  808F06E10   \n","5078318  2022/09/10 23:45     215064  808F06E11   215064  808F06E10   \n","\n","         Amount Received Receiving Currency  Amount Paid Payment Currency  \\\n","1173           52.110000               Euro        61.06        US Dollar   \n","7156           76.060000               Euro        89.12        US Dollar   \n","7925           17.690000  Australian Dollar        12.52        US Dollar   \n","8467           19.430000               Euro        22.77        US Dollar   \n","11529          98.340000               Euro       115.24        US Dollar   \n","...                  ...                ...          ...              ...   \n","5078167     26421.500000             Shekel      7823.96        US Dollar   \n","5078234     47517.490000        Saudi Riyal     12667.62        US Dollar   \n","5078236     11329.850000        Saudi Riyal      3020.41        US Dollar   \n","5078316         0.000006            Bitcoin         0.07        US Dollar   \n","5078318         0.000004            Bitcoin         0.05        US Dollar   \n","\n","        Payment Format  Is Laundering  \n","1173               ACH              0  \n","7156               ACH              0  \n","7925               ACH              0  \n","8467               ACH              0  \n","11529              ACH              0  \n","...                ...            ...  \n","5078167            ACH              0  \n","5078234            ACH              0  \n","5078236            ACH              0  \n","5078316            ACH              0  \n","5078318           Wire              0  \n","\n","[72170 rows x 11 columns]\n"]}],"source":["not_equal1 = df.loc[~(df['Amount Received'] == df['Amount Paid'])]\n","not_equal2 = df.loc[~(df['Receiving Currency'] == df['Payment Currency'])]\n","print(not_equal1)\n","print('---------------------------------------------------------------------------')\n","print(not_equal2)"]},{"cell_type":"markdown","metadata":{},"source":["The size of two df shows that there are transcation fee and transcation between different currency, we cannot combine/drop the amount columns."]},{"cell_type":"markdown","metadata":{},"source":["As we are going to encode the columns, we have to make sure that the classes of same attribute are aligned.\n","Let's check if the list of Receiving Currency and Payment Currency are the same"]},{"cell_type":"code","execution_count":16,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:51:06.994519Z","iopub.status.busy":"2023-10-07T05:51:06.994058Z","iopub.status.idle":"2023-10-07T05:51:07.455980Z","shell.execute_reply":"2023-10-07T05:51:07.454722Z","shell.execute_reply.started":"2023-10-07T05:51:06.994490Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["['Australian Dollar', 'Bitcoin', 'Brazil Real', 'Canadian Dollar', 'Euro', 'Mexican Peso', 'Ruble', 'Rupee', 'Saudi Riyal', 'Shekel', 'Swiss Franc', 'UK Pound', 'US Dollar', 'Yen', 'Yuan']\n","['Australian Dollar', 'Bitcoin', 'Brazil Real', 'Canadian Dollar', 'Euro', 'Mexican Peso', 'Ruble', 'Rupee', 'Saudi Riyal', 'Shekel', 'Swiss Franc', 'UK Pound', 'US Dollar', 'Yen', 'Yuan']\n"]}],"source":["print(sorted(df['Receiving Currency'].unique()))\n","print(sorted(df['Payment Currency'].unique()))"]},{"cell_type":"markdown","metadata":{},"source":["# Data Preprocessing\n","### We will show the functions used in the PyG dataset first, dataset and model training will be provided in bottom section"]},{"cell_type":"markdown","metadata":{},"source":["In the data preprocessing, we perform below transformation:  \n","1. Transform the Timestamp with min max normalization.  \n","2. Create unique ID for each account by adding bank code with account number.  \n","3. Create receiving_df with the information of receiving accounts, received amount and currency\n","4. Create paying_df with the information of payer accounts, paid amount and currency\n","5. Create a list of currency used among all transactions\n","6. Label the 'Payment Format', 'Payment Currency', 'Receiving Currency' by classes with sklearn LabelEncoder\n"]},{"cell_type":"code","execution_count":17,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:53:11.423289Z","iopub.status.busy":"2023-10-07T05:53:11.422843Z","iopub.status.idle":"2023-10-07T05:53:11.432504Z","shell.execute_reply":"2023-10-07T05:53:11.431355Z","shell.execute_reply.started":"2023-10-07T05:53:11.423245Z"},"trusted":true},"outputs":[],"source":["def df_label_encoder(df, columns):\n","        le = preprocessing.LabelEncoder()\n","        for i in columns:\n","            df[i] = le.fit_transform(df[i].astype(str))\n","        return df\n","\n","def preprocess(df):\n","        df = df_label_encoder(df,['Payment Format', 'Payment Currency', 'Receiving Currency'])\n","        df['Timestamp'] = pd.to_datetime(df['Timestamp'])\n","        df['Timestamp'] = df['Timestamp'].apply(lambda x: x.value)\n","        df['Timestamp'] = (df['Timestamp']-df['Timestamp'].min())/(df['Timestamp'].max()-df['Timestamp'].min())\n","\n","        df['Account'] = df['From Bank'].astype(str) + '_' + df['Account']\n","        df['Account.1'] = df['To Bank'].astype(str) + '_' + df['Account.1']\n","        df = df.sort_values(by=['Account'])\n","        receiving_df = df[['Account.1', 'Amount Received', 'Receiving Currency']]\n","        paying_df = df[['Account', 'Amount Paid', 'Payment Currency']]\n","        receiving_df = receiving_df.rename({'Account.1': 'Account'}, axis=1)\n","        currency_ls = sorted(df['Receiving Currency'].unique())\n","\n","        return df, receiving_df, paying_df, currency_ls"]},{"cell_type":"markdown","metadata":{},"source":["Let's have a look of processed df"]},{"cell_type":"code","execution_count":18,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:53:15.266963Z","iopub.status.busy":"2023-10-07T05:53:15.266592Z","iopub.status.idle":"2023-10-07T05:53:56.218064Z","shell.execute_reply":"2023-10-07T05:53:56.216975Z","shell.execute_reply.started":"2023-10-07T05:53:15.266935Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["         Timestamp  From Bank          Account  To Bank        Account.1  \\\n","4278714   0.456320      10057  10057_803A115E0    29467  29467_803E020C0   \n","2798190   0.285018      10057  10057_803A115E0    29467  29467_803E020C0   \n","2798191   0.284233      10057  10057_803A115E0    29467  29467_803E020C0   \n","3918769   0.417079      10057  10057_803A115E0    29467  29467_803E020C0   \n","213094    0.000746      10057  10057_803A115E0    10057  10057_803A115E0   \n","\n","         Amount Received  Receiving Currency  Amount Paid  Payment Currency  \\\n","4278714        787197.11                  13    787197.11                13   \n","2798190        787197.11                  13    787197.11                13   \n","2798191        681262.19                  13    681262.19                13   \n","3918769        681262.19                  13    681262.19                13   \n","213094         146954.27                  13    146954.27                13   \n","\n","         Payment Format  Is Laundering  \n","4278714               3              0  \n","2798190               3              0  \n","2798191               4              0  \n","3918769               4              0  \n","213094                5              0  \n"]}],"source":["df, receiving_df, paying_df, currency_ls = preprocess(df = df)\n","print(df.head())"]},{"cell_type":"markdown","metadata":{},"source":["paying df and receiving df:"]},{"cell_type":"code","execution_count":19,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:57:25.918744Z","iopub.status.busy":"2023-10-07T05:57:25.918280Z","iopub.status.idle":"2023-10-07T05:57:25.929797Z","shell.execute_reply":"2023-10-07T05:57:25.928625Z","shell.execute_reply.started":"2023-10-07T05:57:25.918708Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["                 Account  Amount Received  Receiving Currency\n","4278714  29467_803E020C0        787197.11                  13\n","2798190  29467_803E020C0        787197.11                  13\n","2798191  29467_803E020C0        681262.19                  13\n","3918769  29467_803E020C0        681262.19                  13\n","213094   10057_803A115E0        146954.27                  13\n","                 Account  Amount Paid  Payment Currency\n","4278714  10057_803A115E0    787197.11                13\n","2798190  10057_803A115E0    787197.11                13\n","2798191  10057_803A115E0    681262.19                13\n","3918769  10057_803A115E0    681262.19                13\n","213094   10057_803A115E0    146954.27                13\n"]}],"source":["print(receiving_df.head())\n","print(paying_df.head())"]},{"cell_type":"markdown","metadata":{},"source":["currency_ls:"]},{"cell_type":"code","execution_count":20,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:57:28.907031Z","iopub.status.busy":"2023-10-07T05:57:28.906667Z","iopub.status.idle":"2023-10-07T05:57:28.913761Z","shell.execute_reply":"2023-10-07T05:57:28.912327Z","shell.execute_reply.started":"2023-10-07T05:57:28.907004Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]\n"]}],"source":["print(currency_ls)"]},{"cell_type":"markdown","metadata":{},"source":["We would like to extract all unique accounts from payer and receiver as node of our graph. It includes the unique account ID, Bank code and the label of 'Is Laundering'.  \n","In this section, we consider both payer and receiver involved in a illicit transaction as suspicious accounts, we will label both accounts with 'Is Laundering' == 1."]},{"cell_type":"code","execution_count":21,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:57:31.850839Z","iopub.status.busy":"2023-10-07T05:57:31.850459Z","iopub.status.idle":"2023-10-07T05:57:31.858990Z","shell.execute_reply":"2023-10-07T05:57:31.857826Z","shell.execute_reply.started":"2023-10-07T05:57:31.850810Z"},"trusted":true},"outputs":[],"source":["def get_all_account(df):\n","        ldf = df[['Account', 'From Bank']]\n","        rdf = df[['Account.1', 'To Bank']]\n","        suspicious = df[df['Is Laundering']==1]\n","        s1 = suspicious[['Account', 'Is Laundering']]\n","        s2 = suspicious[['Account.1', 'Is Laundering']]\n","        s2 = s2.rename({'Account.1': 'Account'}, axis=1)\n","        suspicious = pd.concat([s1, s2], join='outer')\n","        suspicious = suspicious.drop_duplicates()\n","\n","        ldf = ldf.rename({'From Bank': 'Bank'}, axis=1)\n","        rdf = rdf.rename({'Account.1': 'Account', 'To Bank': 'Bank'}, axis=1)\n","        df = pd.concat([ldf, rdf], join='outer')\n","        df = df.drop_duplicates()\n","\n","        df['Is Laundering'] = 0\n","        df.set_index('Account', inplace=True)\n","        df.update(suspicious.set_index('Account'))\n","        df = df.reset_index()\n","        return df"]},{"cell_type":"markdown","metadata":{},"source":["Take a look of the account list:"]},{"cell_type":"code","execution_count":22,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:57:34.379521Z","iopub.status.busy":"2023-10-07T05:57:34.378456Z","iopub.status.idle":"2023-10-07T05:57:41.317058Z","shell.execute_reply":"2023-10-07T05:57:41.316062Z","shell.execute_reply.started":"2023-10-07T05:57:34.379481Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["           Account   Bank  Is Laundering\n","0  10057_803A115E0  10057              0\n","1  10057_803AA8E90  10057              0\n","2  10057_803AAB430  10057              0\n","3  10057_803AACE20  10057              0\n","4  10057_803AB4F70  10057              0\n"]}],"source":["accounts = get_all_account(df)\n","print(accounts.head())"]},{"cell_type":"markdown","metadata":{},"source":["# Node features\n","For node features, we would like to aggregate the mean of paid and received amount with different types of currency as the new features of each node. "]},{"cell_type":"code","execution_count":23,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:57:43.694369Z","iopub.status.busy":"2023-10-07T05:57:43.693958Z","iopub.status.idle":"2023-10-07T05:57:43.701141Z","shell.execute_reply":"2023-10-07T05:57:43.699901Z","shell.execute_reply.started":"2023-10-07T05:57:43.694334Z"},"trusted":true},"outputs":[],"source":["def paid_currency_aggregate(currency_ls, paying_df, accounts):\n","        for i in currency_ls:\n","            temp = paying_df[paying_df['Payment Currency'] == i]\n","            accounts['avg paid '+str(i)] = temp['Amount Paid'].groupby(temp['Account']).transform('mean')\n","        return accounts\n","\n","def received_currency_aggregate(currency_ls, receiving_df, accounts):\n","    for i in currency_ls:\n","        temp = receiving_df[receiving_df['Receiving Currency'] == i]\n","        accounts['avg received '+str(i)] = temp['Amount Received'].groupby(temp['Account']).transform('mean')\n","    accounts = accounts.fillna(0)\n","    return accounts"]},{"cell_type":"markdown","metadata":{},"source":["Now we can define the node attributes by the bank code and the mean of paid and received amount with different types of currency."]},{"cell_type":"code","execution_count":24,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:57:45.915808Z","iopub.status.busy":"2023-10-07T05:57:45.915112Z","iopub.status.idle":"2023-10-07T05:57:45.926442Z","shell.execute_reply":"2023-10-07T05:57:45.924963Z","shell.execute_reply.started":"2023-10-07T05:57:45.915734Z"},"trusted":true},"outputs":[],"source":["def get_node_attr(currency_ls, paying_df,receiving_df, accounts):\n","        node_df = paid_currency_aggregate(currency_ls, paying_df, accounts)\n","        node_df = received_currency_aggregate(currency_ls, receiving_df, node_df)\n","        node_label = torch.from_numpy(node_df['Is Laundering'].values).to(torch.float)\n","        node_df = node_df.drop(['Account', 'Is Laundering'], axis=1)\n","        node_df = df_label_encoder(node_df,['Bank'])\n","#         node_df = torch.from_numpy(node_df.values).to(torch.float)  # comment for visualization\n","        return node_df, node_label"]},{"cell_type":"markdown","metadata":{},"source":["Take a look of node_df:"]},{"cell_type":"code","execution_count":25,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:57:48.258031Z","iopub.status.busy":"2023-10-07T05:57:48.257639Z","iopub.status.idle":"2023-10-07T05:57:56.275657Z","shell.execute_reply":"2023-10-07T05:57:56.274417Z","shell.execute_reply.started":"2023-10-07T05:57:48.257999Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["   Bank  avg paid 0  avg paid 1  avg paid 2  avg paid 3  avg paid 4  \\\n","0     2         0.0         0.0         0.0         0.0         0.0   \n","1     2         0.0         0.0         0.0         0.0         0.0   \n","2     2         0.0         0.0         0.0         0.0         0.0   \n","3     2         0.0         0.0         0.0         0.0         0.0   \n","4     2         0.0         0.0         0.0         0.0         0.0   \n","\n","   avg paid 5  avg paid 6  avg paid 7  avg paid 8  avg paid 9  avg paid 10  \\\n","0         0.0         0.0         0.0         0.0         0.0          0.0   \n","1         0.0         0.0         0.0         0.0         0.0          0.0   \n","2         0.0         0.0         0.0         0.0         0.0          0.0   \n","3         0.0         0.0         0.0         0.0         0.0          0.0   \n","4         0.0         0.0         0.0         0.0         0.0          0.0   \n","\n","   avg paid 11   avg paid 12  avg paid 13  avg paid 14  avg received 0  \\\n","0          0.0   1922.000000          0.0          0.0             0.0   \n","1          0.0    480.223333          0.0          0.0             0.0   \n","2          0.0  14675.570000          0.0          0.0             0.0   \n","3          0.0  37340.843333          0.0          0.0             0.0   \n","4          0.0  49649.409677          0.0          0.0             0.0   \n","\n","   avg received 1  avg received 2  avg received 3  avg received 4  \\\n","0             0.0             0.0             0.0             0.0   \n","1             0.0             0.0             0.0             0.0   \n","2             0.0             0.0             0.0             0.0   \n","3             0.0             0.0             0.0             0.0   \n","4             0.0             0.0             0.0             0.0   \n","\n","   avg received 5  avg received 6  avg received 7  avg received 8  \\\n","0             0.0             0.0             0.0             0.0   \n","1             0.0             0.0             0.0             0.0   \n","2             0.0             0.0             0.0             0.0   \n","3             0.0             0.0             0.0             0.0   \n","4             0.0             0.0             0.0             0.0   \n","\n","   avg received 9  avg received 10  avg received 11  avg received 12  \\\n","0             0.0              0.0              0.0       330.166429   \n","1             0.0              0.0              0.0       119.992000   \n","2             0.0              0.0              0.0     14675.570000   \n","3             0.0              0.0              0.0       756.486190   \n","4             0.0              0.0              0.0      3120.573333   \n","\n","   avg received 13  avg received 14  \n","0              0.0              0.0  \n","1              0.0              0.0  \n","2              0.0              0.0  \n","3              0.0              0.0  \n","4              0.0              0.0  \n"]}],"source":["node_df, node_label = get_node_attr(currency_ls, paying_df,receiving_df, accounts)\n","print(node_df.head())"]},{"cell_type":"markdown","metadata":{},"source":["# Edge features\n","In terms of edge features, we would like to conside each transcation as edges.  \n","For edge index, we replace all account with index and stack into a list with size of [2, num of transcation]  \n","For edge attributes, we used 'Timestamp', 'Amount Received', 'Receiving Currency', 'Amount Paid', 'Payment Currency' and 'Payment Format'\n"]},{"cell_type":"code","execution_count":26,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:58:06.006625Z","iopub.status.busy":"2023-10-07T05:58:06.006227Z","iopub.status.idle":"2023-10-07T05:58:06.015211Z","shell.execute_reply":"2023-10-07T05:58:06.014356Z","shell.execute_reply.started":"2023-10-07T05:58:06.006594Z"},"trusted":true},"outputs":[],"source":["def get_edge_df(accounts, df):\n","        accounts = accounts.reset_index(drop=True)\n","        accounts['ID'] = accounts.index\n","        mapping_dict = dict(zip(accounts['Account'], accounts['ID']))\n","        df['From'] = df['Account'].map(mapping_dict)\n","        df['To'] = df['Account.1'].map(mapping_dict)\n","        df = df.drop(['Account', 'Account.1', 'From Bank', 'To Bank'], axis=1)\n","\n","        edge_index = torch.stack([torch.from_numpy(df['From'].values), torch.from_numpy(df['To'].values)], dim=0)\n","\n","        df = df.drop(['Is Laundering', 'From', 'To'], axis=1)\n","\n","#         edge_attr = torch.from_numpy(df.values).to(torch.float)  # comment for visualization\n","\n","        edge_attr = df  # for visualization\n","        return edge_attr, edge_index"]},{"cell_type":"markdown","metadata":{},"source":["edge_attr:"]},{"cell_type":"code","execution_count":27,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T06:00:02.820037Z","iopub.status.busy":"2023-10-07T06:00:02.819644Z","iopub.status.idle":"2023-10-07T06:00:07.880960Z","shell.execute_reply":"2023-10-07T06:00:07.879754Z","shell.execute_reply.started":"2023-10-07T06:00:02.820005Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["         Timestamp  Amount Received  Receiving Currency  Amount Paid  \\\n","4278714   0.456320        787197.11                  13    787197.11   \n","2798190   0.285018        787197.11                  13    787197.11   \n","2798191   0.284233        681262.19                  13    681262.19   \n","3918769   0.417079        681262.19                  13    681262.19   \n","213094    0.000746        146954.27                  13    146954.27   \n","\n","         Payment Currency  Payment Format  \n","4278714                13               3  \n","2798190                13               3  \n","2798191                13               4  \n","3918769                13               4  \n","213094                 13               5  \n"]}],"source":["edge_attr, edge_index = get_edge_df(accounts, df)\n","print(edge_attr.head())"]},{"cell_type":"markdown","metadata":{},"source":["edge_index:"]},{"cell_type":"code","execution_count":28,"metadata":{"execution":{"iopub.execute_input":"2023-10-07T05:58:16.265617Z","iopub.status.busy":"2023-10-07T05:58:16.265045Z","iopub.status.idle":"2023-10-07T05:58:16.274597Z","shell.execute_reply":"2023-10-07T05:58:16.273471Z","shell.execute_reply.started":"2023-10-07T05:58:16.265571Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["tensor([[     0,      0,      0,  ..., 496997, 496997, 496998],\n","        [299458, 299458, 299458,  ..., 496997, 496997, 496998]])\n"]}],"source":["print(edge_index)"]},{"cell_type":"markdown","metadata":{},"source":["# Final code \n","### Below we will show the final code for model.py, train.py and dataset.py"]},{"cell_type":"markdown","metadata":{},"source":["# Model Architecture\n","In this section, we used Graph Attention Networks as our backbone model.  \n","The model built with two GATConv layers followed by a linear layer with sigmoid outout for classification"]},{"cell_type":"code","execution_count":29,"metadata":{"trusted":true},"outputs":[],"source":["import torch\n","import torch.nn as nn\n","import torch.nn.functional as F\n","import torch_geometric.transforms as T\n","from torch_geometric.nn import GATConv, Linear\n","\n","class GAT(torch.nn.Module):\n","    def __init__(self, in_channels, hidden_channels, out_channels, heads):\n","        super().__init__()\n","        self.conv1 = GATConv(in_channels, hidden_channels, heads, dropout=0.6)\n","        self.conv2 = GATConv(hidden_channels * heads, int(hidden_channels/4), heads=1, concat=False, dropout=0.6)\n","        self.lin = Linear(int(hidden_channels/4), out_channels)\n","        self.sigmoid = nn.Sigmoid()\n","\n","    def forward(self, x, edge_index, edge_attr):\n","        x = F.dropout(x, p=0.6, training=self.training)\n","        x = F.elu(self.conv1(x, edge_index, edge_attr))\n","        x = F.dropout(x, p=0.6, training=self.training)\n","        x = F.elu(self.conv2(x, edge_index, edge_attr))\n","        x = self.lin(x)\n","        x = self.sigmoid(x)\n","        \n","        return x"]},{"cell_type":"markdown","metadata":{},"source":["# PyG InMemoryDataset\n","Finally we can build the dataset with above functions"]},{"cell_type":"code","execution_count":30,"metadata":{"trusted":true},"outputs":[],"source":["class AMLtoGraph(InMemoryDataset):\n","\n","    def __init__(self, root: str, edge_window_size: int = 10,\n","                 transform: Optional[Callable] = None,\n","                 pre_transform: Optional[Callable] = None):\n","        self.edge_window_size = edge_window_size\n","        super().__init__(root, transform, pre_transform)\n","        self.data, self.slices = torch.load(self.processed_paths[0])\n","\n","    @property\n","    def raw_file_names(self) -> str:\n","        return 'HI-Small_Trans.csv'\n","\n","    @property\n","    def processed_file_names(self) -> str:\n","        return 'data.pt'\n","\n","    @property\n","    def num_nodes(self) -> int:\n","        return self._data.edge_index.max().item() + 1\n","\n","    def df_label_encoder(self, df, columns):\n","        le = preprocessing.LabelEncoder()\n","        for i in columns:\n","            df[i] = le.fit_transform(df[i].astype(str))\n","        return df\n","\n","\n","    def preprocess(self, df):\n","        df = self.df_label_encoder(df,['Payment Format', 'Payment Currency', 'Receiving Currency'])\n","        df['Timestamp'] = pd.to_datetime(df['Timestamp'])\n","        df['Timestamp'] = df['Timestamp'].apply(lambda x: x.value)\n","        df['Timestamp'] = (df['Timestamp']-df['Timestamp'].min())/(df['Timestamp'].max()-df['Timestamp'].min())\n","\n","        df['Account'] = df['From Bank'].astype(str) + '_' + df['Account']\n","        df['Account.1'] = df['To Bank'].astype(str) + '_' + df['Account.1']\n","        df = df.sort_values(by=['Account'])\n","        receiving_df = df[['Account.1', 'Amount Received', 'Receiving Currency']]\n","        paying_df = df[['Account', 'Amount Paid', 'Payment Currency']]\n","        receiving_df = receiving_df.rename({'Account.1': 'Account'}, axis=1)\n","        currency_ls = sorted(df['Receiving Currency'].unique())\n","\n","        return df, receiving_df, paying_df, currency_ls\n","\n","    def get_all_account(self, df):\n","        ldf = df[['Account', 'From Bank']]\n","        rdf = df[['Account.1', 'To Bank']]\n","        suspicious = df[df['Is Laundering']==1]\n","        s1 = suspicious[['Account', 'Is Laundering']]\n","        s2 = suspicious[['Account.1', 'Is Laundering']]\n","        s2 = s2.rename({'Account.1': 'Account'}, axis=1)\n","        suspicious = pd.concat([s1, s2], join='outer')\n","        suspicious = suspicious.drop_duplicates()\n","\n","        ldf = ldf.rename({'From Bank': 'Bank'}, axis=1)\n","        rdf = rdf.rename({'Account.1': 'Account', 'To Bank': 'Bank'}, axis=1)\n","        df = pd.concat([ldf, rdf], join='outer')\n","        df = df.drop_duplicates()\n","\n","        df['Is Laundering'] = 0\n","        df.set_index('Account', inplace=True)\n","        df.update(suspicious.set_index('Account'))\n","        df = df.reset_index()\n","        return df\n","    \n","    def paid_currency_aggregate(self, currency_ls, paying_df, accounts):\n","        for i in currency_ls:\n","            temp = paying_df[paying_df['Payment Currency'] == i]\n","            accounts['avg paid '+str(i)] = temp['Amount Paid'].groupby(temp['Account']).transform('mean')\n","        return accounts\n","\n","    def received_currency_aggregate(self, currency_ls, receiving_df, accounts):\n","        for i in currency_ls:\n","            temp = receiving_df[receiving_df['Receiving Currency'] == i]\n","            accounts['avg received '+str(i)] = temp['Amount Received'].groupby(temp['Account']).transform('mean')\n","        accounts = accounts.fillna(0)\n","        return accounts\n","\n","    def get_edge_df(self, accounts, df):\n","        accounts = accounts.reset_index(drop=True)\n","        accounts['ID'] = accounts.index\n","        mapping_dict = dict(zip(accounts['Account'], accounts['ID']))\n","        df['From'] = df['Account'].map(mapping_dict)\n","        df['To'] = df['Account.1'].map(mapping_dict)\n","        df = df.drop(['Account', 'Account.1', 'From Bank', 'To Bank'], axis=1)\n","\n","        edge_index = torch.stack([torch.from_numpy(df['From'].values), torch.from_numpy(df['To'].values)], dim=0)\n","\n","        df = df.drop(['Is Laundering', 'From', 'To'], axis=1)\n","\n","        edge_attr = torch.from_numpy(df.values).to(torch.float)\n","        return edge_attr, edge_index\n","\n","    def get_node_attr(self, currency_ls, paying_df,receiving_df, accounts):\n","        node_df = self.paid_currency_aggregate(currency_ls, paying_df, accounts)\n","        node_df = self.received_currency_aggregate(currency_ls, receiving_df, node_df)\n","        node_label = torch.from_numpy(node_df['Is Laundering'].values).to(torch.float)\n","        node_df = node_df.drop(['Account', 'Is Laundering'], axis=1)\n","        node_df = self.df_label_encoder(node_df,['Bank'])\n","        node_df = torch.from_numpy(node_df.values).to(torch.float)\n","        return node_df, node_label\n","\n","    def process(self):\n","        df = pd.read_csv(self.raw_paths[0])\n","        df, receiving_df, paying_df, currency_ls = self.preprocess(df)\n","        accounts = self.get_all_account(df)\n","        node_attr, node_label = self.get_node_attr(currency_ls, paying_df,receiving_df, accounts)\n","        edge_attr, edge_index = self.get_edge_df(accounts, df)\n","\n","        data = Data(x=node_attr,\n","                    edge_index=edge_index,\n","                    y=node_label,\n","                    edge_attr=edge_attr\n","                    )\n","        \n","        data_list = [data] \n","        if self.pre_filter is not None:\n","            data_list = [d for d in data_list if self.pre_filter(d)]\n","\n","        if self.pre_transform is not None:\n","            data_list = [self.pre_transform(d) for d in data_list]\n","\n","        data, slices = self.collate(data_list)\n","        torch.save((data, slices), self.processed_paths[0])"]},{"cell_type":"markdown","metadata":{},"source":["# Model Training \n","As we cannot create folder in kaggle, please follow the instructions in https://github.com/issacchan26/AntiMoneyLaunderingDetectionWithGNN before you start training "]},{"cell_type":"code","execution_count":33,"metadata":{"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["Epoch: 000, Loss: 11203.3604\n","accuracy: 0.9728105437891242\n","Epoch: 001, Loss: 3255.3527\n","accuracy: 0.9727747383147218\n","Epoch: 002, Loss: 2784.3019\n","accuracy: 0.9729629777706701\n","Epoch: 003, Loss: 2315.4741\n","accuracy: 0.9731756547458457\n","Epoch: 004, Loss: 2079.3264\n","accuracy: 0.9731910450296517\n","Epoch: 005, Loss: 2036.6617\n","accuracy: 0.9733516552154903\n","Epoch: 006, Loss: 1957.6401\n","accuracy: 0.9732417582417583\n","Epoch: 007, Loss: 1897.1288\n","accuracy: 0.9732636355458716\n","Epoch: 008, Loss: 1872.3352\n","accuracy: 0.9733338659901924\n","Epoch: 009, Loss: 1860.6035\n","accuracy: 0.9733846576849051\n","Epoch: 010, Loss: 1832.2351\n","accuracy: 0.973404680536403\n","Epoch: 011, Loss: 1819.1463\n","accuracy: 0.9733398593575195\n","Epoch: 012, Loss: 1793.2119\n","accuracy: 0.9734198524379705\n","Epoch: 013, Loss: 1796.1004\n","accuracy: 0.973441997121842\n","Epoch: 014, Loss: 1765.4483\n","accuracy: 0.9734849052925209\n","Epoch: 015, Loss: 1756.6054\n","accuracy: 0.9734638406731221\n","Epoch: 016, Loss: 1758.1362\n","accuracy: 0.9735426747966262\n","Epoch: 017, Loss: 1735.6409\n","accuracy: 0.9734305548619088\n","Epoch: 018, Loss: 1732.0451\n","accuracy: 0.9734943491251387\n","Epoch: 019, Loss: 1722.9502\n","accuracy: 0.9735556266112433\n"]}],"source":["import torch\n","import torch_geometric.transforms as T\n","from torch_geometric.loader import NeighborLoader\n","\n","device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n","dataset = AMLtoGraph('/path/to/folder/of/data')\n","data = dataset[0]\n","epoch = 20\n","\n","model = GAT(in_channels=data.num_features, hidden_channels=16, out_channels=1, heads=8)\n","model = model.to(device)\n","criterion = torch.nn.BCELoss()\n","optimizer = torch.optim.SGD(model.parameters(), lr=0.0001)\n","\n","split = T.RandomNodeSplit(split='train_rest', num_val=0.1, num_test=0)\n","data = split(data)\n","\n","train_loader = loader = NeighborLoader(\n","    data,\n","    num_neighbors=[30] * 2,\n","    batch_size=256,\n","    input_nodes=data.train_mask,\n",")\n","\n","test_loader = loader = NeighborLoader(\n","    data,\n","    num_neighbors=[30] * 2,\n","    batch_size=256,\n","    input_nodes=data.val_mask,\n",")\n","\n","for i in range(epoch):\n","    total_loss = 0\n","    model.train()\n","    for data in train_loader:\n","        optimizer.zero_grad()\n","        data.to(device)\n","        pred = model(data.x, data.edge_index, data.edge_attr)\n","        ground_truth = data.y\n","        loss = criterion(pred, ground_truth.unsqueeze(1))\n","        loss.backward()\n","        optimizer.step()\n","        total_loss += float(loss)\n","    if epoch%10 == 0:\n","        print(f\"Epoch: {i:03d}, Loss: {total_loss:.4f}\")\n","        model.eval()\n","        acc = 0\n","        total = 0\n","        with torch.no_grad():\n","            for test_data in test_loader:\n","                test_data.to(device)\n","                pred = model(test_data.x, test_data.edge_index, test_data.edge_attr)\n","                ground_truth = test_data.y\n","                correct = (pred == ground_truth.unsqueeze(1)).sum().item()\n","                total += len(ground_truth)\n","                acc += correct\n","            acc = acc/total\n","            print('accuracy:', acc)"]},{"cell_type":"markdown","metadata":{},"source":["# Future Work\n","In this notebook, we performed the node classification with GAT and the result accuracy looks satisfied.  \n","However, it may due to highly imbalance data of the dataset. It is suggested that balance the class of 1 and 0 in the data preprocessing. It is expected that the accuracy will dropped a little bit after balancing the data.  We will keep exploring to see if there are any other models give better performance, such as other traditional regression/classifier model."]},{"cell_type":"markdown","metadata":{},"source":["## Reference\n","Some of the feature engineering of this repo are referenced to below papers, highly recommend to read:\n","1. [Weber, M., Domeniconi, G., Chen, J., Weidele, D. K. I., Bellei, C., Robinson, T., & Leiserson, C. E. (2019). Anti-money laundering in bitcoin: Experimenting with graph convolutional networks for financial forensics. arXiv preprint arXiv:1908.02591.](https://arxiv.org/pdf/1908.02591.pdf)\n","2. [Johannessen, F., & Jullum, M. (2023). Finding Money Launderers Using Heterogeneous Graph Neural Networks. arXiv preprint arXiv:2307.13499.](https://arxiv.org/pdf/2307.13499.pdf)"]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.9.17"}},"nbformat":4,"nbformat_minor":4}