You might have worked on various data processing techniques, machine learning algorithms from understanding the business requirement, building the machine learning model to model’s validation process. Through this series, we are going to learn how to build ML model from scratch using Microsoft Azure Services i.e. on cloud for fraud detection.
In part-1 of this tutorial, I have given an brief introduction about cloud computing, how to use Azure notebook and steps followed to create a machine learning model.
Table of Content:
- What’s the Cloud computing?
- What’s the need of cloud services for machine learning?
- What is Azure notebook?
- What are the steps to build the model in Azure notebook?
- What is Microsoft Azure service?
- How to build machine learning model using Microsoft Azure?
What is cloud computing?
Cloud computing is process through which companies offer cloud services through internet. Services may include server, data storage, access to different software etc.
What’s the need for cloud services in machine learning?
Most of the time for learning purpose machine learning model is created and validated on local machine.
Okay! Then what’s the need of using cloud services and pay for it
Reason cloud services needed for:
- Machine learning model require a lot of time and space for training. It’s quite difficult to get access of high computing power devices to everyone with low cost. Cloud services offer cheap solution for such problems. They provides high computing hardware and storage option on real time basis.
For this post we will discuss Fraud detection case. You can download the data from the following GitHub link.
What is Azure notebook:
Azure notebook is an IDE that is used to develop the code in various languages (R, python, Julia). It works in similar manner as Jupyter notebook does.
Steps followed to create a project in Azure notebook:
- To access Azure notebook, first create an Microsoft account, login and start new project from ‘My projects’ section.
- Either create a new notebook or you can upload from local machine using the upload option.
- Import the data required, now notebook is ready to run.
Build machine learning model on Azure notebook
Remember, goal of this article to explain how to use Azure notebook and how to use Azure portal to build machine learning model. So I am skipping the model development part. Still if you are interested you can follow the steps given. You can check the code on GitHub link given below:
To build machine learning model,we need to follow these steps:
- Data pre-processing
- Data cleaning (remove missing values)
- Label Encoding
- Feature Selection
- Apply machine learning model
- Check accuracy of the model
What is Azure machine learning service?
Microsoft Azure service is a platform which allows it’s users to build, train,deploy and manage machine learning models. To use azure services, it’s need to create an account on it’s portal.
How to use Azure machine learning service?
- To use Microsoft cloud service, first make an account on Microsoft Azure portal. Microsoft provides 30 days free trial on login first time.
- Once logged in, Azure dashboard will be opened like this:
- Create new resource and select machine learning category to create workspace
- Once selected, fill the details required. Region specifies the location from where cloud services will run. Here I have selected basic edition of workspace. You can find more details on Microsoft documentation.
- Once details are filled, hit on review + create, then it will review the details and then create the resource.
- Once deployment is done, workspace is created.
- Download the configuration file. This file contains information about subscription_id, resource_group and workspace name. These will be needed to deploy the model later.
- Workspace can be created using Azure notebook as well using following step:
Sometimes, it shows error as azureml.core does not exist. In that case, install the package using command:
!pip install azureml-core
- After creating workspace in Azure notebook, create config file so that this workspace can be used in multiple environment.
- Load workspace from config file
from azureml.core import Workspace ws_new = Workspace.from_config() ws_new.get_details()
- Now train the machine learning model on Azure network and save the model using joblib package in the form of ‘pickle’ file.
# Save the trained model joblib.dump(value=clf, filename='model_fd.pkl')
- After training, register the model on workspace created on Azure portal so that this model can be used anytime to test the data.
from azureml.core.model import Model model = Model.register(workspace=ws_new, model_path="./model_fd.pkl", model_name="fraud_detection")
After running the above command, you can check the model created on portal
So far, we have learnt how to build machine learning model on local machine and how to save it on cloud so that it can be used for further deployment.
In next post, we will discuss how to deploy machine learning model on cloud to use it for production.