Traditional Programming is based on the idea of writing a program, giving it an input and getting an output. This works well for all the tasks where the rules can be clearly defined. Consider the problem of classifying a number as odd or even. This can be done by a simple if-else program.
For problems where the rules can not be clearly defined, we use Machine Learning to generate these rules for us. Consider the problem of classifying an image as cat or dog. Writing a program for this would be very difficult. Machine Learning is the idea where we provide the computer with data and corresponding outputs and get the program. This phase is called training. Now we use these program, along with new data like traditional programming to get an output. This phase is called inference.
Machine Learning is a subset of Artificial Intelligence (AI). While AI aims to imitate human thinking, Machine Learning focuses on using statistics to uncover patterns in data. For instance, in games like chess, AI uses strategies like minimax, similar to how humans strategize, while Machine Learning methods such as Linear Regression aim to draw the best-fitting line through data points, relying on statistics and pattern recognition rather than mimicking human thought processes.
At the heart of machine learning is the quest to find a function f(x) that closely approximates the relationship between inputs and outputs in the real world. Unlike traditional programming, where functions are manually defined, machine learning algorithms learn from data to automatically derive the most suitable function or model for a given task.
The ML-Toolbox is like a toolkit full of different machine learning methods, each offering its own form of f(x). The trick is picking the right one for the job, which is kind of like choosing a setting on a tool – it depends on what we are trying to do. Neural networks are popular, but they're just one tool in the box, giving us outputs in the form of weights and biases.
The core concept behind the ML-Toolbox is to grasp the diverse range of algorithms capable of generating forms of f(x). Some widely used algorithms include Decision Trees, Neural Networks, Support Vector Machines, Random Forests, and K-Nearest Neighbors. The goal isn't to say which method is the best. Instead, it's about knowing when each method works well and when it might struggle. It's like knowing when to use a screwdriver versus a hammer.
👨💻ML-Toolbox
┣ 📂assets
┃ ┣ 📂data // datasets
┃ ┃ ┣ 📄articles.csv
┃ ┃ ┣ 📄gender.csv
┃ ┃ ┣ 📄modified_mumbai_house_price.csv
┃ ┃ ┣ 📄mumbai_house_price.csv
┃ ┃ ┣ 📄student_marksheet.csv
┃ ┃ ┣ 📄titanic.csv
┃ ┣ 📂img
┃ ┣ 📂scripts
┃ ┃ ┣ 📄house price dataset.py
┣ 📂Concept Learning // Classification
┃ ┣ 📄titanic survival prediction.ipynb
┃ ┣ 📄README.md
┣ 📂K Nearest Neighbors // Classification, Regression & Recommendation
┃ ┣ 📄gender prediction.ipynb
┃ ┣ 📄house price prediction.ipynb
┃ ┣ 📄article recommendation.ipynb
┃ ┣ 📄README.md
┣ 📂Perceptron // Classification
┃ ┣ 📄gender prediction.ipynb
┃ ┣ 📄README.md
┣ 📂Naive Bayes // Classification
┃ ┣ 📄gender prediction.ipynb
┃ ┣ 📄README.md
┣ 📂Logistic Regression // Classification
┃ ┣ 📄gender prediction.ipynb
┃ ┣ 📄README.md
┣ 📂Linear Regression // Regression
┃ ┣ 📄house price prediction.ipynb
┃ ┣ 📄README.md
┣ 📂Support Vector Machine // Classification
┃ ┣ 📄gender prediction.ipynb
┃ ┣ 📄README.md
┣ 📂Kernels
┃ ┣ 📂Perceptron
┃ ┃ ┣ 📄gender prediction.ipynb
┃ ┣ 📂Linear Regression
┃ ┃ ┣ 📄house price prediction.ipynb
┃ ┣ 📂Support Vector Machine
┃ ┣ 📄gender prediction.ipynb
┃ ┣ 📄README.md
┣ 📂Gaussian Processes // Regression
┃ ┣ 📄house price prediction.ipynb
┃ ┣ 📄README.md
┣ 📂Neural Networks
┃ ┣ 📄house price prediction.ipynb
┃ ┣ 📄README.md
┣ 📂K Means Clustering // Clustering & Recommendation
┃ ┣ 📄grouping students.ipynb
┃ ┣ 📄article recommendation.ipynb
┃ ┣ 📄README.md
┣ 📄README.md
The following tasks are still in progress:
- Implementation & Verification of Kernels
- Implementation of Neural Networks
- Implementation of Decision Trees
- Implementation of Association Rule Mining
- Mid Term Review
- Video lectures 25 - 34 of Cornell CS4780
- Gaussian Processes
- Big thanks to Prof. Kilian Weinberger for the Cornell CS4780 course, Machine Learning for Intelligent Systems. Majority of the content in this repository is inspired from the lectures.
- MIT 6.036 Machine Learning by Prof. Tamara Broderick.
- Bias Variance Tradeoff by MIT OpenCourseware and The Stanford NLP Group.
- Additional resources to understand kernelizations.
- Neural Networks and Deep Learning Online Book by Michael Nielsen.
- Kaggle for providing several datasets used in this repository.