Data & ML | Scott Miner

IMDb Sentiment Classifier

Thu, 07 Nov 2024 00:00:00 +0000

A sentiment analysis model trained on the IMDb Movie Reviews dataset to classify reviews as positive or negative.

Approach

Preprocessing: tokenization, vocabulary building, and sequence padding of raw review text.
Embeddings: pre-trained GloVe word vectors give the network a head start on word meaning.
Model: a bidirectional LSTM reads each review in both directions, with batch normalization and dropout-style regularization to improve accuracy and generalization.
Training and evaluation: scripted training loop with held-out evaluation and metrics reporting.

What it demonstrates

End-to-end NLP with TensorFlow and Keras: text preprocessing, transfer learning with embeddings, recurrent architectures, and disciplined model evaluation.

Word2Vec Skip-Gram in TensorFlow 2

Tue, 11 Apr 2023 00:00:00 +0000

An implementation of the Word2Vec skip-gram algorithm in TensorFlow 2.0 that learns vector representations of words from a slice of Wikipedia (the text8 dataset).

How it works

The text8 corpus is downloaded (a helper script fetches it on demand) and processed into a vocabulary, dropping rare words below a frequency threshold.
A skip-gram model is trained with stochastic gradient descent and noise-contrastive estimation (NCE) loss.
During training the model is evaluated by finding the nearest neighbors of a set of test words using cosine similarity between their embeddings.

The implementation follows the approach described in Mikolov et al., Efficient Estimation of Word Representations in Vector Space (2013), and builds on Aymeric Damien’s TensorFlow examples.

What it demonstrates

Working knowledge of word embeddings, negative-sampling style objectives, and writing training and evaluation loops directly against the TensorFlow 2 API.

Universal EDA

Sat, 15 Apr 2023 00:00:00 +0000

Universal-EDA provides an easy-to-use interface for exploratory data analysis (EDA) and data loading so that the first hour with a new dataset is spent looking at the data rather than writing boilerplate.

Background

The project started while updating an older linear regression project and discovering that the Boston Housing dataset had been removed from newer versions of scikit-learn. Universal-EDA keeps classic teaching datasets available and wraps the repetitive first steps of any analysis.

Features

DataExplorer: summary statistics, missing-value reports, and quick visualizations for any DataFrame.
DataLoader: loads common datasets from scikit-learn and converts DataFrames to CSV files.

from src import DataExplorer, DataLoader

df = DataLoader().load_toy_dataset('boston')
DataExplorer(df).basic_info()

What it demonstrates

Packaging reusable analysis code into clean classes, and designing small APIs that make day-to-day data work faster.

Sequence Prediction ANN

Thu, 27 Apr 2023 00:00:00 +0000

A compact artificial neural network that learns to predict the next value in a numeric sequence, a small-scale version of the time-series forecasting problem.

Design

The code is deliberately modular so each concern can be swapped or tested on its own:

Data preparation: turns raw sequences into supervised learning windows and splits train and test sets.
Architecture: defines the layers and activation functions of the network.
Training: runs the optimization loop and reports evaluation metrics.

What it demonstrates

The fundamentals of supervised sequence learning, clean separation of concerns in machine learning code, and model evaluation with scikit-learn utilities.

Healthcare Expert System

Mon, 19 Jun 2023 00:00:00 +0000

A rule-based expert system that models the decision logic used in heart failure telemonitoring. Given patient-reported measurements and symptoms, the system applies clinical rules to suggest self-care actions and flag situations that need a clinician’s attention.

How it works

Domain knowledge is encoded as rules in Pyke, a Python knowledge engine.
Patient facts (weight changes, blood pressure, symptoms) are asserted into the knowledge base.
The inference engine fires matching rules to produce recommendations and alerts.

What it demonstrates

Symbolic AI and knowledge representation as a complement to statistical machine learning, and translating a real clinical workflow into explicit, auditable rules.