Emoji Selector

Scratch-trained text-to-emoji experiment. The repository trains a causal language model that accepts arbitrary text and returns a likely emoji or generates text inline with emojis.

Two models are included:

1. emoji-selector-0.005b: A 5M parameter model trained on short phrases to predict a single emoji.
2. son-of-emoji-monster-0.005b: A 5M parameter model trained on generated stories to produce coherent text interweaved with emojis. (Available natively and as an Ollama GGUF model).

This repo is intentionally local-first: the dataset, validation reports, prepared splits, model checkpoints, inference scripts, and training scripts all live in this directory.

Setup & Installation

To install dependencies and get ready to train or use inference, run:

# 1. Create a virtual environment
python3 -m venv .venv
source .venv/bin/activate

# 2. Install dependencies
pip install -r requirements.txt

Datasets

The raw data is parsed from master_training_data.csv. The prepared datasets used for training are already located in data/prepared/:

• data/prepared/all_stories.txt (Used for son-of-emoji-monster)
• data/prepared/train.jsonl and val.jsonl (Used for emoji-selector)

Training Your Own Model

You can train your own models from scratch. Both scripts will automatically build a tokenizer and a GPT-2 architecture before starting the training loop.

1. Training Emoji-Selector

To train the phrase-to-emoji classifier:

source .venv/bin/activate
python scripts/train_emoji.py --config configs/emoji-selector-0.005b.yaml

Checkpoints are saved periodically. You can resume training using --resume-from if needed.

2. Training Son of Emoji Monster (Text Generation)

To train the story generation model:

source .venv/bin/activate
python scripts/train_stories.py --output-dir models/son-of-emoji-monster-0.005b

To resume training from a checkpoint:

python scripts/train_stories.py --resume-from models/son-of-emoji-monster-0.005b/checkpoints/step-26000

Inference

Emoji Selector

To get an emoji prediction for given strings:

./test-inferrence.py "I am so happy today" "This weather is terrible"

Son of Emoji Monster (Batch Generation)

To test the generation model's Token Per Second (TPS) and coherence on a batch of prompts:

source .venv/bin/activate
python scripts/generate_batch.py

Ollama Support

The son-of-emoji-monster model has been converted to GGUF format (son-of-emoji-monster.gguf) so you can load it directly into Ollama:

1. A Modelfile is provided.
2. Run the following command to create the model in your Ollama instance:

   ollama create son-of-emoji-monster -f Modelfile
   

3. Run the model:

   ollama run son-of-emoji-monster