David versus Goliath, but in the AI world: Meet Martel’s researcher training language models on a single laptop while industry giants invest in warehouse-sized computing facilities. Armed with just a laptop and determination, Dr. Amjad Majid is challenging the notion that bigger is always better in the world of generative AI.
Generative AI has undeniably transformed the technological landscape, but its substantial resource requirements often place it out of reach for individual developers and smaller organisations. While industry giants suggest that breakthroughs in Generative AI (GenAI) demand vast computing resources, Dr. Amjad Majid, a senior researcher at Martel Innovate, aims to challenge this notion. Drawing inspiration from OpenAI’s humble beginnings with ChatGPT, Dr. Majid seeks to democratise GenAI innovation by making it accessible on modest hardware.
Driven by the question, “How far can we push GenAI with just a laptop?”, Dr. Majid developed LLM Toaster — a framework designed to facilitate the training of small, transformer-based language models (LLMs) on a single GPU-equipped laptop. LLM Toaster allows users to seamlessly start, stop, and resume training sessions while maintaining comprehensive logging and time tracking. This tool empowers developers to experiment with GenAI without the need for extensive cloud resources or large-scale computing clusters. 🔗 Explore LLM Toaster: GitHub – amjadmajid/llm_toaster
Using LLM Toaster, Dr. Majid trained BabyGPT, a 152-million-parameter model, over 125 hours across multiple sessions. BabyGPT was trained on Hugging Face’s fineWeb-Edu dataset 📚—a 27 GB collection of high-quality educational content, which tokenizes down to 10 GB using the GPT-2 tokenizer. You can review the training logs and a loss chart 📊 here: 🔗 BabyGPT Training Logs: GitHub – amjadmajid/BabyGPT
Try BabyGPT Yourself: Experience prompt completion tasks with BabyGPT at https://baby-gpt.com. While BabyGPT isn’t designed to rival advanced models like ChatGPT, it serves as a proof of concept for achieving meaningful GenAI results on a single laptop.
Conclusion and Next Steps
This project demonstrates that even a single laptop can make a meaningful contribution to GenAI experimentation and development. Although large-scale training may eventually require cloud resources, tools like LLM Toaster open doors for innovation with smaller models and new algorithms at the edge. Next, Amjad plans to focus on optimization and experiment with algorithmic adjustments 🔧 to further harness the power of GenAI on limited resources.
Stay tuned for more updates as we continue to push the boundaries of GenAI on limited resources!
