Beyond GPT-2: Choosing a Modern Model for Our Poem Generator

Blog assignment: Introduction to Large Language Models (LLMs)

In our Natural Language Processing (NLP) course, we developed a Poem Generator using Transformer models. Our project explores three stages: building a small GPT-style model (Tiny GPT) from scratch, experimenting with the pretrained GPT-2 model, and comparing their generated poems.

Although these models helped us understand how language generation works, we wanted to investigate whether modern Large Language Models (LLMs) could produce better poems. In this blog, we compare six models, examine their strengths and limitations, and select one model that fits our project and available computing resources.

Tiny GPT learns from a small poetry dataset. Consequently, it may learn common poetic patterns without fully understanding the relationships between ideas. GPT-2 was pretrained on a much larger text collection, making it more capable of generating fluent text. However, its original context window is 1,024 tokens, and its base version is not specifically trained to follow detailed instructions.

For example, asking GPT-2 to write four rhyming lines about rain at night does not guarantee that it will produce exactly four lines or maintain a consistent rhyme scheme.

We therefore need a model that follows instructions, generates coherent text, respects requested poem structures, and runs on hardware available to our team.

We used five criteria to compare candidate models:

We researched six models from different developers. Their parameter counts, context windows, licences, and intended strengths are summarized below.

Note: Context limits can depend on the model variant and software configuration. Open-weight does not mean every model has an unrestricted open-source licence.

GPT-2, developed by OpenAI, is a useful baseline for our project because it demonstrates how a Transformer predicts the next token to generate text. Its relatively short context window and lack of instruction tuning in the original base model make it less convenient for prompts with several constraints. It remains valuable for comparing modern models against an established text-generation approach.

Gemma 3 is developed by Google. Its 4-billion-parameter instruction-tuned version supports a 128K-token context window and is designed for tasks including text generation, reasoning, and summarization. It is a strong candidate for creative writing, but its Gemma-specific terms differ from standard permissive licences.

Llama 3.2 is developed by Meta. Its 3-billion-parameter text model is designed for dialogue and other language tasks. Its relatively small size makes it attractive for resource-constrained environments. However, its Community Licence and acceptable-use requirements must be considered, and its poetry performance would need to be tested.

Phi-4 Mini Instruct is developed by Microsoft and contains approximately 3.8 billion parameters. It supports a 128K-token context window and is particularly focused on reasoning, mathematics, coding, and instruction following. Although these capabilities are useful, we would need experimental evidence to establish whether it offers an advantage for poetry.

SmolLM3-3B is developed by Hugging Face. It is a compact instruction-tuned model with support for long-context tasks and multiple languages. Its Apache 2.0 licence and publicly documented training approach make it attractive for experimentation. Nevertheless, we cannot assume that it generates better poems without testing it on our prompts.

Qwen3-4B-Instruct-2507 is developed by the Qwen team at Alibaba. It contains approximately four billion parameters and supports a native context window of 262,144 tokens. Its model card reports capabilities in instruction following, text comprehension, reasoning, and open-ended text generation. These features make it a promising candidate for our project.

We selected Qwen3-4B-Instruct-2507 because it offers a useful balance between instruction following, generation capabilities, computing requirements, and licence flexibility.

First, it follows detailed instructions. We can prompt it to write a poem with a specific topic, number of lines, tone, and rhyme scheme. Its instruction-tuned design makes it a better fit for this task than an unmodified GPT-2 base model, although compliance must still be measured.

Second, it is practical to experiment with. The model has approximately four billion parameters. Its published BF16 weights occupy about 8 GB, so loading it on a 16 GB Colab T4 may be feasible. However, model weights are not the only memory requirement: the attention cache, input length, generated tokens, and software overhead also consume GPU memory. We will begin with short prompts and modest generation lengths, monitor GPU memory, and reduce the workload or use quantization if necessary.

Third, its licence supports our research. Qwen3-4B-Instruct-2507 uses Apache 2.0, which permits use and modification subject to its conditions. We can download the model weights and run local inference through supported software without needing a paid inference API or API key. We must still follow the licence and any applicable platform requirements.

Fourth, it fits our existing workflow. The model supports Hugging Face Transformers and uses a chat template for structured prompts. This allows us to build on our existing Python knowledge and keep the experiment manageable.

Finally, its long context window provides room for larger prompts and additional instructions. Our current poems do not require 262K tokens, but the capability is useful if we later experiment with longer creative-writing tasks.

To make our comparison fairer, we will use the same set of poem prompts for Tiny GPT, GPT-2, and Qwen3. We will keep the requested topics, poem lengths, and evaluation criteria consistent. Where possible, we will use comparable generation settings, while respecting each model's recommended configuration.

We will evaluate each output using six criteria:

We can use a simple scoring rubric, such as 1–5 for each criterion, and ask human reviewers to assess the results. We will report the actual scores after running the experiment rather than inventing results in advance. We will also record generation time and peak GPU memory usage to understand the practical cost of each model.

Our comparison has several limitations. First, Tiny GPT, GPT-2, and Qwen3 differ in size, training data, and training methods. Therefore, differences in their output cannot be attributed to model architecture alone.

Second, even an instruction-tuned model can ignore constraints, repeat phrases, or generate awkward rhymes. Fluency does not guarantee creativity or quality. Third, the models may reproduce biases or familiar patterns learned during training.

Finally, Qwen3 requires more computing resources than Tiny GPT. If we want it to imitate the specific style of our poetry dataset, we could explore parameter-efficient fine-tuning, such as LoRA, if sufficient training data and hardware are available.

Our experiments with Tiny GPT and GPT-2 introduced us to Transformer-based text generation. Researching modern LLMs showed us that instruction tuning, training methods, model size, and context length can make a difference to how well a model responds to creative-writing requests.

We chose Qwen3-4B-Instruct-2507 because it combines instruction-following capabilities, a permissive licence, compatibility with our Python workflow, and a model size that may fit our Colab environment with careful memory management.

Our next step is to test it against Tiny GPT and GPT-2 using identical prompts and transparent evaluation criteria. This will help us determine whether the modern model actually improves our Poem Generator rather than relying on assumptions.