Nordic Model of GPT Being Developed

While ChatGPT has taken the world by storm, RISE and partners are developing similar models specifically trained for Nordic languages. This initiative addresses a critical gap: major AI language models are primarily trained on English, leaving smaller languages underserved.

The Development Partnership

The Nordic language model effort brings together:

• RISE
• AI Sweden
• WASP (Wallenberg AI, Autonomous Systems and Software Program)
• NVIDIA

Progress and Scale

According to AI researcher Joakim Nivre at RISE: “It’s this ability to have long dialogues that really sets it apart from other language models.”

The development progress has been impressive:

• Largest completed model: 20 billion parameters
• Development target: 40 billion parameters
• Goal: GPT3-equivalent capability for Nordic languages

For comparison, OpenAI’s GPT3 uses 175 billion parameters.

The Core Challenge

Training large language models requires:

• Massive amounts of training data
• Enormous computing resources
• Significant financial investment

These barriers have limited who can develop such models, making this collaborative Nordic effort particularly important.

Practical Applications

Rather than replacing search engines, Nivre suggests the technology will:

• Generate text drafts for human review
• Complement search tools for real-time information
• Enable Swedish-language AI applications
• Support government and public sector use cases

Key Limitations

Nivre notes important constraints: language models lack knowledge of current events beyond their training data, and occasionally produce factual errors. This means they work best when humans remain in the loop.

Strategic Importance

By developing Nordic language capabilities locally, Sweden and its neighbors reduce dependence on foreign AI platforms and ensure that AI tools work properly for Swedish speakers.